It seemed interesting back then but it's kind of dead since the core developers (Avi Kivity, Nadav Har'El and others) have moved on to another startup called ScyllaDB that is a re-implementation of Apache Cassandra in C++.
The OSv project is indeed now moving much slower than it used to when we had an entire startup company devoted to it, but is not dead - it still has three committers from three companies - myself (Nadav Har'El), Waldek Kozaczuk and Timmons Player. It still works, and you are welcome to try it.
OSv is not as minimal as includeOS, which of course depending on your viewpoint and use case, is either a pro or a con. OSv supports more types of hypervisors, and has support for a larger subset of the Linux ABI, can run pre-compiled, unmodified, Linux shared libraries, and supports multi-core VMs.
I work in an amazon FC (in the US), and not in any leadership roles. I recently moved to a new state, and I found amazon. I applied and within about 2 weeks I was working. I'm about 5 months in and it's actually decent.
> If I’ve learned anything from doing this job, it’s that money can’t replace time. I work four nights a week in an Amazon warehouse near my home
Is this a bad thing ? I'd rather do 4x 10 hour days, than 5x 8 hour days. I can work my ass off for 4 days and sleep for day 5, and still have 2 days off. My co-workers seem to like this as well.
> I have to put away each item in 15 seconds or less, and get through 250 in an hour, or I’ll be given a warning by a manager. Stepping away from my station to, say, get a drink of water can have a big impact on my performance.
Amazon has a lot of fulfillment centers. Customer obsession is #1. In order for customers to get their package on time, amazon needs metrics. A lot of associates work alone or in a small team, also their is minimal supervision. Metrics allow for managers and corporate to know many things, including whether employees are "on-task". The required rate changes in different departments. If his FC has robotics, then they deal with small items. 250 small items an hour isn't bad. I'm not too sure about his building but I have all day to make rate. At the end of the day your average rate per hour should be >= X.
> During my half-hour breaks I rush downstairs to have something to eat. It’s stressful – and it definitely affects my health, standing up for hours on end. I worry I may pass out if I don’t rest during my meal breaks. I’ve lost a lot of weight since I started.
You don't need to rush, at least where I work. It's actually a benefit if you take your time. You have 30 minutes from the time you clock out, and it's about a 45 second walk to break rooms. If you feel like your going to pass out, you should not be working in a warehouse. It can be a workout environment, depending on where you are placed. However, you will be standing anytime your not on break, and that can sometimes get annoying.
Amazon provides amazing time-off flexibility through an employee portal. If I ever feel too tired to work, and I have time-off available, I can just go tell my manager, HEY IM LEAVING. He says ok (or don't leave, I need you!), and the ball is in your court.
Amazon provides benefits from day 1, not usually found in warehouses. They also provide "Variable Pay Compensation". Every month you get a BONUS ! Provided you did not have unscheduled time off, you will get extra money based on your hours. Peak times just doubles how much you receive. I barely worked in December, and my vcp for that month is just under $300. You don't need to do anything special, its always just added to your payout.
One employee can never tell the whole story (including me). Different buildings with different people in all of them.
I use fastmail, and it's pretty fast. I too have wondered what was under the hood, but I figured it out a while ago [1]. The docs are pretty old, and it could use better examples, but its a solid library. If you look at the code, it won't take long to figure out how everything works. They also open sourced JMAP[2] which is the synchronization protocol they use.
traffic is as localized as possible. As a general rule you can think of, if the peer is more local to you than the nearest CDN PoP you'll receive data from the peer, otherwise you'll receive data from the CDN.
>Cool! Does this mean all peer traffic is local ? Also, how can you detect when traffic is metered ?
We combine both client side and server side methods for that.
If the server doesn't have access to the content of emails, then it reverts to a featureless blob store:
Search isn't possible
Previews can't be calculated
If you lose your private key, we can't recover your email
Spam checking on content isn't possible
To access mail on multiple devices, the private key needs to be shared securely between them
It absolutely is, in both theory and practice. The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.
> If you lose your private key, we can't recover your email
This is a damn feature. I had my icloud account social engineered (someone walked into an apple store claiming to be me and they couldn't get their iphone syncing to "their account"). I'll never again trust another company with my private stuff.
> Spam checking on content isn't possible
This is probably your best point. It's definitely harder to do well
> To access mail on multiple devices, the private key needs to be shared securely between them
This is a non-issue. It can easily be derived from a password
Regarding Apple and security, their policy via AppleCare seems to be to ASK (over the phone, for instance) for your cleartext computer administrator password before you send in your laptop for repair, without any warning whatsoever of the implications.
I spilled soda on my mac once, and took it to the repair shop. the receptionist there asked me for my password. I laughed and I said of course not. she was shocked and asked: well, how are we going to test the new keyboard. I don't know maybe try to type random things in the password field?
I had this EXACT experience just this month. I went to have a screen replaced and I even explained (and apologized for the inconvenience) that I was very security focused. I set it up in advance so that would perform the repair in front of me, that the device wouldn't be plugged into any of their computers, and it wasn't to be taken out of sight.
Then, he straight up asked me for my password, "most customers write their password down so we can test that it works."
I feel a little bad because the look I must have given him was pretty absurd. I told him no, I'll just take the risk and test it myself.
They have physical access to the device. They can enter recovery mode by holding down command r when restarting. It gives them a different copy of the os with multiple apps that you can type in (like the terminal) and would allow superuser access to whatever they wanted, however your encrypted home directory would remain encrypted and they would not be able to read it. In the olden days you would stick in a recovery cd and reboot onto that... you could also (historically and presently) stick a USB drive into the thing and boot into an os on that.... NEVER give out your password.
I thought that too, until i bought a cheap netbook that came with windows (7? Student edition? I cannot remember). I plugged in a mouse and windows said it needed to connect to the internet to download the driver for my MS optical mouse. I practically fell out of my chair laughing.
Except the mouse wouldnt work. Either the os couldnt use it, or wasnt letting me. Either way, it was funny seeing one ms product not recognize another.
> It absolutely is, in both theory and practice. The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.
Are you suggesting that to search your mailbox, the client should download every single encrypted message in the entire mailbox and decrypt them all locally to search them?
If not, how does the server get this "encrypted index" without having your private key?
The search doesn't happen on the server. The client (e.g. desktop client) indexes the messages and encrypts the index. Then, when another client (e.g. mobile client) wants to search for a message, it downloads the index, searches it, and then pulls down the appropriate message(s).
If an attacker can tell which part of the index was modified, that gives them enough information to decrypt the index and e-mails.
Clients would always have to download+upload the full index (which needs to be re-encrypted with a new IV). This is a huge problem - the index can easily be hundreds of MB for a large mailbox.
Technically if a client has a full index version X (in plaintext), it could modify X to get X+1, compute a binary diff between X and (X+1) - encrypt and upload the diff.
Another client on index version X could download the diff, and get index (X+1).
Some desktop client should probably do compaction from time to time.
You would have to re-download the index after compaction. But the index and patches would be independent - you apply the decrypted patches to the decrypted index.
I'm not quite sure what you're getting at. It sounds like you're describing a known-plaintext attack, which modern ciphers are not vulnerable to. And what you are describing makes it seem like full disk encryption would be totally useless, but we know that's not the case.
Well, the client has to fetch message once, decrypt it, and upload updates to the index (kept on server).
I guess, it is a reasonable sacrifice for privacy, unless you want searches to be able to search within attachments including documents in big archives.
So what, you hash each word in the e-mail and search for the hash, and this returns which emails include those hashed words? Would that be horribly insecure? I guess it would be impossible to salt those hashes, and it probably risks defeating the whole crypto.
You wouldn't use a hashing algorithm to build the index. They are talking about an inverted index in a binary format, something like what lucene outputs. That binary index would be encrypted with a block cipher (AES, Blowfish) using a secret key and would then be stored on the server.
The mobile client comes along and downloads & decrypts the index in memory, searches it for some terms(s), the index returns 10 result, of which, the user selects one and the mobile client downloads and decrypts to show to the user.
You can mostly get rid of spam by requiring the sender to perform a proof of work if they aren't in your contacts list. I.e. whitelist of senders + proof of work or some kind of configurable per domain quota/proof of work.
Truncated to something like fvv544+user@mydomain.com that would only be valid when sent with the from-adress new-sender@example.com (along with some clever magic to avoid email loops! :-)
I think maybe it was authored by ESR (Eric Raymond) in python - but Google only turns up various dkim schemes...
New senders would have their mail held back, and get a "please reply if you are human"-message - a reply (to the "magic" reply-to alias) would release the held mail and whitelist the sender.
I'm talking about now going forward. There are plenty of ways to make a person opt-in the first time they send you something. That process gets rid of more than 99% of spam.
It wouldn't be so good if email turned into Facebook, where you can only contact somebody if the circuit has been established beforehand (i.e., both parties friend each other). What happens if this is the point of initial contact, and there's no other way to get in touch? I have to conspicuously friend this person from my otherwise dormant account and then not think about whether they're going to feel weird when I unfriend them later on?
I sent an unsolicited email a couple weeks ago. Academic-type, self published, personal webpages where I saw something that needed fixing, and so I did. There was no CMS, just a static site—and not the sort where you're running markdown sources through a generator and have a whole Git hosting service apparatus intertwined with hooks sunk into it. Plain, legacy HTML. I saved a local copy to my machine, made my changes, and mailed them in. They happened to be machine-readable patches, given the recipient, but it could have just as well been a message written in natural language. The changes were made, I was thanked, and done.
I'm still upset that in the 90s I could get in contact with almost anybody by looking in the phonebook, but when I tried doing that a few years ago for an old coworker, it was hopeless. Getting the listings for a city you've moved away from can prove to be harder than it seems, even when you know they have a landline.
These are the kinds of things you lose when you assume the world is already fixed in the form that it should maintain going forward.
From every client that a former client has sent my way. 50% of my clients introduce themselves with some variation of "Something happened, Bill gave me your name. Help!".
> The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.
Going on a tangent, but do you know of any services that offer such a thing?
Also search that only uses email titles is still at least 70% as useful as full title+body search. So I wouldn't count this as such a terrible non-feature of e2e encrypted email.
The interpretation of IP law, at least internally at many big software companies, does in fact work like coodies. Hence things like "clean room implementations" of algorithms, modules, API interfaces etc.
Clean room matters for copyright. It doesn't help for patents, as patents protects the idea, not the expression of it so expressing the same idea in a different way doesn't help. With a patent you want to know the specifics of the original to ensure you make yours sufficiently different to sidestep the claims.
Where not knowing helps with patents it is in that if you infringe, wilfull infringement increases damages up to threefold.
Adding to the iCloud story, I forgot my security questions and I was able to have the AppleCare agent over the phone reset it for me after talking about what apps I've purchased recently.
It's icloud, not personal device storage. It has to be possible if you want recoverable content. If you don't like it, you can back up locally to itunes.
> > To access mail on multiple devices, the private key needs to be shared securely between them
> This is a non-issue. It can easily be derived from a password
How does that change the equation? You're still exposing the thing-that-decrypts to multiple devices, thus (many!) more threat vectors. Lose one, and you lose them all, which is the point of the claim.
No, the (implicit) point in the start of the thread is that multiple other encrypted mediums don't leak as much. Use them instead.
If you care enough to use encrypted email, you should probably seriously consider abandoning email. (signing is different - that's proof of identity and non-modification, useful in many non-private scenarios)
>I had my icloud account social engineered (someone walked into an apple store claiming to be me and they couldn't get their iphone syncing to "their account")
These are just minor problems. End to end encryption should not be forced upon users, but should be used selectively. You don't want to lose your entire inbox because you wanted to send a few encrypted emails.
> The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.
How slow would this be with thousands of emails on a mobile device ?
> This is a damn feature. I had my icloud account social engineered (someone walked into an apple store claiming to be me and they couldn't get their iphone syncing to "their account"). I'll never again trust another company with my private stuff.
Most email providers don't have retail stores where this type of attack can happen.
> This is a non-issue. It can easily be derived from a password
What happens if this password got into the wrong hands or even lost? Is it possible to change? Must I re-encrypt every single email ?
> How slow would this be with thousands of emails on a mobile device ?
Not that much slower than email is now. A B+tree index of (for example) 1 gb of total content and index is only about 1-2 gigs in size. You could just store that shit on your phone encrypted. Or 1-5 megs of index on your phone, with 2 round-trips for any email/first page search result per word.
> What happens if this password got into the wrong hands or even lost? Is it possible to change? Must I re-encrypt every single email ?
Overall yes, re-encryption is necessary. But that can be a batch process done while your phone is charging and on wifi over night. Is a drawback, but re-keying always is. But just because you got to change the keys every once in a while, doesn't mean you toss the locks.
> Overall yes, re-encryption is necessary. But that can be a batch process done while your phone is charging and on wifi over night. Is a drawback, but re-keying always is. But just because you got to change the keys every once in a while, doesn't mean you toss the locks.
Encrypted hard disks utilize a simple scheme whereby the password/key derived from the password decrypts a small set of keys or single key that's just randomly generated, and decrypts the rest of the drive. So, to change the password, all you need to reencrypt are the real keys.
Password -- decrypts --> master key -- decrypts --> rest of hard drive
^^^^^^^^^^
this is what gets re-encrypted on password change
and is small; ~a few KiB
Similar could be had for email.
The approach above also lends itself well to supporting multiple passwords. (If you share an account w/ someone, for example, though email has other tools like mailing lists that might be better suited.)
Lots do, though, and the ones that don't often offer customer support over the phone. And anyway, social engineering doesn't only happen in stores or when talking to CS.
Really? Name any of the top 10 email providers that have stores, or a responsive 800 number for email support. I'll give you Apple, but they are the clear outlier. MS, Google, have stores, but not with email support, and questionable phone support at best for any non enterprise product.
AT&T? I don't have a list of top 10 email providers, but them and Verizon and Yahoo have all had people report social engineering attacks against them to gain access to accounts.
> How slow would this be with thousands of emails on a mobile device ?
This in itself is not a problem. Btrees are O(log n) for search, so thousands of emails can be looked up in 3-4 requests with the most naive way to implement remote tree walk.
The fact you're revealing the structure and index contents to the server as you search would be a bigger issue.
Search is still possible. You can stem words and store their weights without storing the actual unencrypted text. This isn't perfect security, but it's good enough that it would be difficult for the government to successfully use the search metadata in a case against you without a lot of other evidence.
This is what we do on FWD:Everyone for email threads shared within private repositories.
Security is rarely good or bad in an absolute sense, but rather is judged by considering the assets under protection and the threat models you're protecting against.
Just because certain people need the sorts of protections provided by Lavabit doesn't mean that products like Gmail are bad or insecure. They're each secure for the use cases they're targeting.
I've never tried it but I'd bet you could use instead use a key derived from the master somehow to salt a hash for all the stems too. I think that'd let you still do the search and possibly have a good index for it still but it'd add some hurdles to the whole thing so that you couldn't make a lot of good guesses about the actual contents of the emails. It would still leak information a out related emails (say there's a thread discussing a unique term in it), but it'd still be better than otherwise.
In theory you could HMAC all the stems, but at that point you really need to ask yourself what threat model you're trying to protect against. E.g. keeping only the stems is more than enough to prevent competitors from learning anything useful about your business if your database gets compromised, but using an HMAC probably isn't going to significantly slow down a state-level actor who has a warrant or zero day for your web servers. So on the balance I think the technical debt that would be introduced would likely make the overall system less secure, at least for most use cases.
I would argue the biggest downside to encrypted email is the sender, recipient, date, and size are known to all intermediaries.
First of all, sometimes knowing who you talk to is more important than knowing what you say. Traffic analysis lets you draw all kinds of conclusions.
Secondly, there are two parties, so double the chances for rubber hose cryptography. An opponent can approach either party and ask for details, so both parties are trusting the other can't be broken by the opponent.
Of course, we should all be encrypting everything anyway, no point in giving out free message bodies.
If you don't trust the other party you should not be sending him sensitive data. He has to be able to see it so of course he will be able to reveal it.
It is not that you don't trust your party, but that anyone with access to the servers or network between you can learn about who you are communicating with and when. Such metadata are often more useful to use against someone than the actual content.
Well, you could always use an old school local email client and would get search and previews for free. And some 3rd party not being able to "recover" my private correspondence sounds more like a feature than an issue.
Oh no, you're describing the solution rather than the problem: a global search, copy, and seizure converted to a local one that happens on per-target basis. Way better than mass collection with FISA warrants and systems like QUANTUM.
It's the NSA system that operates globally that can automatically fire attacks at systems based on patterns the operations people put in. Almost all the European countries are part of a SIGINT-sharing alliance per one slide. The exceptions were Switzerland, Iceland, and one I can't remember.
I'm downloading my mail from POP3 servers, so I'm searching locally. I'm copying access password to my tablet and phone (K9 client) and I don't keep more than a few dozen of messages there because the POP3 mailboxes are cleaned up when I download from the laptop. I backup my mail to a remote server, encrypted with duplicity.
I understand that this is not acceptable for 99.99+% of people, even the technical ones, but I think I could use a fully encrypted mail store. No problem with the mail provider ending up as a blob store: privacy-wise it's what they should be anyway. No harm to their business, if all the money they make are from users and they're not selling data.
The only problem is centralized spam checking. Running an antispam engine locally wasn't very effective years ago and Thunderbird was sub par. Is there anything that's on par with email providers right now?
At the moment, encrypted spam doesn't exist. But only because there aren't enough people using encryption to make it worthwhile for the spammers to invest time doing it.
Theoretically, a spammer could scan the public keyservers for email addresses and public PGP keys and then send encrypted spam to all of those people. If ever a well targeted spam mailshot was sent, that would be it. But would you really want to get on the bad side of thousands of tech nerds?
I think email needs to stop being treated like a database. It should be used for sending and receiving messages, the rest should be handled by other software... like a database for example.
I personally think it's funny that people mix secure communication and email. Secure system should be built to be secure. Email isn't a good option for that. Most systems are designed to be insecure and email is just a great example.
