The govt comes by with a subpoena (secret, classified, or public) and requires Microsoft or the customer company to produce communication records that exist in a form that may be used as evidence. Failure to do so is best contempt of court and worst obstruction of justice. No 5th amendment privilege for other people's crimes. So everyone who chooses to store or process messages makes it so the encryption is reversible and they can honor court requests. Nothing is private as a result.
EDIT: I should make it clear I don't agree with the current status quo. Let me answer two very good questions.
> Would they also do that if logging failed during the period requested?
If it can be shown that there is a willful neglect of collecting logs then the govt in the past has gone after companies for some form of conspiracy (most famous: MegaUpload, but Microsoft customers have had their fair share for accounting and securities fraud) or criminal negligence. There is a prevailing theory that companies are responsible for employee actions, and failing to log is seen as unacceptable.
It's only recently (within the last year) that the courts have ruled whether compelling passwords is protected by the 5th amendment, and most systems in place were designed and built using previous assumptions from 10 years ago.
> If this is true, why is OTR offered at all?
It's a checkbox feature required for HIPAA, PCI, and similar. "Must have encryption" -- the standards and IT departments don't say how the keys are managed.
They would get them for contempt of court or obstruction of justice for being not unwilling but rather unable to fulfill the request?
That seems wrong. Would they also do that if logging failed during the period requested? Or if they simply neglected to log in the first place?
Edit to your Edit: I don't see how the 5th is involved in the slightest. If party A has a secret, and the courts subpena party B, then party B's inability to comply has nothing to do with pleading the 5th. They simply do not have the information requested, and in fact never possessed it in the first place. Furthermore, party B cannot possibly be said to have been negligent. The courts subpenaed the wrong person.
The point about the 5th amendment is that prior to last year's ruling, it was assumed that passwords (or private keys) could be compelled for any reason in the US, and systems were built with that assumption. I recommend you look into legal disclosure or e-discovery products. Not right or wrong, but that's the design assumptions backed by a bunch of lawyers in govt and private sector.
The example is incomplete. In messaging systems, Party A uses Party B to send messages, which may be useful for a court case. The government may reasonably subpoena Party B to produce Party A's records since they may not want to alert Party A that it is under investigation.
If Party B (the service provider) has any feature that makes use of the content of Party A's secrets (read: url-checking, auto-loading thumbnails, indexing for search, some types of routing, etc), then there is little ground for Party B to argue it can't decrypt Party A's records for the subpoena. Further, Party B may become a co-conspirator if it keeps incomplete records or destroys records too quickly. Even if the co-conspirator charge is remote and hard to convict, most service providers would prefer to avoid the publicity of such a court case and add decryption anyway.
The entire point of this discussion is that Party B should be constructing their service in such a way that Party A never gives them their secret. Party B would then be able to keep complete and perfect records, turn over all of those records at the drop of a hat, and nevertheless be unable to reveal Party A's secret.
Think PGP + Gmail, except that unlike usual, Google provides you with download of PGP.
The court could subpena Google, and Google could give the courts my PGP encrypted communications. However they would be unable to give them my private key and declaring them to be in contempt of court for that would be a massive miscarriage of justice.
PGP is a pain in the ass to use. However we have more streamlined technologies currently available that provide the same properties in this scenario. What is upsetting to people in this discussion is that companies like Skype are not employing such systems. We know Skype is not because they are employing one such "feature" that you mention (url-checking). They are therefore employing a system that does leave them open to having private information subpenaed.
You're absolutely right in describing the theory behind PGP (or GPG for the purists :), but unfortunately there is not yet a way to build a messaging service that has both features AND privacy. The should in your statement "Party B should be constructing their service" implies and expects a capability that is not (yet) possible to build. The point of my posts were to illuminate the reasoning companies make it easy to decrypt for the US government due to their exposure from in-demand content-aware features and fear of legal action.
Like you said, PGP+Gmail sucks for all parties included on a chain. Clients stop working. Non-users can't read the emails. Gmail spam filtering, ads, search indexing, and labeling all break. The same is true for PGP+Exchange, and most corporate customers much prefer Exchange features to the privacy offered to individuals with PGP.
I'm also not aware of 'more streamlined services' that offer true privacy -- please illuminate them if they exist. Services like Voltage suffer from the same root cause of reversible encryption.
So, customers have to choose: use a service with content-aware features OR use a dumb service that (currently) does not have the features. Most people choose features, and I would venture in this case Microsoft opted for features over pure privacy.
I would be among the first to welcome a way to accomodate both pure privacy and features in a service, and I encourage all to find a way. Please build it!
But if you're referring to Google's "Off the Record" feature, that's not OTR encryption. They just don't save that conversation in your logs. But they still have access to it themselves.
The govt comes by with a subpoena (secret, classified, or public) and requires Microsoft or the customer company to produce communication records that exist in a form that may be used as evidence. Failure to do so is best contempt of court and worst obstruction of justice. No 5th amendment privilege for other people's crimes. So everyone who chooses to store or process messages makes it so the encryption is reversible and they can honor court requests. Nothing is private as a result.
EDIT: I should make it clear I don't agree with the current status quo. Let me answer two very good questions.
> Would they also do that if logging failed during the period requested?
If it can be shown that there is a willful neglect of collecting logs then the govt in the past has gone after companies for some form of conspiracy (most famous: MegaUpload, but Microsoft customers have had their fair share for accounting and securities fraud) or criminal negligence. There is a prevailing theory that companies are responsible for employee actions, and failing to log is seen as unacceptable.
It's only recently (within the last year) that the courts have ruled whether compelling passwords is protected by the 5th amendment, and most systems in place were designed and built using previous assumptions from 10 years ago.
> If this is true, why is OTR offered at all?
It's a checkbox feature required for HIPAA, PCI, and similar. "Must have encryption" -- the standards and IT departments don't say how the keys are managed.