I guess I imagined that there could be some kind of feature in a given OS that detects when a program wants to encrypt a user file (shouldn't that be possible?) and a dialogue box could pop up with a warning to the effect "This program wants to encrypt your file. Do you want to proceed? Yes/No".
As far as I am concerned, it is not possible to determine whether a modification of a user file counts as encryption or something else. What is possible is detecting if an application modifies a large amount of files in rapid succession, but that also won't help you protecting your data.
Because if this is simply not possible, then the situation is pretty dire - doing backups is not going to be 100% effective, because the ransomwear can easily infect backups on external drives - in that show I watched, all the woman's backups on all external drives and even dropbox were attacked and encrypted. So having Time Machine is not exactly a solution - unless Time Machine has some kind of sandbox wherein once a file has been saved by Time Machine, the only operation that is allowed on that file is a copy/paste one where you can do restoration, but you cannot alter the file on TM in any other way.
While it is certainly possible to mess up the TM backup, it is extremely time-consuming (you'd need to encrypt every file version, which even on a fast external drive might take hours). Besides, you should use multiple rotating backup destination (I have 4), so there should be a working backup even if your current one gets contaminated. This issue is solvable with a proper snapshot-based file system though — as you already said that, where the previous versions cannot be tampered with by an unauthorised user anymore.
Also, I think that merely "being cautious" is not going to save you. The woman in that show who got all her files everywhere, including external drives and dropbox encrypted by ransomware clicked on a link to a video allegedly sent by her friend (and according to the show, it could be any file type, like f.ex. pdf files etc.).
Well, this is just nonsense. OS X will not launch an application disguised as a PDF or a video file. Besides, these kind of attacks are easily blocked with Gatekeeper — and that is why I have it turned on by default, even though good 80% of the apps I am using are not properly signed. I have read that Gatekeeper can be easily fooled, but so far, I could not find any substantial demonstration of this.
Now, you could say, "don't click on any links even from friends" - but how practical is it, when the bad guys can stage a Man In The Middle attack wherein they hijack an email or whatever and substitute a good file with ransomware... so f.ex. your friend could be on the other end of the phone, saying "OK, I'm sending you a pdf file 'Birthday Party Preparations.pdf'" and because his/her system is compromised, some malware automatically substitutes the ransomware with the same ostensible name "Birthday Party Preparations.pdf". Now you click, and you're hosed.
If that happens, what with MITM attacks, you could never click on any link to anything even with a person at the other end of the phone, not to mention any website. How practical is that?
Regarding MITM: that is why we have secure channels everywhere
That's why I wonder if it is possible for the OS to analyze a program before launching to see if it's encrypting and issue a warning or some other solutions, like the sandboxed TM I mentioned above.
As I said before, my hunch is that this problem is not decidable. Now, I'd love to write a nice proof of this similar to this one:
http://www.lel.ed.ac.uk/~gpullum/loopsnoop.html, but I am already past deadline on too many papers as it is