Tuesday, 22 May 2012

WhatsApp Considered Insecure (2012)

The views expressed here reflect the views of the author alone, and do not necessarily reflect the views of any of their organizations.

Summary

Since this post gets many hits let's start right away with the conclusion: I consider WhatsApp to be insecure. Personally I'd never use it to send serious/secret/sensitive messages. And you should never blindly trust incoming messages!

Introduction

Quick links: WhatsApp Security Advisory MVSA-1 and MVSA-2.

During my internship at a Ernst & Young I created a methodology to test the security of mobile applications.  After I finished it, and after completing my internship successfully, I decided to take a look at WhatsApp and apply the methodology I created on my own. Several new vulnerabilities were found, including a very severe one that even affected people not using WhatsApp. But before going into detail let's first investigate the security history of WhatsApp.

WhatsApp Security History

Over its lifetime WhatsApp has gotten some bad security attention. One of the first vulnerabilities was found in May 2011 where an authentication flaw was found making it possible to register any phone number. This video demonstrates the flaw. As will be explain in this post, there were (are?) still flaws in the phone number verification process.

Around the same time it was found that WhatsApp doesn't encrypt the messages you send and receive, meaning that if you use an unsecure wireless network people can sniff your WhatsApp messages. A few days ago a script was released demonstrating how an attacker can abuse this flaw to sniff all the messages you send and receive [Article]. At the time of writing this post this vulnerability has been fixed for some clients but not for all (notably it's fixed for Android).

In September 2011 two new vulnerabilities were found in the registration process of WhatsApp. The first was found by Andreas Kurtz and the second by SEC Consult. The one found by Kurtz is a symptom of a fundamentally flawed registration process. In essence during the registration process the servers of WhatsApp fully trust anything the client says. However everything happening on the phone (= client) can be manipulated by someone with sufficient technical skills. In security it's well known to never trust anything the client is telling. Unfortunately that's exactly what WhatsApp is doing. Again this flaw in the registration process has not been adequately fixed.

The second vulnerability found by SEC Consult also allows an attacker to register any phone number he or she wants (by brute forcing the challenge PIN number). WhatsApp has implemented a basic fix, but as SEC Consult already mentions it provides insufficient protection. An attacker targeting a group of, say, 100 people, will on average still successfully compromise at least one individual.

During their research SEC Consult also found another vulnerability which makes it possible to change the status of any WhatsApp user. They reported this issue to WhatsApp but after waiting 3 months WhatsApp still didn't fix the issue, so they decided to make the information public [Source]. As a result someone created the website whatsappstatus.net where anyone could enter someone's phone number and the new status message of the user [Article, Blogpost, Script]. This removed the small technical knowledge needed to execute this attack. Fortunately this has now been fixed and whatsappstatus.net is no longer online.


To provide some more background the research paper "Evaluating the Security of Smartphone Messaging Applications" is also interesting. It shows that nearly all mobile messaging applications have some security vulnerabilities. So although WhatsApp will be subject of this post, remember that other mobile applications might also contain several vulnerabilities. If you want to have some fun yourself I suggest checking out the applications mentioned in that paper ;)

Anyway all this shows that WhatsApp had some severe vulnerabilities in the past. The phone number registration flaws are a symptom of a fundamentally flawed design. Their late response to the work of SEC Consult also isn't good. Another big negative is that they never mention security updates on their website. As a security researcher you don't know whether or not a vulnerability has been fixed. And as a user of WhatsApp you are never warned of potential problems!

Man, this section turned out longer than expected (actually this whole blog post is a lot longer than expected :). Time to move on the real stuff!

Authentication Design Flaw(s)

When researching WhatsApp I started by investigating the registration process of WhatsApp. And this was also where the first severe vulnerability was found. To understand this vulnerability we'll first have to understand the registration process. It begins with the user entering his or her phone number, after which WhatsApp will issue the following HTTPS request:
https://r.whatsapp.net:443/v1/exist.php?cc=<cc>&in=<phone>&udid=<id>&sim=<sim>
Here <cc> and <phone> stand for the country code and phone number entered by the user, <id> for the hardware identifier of the device, and <sim> for the MSISDN of the device (the real phone number of the phone). The sim parameter is not used and can be dropped. Essentially this request checks if the phone number has already been registered on the device. If it hasn't been registered the reply will be as follows
<?xml version="1.0" encoding="UTF-8"?>
<exist><response status="fail" result="incorrect"/></exist>
This mechanism is interesting in its own right. But for now we'll ignore this request and come back to it later (see section "What's the Password?"). So let's say the phone number hasn't yet been registered on the device. WhatsApp continues by making a second HTTPS request:
https://r.whatsapp.net:443/v1/code.php?cc=<cc>&to=<cc><phone>&in=<phone>&lg=en&lc=US&mnc=260&mcc=310&imsi=<imsi>&method=self&reason=&token=<token>&sim=<sim>
An example value of <token> is 9fe2a4f90b4acff715d1daf84428bddd. You'd think this token is going to be used in the next stages somehow. But it's not. WhatsApp has made several of these strange implementation decisions, so don't be surprised when more strange behaviour is going to be discussed. Also note the parameter method which specifies the authentication method used. WhatsApp offers several authentication methods in an attempt to assure one of them will always work. The "self" method is used first. Anyway the default response to this request is of the form
<?xml version="1.0" encoding="UTF-8"?>
<code><response status="success-attached" result="613"/></code>
The success-attached parameter is interesting and it's corresponding value, here 613, will be of importance during the next stages of the registration process. When the response is received WhatsApp continues by asking the user if he or she really wants to register the given phone number.


When the user clicks OK WhatsApp will attempt to send a special SMS message to the phone itself. The code that achieves this is:


As you can see the SMS message "WhatsApp <code> WhatsApp internal use - safe to discard" is sent to a random port of the phone number the user is trying to register. An example of the included <code> is ROGEuMirJNfCqXnuFdSUrwGYYfeA8G36. So essentially we're sending an SMS message to ourselves. After sending the message WhatsApp will monitor incoming messages. Unfortunately I'm using the Android emulator and can't fully emulate the required behavior yet (intercepting outgoing SMS messages on the Android emulator, and sending data SMS messages, is a good subject for a new blog post). My guess is that if WhatsApp detects that an SMS message is received from the phone number the user is trying to register, and has the exact same <code> as previously transmitted, it will complete the registration.

Now after getting this far we can do some very targeted google searches to see if other researchers are also working on this. And indeed! We find a Spanish blog post of Jose Selvi where he has done similar research. His post confirms our guess and even has a small video demonstrating the process.


So when WhatsApp receives the SMS it sent itself, it continues by registering the phone number. This contains a huge design flaw. The client (so your phone) is fully trusted! Basically the client does some stuff, and then tells the WhatsApp servers "hey register this number". But like we mentioned the client can't be trusted. And this has been demonstrated by modifying the WhatsApp client so an attacker can register any phone number he or she wants [Source].

Selvi has reported this vulnerability to WhatsApp and apparently they "fixed" it. But how can it be fixed if they're still using the method and still trusting the client?! I contacted WhatsApp myself and asked them about this. They answered that the "self" authentication is still in use, and that "the described method as published in the url has been resolved". Of course it's not because the exact method described by Selvi no longer works that it's safe! As mentioned on the blog:
"No obstante, no se ha realizado una auditoría profunda de la aplicación más allá de la prueba de concepto que se realizó como demostración de la charla, por lo que es posible que otras vulnerabilidades similares, o de otro tipo, puedan existir."
Roughly translated he's saying that no real audit of WhatsApp has been made and that it's possible vulnerabilities still exists (thanks google translate). In a comment on his blog it was even mentioned that more vulnerabilities were found. However these are not published so we have no idea how they work exactly.

After all this it's no longer our task to demonstrate these vulnerabilities to convince WhatsApp that they should improve their security. It's a waste of time. It's now WhatsApp responsibility to prove their registration process is secure. Not the other way around. I consider it completely broken and insecure until proven otherwise. I contacted WhatsApp and explained that the client can't be trusted to prove the phone number actually belonged to the user. As a reply I got the message "understood", so let's hope they'll actually remove this inherently flawed authentication method in the future.


Server Sent Verification SMS

In case the "self" authentication method fails a second method is used. It begin by requesting the HTTPS page
https://r.whatsapp.net:443/v1/code.php?cc=<cc>&to=<cc><phone>&in=<phone>&lg=en&lc=US&mnc=260&mcc=310&imsi=<imsi>&method=sms&reason=self-send-timeout&token=<token>&sim=<sim>
Note that the method parameter is now set to sms. When the WhatsApp server receives this request it will send a text message containing "WhatsApp code <pin>" to the phone number specified in the HTTP request. Here <pin> is a 3 digit number. WhatsApp will attempt to detect this incoming message. If the phone number entered by the user is indeed his/her number the text message will arrive. When the message is received WhatsApp will prove this to the server by sending the challenge <pin> number using the following request
https://r.whatsapp.net:443/v1/register.php?cc=<cc>&in=<phone>&udid=<udid>&code=<pin>
If the pin is correct the phone number will now to tied to the given hardware id (udid). Essentially the phone number will be the username and the hardware id will be the password. This is another mistake: the hardware identifier (i.e. the password) can be read by other applications! More on that later though.

Remember the number included in the success-attached response earlier? Go look again, it's right above the "We will be verifying the phone number XXX" screenshot. In our case the number was 613. Well guess what, this is the number WhatsApp (used to) include in the challenge SMS message! Hence you don't have to own the phone number to register it. Simply write down the number in success-attached and request the register.php page with this pin number. The vulnerability has been demonstrated on a test phone number given by WhatsApp and worked.

This vulnerability has been reported to WhatsApp and has been fixed as explained in my security advisory. Important is that this affected people not using WhatsApp too! An attacker could register your phone number and send messages in your name to all your friends who use WhatsApp. He or she would also receive all messages send by people using WhatsApp.

Sent PIN by Voice Call

Finally it's possible to let WhatsApp send the PIN by a voice call. This is basically the same as the server sent verification SMS, except that the user now has to answer the voice call and manually enter the pin number.

Responsible Vulnerability Disclosure

After finding the registration vulnerability (and a few others) I decided to contact WhatsApp. The plan was to get these vulnerabilities fixed before making them public. This is something where WhatsApp did do well: they responded timely to all my emails and remarks. The security advisory includes a timeline of messages sent. Based on this I had good hope to get most of these issues resolved.

After contacting them they requested a demonstration of the phone number registration vulnerability on a test phone number. So I registered the test phone number using the technique we just covered. As a response they said an updated client for Android was available which solved the registration vulnerability. This was strange since the vulnerability can only be fixed server side! And indeed the vulnerability was not fixed.. So I investigated what can be done to fix the vulnerability. Apparently you can't simply leave out the success-attached parameter in the response since the client requires this parameter to be present. Hence I suggested to simply include a random PIN number for the success-attached parameter. They agreed. And so this vulnerability was fixed :)


Another vulnerability I discovered was that the Android client didn't verify the Common Name of the HTTPS certificate. Remember that all request made when registering the phone are done over a secure HTTPS connection. Unfortunately this vulnerability allows an attacker to intercept all traffic when he or she is performing a man-in-the-middle attack. Any valid certificate would be accepted by the Android client. This vulnerability was also reported to WhatsApp and has been fixed in the latest Android Client.

Finally a less severe vulnerability was also reported and patched. It was a full path disclosure in a PHP warning. The functions producing the warning appeared not to be further exploitable, although this hasn't been explicitly tested. A redacted screenshot of the warning can be seen here.

I also asked them when encryption will be implemented for sending and receiving messages. Apparently they already submitted an Android version using an encrypted channel on May 7. An update for the Windows Phone was submitted to the marketplace on May 11. They can't say when the other versions will be patched because each version has its own software stack and development cycle. Although I cannot test the versions currently in the marketplaces, the Android version available on the WhatsApp website indeed uses encryption to send its messages. The actual strength of the encryption system has not been investigated.

Brute Forcing the PIN

Currently WhatsApp uses a basic brute force protection which allows the user to guess the PIN number only 10 times. After more than 10 attempts the phone number is blocked and one must contact WhatsApp in order to register the number. However the pin number has only 10^3 = 1000 possible values. As already mentioned by the security advisory of SEC Consult this doesn't provide sufficient protection.

The problem is that it's still feasible to attack a group of users. For example when attacking a group of 30 users the probably that all brute force attempts fail is (990/1000)^30 = 76%. In turn this means the probability of at least one successful brute force attempt among the 30 users is 24%. This is too high. The PIN number should be a lot larger! In a response to this WhatsApp stated that they are actively working on brute force protections. So, although the current brute force protection is a lot better than nothing, hopefully it will be improved in the future.

Auto Update Vulnerabilities

This one only applies to the Android version when directly downloaded from the website of WhatsApp. Versions downloaded from the marketplace use a different updating mechanism. Anyway, during startup it checks the following URL to see if an update is available:
www.whatsapp.com:80/android/9/WhatsApp.version
This page returns the version number of the latest available client for Android, for example 2.7.7435. If this version number is higher than the installed version WhatsApp will first request the checksum of the latest .apk file of the Android file by loading the URL
www.whatsapp.com:80/android/9/WhatsApp.cksum
An example response is 4273431032. It then continues by requesting the new .apk file
www.whatsapp.com:80/android/9/WhatsApp.apk
The problem is that all this is done over an unsecured HTTP connection. And unfortunately WhatsApp doesn't verify the downloaded .apk using a digital signature or similar. Thus an attacker can intercept these requests and force the user into downloading a malicious .apk file. So that's the first vulnerability: an attacker can intercept the HTTP traffic and injecting his own application. When further investigating the auto update functionality we find more. The downloaded .apk file is saved to
/mnt/sdcard/WhatsApp/WhatsApp.apk
and accompanied with the empty file called WhatsApp.upgrade in the same directory. The next time WhatsApp starts it will look for these files, and if both are present display a dialog asking if the user wants to install the update:



Unfortunately this can be abused by a 3rd party application. A 3rd party application can write the files WhatsApp.apk and WhatsApp.upgradate to the directory /mnt/sdcard/WhatsApp, where WhatsApp.apk can be a malicious application. This will trigger the update dialog of WhatsApp as shown above. The user will them be prompted to install the malicious application from the context of WhatsApp. So luckily the user still has to approve the installation. But because he or she thinks the program is an update of WhatsApp it's more likely that the user will agree to install the application, especially if the name would be similar to WhatsApp and uses the same logo. This is not something you want!

What's the Password?

Time to return to the very first HTTPS request sent when registering a phone number. It was
https://r.whatsapp.net:443/v1/exist.php?cc=<cc>&in=<phone>&udid=<id>&sim=<sim>
Where the sim parameter can be dropped. What's interesting is that if the number was previously registered on the phone this request will return
<?xml version="1.0" encoding="UTF-8"?>
<exist><response status="ok" result="<cc><phone>"/></exist>
and WhatsApp will start and log you in. No further information is needed! From this behavior we can derive that the phone number can be seen as the username and the udid as the password. The udid is a "Unique Device IDentifier" of the phone (a hardware id).

There's one problem with this design, namely that every application on your phone can read this unique device identifier (given the appropriate permissions). For example on Android an application having the permission android.permission.READ_PHONE_STATE can read the udid. Let's phrase this differently: another application can read your WhatsApp password and impersonate you. WhatsApp has responded that they are "working on unique 160-bite [sic] random passwords, but is going to take some time".

Conclusion

Let's recall the research paper "Evaluating the Security of Smartphone Messaging Applications" where it was shown that nearly all mobile messaging applications contained security vulnerabilities. In this post we specifically targeted WhatsApp. We found that it contained numerous vulnerabilities, and I personally consider WhatsApp an insecure application that I wouldn't use. On the bright side WhatsApp is taking these security issues more seriously and appears to be in the processing fixing all mentioned vulnerabilities. Only time will tell if they actually manage to make it a secure application or not. It's also an open question how insecure the other messaging applications really are.

In general mobile developers need to learn more about security!

Monday, 27 February 2012

TEDxUHasselt Salon: How you could be hacked

TEDxUHasselt is an independently organized TED event by three of my fellow students. In order to extend the local community around it, and attract passionate people, they recently organized TEDxUHasselt Salon. As they state: "TEDxUHasselt Salon is an informal event aimed at building a local community of passionate people". Basically anyone with an interesting idea or passion can register and give a talk of 10 minutes.

I decided to give a talk as well, with as subject something related to computer security. To keep it interesting and relevant for the audience I had chosen to work out a realistic example of how a cyber criminal could attack TEDxUHasselt attendees. For the reader who's familiar with security tools it was essentially a small demo of metasploit and meterpreter, combined with a realistically forged email designed to manipulate the target into clicking a deceptive (spoofed) URL.


The presentation started with a few examples of how cyber criminals are trying to steal money. Then we layout the groundwork for a fictive attack on the TEDxUHasselt attendees. Arguably the most interesting part was a video showing a demonstration of the victim reading the spoofed email and following the link, which results in the victim being hacked. Then we show everything a hacker can do with your computer, including downloading sensitive files, logging all the keys the victim presses, and even recording the microphone of the victim!

The slides of the presentation can be seen here:





The actual execution of the fictive attack has been recorded and you can see it in the youtube video below. As you'll notice it isn't hard to launch the attack and to retrieve sensitive information once you have control of the victim's computer. Note that the video immediately starts at the part that is relevant for all users, the first part of the video consists of the hacker configuring his tools to launch the attack.






The presentation itself was a big success and everyone seemed to like it! People are interested in security, you just have to make it understandable and relevant for "the common man". A thank you goes out to Rutger, Niels, and Bob for organizing TEDxUHasselt Salon!

Wednesday, 1 February 2012

Foundations of Privacy

Privacy is a difficult concept and there are many sides to the privacy issues we are facing in our digital age. For my master thesis I studied privacy in databases, where the goal is to find a mathematical definition of privacy. But in this post I won't focus too much on the math behind it all, instead I'll go over some interesting observations I have made during my work, and explain some of the basic concepts. To get started we'll look at some privacy fiasco's that occurred in the past.

1. AOL Search Fiasco

We begin with the AOL search fiasco where AOL released around 20 million search queries from 65,000 AOL users. In an attempt to protect privacy, the username of each individual has been changed to a random ID. However, different searches made by the same person still have the same ID. Shortly after the release the person with ID 441779 was identified as Thelma Arnold. By combining all the searches she made, it became possible to find out her real identity. This already shows that simply removing unique identifiers such as a person's name, address or social security number do not provide privacy, and more generally that anonymity does not imply privacy (more on this later). No real care was given to anonymizing the search results. So this is not a true example that removing identifying attributes (eg., name, address, etc.) fails to protect privacy, as no real care was given to anonymizing the dataset. This can be deduced from the observation that the person who was responsible for releasing the data was quickly fired.

2. The HMO Records

In this attack the medical records of the governor of Massachusetts were revealed. The attack begins with the following observation: One can uniquely identify 63% of the U.S. population with only knowing the gender, ZIP code and date of birth of an individual. We now turn our attention to two different datasets. The first one is the voter registration list for Cambridge Massachusetts and includes information of each voter. A bit simplified the dataset can be represented using the following relation:
VoterRegistration(ZipCode, BirthDate, Gender, Name, Address, DateLastVoted)
The second dataset is the Massachusetts Group Insurance Commissions medical encounter data, containing medical information on state employees (and thus also contained the medical records of the governor of Massachusetts). A small part of this medical data, without all identifying information such as name, address and social security number removed, were released. It can be represented using the following relation:
MedicalData(VisitDate, Diagnosis, Procedure, ZipCode, BirthDate, Gender)
As we can see both datasets can be linked to each other by combining gender, ZIP code and date of birth! So even though the names were removed in the medical dataset, researchers were still able to link the data back to individuals. As a result the medical records of the governor of Massachusetts were found.

3. Kaggle

Finally we come to an example that shows it's possible to find correspondences between social networks merely based on the structure of underlying friendship relations. Even more interesting is that the art of "de-anonymizing" a dataset was used to win a competition. Kaggle organized a competition where participants were given a dataset of people and their underlying relations, which we will call a social graph (each node is represented using a random ID and no information such as username, name, location, etc. were included). An example of a social graph is shown in the figure below:
The circles/letters represent people and a line is drawn between them if they are friends. However not all relationships were given to the participants. In fact, the goal of the competition was to determine whether certain given relationships, which were not present in the original social graph given to the contestants, were fake or real. Of course if we'd knew who the individuals behind all the nodes are, we could just look up if two nodes are really friends or not! So if we manage to de-anonymize the social graph given by Kaggle we can game the competition: Instead of making a machine learning algorithm to predict the answer we simply look them up.

It was found out the social graph was created by crawling Flickr. So the researchers made their own crawl of Flickr and based on the structure alone created a mapping between their own crawl and the Kaggle social graph. In other words they now know which individuals correspond to the supposedly anonymous nodes, and thus they identified the individuals using only the structure of his or her social graph.

The Difficulty of Absolute Privacy Protection

It should be obvious by now: Assuring privacy is hard. We can't simply remove attributes such as name, address, social security number, etc. from a dataset. The reason is that seemingly innocent information such as gender, ZIP code, and birthdate can still be used to uniquely identify an individual. Clearly the need for a rigorous privacy definition is needed. Researchers have proposed theoretical models such a k-anonymity, but it turned out to have problems so L-diversity was suggested. I turn weakness were found in L-diversity, resulting in a new definition called T-closeness. But still there are problems even for T-closeness. So in practice assuring privacy is hard, and even finding a robust mathematical definition appears to be a challenging task. What can be done?!

Before answering that question we're going to argument that the situation, at least from a theoretical point of view, is even worse than one might already think. There is in fact theoretical evidence suggesting that absolute privacy protection is impossible. This proof was heavily based on an earlier attempt at the proof. Now of course not releasing your information does provide absolute privacy. But what they have proven is that if a database is sufficiently useful there is always a piece of external information that, combined with the output of the database, violates the privacy of an individual. An example explains this best. Assume that learning the salary of an individual is considered a privacy violation. Further assume that we know the salary of Alice is 200 EUR higher than the average salary of a Belgian citizen (this is the external information). So we don't know her exact salary. Let's say we now receive access to a database containing the salary of every Belgian citizen. From this database we can calculate the average salary of a Belgian citizen, say 2500 EUR. Combining this with the external information teaches us that the salary of Alice is 2700 EUR! Et voila, the privacy of Alice has been violated. All due to gaining access to a database from which we merely learned an average of a particular value.

So it seems it's very difficult (impossible) to assure that there will be no privacy violation whatsoever. What can we do? The answer is simple. We will reduce the probability of a privacy violation as much as possible.

Differential Privacy

One of the promising definitions in the privacy research community is called differential privacy. It provides relative privacy prevention, meaning that the probability of a possible privacy violating occurring can be greatly reduced. Important is that when using differential privacy the dataset is not handed out to the public. Instead users can pose questions (queries) to the dataset (eg., over the internet), and answers will be given in such a way to reduce the probability of a privacy violation. In practice this is done by adding a small amount of random noise to the answer. Note that there is always a tension between accuracy of the data, and the privacy guarantees provided by the data release. The higher the privacy guarantees, the lower the accuracy.

Differential privacy assures that, when you provide your personal information to the database, the answers to queries will not differ significantly. In other words handing over your information should not be visible in the calculated answers. This results in a more natural notion of privacy: When giving your information your privacy will on be very minimally reduced (remember that we consider absolute privacy protection impossible).

More formally, when you have two databases, one with your personal information (D1) and one without your personal information (D2). Them the probability (Pr) that an answer mechanism (K) returns a specific answer (S) should be nearly identical (multiplication by exp(epsilon)) for both databases. Mathematically this becomes
The parameter epsilon defines how much the probabilities are allowed to vary. A small epsilon such as 0.01 means the probabilities should be almost identical (within multiplicative factor 1.01). However for a larger epsilon such as 1 the probabilities can differ by a larger amount (they must now be within multiplicative factor 2.71). The reason we use the exp(epsilon) instead of just epsilon is because manipulating formulas that use exp(epsilon) is a lot more straightforward.

We can now design algorithms that answer queries while assuring differential privacy. In short we can now prove we assure a certain definition of privacy! There are still some difficulties in implementing differential privacy in practice. There first one is that it's not clear what a good value for epsilon would be. The second problem is that you cannot ask an unlimited amount of questions (queries) under differential privacy. So in practice a reliable mechanism must still be designed to ensure only a limited amount of queries are answered.

Another downside is that the actual dataset is not released. For researchers being able to visually inspect the dataset and first "play around with it" can be important to gain a better understanding of it. Therefore the problem of releasing an anonymous dataset, while minimizing the probability of a privacy violation, is still an important topic.

Anonymity vs. Privacy

Another important observation is that anonymity does not imply privacy. Take for example k-anonymity. It states that an individual must be indistinguishable to at least k-1 other individuals. Below we give an example of a 4-anonymous database. Based on the non-sensitive attributes, which are used to identify an individual, we notice there are always at least 4 rows having the same values. Hence each individual is indistinguishable with at least 3 other individuals.
If you know a person having Zip code 1303 and an age of 34 is present in the database, he or she can correspond to any of the four last rows. So the anonymity of that individual is preserved. However all four rows specify that he or she has cancer! Hence we learn our targeted individual has cancer. Anonymity is preserved while privacy was violated.

The Ethics of Using Leaked Datasets

Another observation made during my master thesis is that it's hard for academic researchers to obtain dataset to test their algorithms or hypothesis. Even worse is that most are forced to create their own datasets by crawling public profiles on sites such as Twitter, LiveJournal and Flickr. This means their test data only contains public profiles, creating a potentially significant bias in their datasets. Another problem is that researchers won't make these dataset publically available, probably out of fear of getting sued. And that is not an irrational fear, demonstrated by the fact that Pete Warden was sued by facebook for crawling publicly available data. Because these datasets are hard to obtain, peer review becomes difficult. If another, independent, researcher doesn't have access to the data he or she will have a very hard time trying to reproduce experiments and results.

But more interestingly is that there are public dataset available! It's just that no one seems to dare to use them. As mentioned the AOL  dataset was publicly released, but researchers are hesitant to use it. Another dataset, called the Facebook100 data, is also a very interesting case. Researchers were given social graphs of 100 American institutions in anonymized form (private attributes such as name, address, etc. were removed). Amazingly the dataset contains all friendship relations between individuals present in the dataset, independent of their privacy settings on facebook. As we've seen an unmodified social graph can be vulnerable to re-identification attacks (see the Kaggle case). A week after the release facebook requested the dataset to be taken down. Deleting data on the internet is hard however, and copies of it are still floating around. Nevertheless, because facebook requested to dataset not to be used, researchers seem to be hesitant to use this dataset.
"Privacy issues are preventing a leap forward in study of human behavior by preventing the collection and public dissemination of high-quality datasets. Are academics overly-sensitive to privacy issues?"
It's clear why researchers aren't touching these datasets. Either they include too much personal information, the data provider has requested to take the dataset down, or out of fear of getting sued. The question is if this behavior really benifits the user, the one who we're trying to protect. The answer is NO. Let's look at all the players in this game:
  1. Researchers don't conduct research on the "leaked" datasets. This hinders scientific progress.
  2. Users are less aware of vulnerabilities as researchers can't demonstrate them.
  3. Companies benefit as they don't get bad press about leaked data & possible privacy issues.
  4. Malicious individuals benefit since vulnerabilities remain unknown, which in turns morivates users to continue sharing information publicly. They are also not limited by moral objections and will use the leaked data if valuable.
Currently only the companies and malicious individuals benefit from this behavior. Scientists and users are actually in a bad position by not allowing/doing research on public/leaked datasets. A utopian view would be that researches conduct analysis on the datasets and make advancements in their field. At the same time users can be warned about potential vulnerabilities caused by the leaked data before hackers will abuse them.

Fear Based Privacy

A mistake to watch out for, at least in my opinion, would be what I call fear based privacy. Yes, you have to remain diligent to make sure the government and companies don't collect too much information. And yes, you should be careful about the information you provide to the public! But one must first carefully consider the arguments for, and against, making information public. A knee-jerk reaction saying that any information you share might potentially be abused by a malicious individual is not a valid argument. It's an argument purely based on fear: "Even if we don't know yet how a hacker could abuse certain information, who knows he or she might find a method in the future, so it's better not to share anything."

Now don't get me wrong here. Currently I believe that an absurd amount of information is being collected, especially on the internet, and that this is not a good thing. But a good balance must be found between the benefit of sharing information, and the potential risks for sharing said information. Compare it to driving a car. It involves the risk of possibly getting in an accident. However, the benefit that a car provides outweighs its risks, and instead of completely avoiding cars we attempt to reduce the probability and severity of accidents.

Interesting Links

    Special thanks goes to Jan Van den Bussche of University Hasselt (Belgium) for helping me during my master thesis.