Facebook – The Clash Between Facebook and Independent Researchers – The Journal.
This transcript was prepared by a transcription service. This version may not be in its final form and may be updated.
Ryan Knutson: Laura Edelson is a Ph.D. Candidate in Computer Science at NYU. Last week, she saw a notification on her phone that Facebook had sent her an email.
Laura Edelson: I didn’t really think about it because I was just about to sit down to dinner with my family. And then when I got up from dinner, I saw I also had an email from a reporter. And I read that email from Facebook that said my account was suspended. So that was how I found out.
Ryan Knutson: The reason her personal Facebook account had been suspended, had to do with a research project she was leading. The project, which is called Ad Observatory, had been studying how good Facebook was at identifying and labeling political ads on its platform. In a blog post that same day, Facebook said it had been warning the NYU researchers for months about a tracker they were using to collect data. And so Facebook finally made a move.
Laura Edelson: It is effectively ending Ad Observatory. That’s what’s been shut down.
Jeff Horwitz: This story is about how Facebook is responding to entities that are trying to figure out how the platform works and what it’s doing. There’s also just kind of the bigger question of whether outsiders have the right to study the platform on their own.
Ryan Knutson: Welcome to The Journal, our show about money, business and power. I’m Ryan Knutson. It’s Monday, August 9th. Coming up on the show, the tension between Facebook and the researchers trying to study it.
In the past decade, as Facebook has become bigger and more widely used, a number of academics and researchers have tried to study the platform more comprehensively.
Jeff Horwitz: There are efforts to systematically collect data about what ads people see, whether there might be bias. There are efforts to systematically look at public groups.
Ryan Knutson: That’s our colleague, Jeff Horwitz, who covers Facebook.
Jeff Horwitz: Everyone’s trying to figure out ways to ask a question that can be answered through the collection of data on the platform.
Ryan Knutson: One part of Facebook that’s garnered a lot of interest is the company’s ad library. It’s a tool that Facebook launched in 2018 to try and be more transparent about who’s paying for advertising and who those advertisers are trying to reach.
Jeff Horwitz: They produced what was a near comprehensive database of all the ads that ran. You could take a look at them, see who paid for them, and get a sense of some very basic information about where they were shown, right? Like were people in Arkansas? Or were they old? Were they young?
Ryan Knutson: The ad library has turned into something that a lot of researchers rely on for all kinds of studies, like figuring out who gets targeted for which kinds of ads. But Laura and her team wanted to study the ad library itself, and specifically how the ad library identified and labeled political ads.
Jeff Horwitz: She got really curious about how the ad library worked, and has pulled together a whole bunch of research and data using the public access to that ad library that she has.
Laura Edelson: I think the biggest question that we were trying to answer is just that very simple, how good a job does Facebook do at identifying political ads?
Jeff Horwitz: She has caught some instances where Facebook’s ad library wasn’t doing the things it said it was supposed to do.
Laura Edelson: The premise here is that Facebook is very good at identifying political ads and all the political ads are in that ad library. If Facebook can’t do that well, then it kind of puts an asterisk next to any research you might do with that ad library and any findings you might come to.
Ryan Knutson: So Laura and her team built something called a browser extension for Chrome and Firefox, which could gather data about the ads popping up on Facebook.
Jeff Horwitz: People would install the browser extension. And the browser extension would basically capture the ads they saw on Facebook, as well as information about why they’re seeing the ad.
Ryan Knutson: Volunteers who install this browser extension, named Ad Observer, would have their data sent back to Laura’s team to be aggregated and analyzed.
Laura Edelson: So when users who’ve installed our browser extension are shown ads on Facebook, those ads, as well as the ad targetings are sent back to our servers. What we then do is, we can look at that ad and just see, how was it shown to the user? Was it disclosed to the user as a political ad? So we can see, are political ads being disclosed consistently and effectively?
Ryan Knutson: Laura says that the Facebook platform does a pretty good job of catching and labeling political ads, but it still misses some.
Laura Edelson: In our judgment, looking at the data we’ve collected via Ad Observer and via the ad library itself, we think that Facebook captures about … during the 2020 election, they seemed to catch about 90% of political ads. And that’s pretty good. If we’re judging machine learning algorithms, 90% accuracy is quite good. But it does mean that there are hundreds of thousands of ads that are missed.
Ryan Knutson: And when the NYU team found a political ad that had slipped through the cracks, they added it to a database they were building that combined the ads in Facebook’s ad library with the ads that Laura’s team was catching.
Do you have any examples of things that you discovered, like examples of ads that you could tell us about?
Laura Edelson: So we found actually a pretty wide range of things that Facebook had missed. We found ads encouraging people to use mail-in balloting. We found ads about mail-in balloting and how maybe it’s not so trustworthy. We found ads about ballot measures. We found ads aimed at specific communities, like native Americans. So it was kind of a little bit of everything.
Ryan Knutson: And none of which were labeled as political advertising?
Laura Edelson: Right. It seems like across the board, there’s just stuff that slips through the cracks.
Ryan Knutson: But there was one problem. Facebook had warned Laura that she and her team were violating the platform’s terms of service. That’s after the break.
Laura says that she notified Facebook about her research project when they started, and that it seemed like the company didn’t have major objections.
Laura Edelson: One of the things that’s been a little bit strange about this entire affair is, until Tuesday, we had good and regular communication with various people who work at Facebook about the kind of security issues that we work on. So, it’s complicated.
Ryan Knutson: But Facebook says it gave her warnings from the get-go. And last October, citing privacy concerns, Facebook sent her a cease and desist letter, saying the project violated its rules against bulk data collection, also known as scraping.
Jeff Horwitz: So by the time the cease and desist letter came, they had already collected targeting data behind more than 200,000 ads. And they were saying that it was being useful to expose areas where the publicly available archive of ads was either failing to log ads or not providing enough information.
Laura Edelson: Very often our findings are not flattering to Facebook. But we see ourselves as cybersecurity researchers who are trying to collaborate with platforms to make their systems better. You don’t make systems better by ignoring the problems that are there.
Ryan Knutson: Why might Facebook have been concerned about the work she was doing?
Jeff Horwitz: The company has, for a long time, had some privacy issues. They, I think by their own admission, were not great about it for a very long time. So Facebook’s position on this stuff is that basically, any gathering of data in a bulk fashion from its site is not okay, unless it has Facebook’s permission.
Ryan Knutson: But rather than slow Laura down, the letter actually drew more attention to her project and led to even more volunteers installing the browser extension.
Laura Edelson: There were a lot of users who maybe wanted to, in the face of that, affirmatively say that they wanted to be part of a project like ours. So we actually got up to about 16,000 users back then.
Ryan Knutson: The cease and desist letter gave Laura and her team 45 days to comply with the company’s policy. But Laura says she didn’t want to let Facebook dictate the terms of her research, and that she and her colleagues were taking steps to protect user privacy. So they kept going. Until last week.
Laura Edelson: I really did not see this coming, to be honest. I’m still in shock, I think.
Ryan Knutson: Laura thought the company might take action against the browser extension, but not her personal Facebook page.
Laura Edelson: I was expecting them to take a step against Ad Observer. To sue us, or to try to get the Chrome and Firefox store to take our extension down. Or to try to block us technologically. I wasn’t expecting them to suspend my personal Facebook account and all the other research that we do that makes content transparent to the public, and through which we provide data to two dozen other researchers and journalists.
Jeff Horwitz: Facebook’s line has been that they respect and understand the importance of independent research, but that it needs to be done under terms that are in keeping with the company’s privacy policies.
Ryan Knutson: By suspending her personal account, and the accounts of some of her colleagues, Facebook actually didn’t stop the browser extension. The browser extension is still able to collect data. But without their Facebook accounts, Laura and her team, aren’t able to access the tools they need to analyze the data.
Jeff Horwitz: So there’s a bit of irony to what did get shut down, which is … the thing that Facebook took issue with was the collection of data from these browser extensions that users had volunteered to install. And that’s not something that Facebook was able to shut down here.
Ryan Knutson: Why not?
Jeff Horwitz: It’s unclear whether Facebook can functionally stop this sort of stuff. It’s pretty hard to stop an outside entity, particularly an outside entity that has the full permission of Facebook users and is attached to their browser, from gathering data about your platform.
Ryan Knutson: So Facebook says that you were violating its terms of service. Do you disagree?
Laura Edelson: Facebook, to my knowledge, has a term of service that prohibits users from collecting data by automated means. So I think in a very narrowly construed sense, yes, our browser extension enables users to collect data via automated means. That is true.
I think though, there is another term in that term of service that says, unless it is authorized by Facebook. So Facebook could choose to authorize what we do. And they are not willing to do that.
Ryan Knutson: In a blog post last week, Facebook said it welcomes research that helps keep the company accountable, so long as it doesn’t compromise the security of the platform or violate the privacy of other users. Facebook also said there was another reason to shut the project down. Because it was legally obligated to.
In 2019, the company settled a Federal Trade Commission investigation into the Cambridge Analytica scandal. And as part of that FTC settlement, Facebook promised to police user privacy more closely.
Jeff Horwitz: The Cambridge Analytica situation loomed very large and led to this $5 billion settlement, which has some fairly steep requirements about not just what information Facebook lets off the platform, but also what gets taken from it. And how it handles instances where information is removed without its permission.
Ryan Knutson: Two days after Laura’s Facebook page was shut down, a senior FTC official wrote in a public letter to Facebook CEO Mark Zuckerberg, that the 2019 settlement did not require the company to shut down Laura’s project. In his letter, the agency official wrote, quote, “We hope that the company is not invoking privacy, much less, the FTC consent order as a pretext to advance other aims.” The official said that that settlement does allow for outside research, and that, quote, “I am disappointed by how your company has conducted itself in this matter.”
How do you think this move to ban your personal Facebook account will affect all of this kind of research that you and other groups are trying to do to look at Facebook and how it functions?
Laura Edelson: I think this is going to have a chilling effect on the research community. I have an email in my inbox that I got from another researcher who I know had been planning to set up a similar project, who I had been talking to about just some of the technical details of how to do this. Who told me that his university’s legal risk department, maybe it wasn’t going to allow the project to continue. And I completely understand that. It’s very difficult for an individual institution to stand against a company with the size of … might of Facebook.
Jeff Horwitz: In some ways you can make a case that the Ad Observatory is a relatively small project, focused on a very specific corner of Facebook. And even a specific corner of that corner, right? Which is the targeting criteria for ads. But there is, I think, a lot of concern here about precedent in general.
There’s a sense that to be objective and independent, research into the platforms needs to be able to be conducted without the full permission of the platforms. Waiting for Facebook to tell you what you can study isn’t a particularly appetizing choice, if you’re trying to be independent.
Ryan Knutson: That’s all for today, Monday, August 9th. The Journal is a co-production of Gimlet and the Wall Street Journal. Additional reporting in this episode by Meghan Bobrowsky. Thanks for listening. See you tomorrow.
Facebook – The Clash Between Facebook and Independent Researchers – The Journal.
Tags: Facebook – The Clash Between Facebook and Independent Researchers – The Journal.