The Chinese government’s pervasive surveillance of Xinjiang’s Muslim Uyghur population makes any contact Uyghurs have with friends and relatives overseas a potential red-flag for police scrutiny. When four family members died in succession in recent years, a heartbroken Aksu refrained from calling home to ask about the circumstances of their deaths to shield relatives from potentially dangerous state scrutiny for having overseas ties.

So when I talked to Aksu in November, I made sure to use Signal, an encrypted phone app, to protect our discussion about psychological trauma afflicting Uyghurs overseas.

The next day, I received an odd note from Otter.ai, the automated transcription app that I had used to record the interview. It read: “Hey Phelim, to help us improve your Otter’s experience, what was the purpose of this particular recording with titled ‘Mustafa Aksu’ created at ‘2021-11-08 11:02:41’?”

Three responses were offered: “Personal transcription,” “Meeting or group collaboration,” and “Other.”

I froze. Was this a phishing attack? Was Otter or some entity that had access to Otter’s servers spying on my conversations?

I contacted Otter to verify if this was indeed a real survey or some clever phishing ruse. An initial confirmation that the survey was legitimate was followed by a denial from the same Otter representative, laced with a warning that I “not respond to that survey and delete it.” My communications with Otter were all restricted to email and were sporadic, often confusing and contradictory.

In the three months since that initial exchange (and there was more to come), I’ve gone down the rabbit hole — talking to cybersecurity experts, press freedom advocates and a former government official — to try and understand what vulnerabilities and risks are present in this app that’s become a favorite among journalists for its fast, reliable and cheap automated transcription.

We make privacy versus utility tradeoffs all the time with our tech. We know Facebook sells our data, but we still post baby pictures. We allow Google maps access to our location, even though we know it leaves an indelible digital trail. And even savvy, skeptical journalists who take robust efforts to protect sources have found themselves in the thrall of Otter, a transcription app powered by artificial intelligence, and which has virtually eliminated the once-painstaking task of writing up interview notes. That’s an overlooked vulnerability that puts data and sources at risk, say experts.

“The fact that these AI-powered services exist and can turn a couple hours of audio into a reasonable written transcript often in a matter of minutes is a complete game changer,” Susan McGregor, researcher at Columbia University’s Data Science Institute, said of transcription apps. “These run on machine learning, which means that they expose your data to the algorithm that is both transcribing your text and almost certainly using your text and audio to improve the quality of future transcription.”

Otter and its competitors, which include Descript, Rev, Temi and the U.K.-based Trint, are digital warehouses whose advantages of speed and convenience are bracketed by what experts say can be lax privacy and security protections that may endanger sensitive text and audio data, the identities of reporters and the potentially vulnerable sources they contact.

Trint, Otter, Temi and Rev all claim compliance with all or part of the user data protection and storage standards of the European Union’s flagship data privacy law, the General Data Protection Regulation. But cybersecurity experts say that the sharing of user data with third parties creates privacy and security vulnerabilities.

Otter “shares your personal data with a whole host of people, including mobile advertising tracking providers, so it strikes me that there’s an awful lot of personal data and the potential for leakage of sources for journalists,” said Paul Rosenzweig, former deputy assistant secretary for policy in the Department of Homeland Security, and founder of Red Branch Consulting. “They also quite clearly say that they respond to legal obligation [law enforcement data requests], so any journalist who transcribes an interview with a confidential source and puts it up on Otter has got to live with the possibility that Otter will wind up giving that transcript to the FBI.”

Or to security agencies of a foreign country. When I asked Otter to clarify whether it shares user data with non-U.S. government or law enforcement agencies, the answer wasn’t comforting. “We disclose Personal Information if we are legally required to do so, or if we have a good faith belief that such use is reasonably necessary to comply with a legal obligation, process or request,” Denise Mutch, an Otter service team member, told me via email on Dec. 16.

Otter distances itself from what might happen to personal data the company shares with third party users. “We are not responsible for the content, privacy and security practices and policies of such third parties or App providers,” Otter’s privacy policy warns.

Journalists are, by and large, aware that nothing is really ever private in this day and age. But the dangers of transcription apps are mostly flying below the radar. The Committee to Protect Journalists’ Digital Safety Kit, last updated in October, doesn’t mention transcription apps.

“This is a very under-discussed issue in a lot of newsrooms because of how much people are dependent on transcription services and just how much of your personal interview data these services can access,” said Martin Shelton, principal researcher at the Freedom of the Press Foundation, which last month published a security analysis of transcription apps.

The report acknowledged that Otter and its competitors use encryption that makes user data “less likely to be compromised by an attacker outside of the organization.” However, unlike the end-to-end encryption offered by messaging apps like WhatsApp and Signal, Otter’s algorithm decrypts data in order to access the audio.

Some transcription apps have other technical vulnerabilities, including the absence of two-factor authentication, that create the potential for what Shelton calls “a nightmare scenario” of hackers accessing one’s account. Of the five main transcription app services, only Rev and Otter offer two-factor authentication for users, and Otter limits it to upper-tier business plans.

These aren’t new concerns. Tech media platform ZDNet sounded the alarm on problems with Otter’s privacy policy back in 2018. Privacy and technology lawyer Aaron Baer warned in August 2020 that transcription app users “will never truly be able to know what happens with their data, in spite of how trustworthy a company can appear.”

Experts say that the trove of data those apps store in their cloud servers make them hacker magnets. “Anytime you have a high concentration of otherwise nonpublic original information, it is going to be a target,” McGregor said.

The long arm of Chinese hackers

My concern was that my interview with Aksu had somehow caught the attention of the Chinese government’s international efforts to surveil and harass Uyghur activists — either through a hack or a government demand to the company.

Facebook revealed in March that it had blocked from the platform a group of China-based hackers that had targeted Uyghurs living abroad with “malware to enable surveillance.” Facebook didn’t directly implicate the Chinese government in those efforts, describing them as “a well-resourced and persistent operation while obfuscating who’s behind it.”

A Washington Post analysis of hundreds of Chinese government documents published in December revealed at least 300 official projects dedicated to “collect data on foreign targets from sources such as Twitter, Facebook and other Western social media.” An analysis of hacking of News Corp. revealed Feb. 5 via a Securities and Exchange Commission filing indicated the hackers had China links. Cybersecurity firm Mandiant concluded that the hackers sought information “to benefit China’s interests.” Chinese Foreign Ministry spokesperson Zhao Lijian responded by stating that “China firmly opposes and combats all forms of cyber-attacks.”

After repeated requests about the purpose of the survey, an Otter representative finally acknowledged that they had sent it to me — something verified by Eva Galperin, director of cybersecurity at the Electronic Frontier Foundation, who checked the email headers. Otter initially described the inquiry as a “Speech Level Survey … requested by our engineering team.” Otter said that the fact that Aksu was the focus of the survey was only because I’d entered his name as the recording’s title.

“For that particular conversation, the conversation’s title included personal information about someone, which was referenced within the survey,” Allen Lai, an Otter service team member, told me via email on Jan. 14. “We want to ensure [sic] you that we are not monitoring your account or content and that we referenced the title of the conversation for you to be able to recall that specific recording.”

Apparently I wasn’t the only Otter user worried about this kind of scrutiny. “This survey has been discontinued over concerns that some customers (such as yourself) may include personal or sensitive information within the title of the conversation, and including this information within a survey may cause some concern,” Lai said.

Take appropriate precautions

The privacy policies of transcription apps offer little comfort for the data security-minded. The Otter privacy policy claims the right to use users’ personal information — including “any text, images or videos that you upload or provide to us” — both to help improve the transcription algorithm as well as for sharing with third parties. Otter said those third parties include 11 different types of entities including “platform support providers”, “advertising partners” and “law enforcement agencies, public authorities or other judicial bodies and organizations.” For that reason, the privacy policy cautions that subscribers get informed consent from anyone who they record using the app.

Otter specifies in its privacy policy that users have the right “to receive an explanation of how we use [user data] and who we share it with.” I asked Otter if they had ever shared my “personal information” — such as the audio and text file of my chat with Aksu — with any foreign government or law enforcement agency. “I would imagine if we had, you would have been informed in some kind of manner, but I will submit the question for verification,” an Otter service representative told me via email on Dec. 16.

On Tuesday, Feb. 15, Otter finally gave me a more substantive response to that query. “We have not and would not share any data, including data files, of yours with any foreign government or law enforcement agencies,” Otter’s Public Relations Manager, Mitchell Woodrow, told me via email. “To be clear, unless we are legally compelled to do so by a valid United States legal subpoena, we will not ever share any of your data, including data files, with any foreign government or law enforcement agencies.”

That provided some measure of reassurance.

“These services generally pledge not to look at your transcriptions unless absolutely necessary. In some cases, they have an AI-automated transcription, and report they have policies and technical controls in place to limit when employees can look at your data. In some cases, they have human transcriptionists, and typically require those to sign confidentiality agreements,” said Shelton of the Freedom of the Press Foundation.

“The bottom line is they do have access to the plain text or the human readable version of the transcript and also the human listenable version of the interviews that you upload [and] this access is necessary to provide many of these services,” Shelton added.

The Freedom of the Press Foundation report recommends that users protect the integrity of data that they commit to transcription app cloud servers with strong passwords and choosing providers that offer two-factor authentication. And it advises users to download and then delete their audio transcripts — cutting and pasting it to another platform such as Word or Google docs — to remove them from company servers to reduce exposure risk. But those are individual stopgap solutions in the absence of what cybersecurity experts say is a much-needed federal data privacy law that covers all corporate use of consumer data.

“For myself, if I’m doing something really sensitive, I still have a standalone digital recorder that does not connect to the internet,” McGregor said. “But if we care about these things, we have to make laws about them … [because] existing law is outdated, mostly predates the consumer internet and until we update it, then we’re sort of stuck with this patchwork of interpretation based on technologies that were mostly invented in the last century.”

Congress remains at a standstill on a comprehensive federal privacy bill. While bipartisan negotiations continue in the House and Senate, partisan disagreements remain over whether a federal bill should preempt existing state privacy laws and whether to allow individuals to sue over privacy violations. In the absence of a federal bill, multiple states have passed their own privacy laws, including California, Virginia and Colorado.

Until those laws change, journalists and others who rely on transcription apps need to carefully consider the potential dangers.

Journalists “really have to engage in a risk assessment about the threats versus the efficiency and utility of using [transcription apps],” said Rosenzweig. “If you’re a journalist who routinely interviews Uyghurs or dissidents in Russia, no, probably not.”

I asked Aksu how he felt about the possibility that our conversation might have been vulnerable. With years of experience of Chinese government surveillance, he expressed resignation about the possibility that our communications had been compromised. “I get harassed constantly by the Chinese government, so I kind of get used to it,” he said.

Rebecca Kern contributed to this report.