People

TestFlight forever: Developers are building a better world outside the App Store

This club has everything: Cool unreleased apps. Hardly any rules. Endless mystery and tech cred for those who get in.

TestFlight

Apple built TestFlight as a way for developers to beta-test apps, but it's become something much bigger.

Image: Apple

Jordan Singer makes a bunch of iOS apps, but you'll never find most of them on the App Store. Singer calls them "lil apps," and each one is just that: an ultra-basic tool for to-do lists or sports scores or news reading. All the lil apps are side projects for Singer, who works as a product designer for Square's Cash app, so it didn't bother him too much when most of them were summarily rejected from the App Store. "Apple's response," he said, "for the majority of them, was basically … this does not meet the minimum functionality requirement of the App Store."

Singer wasn't interested in adding more functionality to his apps. Doing very little was the point of lil apps. So he did what a lot of developers are starting to do: He published the app to TestFlight, Apple's tool for testing apps, and left it at that. Anyone can download and use it, but you have to know where to look.

Apple considers TestFlight a developer tool, a way for app makers to test their own software and get feedback from a few trusted testers before launching into the Wild West of the App Store. Developers are starting to see it as something else: A place to have apps just for them, or just for their friends, away from the prying eyes and poking fingers of a billion iOS users. It's also increasingly a place to build apps and try things outside of Apple's stringent guidelines for the App Store.

Building an app outside the App Store has been possible for years, of course. Developers have long posted code to GitHub, which anyone with a developer account could compile in XCode and install on their phone. Apple's had testing systems for years, too, but they used to require email addresses or device IDs to work. Now, as Apple has simplified the system, anyone with an app can just share a TestFlight link and have thousands of strangers sign up.

The only restriction Apple places on TestFlight apps is that each one can be used by a maximum of 10,000 people at a time, which suits many developers just fine. People are clamoring for access to the cooler apps still in private beta, while indie developers are using TestFlight to find an audience they'd never find in the bowels of the App Store. And there's no cooler way to show your status than to post a screenshot of your own TestFlight, chock-full of the hottest apps nobody even knows about yet.

All this buzz does actually help developers accomplish one of the stated goals of TestFlight: finding people to test their apps. "One of the challenges with testing early products is that some products need users before you can really get a sense for how they might function — go figure," Lowercase Capital's Matt Mazzeo wrote in 2015. Mazzeo was announcing a product called Lowercase Alpha, which invited people to sign up to get texts whenever the Lowercase team came across a cool beta. If you saw something you liked, you'd just respond "GET" and get a link to download the app. You could send feedback with a text, too, and Lowercase would make sure the developers got it. It served a practical purpose, but Mazzeo also understood the in-crowd appeal: "Sign up for Lowercase Alpha and become user zero of the most fun and exciting products before they hit the market," he wrote.

More recently, a few developers have tried to open the TestFlight doors even wider. Thomas Weigt, an iOS developer and consultant at BitSuites, said he went looking for a TestFlight community about a year ago, when he had an app and needed some testers. "I don't really have a huge following or anything like that," he said, so instead he bounced around Reddit and other communities looking for people who might want to give him some early feedback. He noticed lots of people were interested in testing new apps but didn't know where to go.

So Weigt built a website called Departures, which just lists all the open TestFlights he could find. They run from the big companies — right now, users can jump into the TestFlights for Microsoft Word, Spotify and TikTok — to indie productivity apps, developer tools, and an inexplicably large number of VPNs. At first, Weight found them himself, mining Twitter and Google for links to open TestFlights, but more recently developers and users are submitting them directly.

At first, Weigt said, everybody wanted the same thing: Dark Mode. That was the big feature of iOS 13, and lots of users wanted to get their hands on the new low-light hotness before their friends. Now, with iOS 14 around the corner, users seem to really want to test the new home screen widgets.

Apple didn't respond to a request for comment, but its stance on the way apps should work on iOS is fairly clear. TestFlight is not an alternative to the App Store, it's a staging ground on the way there. Developers told me Apple doesn't review TestFlight apps very intensively, other than to make sure they're not fundamentally broken or obviously malicious. And if Apple's already reviewed, say, version 1.0 of your app, they say it won't even look at 1.0.1. It doesn't think of TestFlight as a long-term home for apps.

Yet that's increasingly what TestFlight is becoming. Developers are using it to build things that would never be allowed in the App Store, to get around some of Apple's more-restrictive policies and more-expensive commissions, and to create an app ecosystem that feels smaller and more intimate. The App Store forces every app to try to become a multinational conglomerate, but TestFlight lets them stay mom-and-pop stores.

Used correctly, TestFlight is also an incredible hype machine. Clubhouse, the hottest app in Silicon Valley, is still TestFlight-only even after months of use and buzz. Clubhouse founder Paul Davison said the app hasn't gone public yet while the small team tries to finish it, but keeping the app exclusive also solves a separate problem. "We think it's important to grow communities slowly, rather than 10x-ing the user base overnight," Davison wrote on Clubhouse's blog last month. "This helps ensure that things don't break, keeps the composition of the community diverse, and allows us to tune the product as it grows." If Clubhouse tried to put its app on the App Store without allowing everyone in, Apple would reject it out of hand. (Just ask the folks who built Hey.) TestFlight gives Clubhouse a way to control growth and to keep hold of its own narrative as long as possible.

There are other, more nefarious reasons a developer might want to avoid App Store scrutiny. Facebook's Research app didn't use TestFlight but went through an Enterprise Certificate program that allowed Facebook to break lots of Apple's rules on data collection and device access in the name of understanding how teens use their phones. (Apple ultimately stopped Facebook from doing this.) In general, TestFlight lets developers collect more data on users — because it's ostensibly for figuring out what's not working in their app — and gives them more leeway.

This kind of exclusivity can change the network, too. The people who know what TestFlight is have the app downloaded and are willing to jump through hoops and deal with bugs tend to be a specific kind of early-adopter. That's part of why Clubhouse has the reputation of being a social network just for VCs. For some developers, that self-selecting level of tech knowledge is exactly the point, but it leaves out a big part of the community.

That's part of what Weigt tried to solve with Departures, making these apps more accessible to more people. Singer's working on something similar with an app called Airport, which operates like an app store for TestFlights. "It's been unreal to see the hype" for TestFlights, he said, "especially around those that have this exclusivity and mysterious vibe to them. and I really wanted to try to lean into that." He's still building the app, but it's already available for download, too. On TestFlight. If you have the link.

Eventually, an app like Clubhouse will likely end up on the App Store. (Hard to be worth $100 million with 10,000 users.) But not every app needs — or wants — access to a billion pockets. Some apps are designed for a group of friends, or a single company, or a small neighborhood. Rather than submit to the App Store and hope nobody notices, those apps are increasingly turning to TestFlight. Other apps might require celebrity status or want to skirt Apple's payment systems.

There's also a whole class of developers building apps just for the fun of it, as a form of creative expression. "They don't want to put it on the App Store," Weight said. "They might just want to share with their friends. It's a creative outlet just like painting or anything else." Ryan Hoover, the CEO of Product Hunt, said he's seen an increase in side projects and experiments as well. "People are experimenting and tinkering more, and some people might either be laid off or just thinking about their next thing," he said. As lockdown changes the nature of work and life, too: People are coding their own solutions to life's annoyances.

Most people I talked to are worried Apple might crack down on how TestFlight is used, preventing them from using it as a smaller, more curated and permissive alternative to the App Store. (More than one also pointed out that Apple probably has much bigger App Store things to deal with right now.) But for now, TestFlight is the perfect place for developers to play.

An entire ecosystem of apps lives under the TestFlight surface. Hoover said he's using a cool journaling app and a messaging app he really likes (but didn't want to tell me too much about, such is the hush-hush nature of the TestFlight circles). Mazzeo, the investor who has hoarded and shared TestFlights for years, is currently building an app called Roadtrip that turns commute playlists into collaborative projects. It's already available for download, and users seem to love it. But it's not in the App Store.

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins