Making is Show Business now
Two Truths and a Take, Season 2 Episode 34
Nadia Eghbal’s new book, Working In Public: the Making and Maintenance of Open Source Software, may not have been on your short list of books to read this year. It’s admittedly a nerdy topic: it’s about open source projects, roles and responsibilities; the rise of GitHub as a developer platform; and how developer culture is evolving around the new power of creator platforms.
I recommend you get it. It is mostly about software development, but the core insight of the book is bigger: Eghbal clearly sees and articulates something important about the way we make things, and how that’s changing.
Who does the work?
Working in Public opens by challenging a common perception about open source today: the idea that it’s collaborative.
The widespread perception of open source is that it’s community work. Looking back at the first heyday of open source in the 90s, you picture these loosely organized, squabbling, collaborative efforts between a closely-knit group of nerds. Some special technology, notably Linux, emerged from that era as evidence that self-motivated teams of volunteers could build world-class products.
One of the radical ideas from this period was that loosely organized, socially motivated volunteer communities could accommodate more useful contributors than formally managed teams. Traditionally, the rule in software engineering is that “adding more engineers to a project makes it ship later”, popularized in Fred Brooks’ 1975 book The Mythical Man-Month. It makes sense: beyond a core team, adding marginal engineers to a task creates more costs (getting them up to speed, dealing with team complexity) than benefits (any useful output they manage to ship).
The open source experience in the 90s violated that rule. Self-organized communities, coordinating via forums and email lists, learned how to accommodate hundreds of productive contributors. We rethought Brooks’ law: if they’re all genuinely self-motivated, then the more the better; they’ll figure out how to make it work. The corollary to this understanding became: if you need to get more work done, go attract more contributors.
Those lessons have carried forward to today, where open source has become so widespread we’ve dropped the name: it’s just “The Ruby community”, or “the Python community". Eghbal writes: “The default hypothesis today is that, faced with growing demand, an open source “maintainer” - the term used to refer to the primary developer, or developers, of a software project - needs to find more contributors. It’s commonly thought that open source software is built by communities, which means that anyone can pitch in, thus distributing the burden of work. On the surface, this appears to be the right solution, especially because it seems so attainable. If a lone maintainer feels exhausted by their volume of work, they should just bring more developers on board."
Unfortunately, for many maintainers, that collaborative mindset no longer reflects reality.
"However, in speaking to maintainers privately, I learned that [initiatives to bring on more contributors] caused them to seize with anxiety, because such initiatives often attract low-quality contributions. This creates more work for maintainers - all contributions, after all, must be reviewed before they are accepted. Maintainers frequently lack infrastructure to bring these contributors into a “contributor economy”; in many cases, there is no community behind the project at all, only individual efforts.”
Something important changed between the 90s and today. If you look at most open source projects now, the distribution of who’s doing the work versus who’s simply there is skewed dramatically: it’s common to see projects where 95% of the work is done by a nucleus of people, perhaps even a single developer, with a long tail of “contributors” who are really more like users or consumers: they might offer occasional pull requests, or flag some issues occasionally, but they’re more like fans watching a football game than the main players.
“Fans” is a useful comparison. They serve an important purpose, they’re excited about the project, and they want to be a part of it. It's like how fans of a band want to “be a part of something”, while understanding it’s not really their show. You’re happy they’re there; the point of creating open source code is sharing it, and there’s no better sign that you’re doing something useful than fans showing up. Fans are your distribution. But fans are costly. They want to engage with you; they need interaction, and they need your time. Having more fans is great, but it doesn’t make your life easier.
Today, the natural constraint on open-source projects has become like an upside-down Tragedy of the Commons problem. The classic commons problem involves a self-renewing but finite public good, like the town green for grazing animals. Here, the commons at risk of being overdrawn isn’t the product; the code itself can be consumed limitlessly. The risk is that the creators become overdrawn and overwhelmed with participation, and with demands on their time.
Eghbal puts it succinctly: “It is not the excessive consumption of code but the excessive participation from users vying for a maintainer’s attention that has made the work untenable for maintainers today.” As the creator of code, you aren’t a community member anymore. You’re a performer, and you’re on stage.
Communities are for collaborators; Platforms are for creators
So how did this happen? How did we go from Brooks’ Mythical Man-Month, where adding more contributors were a net cost on production, to the 90s source ideal where adding new contributors magically worked, and then back to this new present situation, where they’re a cost again?
Here’s where we get to the real insight of the book, which isn’t actually an insight about software. It’s an insight about the relationship between creators and platforms.
If you go back to the 90s, when we first figured out how communities built software, it took a lot of work to use the internet. It wasn’t fast or easy, but that made it special: it meant that everyone on the internet cared about it, a lot. You were really committed to being there. Online communities back then were like a federation of villages; each one had its own culture, its own customs, and its own values. The open source community was like that too, and remained so for a long time.
In these kinds of environments, attracting more users really did advance open source projects, because the costs were usually worth it. When a new member joined a community, they were probably serious about it. There weren’t many “tourists” back then, so there was a real environment of camaraderie. Existing users were happy to onboard you and teach you things, because their effort would likely pay off as a good investment.
Since community members joined slowly and stuck around, there was a lot of trust and shared context in the group. Every community had a different way of working, so there was a fair amount of friction preventing users from jumping around or “surfing” from project to project. Groups could preserve and maintain their collective motivation to keep shipping; they weren’t getting paid, so that motivation was everything.
It sounds pretty idyllic, and for many users back then, it was. As the internet grew more popular, the old timers reliably complained every September as a new crop of college freshmen gained access for the first time, not knowing any of the social conventions. AOL opened the floodgates in 1993, which Usenet bitterly declared “Eternal September”, and the internet veterans have been complaining about it since.
We know what happened next with online content. The tapestry of forums and newsgroups that made up the early internet flourished for a while, but in the 2000s an invasive species arrived: the platforms. The platforms made it so easy to create, share, distribute and discover that everyone joined them, smushing everything together into common, user-friendly formats without the local context or nuance of the smaller groups.
In the open source community, GitHub changed everything. Before GitHub, open source communities didn’t only differ in their social conventions and quirks; they also differed in how they did the work; specifically, in how they managed something called version control while developing software. Once you’d identified your favourite way, you’d likely stay loyal to the projects who used it.
GitHub changed all that. In platform fashion, GitHub started with a simple and convenient set of tools for hosting and version control, but quickly grew into a network. If open source before was like a cluster of distinct villages, GitHub was like a highway connecting them all: developers loved it, because GitHub made it easier to create and distribute and discover. It quickly became the standard way you interacted with projects, and with other developers.
This transition was really tough for the old way. Having one standard set of tools and practices meant a continual revolving door of users, all doing exactly what they’d been told to do: “Want to get into open source? Go check out projects! Make some pull requests! Get involved, be humble, and ask to learn!” With no friction associated with joining or leaving a project, popular projects would get immediately overrun with “help”, but see little real benefit; only irritation and burnout for maintainers, overwhelmed by all of the well-intentioned but short-lived visitors. Developer Nolan Lawson described his experience in the book as “a perverse effect where, the more successful you are, the more you get ‘punished’ with GitHub notifications.” It’s Brooks’ law again, bizarrely reincarnated.
On the other hand, platforms like GitHub are incredible for creators who understand how to work a crowd. Like Twitter or any other social media format, GitHub helps creators to create franchises around themselves, and become known for who they are, rather than what they’re working on at any given moment. Platforms give power to individuals, but it comes with an imperative: you cannot treat your collaborators as peers. You have to treat them as fans, and give them what they want, while protecting your ability to keep creating.
As Eghbal notes: “Like any other creator, these developers create work that is intertwined with, and influenced by, their users, but it’s not collaborative in the way that we typically think of online communities. Rather than the users of forums or Facebook groups, GitHub’s open source developers have more in common with solo creators on Twitter, Instagram, YouTube, or Twitch, all of whom must find ways to manage their interactions with a broad and fast-growing audience."
We Shape our Tools, and They Shape Us
We used to think about software as a product with inherent scarcity: we sold licenses, on floppy disks and CDs, like it were a physical good. We recreated the familiar real-world scarcity we were used to, for no reason aside from that it was how we knew how to sell products. We’ve now learned that code isn’t scarce; it’s the making part that’s scarce. Code is simply a byproduct; the makers and maintainers are what’s valuable.
Code is clearly a kind of technology. It’s a production factor; it gets things done. And yet, in the creator’s mindset, code is also unmistakably content. There’s a cost to making it, and there’s a cost to making it big; but that cost is not because marginal consumption itself is expensive. It’s because success brings attention, interaction, and maintenance - both of the code itself, and of the creator’s reputation. This all takes work, and it’s often not the kind of work the creators like doing.
When you create great products, and people get engaged, you become a show. Shows create their own success: fans create more fans, just like product use creates more of itself. Conversely, if no one’s at your show, no one will go next time. If no one uses your product, it won’t improve. There’s no business on earth with a greater divergence of outcomes between the winners and everyone else.
There’s an old bit of wisdom that nothing is harder to sustain than an emotion. Momentarily putting on a great show is one thing; sustaining that environment and that emotion, not just for the fans but especially for the creators, is incredibly hard to do. Show business, in a nutshell, is the art and science and orchestration that happens around the show that keeps those emotions renewed, and not exhausted.
It feels strange to juxtapose these two kinds of vocations. Making technology seems like a world apart from entertainment and show business. But in this new world, making is show business. Look at what founders do all day! And I don’t mean that in a derogatory way. It’s hard work to create a product, create attention, and then - most importantly - manage that attention so it feeds you momentum but doesn’t burn you out. That’s the hard part of show business; the part they don’t tell you about.
The hardest thing to do is to create, and then keep creating. The people who make the greatest impact are the people who can keep creating, one thing after another, growing in impact each time. If you look at the world’s great makers - Elon Musk, Beyoncé, whoever you like - they don’t stand apart because of their raw talent. They stand apart because they’re able to sustain their creative output for year after year. They manage to keep shipping and compounding their success while leveraging the show around them.
Platforms reward that skill set. At their extreme, it becomes the primary thing that matters. Does that mean all making will trend this way? Not necessarily; it’s a choice. But it’s a choice we make lot. Startups begin and grow within the context of a local scene, not unlike the way bands get started and level themselves up. Many modern tech companies have an average employee tenure of around 2 years; this would be disastrous for an older company or community, but in this new creator-driven model, it’s what we’ve optimized for.
In the old world, you maximized engagement and retention of your community by giving them trust, and giving them context, and making them peers. Platforms make it easier to create and discover and distribute, but at a cost: too many people, too much attention, and too little friction to join or leave. The cost of platforms is less overall trust, and less overall context, for everyone but the core nucleus. Instead of fighting that tradeoff, we accept it: we find new ways to build, like microservices, which make it easier for low-trust, low-context participants to be productive anyway.
You can see what’s happening here: it’s We Shape our Tools, and They Shape Us (and then We Shape Our Tools Again). Our desire to create and discover leads us to build tools and platforms that make creation and discovery easier. This dramatically levels the playing field for participation, and you get way more participants, but they’ll be coming in without context, and without trust. So we adapt in order to make them useful anyway: software is getting more modular, more plug-and-play, and less monolithic. It’s not necessarily better software, but it accommodates the way we build now.
Working in Public is a great book, because it walks you through in detail how the software community implements this philosophy in practice. It’ll be everywhere, soon enough.
Permalink to this post is here: Making is Show Business now | alexdanco.com
Some news out of Shopify this week that, for some of you, has been years in the making: subscription selling is now finally available, and it’s built really well:
Introducing Shopify subscription APIs
Subscriptions are a great example of a hard problem that looks on the surface like an easy problem. And it’s a window into one of the fundamental tensions we face every day building products.
On the one hand, we want to enable as many different use cases, business models, and opportunities for our merchants, partners, and platform developers as we can. Commerce is a tapestry; we want to make all commerce work beautifully, not just certain merchant models. But on the other hand, we don’t Shopify to fragment into different building blocks, custom checkout flows, duplicate admin panels, embedded apps, and cascading complexity. The art of building great products here is nailing both: supporting more use cases, with less complexity.
If you check out how we’ve built Subscriptions on Shopify, you’ll see a great example of what a good job looks like here. Subscriptions aren’t just a new product feature, they take advantage of a new internal architecture inside Shopify built to support any selling model. We mad this possible with an app extension model called Argo that lets custom app features render natively inside Shopify, harness native Shopify checkout, and more. Check it out; it’s really impressive work and it’s a good reminder of how lucky I am to be around a team that ships product like this, so frequently and at such a high standard.
Other reading from around the internet:
Crypto market structure 3.0 | Arjun Balaji This is a well-explained and highly informed picture on the evolution of cryptocurrency trading markets, how they’ve evolved over the past decade, and what’s on the horizon. It’s a rose-coloured glasses version, to be sure, but that doesn’t take away from its utility or merit. Do read this if you’re interested in learning more about where crypto is at these days.
A columnist makes sense of Wall Street like no other (see footnote) | Emily Flitter, NYT Here’s a lovely profile of Matt Levine in the NYT, who as you know from reading this newsletter is one of my favourite writers today. (I enjoyed Gary Shteyngart’s description of him as “the least offensive person in finance.”)
Speaking of the New York Times, here’s an interesting slide deck from Mine Safety Disclosures walking through their new business model in the 2020s, and how to think about them as an investment.
And finally, this week’s Tweet of the Week which made me laugh the hardest:
Have a great week,
Alex