Building in the Open

Around the beginning of this year, I started on a new project called CTZN. It was a new attempt at decentralized Twitter that grew out of some ideas I had bouncing in my head.

Here’s what I published at the time:

• The Anti-Parler: Principles for decentralized social networking
• Decentralization and Decency (FOSDEM 2021 talk)
• The CTZN design document

As an experiment in “radically open” development, I decided that I would live-stream all of CTZN’s development on youtube. This was so consistent that I was able to archive the livestreams by commit, which you can see here. (Fun, right?)

Livestreaming CTZN was, in many ways, wonderful. I found a community of supportive, kind, and interesting people who joined me (some daily!) on the streams and made working on CTZN a real joy for me.

It was also, in a few ways, pretty tough.

You might think it would be the effort of streaming that made it hard, but I’m not sure that was it. Sure it took a little more energy than a typical work day — plus, I had to sit at my desk to code while I normally prefer sitting on my couch — but honestly the streaming wasn’t that tough. Especially during pandemic lockdown, it was a really nice way to spend time with people and get a near coworking-like experience.

No, what made CTZN’s development hard was the same thing that always makes all “development in the open” hard: changes in the plan.

CTZN postmortem

Ultimately CTZN didn’t accomplish what I was aiming for. Features and tech decisions didn’t work like I hoped. The high concept fell flat. We began to approach time for the beta and I didn’t love the product.

At some point I’m going to give a talk about “Productizing P2P” because it’s what I’ve been working on for a decade and it’s hard work. Many of us decentralizers are mission-driven to replace current tech with open & decentralized alternatives, but any such product needs to win in the market. “Replacing current tech” often translates into “directly challenging the market leader;” did I not describe CTZN as a “decentralized Twitter”? The expectation is that new tech will lead to new breakaway features, but what happens when it doesn’t? Then you’re in trouble. Then you’re just attacking a market which is 100% dependent on network effects and which has a leader so strong that its brand-name is equivalent to the action (microblogs = tweets).

Let’s talk a little about why CTZN didn’t work like I hoped.

There were two ideas I was chasing after having worked on Beaker browser. The first was to take distribution very seriously. My go-to phrase was “no novel pains in the butt.” People expect to use social media on any device, to link any content without having to download an app first, etc. A lot of p2p projects are working around the novelty of the (still maturing) tech and that often gets pushed onto the user, so I figured that this time, we’d use p2p + servers and that would solve it.

The second idea was really nerdy; it was this idea of “social constitutions.” The tech CTZN uses, the Hypercore protocol, is a network of signed, append-only data structures. Even without decentralized consensus, you can play back the state changes in hypercores’ databases against a program and establish whether or not the state-changes match the rules of the code. This kind of remote-auditing is similar to what Certificate Transparency does. I figured that we could take advantage of that system to create a light-weight smart contract, where forums set their “constitution” and then moderation, algorithms, and even fun features like virtual items could be executed with full transparency. It seemed like a compelling reaction to issues of trust with social media.

We implemented both of those ideas well enough, I think, but I had a really hard time translating the constitutions into a user experience that felt new and interesting. At the end of the day, CTZN felt like a Twitter with a quirky virtual items system baked in, which in practice was not far off from Reddit gold where people rewarded posts by granting items. We might have been able to build that into some interesting ideas around attention-algorithms since the items had baked-in scarcity, but there were also struggles with some of the implementation mechanics. (Mainly: items couldn’t be gifted outside of the sub-communities they were created in.) Trying to make these ideas work took me everywhere; I started to veer into developing a no-code forum-builder slash Notion competitor? It became a mess.✝

We did have some fun with it, for what it’s worth. The first item I created was a tongue-in-cheek crypto currency called PaulBucks, and we’d celebrate new features by doing “PaulBucks drops” on the livestream. I love that kind of silliness. Then, around the time I launched the items system, the server died under mysterious CPU load and had to be re-architected, an event that became known as “CTZNpocalypse.” We had commemorative “CTZNpocalypse survivor” items minted and given to everyone. That was all quite fun, but it was mainly fun in relationship to people who were involved in the dev process and the live-streams. It wasn’t “in-product” fun, so to speak. The stream couldn’t be the product forever.

At some point I found myself going through “one more feature” sprints that were designed to make CTZN a stronger product, and that’s usually a give-away that the core premise isn’t working.

Failing in the open

Here’s what really makes building in the open hard: it’s not a solitary exercise. Even with a solo lead dev like with CTZN, there’s inherently a dialogue with the community. You have to include your people in your decision-making, if just to explain your thinking, so that they know why the heck you changed X, Y, or Z.

When there’s a plan and you’re just implementing the code, that’s easy street. When there’s a difficult decision, it’s not so bad; you explain your thinking, step through the choices, get some feedback, and then keep going. But when the plan fundamentally comes into question, that just sucks.

My personal theory is that open community development can only work post-PMF because it’s the only way that the community can work together under loosely-linked collaboration. Everybody has their own ideas and many of them can be pretty good, but do they all fit?

© Fiat

It’s like you’re building a car but don’t know it yet — all you know is that you’re building some kind of vehicle — and then somebody shows up with this great concept for life-rafts. If you don’t have the clarity to say wait, no, this vehicle only travels on land, then you can find your project sliding into a boat that doesn’t float. Shared clarity usually comes with PMF.

Sometimes it wasn’t community members showing up with the “better life-raft.” Sometimes it was me. I’d build it and show it off, and folks would say “hey that’s cool,” and then a little bit later I’d realize it doesn’t fit. Then I would have to explain, sorry, that was a mis-step and I have to take away a feature you might like. This makes public experimentation really difficult, and even when your community is understanding, it feels bad.

About 4 weeks ago I started doing some big re-writes, and then 2 weeks ago I hit the brakes entirely to figure out where things need to go. It’s been a pause, not an end, because the story is certainly not ending here.

The community has been wonderful about this. You’ve all been super supportive. If you have to fail in the open, do it with people like the CTZN community.

So what’s next?

Bo Burnham put out this great special recently, “Inside.” Some lyrics from Welcome to the Internet really resonated with me:

Not very long ago, just before your time,
Right before the towers fell, circa '99,
This was catalogues, travel blogs, a chat-room or two.
We set our sights and spent our nights
waiting for you -- you -- insatiable you.
Mommy let you use your iPad;
you were barely two.
And it did all the things we designed it to do.
Now look at you - ah, look at you, you, you!
Unstoppable, watchable, your time is now, your inside's out,
Honey how you grew!
And if we stick together, who knows what we'll do?
It was always the plan
To put the world in your hand.
(*Laughs*) Could I interest you in everything all of the time?

There’s a lot of great tech in CTZN that we can apply in some different ways and that’s what I’m thinking about right now. A theory of change for decentralizers has to involve success in the marketplace. Solving for distribution, as I did with CTZN, was not a bad idea, but what matters even more is making something that some people really love.

I don’t need more content, more conversation, more engagement; I have that already. What I really need is something that feels like a personal space, perhaps something that runs at a slower pace, perhaps without feeds or comment sections. I’d love to never (I mean never) worry that I’m being tracked or surveiled. I’d love to have a guitar tabs database that doesn’t get overrun with ads. I’d love to search for medical conditions without feeling like I just gave away something embarrassing to a third party. I’d love to share family photos or documents without wondering when the service will shut down and take my data with it.

I don’t know what, exactly, is on my mind. It’s not social media, but it is inter-personally connected. It’s not a Web browser, but it does share and browse and search and curate information. It’s not email, but it does do messaging. It’s not a PaaS, but it does run software.

© Synology

I was talking to my brother about how useful P2P tech could be if we all had personal servers in our homes — something that was always on and has plenty of disk space, like a NAS designed for Hypercore.

P2P is useful for home servers because it automates configuration and NAT traversal, so it makes a good plug-and-play cloud and it’s able to internetwork with the other home servers. Running P2P on your laptop isn’t bad, it just eats CPU/memory and doesn’t have a perfect uptime guarantee, so the really ideal solution would be a box in a closet somewhere where the data would be kept private and you are free to run your own programs. Alas, most people don’t have home servers.

My brother’s response was, well, how realistic is it that people could have home servers in 5 years? And I think it’s pretty realistic. The hardware is affordable. You really just need the software to be right and to package it up. His suggestion was to go ahead and make that bet; if the software really shines when there’s a home server, then build the software and trust that the hardware can happen.

I’m thinking a lot about what that would look like. It’s going to take a little bit of time to formulate the software. I’ll keep trying to share ideas as they form, but I don’t want to have to keep going back on them after sharing them. That is, after all, the hard part of building in the open.

The name “Unwalled Garden” is on my mind. Is that a bad name?

✝ Something I didn’t discuss here, but which factored heavily, was how moderation factored into CTZN’s design flaws. All I can say is, nobody wants to be tasked with being a moderator, nobody wants to be moderated, and moderation is crucial to any online forum. Improving on the current state of things is non-trivial and I’m unsure whether decentralized tech offers any unique benefits or costs; it seems to be roughly the same challenge as it ever is.