A few days into making One Million Checkboxes I thought I’d been hacked. What was that doing in my database?

A few hours later I was tearing up, proud of some brilliant teens.

But let’s back up.

Note - I’m trying something new; I’ve also made a YouTube video that tells this story. I’m trying to decide whether I’m interested in making videos; check it out if you’d like!

What was One Million Checkboxes?

On June 26th 2024, I launched a website called One Million Checkboxes (OMCB). It had one million global checkboxes on it - checking (or unchecking) a box changed it for everyone on the site, instantly.

My expectations for the site were very low and very wrong. I thought hundreds of players would check thousands of boxes - instead, 500,000 players checked over 650,000,000 boxes in the two weeks that I kept the site online. The site made it into the New York Times and the Washington Post; it’s on Know Your Meme and Wikipedia. The whole thing was a wild ride.

A separate blog of mine covers the technical details behind OMCB. This blog covers my favorite story from running the site.

But to tell you that story, I need to give you two bits of context.

Bit of context 1: OMCB made it hard to draw

I like to make games that help people interact on the internet. Some people are assholes when they interact on the internet. So when I make games like this¹ I try to add constraints to make the average interaction a little more pleasant.

I’ve been around long enough to know what people will draw if you put an unrestricted canvas on the public internet, so for OMCB I wanted to constrain drawing.

To do this I scaled the number of checkboxes in a row to the size of the browser window. Here’s what that looks like in practice:

excuse the kerning

Here I’ve written “EXAMPLE” - but you can only see it if the browser is exactly the right width! As soon as the number of checkboxes in a row changes the message disappears.

This meant that if you drew something rude on your phone it wouldn’t show up for me on my laptop and vice-versa: your graffiti was only visible to people with the exact same display as your own. This constraint was particularly nice because it was subtle; you might never realize that your writing wasn’t visible to anyone else on the site!

I got a lot of comments asking for me to “fix” this so that people could draw. But the choice here was very intentional.

I’ve found that the best stories from my stranger-interaction games come from how people work around the constraints that I add.

That was foreshadowing.

Bit of context 2: how I stored state

OMCB had a million checkboxes on it. One million is a big number! So I wanted to store and transmit my state efficiently.

To do this I said - a checkbox has two valid states. It’s checked or it’s unchecked. That’s like a bit - a bit is either 0 or 1. So I just stored the state for each checkbox as a bit. Bit 3 was “1” if checkbox 3 was checked and “0” otherwise.

That’s a million bits. There are 8 bits in a byte, so that’s 125,000 bytes, which is 125KB - not even the size of an MP3! Totally workable.

checked boxes are 1s, unchecked boxes are 0s

I stored this data in Redis (an easy to use database) and base64-encoded² it when transmitting it to clients.

I promise this matters. Let’s get back to the story.

Have I been hacked?

A few days after launching OMCB, I rewrote the backend in go (with the help of my friend Eliot) to keep up with the load. And then, for some reason, I dumped an ascii encoding of the raw bytes in my database. I don’t know why - I just did it.

The data I saw looked like this:

what in the world

And my reaction to the data looked something like this:

sheer terror

I panicked. There were URLs in my database! There were URLs pointing to catgirls.win in my database!! Something was very very wrong.

I assumed I’d been hacked. I poured over my logs, looking for evidence of an intrusion. I read and re-read my code, searching for how somebody could be stuffing strings into a database that should have just contained 0s and 1s.

I couldn’t find anything. My access logs looked fine. My (very simple) code was ok. My heart rate increased. My girlfriend patiently waited for me to join her for dinner. And then - wait.

Wait!

I saw it.

I looked at the checkboxes that corresponded to the sketchy URLs in my database.

the "h" is one byte - 8 bits. 8 checkboxes. those 8 checkboxes

That H - it represented one byte. One byte represented 8 bits. 8 bits represented 8 checkboxes.

Those chunks of 8 checkboxes formed a repeating pattern that lined up with the URLs. And if I changed something - if I unchecked a box - the pattern immediately reappeared.

spooky

I hadn’t been hacked.

Someone was writing me a message in binary.

What does that mean?

When I dumped my database, redis converted the data to ascii.

To do that, it read the data one byte - 8 bits - at a time. It converted that byte to a number between 0 and 255 (2^8 - 1). And then it checked whether that number was in the printable ascii character range (32 - 127). If it was, it printed out the corresponding character; otherwise it printed the byte’s hex representation (e.g. \x00 for 0).

104 represents an 'h' and '116' represents a 't'

So someone was:

• Checking boxes
• To flip bits
• To form numbers
• That formed letters
• That spelled out the URL

And they were doing it with thousands of other people on the site.

I was impressed.

Down the rabbit hole

So.

<a href="https://catgirls.win/omcb" rel="nofollow">https://catgirls.win/omcb</a>

catgirls dot win

I hemmed and hawed. I googled around. And then I clicked the link.

interesting

The link went to a discord! And the discord was called “Checking Boxes”³. I joined the discord.

saying hello

And someone was really excited to see me! We chatted for a bit. And then they asked me something that blew my mind:

"Have you seen your checkboxes as a 1000x1000 image yet?"

I said no. They showed me what they were up to:

goodness

They were downloading data for all million checkboxes and rendering them as a 1000x1000 grid (the unchecked boxes are white; the checked boxes are black).

There’s a lot going on here! We’ve got “be gay do crime” - love that - but there’s some interesting technical stuff here too.

double goodness

The repeated noise at the bottom is the binary message I found. Above that is a base64 version of the same message - remember that base64 is what I used for transport. And on the left side is a QR code (with full error correction!). All of these messages linked to the discord.

The discord was full of some very sharp teens⁴, and they were writing these secret messages to gather other very sharp teens to talk about botting the site. Anyone who was writing a bot would probably be looking at either the base64’d version of the data, the binary version, or the 1000x1000 image version; they were covering all the bases.

And this worked! The discord grew from under 20 people when I joined to over 60 by the time I shut the site down.

So what’d they do?

Well, they drew a whole lot! As they built better systems for drawing (and better reverse-engineered my rate limits⁵) their drawings became more complex.

A windows blue screen of death - pre CrowdStrike incident!

Over time they experimented with animations and even tried out some protocols for adding color - like treating adjacent cells as the red, green, and blue channels of a color and drawing to a smaller grid.

The grid at its most chaotic - someone was a Jake Gyllenhaal fan.

I gave the discord a warning before I sunset the site. The night before doing it I removed all my rate limits to see how much traffic the site could handle and what they could do. We ended up with some really cool animations - my favorite was a Rickroll (this clip is not sped up).

Look at Rick go

Is botting good?

Lots of people were mad about bots on OMCB. I’m not going to link to anything here - I don’t want to direct negative attention at anyone - but I got hundreds of messages about bots. The most popular tweet about OMCB complained about bots. People…did not like bots.

And I get it! The typical ways that folks - especially folks who don’t program - bump into bots are things like ticket scalping and restaurant reservation bots. Bots that feel selfish and unfair and antisocial.

And there certainly was botting that you could call antisocial. Folks wrote tiny javascript boxes to uncheck every box that they could - I know this because they excitedly told me.

ok!

I expected this - I’m a programmer! - but people told me that it ruined the site for them and I suppose I understand that.

So, sure. This drawing probably degraded the experience for “regular” users - although the botters did have some rules around where to bot, and I would occasionally chime in to ask them to dial things down.

debatable how well this worked

So there are some caveats. I understand why people don’t like bots; maybe this wasn’t an unassailable good. But man…

I found this so moving!

In highschool, I wrote a recursive mail rule that sent a friend of mine millions of messages as a joke. I (accidentally!) repeatedly crashed the school’s mail server⁶

The adults in my life⁷ were largely not mad at me. They asked me to knock it off, but also made me a t-shirt. I don’t think I’d be doing what I do now without the encouragement that I received then.

What this discord did was so cool - so surprising - so creative. It reminded me of me - except they were 10 times the developer I was then (and frankly, better developers than I am now). Getting to watch it live - getting to provide some encouragement, to see what they were doing and respond with praise and pride instead of anger - was deeply meaningful to me. I still tear up when I think about it.

I’m proud to have made something that this discord decided was worth playing with, and I’m even more proud of what they did with it.

I can’t wait to see what they go on to make.

Again - I’ve made my first YouTube video to tell this story. Given how emotional I found the whole thing, I thought being able to use my voice would be nice. Check it out if you’re interested!

Democrats have expressed hope that putting Walz, a Midwesterner who grew up working summers on a Nebraska farm, on their ticket will help them win over rural voters. But a close look at Walz’s former district — a prime example of how America’s huge urban-rural cultural divide shapes the nation’s politics — shows just how difficult that task will be.

In the 18 years since Walz made the life-changing career change from high school teacher to politician, Minnesota has grown more liberal as a state, but the district that gave him his start has lost almost all of the blue precincts that once dotted its farmlands. The cities in the 1st Congressional District have grown, but in the expansive rural heartland (pigs outnumber residents of the district seven times over, according to 2022 census data), populations have decreased and the people remaining have grown more conservative.

The major upset that Walz delivered in 2006 in his first run here — he beat an incumbent Republican by six points, becoming only the second Democrat in a century to flip the seat — seems even unlikelier now.

A Republican took back the district in 2018, the same year Walz was elected Minnesota’s governor. Walz lost here in both his statewide campaigns, and Republicans are expected to win here again in November. In the district’s cities, many voters — often Walz superfans — want liberal representation, but rural voters are trending ever more Republican, and they are angrier at the left than they were when Walz was their congressman.

Tales of Walz’s days as a teacher and high school football coach trip off the tongues of almost everyone in Mankato, the college town where he and his family lived, but mentions of Walz were widely met with a roll of the eyes this month at the Nicollet County Fair. Mankatoans feel energy and pride for their governor, but at the fair just 20 minutes outside town, some people were more excited by the prospect that if Walz becomes vice president, he might finally leave the state.

“I am not a Walz fan,” declared John Luepke, a farmer and former Nicollet County commissioner. While watching 4-H children show their chickens — “the equivalent of Little League in Minnesota,” an onlooker explained — he quickly deployed one of the Minnesota GOP’s favorite Walz attack lines.

“We had a $17 billion surplus and he just … spent it. It’s not like I got any rebates,” Luepke said.

As part of last year’s budgeting process, Walz agreed to spend almost all of the projected $17.5 billion surplus over the next two years, directing it toward K-12 education, infrastructure projects, and tax cuts and credits for working families. GOP lawmakers said the size of the surplus was evidence of overtaxation and accused Democrats of squandering the money and setting the state up for deficits.

In rural Minnesota, even some voters who liked Walz’s work in Congress complain that as governor he was far too liberal. Some conceded, though, that he knows more about agriculture than most vice-presidential or presidential nominees.

In the livestock enclosure, farmers grumbled over new agricultural controls — designed to curb water pollution — that have affected farmers under Walz’s governorship, including permits for feedlots and increased regulation of the spread of manure and fertilizer.

“Ninety dollars to vaccinate a farm cat!” Luepke exclaimed. Last year he had been able to buy the shots over the counter, he said, but this year he would have to take the cat to the veterinarian for a “wellness check” first, bumping up the price. “They’re farm cats. There are predators and roads — they only have about a 50 percent chance of making it to a year or two old,” he said. He said he wouldn’t have the same problem in South Dakota, where he bought the cat from a farm store. “This state wouldn’t even let them be sold in stores like that,” he added.

The biggest problem, according to the farmers — amid low crop prices and waterlogged fields — is high taxes and high government spending.

“We have some of the highest income taxes in the country. South Dakota doesn’t have any revenue tax,” Luepke continued, to nods from the farmers around him. “Iowa has very little. Wisconsin isn’t too bad either.” Minnesota has the highest top corporate tax rate in the country and the sixth-highest top bracket for income tax, but the state has lower taxes on poor families than many other states do.

Voters here are quick to relay the attacks local Republicans use against Walz — particularly, in a community of family farms that prides itself on pinching pennies, a school food-voucher program fraud scandal that they say falls on Walz’s shoulders.

In one of the largest pandemic-related fraud cases in the country, 70 people have been charged in an alleged scheme to fraudulently claim more than $250 million in federal funds. Walz had no direct role, but critics have said that, as governor, he should have prevented it.

Many voters in the rural areas of Walz’s old district no longer credit him for his moderate voting record in Congress and his time on the House Agriculture Committee.

“They’re portraying him to the whole United States as a moderate,” said Rose Oachs, a 76-year-old retired public health nurse, who was visiting the pig enclosure. “And as far as living in Minnesota all my life, I don’t feel he’s a moderate. He’s far left.”

But southern Minnesota has changed over the past two decades.

“The district was divided when he won in 2006,” said Nick Frentz, a state senator from North Mankato who belongs to the Democratic-Farmer-Labor Party, Minnesota’s state-level affiliate of the national Democratic Party. “But by the time he’s running for the final of his six terms … it’s red red, but he wins. Barely.” By 2016, Walz — who in 2008 won by nearly 30 percentage points — hung on by less than one.

Election maps show the rural parts of Walz’s district growing increasingly Republican during his six terms in Congress. The cities and area colleges and universities have grown, helping Democrats. (Rochester, the district’s biggest city, grew by more than 17,000 people over Walz’s 12 years in Washington, according to census data.) But rural areas have struggled economically and lost population, Frentz said.

For Walz’s supporters, his ability to flip and hold on to a red district that was growing more conservative is evidence that he can appeal to a large swath of the electorate, including voters who disagree with him on some issues.

People Walz regularly interacted with as a congressman said he held on to his district by reaching across the aisle, maintaining a moderate voting record, staying involved with agriculture and taking the time to speak to constituents who disagreed with his politics. Being endorsed by the National Rifle Association — which later rebuked him after he changed his position on gun rights — and serving in the National Guard also helped, Frentz said.

Since being elected governor in 2018, Walz has signed a laundry list of Democratic priorities approved by the narrowly divided state legislature, including legal protections for abortion rights, free school meals, a child tax credit, legal marijuana and protections for gender-affirming care.

That record excited liberal Democrats but drew opposition from many voters in his old congressional district.

Walz didn’t abandon his rural roots, Frentz argued. “Those in the 1st District who had Tim as a congressman and were willing to vote for a Democrat, even with conservative leanings, sometimes they’re disappointed at the stuff that passes at state level that he signs,” said the legislator, who has known Walz for 20 years. “But he doesn’t just represent the … people in the district anymore, and part of the state is a good deal more liberal than down here.”

Even in this now GOP-held district — the Republican incumbent, Brad Finstad, won by 12 points last cycle — tens of thousands of people still vote for Democrats. Although Finstad benefits from how red the rural areas have become, he has to win enough votes in those places to offset the growth of the district’s blue cities.

In Mankato, many people are enjoying watching their high school coach rise to the national stage. On a chalkboard at a local bar that lets people buy each other drinks for the next time they come in, someone has bought Walz a beer, marked with a heart “for democracy.”

Walz’s long tenure coaching and teaching has made him a firm fixture of the community. Every person in Mankato who spoke to The Post either knew Walz personally or knew someone who did.

Nick Maxwell, a personal injury lawyer, moved to Mankato shortly after Walz was first elected to Congress. “There was a lot of excitement at that time and it just carried over … which was interesting because as soon as he stepped out to become the governor, we lost that seat,” he said. “That’s what you see at a macro level: He’s just got something extra that resonates with people and a way of speaking that people can grab on to.”

Walz’s “authenticity” as a politician who behaves like a “normal person” has allowed him to heal rifts between communities that have grown apart, Angie and Dan Bastian, longtime friends and neighbors of the Walz family, said in an interview. Vice President Kamala Harris and other Democrats were right to bet that Walz could bridge divides between rural and nonrural communities nationwide, too, they argued, and offered an example.

Last year, they said, Walz unexpectedly showed up at the wake for Dan Bastian’s father. Bastian’s rural family whispered and stared. Some were not happy. But Walz talked to everyone anyway.

“He wants to listen, he wants to incorporate. He’s firm on his ideas and his policy, but he is very open to bringing people together,” Angie said. “He talked to them. He built a bridge. That’s his magic.”