When to Stop Hosting the Prototype and Rewrite the Deploy by Hand

Our neighbourhood tool library runs out of a converted garage two streets from my house. Volunteers lend drills, ladders, tile saws and a pressure washer that everyone wants on the first sunny Saturday of spring. For years the booking system was a shared spreadsheet and a lot of goodwill. Then Hal, who coordinates the volunteers and had never written code, spent a rainy weekend with an AI app builder and came out with a real booking app. Members could log in, reserve a tool, and get a reminder email the day before it was due back.

The builder hosted it too. One button published it to a subdomain. The database was the builder’s managed one. Environment variables lived in a settings panel. The reminder emails ran from a scheduled job that Hal set up by clicking through a form. For eight months that was exactly right, and I would defend that choice to anyone. Then one Friday night it stopped being right. The work of getting off it taught me more about when to rewrite a deploy than anything I have read on the subject.

The Friday it stopped being a prototype

By the time it broke, the app had grown from 38 households to a little over 400, with about 60 loans a week, most of them collected on Saturday mornings. On Friday evening Hal asked the builder’s agent to add a waitlist for the popular tools. The agent wrote the feature, added a database column, and published. The publish went green. The booking form then failed on every submit.

Hal did the sensible thing and restored the previous checkpoint. The code went back. The database did not. The migration the agent had run stayed in place, and the old code choked on a table it no longer recognised. The platform’s log view showed request counts and a wall of generic 500s. Saturday pickup ran off a paper list taped to the garage door. Hal called me on Sunday because I am the person on our street who runs her own servers, which in a neighbourhood is roughly like being the person who owns a trailer.

I fixed the immediate problem in about forty minutes by adding the column back in a way the old code could tolerate, and the app came up. What kept me at Hal’s kitchen table for another two hours was a set of questions neither of us could answer.

Five questions the platform could not answer for us

I now ask these about any prototype that has started to matter to people. None of them is about which host is better. They are about whether anyone understands the deploy.

Which version is live right now? The builder showed checkpoints with timestamps, but not which commit in the synced repository they matched. We could not say with confidence what code the members were running.

Can we redeploy last week’s version without touching the data? Friday had already answered that one. Code and schema moved separately, and only one of them had an undo button.

Can we rotate the email provider’s key without the builder’s panel? No. The key existed in one place, a settings screen tied to Hal’s personal login.

What runs on a schedule? Hal said “the reminders”. He was not sure whether anything else did.

Could anyone besides Hal do any of this? No. If Hal went on holiday, the tool library’s booking system went with him.

Five noes did not mean the builder was bad. They meant the deploy had become something only the platform understood, at the same moment 400 households had started planning their weekends around it. That combination is my trigger. It is not traffic, cost or code quality. It is people depending on the app on a schedule while nobody can describe how it gets to them.

Writing the runbook before touching the host

“Rewrite the deploy by hand” sounds like it means moving to a VPS. For me it means something smaller and more useful first: writing down, step by step, everything the platform does between “Hal clicks publish” and “a member sees the booking page.” Where it ends up running is a second decision, and a much easier one once the list exists.

The builder had a GitHub sync, so I had the code. I read it the way you read a house you are about to buy, looking for the things the seller forgot to mention. Over about eleven hours across two weeks of evenings, I found these:

• The agent had put the migration call inside server startup. Every boot ran every pending migration, so every publish was also a schema change, whether anyone meant it to be or not. That was the real cause of Friday.
• Seven environment variables were set in the panel. The code read nine. The two missing ones fell back to hardcoded defaults, and one of those was the sender address for every reminder email, pointing at a test address from the first weekend.
• Static files were served through the platform’s CDN with cache headers the repository never mentioned. That is why members sometimes saw the old layout for a day after a change.
• The scheduled job was a form in the panel. It called an endpoint in the app with a shared secret header once an hour.

The runbook came to fourteen steps. Three of them were things the platform did that nobody had asked it to do, and one of those three had caused the outage. The document was worth writing even if we had decided to stay on the builder.

What “by hand” turned into

We did not stay. The deciding factor was not price, though the builder’s plan cost $25 a month and we now pay nothing extra. It was the second and fifth questions. We needed a deploy where code and schema changes were separate, deliberate steps, and where a second volunteer could follow the instructions without me on the phone.

I run a small VPS for a few community services already, so the app went there. It got a Postgres database on the same box, a nightly dump copied to a second machine, and Caddy in front for certificates. The deploy itself is a 70-line shell script that lives in the repository. It pulls a tagged commit, installs dependencies from the lockfile, builds, restarts the service, and waits for a health check. Migrations are a separate command with their own section in the README, and the script refuses to deploy if there is a pending migration nobody has run. I took the migration call out of server startup myself. The reminders became a systemd timer defined in a file in the same repository as the code it calls.

I did consider putting a self-hosted PaaS on the box to get Hal his one-click button back. I decided against it for the same reason we were leaving: a friendly button was what had hidden the migration in the first place, and a PaaS UI can hide a failed deploy just as well as an app builder can. A script is uglier, but everything it does is written down.

Hal ran the deploy twice while I watched and once on his own. The second volunteer, a retired electrician named Rosa who reads manuals for pleasure, ran it the week after. That mattered more to me than any uptime number.

The job I left behind in a panel

The cutover went cleanly on a Wednesday night. Bookings worked and emails sent. I went to bed pleased with myself.

No reminder emails went out on Thursday, Friday or Saturday. I had written the systemd timer and tested it against a copy of the database. I had not enabled it on the real box, because my notes said “reminders: timer” and never said “reminders: enable the timer.” The old scheduled job lived in the builder’s panel rather than in the code, so nothing in the repository reminded me that it was missing. I had also switched the old app off, so its job was not quietly covering for me.

We found out on Sunday. A member returned a hammer drill twelve days late, and the person on the waitlist for it had rented one from a hire shop to finish a job. Nobody was angry, which somehow made it worse. The fix took one command. The lesson took longer to absorb: anything configured in a platform’s panel is part of the deploy, even if it never appears in the repository. The panel should be exported before the code. I had spent eleven hours reading code and ten minutes reading the settings screens, and the settings screens were where the missing step was.

The runbook now has a section headed “things that ran without being asked” that lists the scheduled job, the CDN headers and the startup migration, along with what replaced each one. The cutover checklist has a line for each of them, including “confirm the timer fired.”

Where the agent fits now

Hal still uses the builder, and I encourage it. When he wants a feature, he prototypes it there, and the agent’s changes come across as a pull request to the repository. I read it, and if it touches the schema, the migration goes out as a deliberate step before the code that needs it. The agent is still very good at writing a waitlist. It just no longer has the publish button, and that is the whole change.

The honest cost is on my side of the ledger. Hosting the app myself means that when the box has a problem at eleven at night, I am the one who finds out. I have argued for years that self-hosting is sometimes a stance and sometimes a chore, and this is the chore half. Having Rosa able to run the deploy halves it. It does not make it go away.

When I would leave a prototype exactly where it is

The same month, a friend asked me to look at a rehearsal sign-up app her choir had built with the same kind of tool. I wrote the runbook for it. It came to four steps. There were no migrations after the first, no scheduled jobs, and one environment variable that was in the repository’s example file as well as the panel. The data was a list of names that could be retyped in an afternoon. If it went down on a Tuesday, the choir would send a group message.

I told her to leave it where it was. Writing the runbook had not been a step towards moving it. It had been the way to find out whether moving was worth doing, and for her it was not.

That is the rule I use now. Keep the prototype on the builder while you are the main user, the data can be recreated, and downtime costs nobody their Saturday. Once other people plan around the app, write the deploy down, whether or not you intend to move it. Rewrite it by hand when the written version shows you steps nobody chose, like a migration on every boot, a key behind one person’s login, or a job that exists only in a form. The host matters less than the list. Hal’s first weekend was not a mistake. It got 400 households a booking system, and it only became a liability when nobody could say how it was deployed.