Object Storage vs Committing Uploads: The Folder an Agent Will Happily Push

The client called on a Tuesday to say the site was slow to deploy. That was the whole complaint. Deploys used to take ninety seconds and now they took eleven minutes, and sometimes failed with a timeout. Could I look?

The repository was 2.3 gigabytes. The code in it was maybe four megabytes. The rest was a folder called public/uploads/, containing every file a customer had ever sent through the site’s quote-request form: photos of damp walls, floor plans, PDFs of surveys, and, because the form said “attach any relevant documents,” a few dozen scans of tenancy agreements and one passport.

All of it was in Git. All of it was in the history of a private GitHub repository that three people had cloned, that a subcontractor had access to, and that the hosting platform pulled on every deploy.

The client had been maintaining the site with a coding agent since I handed it over. When I read back through the commit log, I found the moment it happened, and it was not carelessness. It was the agent solving a problem it had been asked to solve.

How a folder of customer files ends up in Git

The site was a small Node app deployed from Git to a platform that builds a fresh copy of the repository on each deploy. The quote form saved attachments to public/uploads/ on the local disk, which is what the original template did.

On that kind of platform, each deploy replaces the application directory with a fresh checkout. Anything written to disk at runtime is gone after the next deploy. So every time the client pushed a change, the uploads from the previous weeks vanished. Customers’ photos turned into broken image links in the admin panel.

The client noticed and asked the agent: “Uploaded files keep disappearing after I deploy. Please fix this so they persist.” The agent investigated, correctly worked out that the deploy replaced the directory with the repository contents, and made the files part of the repository contents. It removed public/uploads/ from .gitignore, added a small script that ran before each deploy to pull the current uploads down from the live server and commit them, and wrote a clear commit message: “Persist uploads across deploys by tracking them in version control.”

It worked. Uploads stopped disappearing. And from that day forward, every customer file became a permanent part of the project’s history.

I do not think the agent did anything a hurried junior developer would not have done. The request was “make them persist,” and Git is a thing that persists. The trouble is that “persist” hid three separate questions that nobody asked: where should these files live, who should be able to read them, and how long should they exist?

Why Git is the wrong home for uploads

It is worth spelling out, because to someone who is not a developer, “it is saved in the repo” sounds like a backup.

Git keeps everything forever. Deleting a file from the repository removes it from the latest version, not from history. Every clone contains every version of every file ever committed. The passport was on the client’s laptop, my laptop, the subcontractor’s laptop, GitHub’s servers, and the hosting platform’s build cache.

Access to code becomes access to customer data. Anyone you give repository access to, like a freelancer fixing a CSS bug, now has every document your customers ever sent. That is a data protection problem in most jurisdictions, and it is exactly the kind of thing you do not want to explain after the fact.

Repositories are not built for binary files. Git is very good at text and bad at large binaries. Each photo is stored in full, clones get slower, and deploys that pull the repository get slower with them. That was the Tuesday phone call.

Deploys start carrying data. Rolling back a code change now also rolls back the uploads folder to an older state. Deploying from a stale branch could remove recent customer files from the live site.

The two real options

Once I had calmed down, the fix came down to one of two approaches. Both are sensible, and which one fits depends on how the app is hosted.

Option one: object storage. Put uploads in an S3-compatible bucket, whether AWS S3, Cloudflare R2, Backblaze B2, or a provider’s own equivalent. The app either uploads to the bucket from the server, or better, hands the browser a short-lived presigned URL so the file goes straight to storage without passing through the app. The database stores the object key, not the file. The bucket is private, and the admin panel generates a signed link when someone needs to view a file.

This works on any hosting setup, including platforms with ephemeral disks and serverless functions. It separates code from data completely. Deploys never touch uploads. It also gives you controls you do not get from a folder: lifecycle rules to delete files after a set period, versioning to recover from accidental deletions, and access logs.

Option two: a persistent directory outside the release. If the app runs on a VPS you control, keep uploads on disk but outside the directory that deploys replace. The classic pattern is a shared/uploads directory next to the releases, with the app’s public/uploads symlinked to it. Deploys swap the code; the shared folder stays. You back it up separately.

This is simpler and costs nothing extra, but it ties uploads to one machine. If you ever move to a platform without a persistent disk, or run a second server, you are back to needing object storage.

For this client, on a platform with no persistent disk and handling documents that should not stay around forever, object storage was the obvious choice.

The cleanup, in order

Moving future uploads to a bucket took an afternoon. Cleaning up the past took longer, and the order matters.

1. Stop the bleeding. I restored public/uploads/ to .gitignore, deleted the pre-deploy sync script, and pushed that change first, so no new customer files would be committed while I worked.
2. Copy the files out. I copied the current uploads folder into a private bucket, keeping the same relative paths, and updated the database records to point at object keys.
3. Switch the app. I asked the agent to change the upload handler to use presigned uploads to the bucket, and the admin panel to generate signed viewing links that expire after fifteen minutes. I reviewed that diff line by line, because it touched both storage credentials and customer data.
4. Rewrite history. With git filter-repo, I removed public/uploads/ from every commit, force-pushed the cleaned history, and asked everyone with a clone to delete it and clone again. I also asked the hosting platform’s support to clear the build cache. The repository dropped from 2.3 GB to under 5 MB.
5. Deal with the data, not just the repo. This is the step people skip. Rewriting history does not un-share what was already shared. The subcontractor confirmed in writing that they had deleted their clone. The client decided, with advice, to contact the customer whose passport had been uploaded. And we agreed a retention period: quote attachments are now deleted from the bucket automatically after twelve months, using a lifecycle rule.

What I tell the agent now

On every project where an agent can commit, I add a short section to the instructions. It has stopped this from happening again on three sites.

• Never commit user-uploaded files, generated files, database files, or logs. If they need to persist, stop and ask where they should live.
• Never remove entries from .gitignore without explaining why in the pull request.
• Uploads go to object storage through presigned URLs. The bucket is private. The database stores keys, not URLs.
• If a runtime file disappears after deploy, that means the host has an ephemeral filesystem. The fix is external storage, not version control.

That last line is the one that matters. The agent’s original reasoning was sound, as far as it went: the files disappear because the deploy replaces the directory, so make them part of what gets deployed. It just did not know that the files were somebody else’s documents rather than part of the product.

I also added a check in CI that fails if any file over 1 MB, or any file in an uploads, storage, or data directory, is added to a commit. It has fired twice since, both times on a large image someone meant to add to the design folder. Both times, it was right to ask.

The cost comparison nobody makes

The client’s first question, after “is the passport thing bad?”, was whether object storage was expensive. For this site, the bucket holds about 2 GB and serves a few hundred downloads a month. The bill is well under a euro. The Git approach looked free, but it cost eleven-minute deploys, an afternoon of history rewriting, and a letter to a customer about their passport.

If your app accepts files from users, decide where they live before the first upload arrives. The agent will ask where to put them if you tell it to. If you do not, it will find somewhere they persist, and Git is always there, always persistent, and always willing.