Mailu vs Mail-in-a-Box: When the Docker Email Stack Still Can’t Clear Gmail

The search is almost always the same: someone already runs Docker, they want mail that is not Gmail, and they are trying to decide whether Mailu is “the grown-up Mail-in-a-Box.” They paste the compose file, they get a green admin UI, they send a test to a personal Gmail account, and the message lands in Spam. Sometimes it never lands at all. Then they swap stacks, as if Postfix-in-a-container versus Postfix-on-Ubuntu was the variable Gmail cares about.

It is not. I used to close tickets for small firms that had done this swap three times. Mailu did not fail because it is Docker. Mail-in-a-Box did not succeed because it is an appliance. Both of them will hand you a 10/10 on mail-tester.com and still bounce off Google’s bulk filters if the IP is new, the PTR is a generic cloud hostname, or the last tenant on that address sent pharmacy mail. The stack is the easy part. Clearing Gmail is the product.

What people think they are choosing

Mailu is a Compose stack: Postfix, Dovecot, Rspamd, Redis, an admin front end, optional Roundcube or SnappyMail, optional webmail, optional antivirus. You bring the host, the reverse proxy, the volumes, and the DNS. You can run it next to Immich and n8n. That is the attraction. It looks like the rest of the homelab.

Mail-in-a-Box is a different bet. It wants an Ubuntu box it can own. It stands up nsd, Postfix, Dovecot, Nginx, Nextcloud for contacts and calendar, and a status page that yells at you in English when DKIM is missing. You do not compose it. You do not “just add a label for Traefik.” You give it a VPS and you follow the DNS checklist it prints.

Those are real operational differences. They change how you upgrade, how you back up, and how angry you get at 2 a.m. when Let’s Encrypt renews the HTTPS cert and forgets the mail names. They do not change whether smtp.gmail.com will accept a message from a DigitalOcean droplet that was a scanner last month.

Mailu’s honest advantage is not deliverability

If you already think in compose files, Mailu is less of a lifestyle change. You pin versions. You put mail data on a named volume. You keep Nextcloud somewhere else because you already have it. You can put the admin UI behind Authelia. You can turn Rspamd’s greylisting up without learning Mail-in-a-Box’s opinion of greylisting.

That flexibility has a cost. Mailu will not walk you through a glue record. It will not refuse to start because your MX points at the wrong hostname. It will start, accept mail on 25 if the cloud provider even lets you bind 25, and queue messages that look fine in the UI. I have watched people treat the Mailu dashboard the way they treat Uptime Kuma: green means healthy. Green means the containers answered HTTP. It does not mean Gmail issued a 250.

The Docker-specific failures are boring and they still eat weekends:

• Port 25 is filtered at the hypervisor. Compose is “up.” Postfix is listening. Nothing inbound or outbound actually moves.
• Someone fronts SMTP with Traefik or Caddy because every other service is HTTP. STARTTLS breaks. Clients that expect a real mail listener get a reverse-proxy greeting.
• Volumes were declared for /data but the queue or the DKIM keys landed in the container layer. A recreate “fixes” TLS and silently mints new keys. SPF still passes. DKIM starts failing. Gmail notices first.
• IPv6 is on by default on the VPS. Mailu sends from an AAAA that has no PTR. Google is less sentimental about that than your mail-tester run, which often tested IPv4 only.
• Rspamd’s Bayes is empty. You turned the spam score down so your own aliases would come through. Now your outbound looks like a host that does not know what spam is.

None of that is an argument against Docker. It is an argument against treating a mail stack like another stateless app. Mail has memory: keys, queues, reputation, and the last fifty messages you sent from that IP.

Mail-in-a-Box’s honest advantage is a checklist, not Gmail

Mail-in-a-Box still wins for a certain person: one domain, one cheap VPS, no desire to become a Postfix person. The status checks are the feature. They tell you the PTR is wrong, the TLSA record is stale, the DNSSEC DS is missing at the registrar. That is more than Mailu will volunteer.

It also hides things you will need later. The box wants to be the nameserver. That fights people who already use Cloudflare or deSEC and only wanted MX and TXT. The Nextcloud it ships is not the Nextcloud you already run. Upgrades are “wait for a release, take a snapshot, hope.” If you outgrow the appliance — extra domains, a catch-all that should not exist, a need to relay through Amazon SES for the messages that actually have to arrive — you are fighting the product, not configuring it.

I have migrated more Mail-in-a-Box installs off a burned Hetzner IP than I have migrated Mailu installs. That is not because Mail-in-a-Box is worse at SMTP. It is because Mail-in-a-Box is what people install when they believe the wizard will also purchase reputation. It will not. The status page can be all green and Gmail will still fold the first invoice reminder into Promotions, then into Spam, then into a silent drop after a user hits “Report spam” once.

The test that lies, and the inbox that doesn’t

mail-tester, MX Toolbox, and the Mailu “send a test” button measure the records you control: SPF alignment, DKIM signature, DMARC policy, rDNS that matches the HELO, a clean blacklist check at the moment you looked. Those checks matter. Skip them and you will fail for a reason you can fix in twenty minutes. Pass them and you have only proven you are not a complete amateur.

Gmail’s decision is downstream of reputation systems you do not get an API for. Google Postmaster Tools will show you domain reputation, IP reputation, and spam rate — after you have enough volume for the graphs to exist. A household domain sending twelve messages a day never gets a useful graph. You are flying on whether your sister-in-law saw the birthday mail.

Outlook.com is often stricter than Gmail for new cloud IPs. Yahoo still wants you in their complaint feedback loop if you send anything that looks like a list. None of those providers care that your compose file is prettier than Josh’s Ubuntu installer.

What neither stack can fix

If you take nothing else from this comparison, take the list of things Compose cannot dockerize:

• PTR on a generic cloud IP. You open a ticket. The host sets mail.example.com or they refuse because you are on a shared range. Mail-in-a-Box will scream about this. Mailu will not. Gmail will quietly punish both.
• A neighbor’s spam. /24 reputation is real. The previous customer on your “dedicated” address ran a form-to-mail script. You inherited their aftertaste.
• Port 25 policy. AWS, GCP, and a surprising number of “unlimited” VPS shops block outbound 25. People discover this after they have already pointed MX at the box.
• A domain that is too new, or a domain that used to be a newsletter. Google remembers. So does Microsoft.
• Content that looks like a campaign. HTML invoices from a domain with no prior human conversation, sent to twenty Gmail users on day one, will get the same treatment as a cold list. Your stack did not cause that. Your behavior did.
• IPv6 without matching rDNS. This one still surprises people who “fixed IPv4” and then watched half the messages leave on the other family.

I have seen Mailu operators spend a week tuning Rspamd while the actual problem was a DigitalOcean PTR that still said ubuntu-s-1vcpu. I have seen Mail-in-a-Box operators pass every status check and still fail because they sent from the box’s IPv6 address, which the provider never wired to the hostname the IPv4 PTR used. Different UIs. Same hole.

When Mailu still cannot clear Gmail — even after you did it “right”

This is the failure that produces the title. You followed the Mailu docs. You published SPF, DKIM, DMARC at p=none then quarantine. You registered the domain in Google Postmaster Tools. You sent only to people who asked. Two weeks later, messages to Gmail sit in Spam or they arrive after a six-hour greylist that is not your greylist.

At that point people switch to Mail-in-a-Box, or they switch from Mail-in-a-Box to Mailu, and they lose the DKIM selector in the move. That is how you turn a reputation problem into a DNS problem. The honest fork is not another all-in-one. It is one of these:

Relay the mail that has to arrive. Keep IMAP and inbound on Mailu if you enjoy owning the store. Hand outbound to Amazon SES, SMTP2GO, Mailgun, or a small provider that already paid for the IP warmup. Your users still hit Dovecot. Gmail sees a sender it already knows. This feels like cheating until you price a weekend against a $10 relay bill.

Warm a dedicated IP you actually control. A provider that will set PTR, will not yank 25, and will not recycle the address every time you resize the droplet. Send like a person for weeks. Do not import a CSV of “friends.” Do not send the same HTML blast to twenty addresses on day one. Mailu will not do this for you. Neither will the appliance.

Stop sending from the homelab IP. Residential fiber, CGNAT, and Starlink are not mail networks. If the MX points at a house, inbound can work with a tunnel. Outbound to Gmail from that path is a hobby that ends in the spam folder. I say this as someone who tried it so I could tell clients to stop.

If you are still choosing between appliances after those three, you are choosing a UI for a problem the UI cannot see.

Mailcow, Stalwart, and the next swap

The next search after Mailu versus Mail-in-a-Box is usually Mailcow, because Mailcow is also Docker and it looks more “complete”: SOGo, a busier admin, more knobs. Completeness is not reputation. A fuller suite gives you more containers to back up and the same SPF record. I will not pretend Mailcow is the Gmail fix either.

Some people leave both Mailu and Mail-in-a-Box for Stalwart, which is a different shape — one binary, JMAP, fewer moving parts — and then discover the PTR is still wrong. If you are already comparing the appliance model to a modern all-in-one, the useful next question is not “which webmail looks nicer.” It is whether the status page can even see a broken reverse DNS, which is the actual plot of Mail-in-a-Box vs Stalwart when the stack hides a broken PTR.

Who should still pick Mailu

Pick Mailu if you already operate Compose, you want the mail data next to the rest of the stack, you are willing to own DNS yourself, and you have a plan for outbound that is not “this new VPS will be fine.” Pick it if you refuse to let an installer become your nameserver. Pick it if you want Rspamd in a container you can replace without reinstalling Ubuntu.

Do not pick Mailu because you think Docker will look more legitimate to Gmail. Gmail does not parse your compose version. Do not pick it because a blog said it is what “production” people run. Production people who need mail to arrive pay for a sender, or they have a netblock they have been warming since before the domain existed.

Who should still pick Mail-in-a-Box

Pick Mail-in-a-Box if you want a single SSH session and a status page that speaks in checklists. Pick it if you do not already have a reverse proxy religion. Pick it if the alternative is a pile of half-configured Mailu services and a DKIM key you cannot find.

Do not pick it because the setup wizard includes DNS and you assume DNS includes reputation. Do not pick it if you already run a careful Cloudflare zone and you will spend the first Saturday fighting nsd. Do not pick it as a Gmail strategy. It is an operations strategy.

The decision I actually give people

If the inbox that matters is Gmail, and the users who matter will not check a second webmail, start with a host that already clears Gmail: Fastmail, mailbox.org, Migadu, or Google Workspace if you are already in that religion. Self-host the archive later if you still care.

If you are going to self-host anyway, pick the stack that matches how you already operate — Mailu for Compose people, Mail-in-a-Box for appliance people — and budget a relay or a clean IP from day one. Treat mail-tester as a lint step. Treat a message that arrives in a Gmail inbox, not Promotions, not Spam, from a recipient who did not whitelist you, as the only acceptance test.

I ran mail for other people long enough to be bored by stack wars. Mailu will not save you from a dirty /24. Mail-in-a-Box will not save you either. The Docker email stack can be tidy, documented, and fully green, and Gmail can still decide you are not a person. That decision is the article. The compose file was never the article.