Stalwart 1.0 Benchmarks: Read the Durability Promise Before the Speedup
A faster mail server is useful only if it meets the service promise your organization needs. Stalwart’s new benchmark publication gives administrators concrete results to inspect, along with qualifications that matter when turning a laboratory comparison into a purchasing or deployment decision.
What is new
Stalwart announced its first stable 1.0 release on October 26, 2026. The project supplies a mail and collaboration server and presents substantial performance improvements. Its accompanying comparisons are maintainer-produced evidence, not an independent certification that one product is fastest for every workload.
Stalwart: October 26 release announcement
Check what an acknowledgement promises
The benchmark’s fairness document describes a consequential difference. Postfix syncs its incoming queue file to disk before acknowledging a message. The tested Stalwart setup writes its log record to the operating system without an fsync on each commit; the document says this can leave recent acknowledged writes vulnerable to a kernel crash or power loss. Dovecot uses mail_fsync = never in the main comparison, with an additional always-sync variant for write tests. The authors explicitly identify the SMTP durability difference as favoring Stalwart and do not isolate its numerical contribution.
Benchmark fairness: acknowledgement, disk sync and configuration differences
Before comparing throughput, write down your own requirement: what must still be recoverable after the failures your service is designed to withstand? Ask how each candidate configuration satisfies that requirement. A throughput result cannot answer that question on its own, and a backup procedure should be evaluated separately from the acknowledgement path.
Read the slow end as well as the typical result
The repository reports Stalwart ahead in 35 of 38 headline comparisons, but also documents exceptions. In the test sending 1 MB messages over 16 connections, the median favors Stalwart while the p99 favors Postfix. Dovecot wins the tests marking a whole mailbox as read. Some SMTP differences depend strongly on client acknowledgement behavior. These details make a single overall multiplier a poor description of the comparison.
Benchmark overview, results and stated limitations
For your evaluation, set an acceptable response-time threshold before running tests. Record how often operations exceed it, plus errors and unfinished work. A typical response can improve while the slowest requests become less acceptable to users. Keep each workload’s result separate instead of averaging unrelated activities into a product score.
Match the test to the service you actually run
The published methodology uses Azure servers with eight vCPUs, 32 GiB of memory and local NVMe for each mail-server side. It supplies a fixed generated mailbox corpus, alternates runs between systems, and includes warm-up and repetition rules. This is a defined laboratory setup; it does not reproduce your particular messages, storage, client population or operating procedures.
Benchmark methodology: hardware, corpus and measurement procedure
The overview also states that the published comparison uses one server per side. Its storage-backend and clustered comparisons are still in preparation. Spam filtering is disabled for the comparison, and the Stalwart build targets the tested CPU while the competing builds are generic. Treat those boundaries as part of the result, not as details to discard.
A practical evaluation record
- Record the exact software builds, storage arrangement and effective configuration for each candidate.
- Choose representative tasks: receiving mail, retrieving messages, searching and handling the mailbox sizes your users actually have.
- Include the production features that matter to you, and document anything intentionally excluded from the test.
- Repeat the workload at increasing load; retain latency, error, queue and resource measurements rather than just a winning number.
- Evaluate recovery and restore behavior with disposable data under an approved test plan before making a production decision.
These are suggested evaluation steps, not results from a benchmark performed for this article. The useful next action is to turn the published results into a small set of questions about your service. Select a configuration when its performance, recovery behavior and operating requirements meet those questions together.
