3. Adding a Durable Outbox
Tutorial · Applies to Brighter V10 · Prerequisites: Your First Message Over a Broker
Write your business data and the message announcing it in one Postgres transaction, then let the Sweeper deliver the message once that transaction has committed.
This is the third rung of the ladder. Rung 2's sender called Post and hoped: if the process had died between writing its data and reaching the broker, the data would exist and the announcement would not, permanently. Here the message is written to a database table — the Outbox — inside the same transaction as your own row, so the two commit together or not at all. A background service sends it afterwards.
The cost is a delay of a few seconds before the message goes out. What you buy with it is that your data and your messages can no longer disagree.
What You'll Build: A Durable Outbox
Rung 2's three projects, unchanged except for the sender, plus Postgres alongside RabbitMQ:
Greetings
unchanged
GreetingsReceiver
unchanged — it does not know or care how the message was sent
GreetingsSender
a command and a handler, a Postgres Outbox, a Greeting table of its own, and the Sweeper hosted in-process
The sender prints Committed. at once and then keeps running. About ten seconds later the receiver prints Received: Hello from the sender. That gap is the whole point: the send is no longer on the request path.
Then you run it again with --fail, which throws after both writes and before the commit, and find neither of them in the database.
Before You Start the Durable Outbox
Rung 2 complete. Your First Message Over a Broker built the three projects this rung starts from, and only the sender changes here. Work in that solution; this page does not rebuild it.
The .NET 9 SDK. Check with
dotnet --version.Docker Desktop, running, with port 5432 free as well as rung 2's 5672 and 15672.
A Postgres client. Everything below uses
psqlinside the container, so you need nothing installed.About twenty-five minutes, most of it reading. The machine work measured 9.2 seconds to add the six package references and 1.3 seconds to build, starting from a rung 2 solution and a NuGet cache holding only rung 2's packages. Pulling the
postgresimage the first time takes longer and depends on your connection.
Step 1: Start Postgres
Rung 2's docker-compose.yml gains a second service. Replace the file with this:
Expected result: docker ps shows two containers running.
Neither service has a healthcheck, so that command returns before either accepts connections. RabbitMQ is ready when http://localhost:15672 answers. Postgres is ready when this prints a row:
Step 2: Add the Packages
All six go to the sender. The receiver and the shared library do not change.
Five of the six are Brighter's. Two are worth pausing on:
Paramore.Brighter.Outbox.Hostingis whereUseOutboxSweeperlives. The Sweeper is the feature this rung is named after, and it ships in its own package rather than in the Outbox one.Npgsqlis not a Brighter package at all. You need it because you are about to talk to Postgres directly — theGreetingtable is yours, and creating it is your code's job. See step 3.
Expected result: GreetingsSender.csproj carries six more PackageReference lines, and all three projects still build with 0 Error(s). No code has changed yet.
Step 3: Create the Greeting Table
Two tables will live in this database and they have different owners. Brighter's provisioning creates the Outbox table and nothing else. Greeting is your table: your schema, your migrations, your problem. A reader who conflates the two goes looking for a schema-management feature that Brighter does not have.
Your table is as small as a table gets:
The sample runs that in plain ADO.NET at the bottom of Program.cs, deliberately not through Brighter, because it is not Brighter's table. You will see it in the next step.
Expected result: nothing to check — you have written no code and the table does not exist yet. Step 4 adds the code that creates it.
Step 4: Configure the Outbox
Replace GreetingsSender/Program.cs:
That will not compile yet — AddGreeting and its handler arrive in step 5. What is new since rung 2:
Three lines on
AddProducers. Rung 2 set onlyProducerRegistryand got the default in-memory Outbox: enough to makePostwork, gone the moment the process is.Outbox,ConnectionProviderandTransactionProviderreplace it with Postgres. The transaction provider is the important one — it is what lets your handler hand Brighter a transaction that the handler opened.AddSingleton<IAmARelationalDatabaseConfiguration>, which is easy to leave out.TransactionProvideris given as a type, not an instance, so the container activatesPostgreSqlTransactionProviderand its constructor asks for that interface. Without the registration the host starts happily, provisions the Outbox, logs the Sweeper ticking, and only then throws — on the first attempt to resolve a command processor, naming a type your code never mentions.UseBoxProvisioningcreates and migrates the Outbox table at startup, which is why there is no migration project and no second terminal here. It needs rights toCREATE TABLEandALTER TABLE. The Docker Postgres above has them; a production database very often does not, and Box Provisioning is one of two options for exactly that reason — see Further Reading.UseOutboxSweeperhosts the Sweeper in this process. It is anIHostedService, which is why the sender now runs a host instead of building one and throwing it away as rung 2 did.StartAsync, notRunAsync. There is work to do between starting the host and waiting on it, and starting is what provisions the table and starts the Sweeper — so it has to happen before the send rather than after.
Both Sweeper values are the defaults, spelled out because the delay they produce is the thing this rung teaches: a message is picked up on the first tick that finds it at least MinimumMessageAge old.
Expected result — the one step on this page where the sender does not build. dotnet build GreetingsSender reports error CS0246: The type or namespace name 'GreetingsSender' could not be found. That is the using GreetingsSender; line above: the namespace belongs to AddGreeting.cs, which you write in step 5. The other two projects still build.
Step 5: Write and Deposit in One Transaction
Rung 2's sender called Post straight from Main. A transaction needs somewhere to live, and a handler is where Brighter puts one — so the work moves behind a command.
Put this in GreetingsSender/AddGreeting.cs:
And this in GreetingsSender/AddGreetingHandlerAsync.cs:
Read the middle of that handler as three numbered acts, because that is the entire argument of this rung:
Your write, to your table, on a connection and transaction you asked the provider for rather than opening yourself.
Brighter's write, to the Outbox table, on that same transaction.
DepositPostAsynconly stores the message; nothing has gone to RabbitMQ.Commit — and both land, or the
catchrolls back and neither does.
There is no window in which the greeting exists and the message does not, or the other way round. That is not achieved by retrying, or by being careful; it is achieved by there being only one write as far as the database is concerned.
The absent call the comment flags —
ClearOutboxAsync— is what most production code does. Called after the commit, it dispatches on the spot and the ten-second wait below never happens; the transactional guarantee is unaffected either way, because the row is already committed by then. This page leaves it out so that the Outbox is visible as a thing with contents rather than a formality. See Transactional Messaging with the Outbox for the version you would ship.
Expected result: all three projects build again, 0 Error(s) — step 4's CS0246 is gone now that the namespace it named exists.
Step 6: Run It
Three terminals this time: one for each app, and a third for the database — the sender no longer exits, because it is hosting the Sweeper. The receiver first, as before; it is still the process that declares the queue and the binding:
Expected result — the sender, which commits at once and then stays up:
And the receiver, about ten seconds later:
Both listings are trimmed; identifiers and timings differ on every run.
Found 0 to clear and then Found 1 to clear are the Sweeper's ticks: it woke, looked, found nothing old enough, and on a later pass found your message and sent it. Those two lines are the only place the delay is visible in the log.
Ten seconds is a long time for a message. It is also the only thing you gave up. While you wait, look at the two tables:
dispatched is null. That single column is the durable Outbox doing its job: the message is committed, it is not yet sent, and nothing about it depends on this process staying alive. Run the same query a few seconds later and it carries a timestamp.
You do not have to time it by hand — the row records both moments:
TimerInterval and MinimumMessageAge are five seconds each, so a message waits out its minimum age and is then caught by the next tick: about ten seconds, give or take where in the cycle it arrived.
Step 7: Make It Fail
This is the step the page exists for. Stop the sender and run it again with --fail:
The handler writes the greeting, deposits the message, and throws before the commit. The greeting it writes says something different on purpose, so you can look for its absence rather than counting rows:
Save request is in that output, and the message was still not saved. DepositPostAsync really did write the Outbox row — inside a transaction that then rolled back. The log records what your code asked for; the table records what survived, and only one of the two is the authority.
Now query both tables again:
Neither table gained a row. This greeting will not survive is in neither, and the receiver stayed silent — no message was ever sent, because no message was ever committed. The two writes were never two writes; they were one transaction.
Greeting.Idskips a number after a failed run. Postgres allocates from the sequence before the rollback and does not hand it back, so the ids go 1, 3. Nothing was lost — ids are not a count.
Stop both containers when you are done. The -v discards the Postgres volume, so the next run starts from an empty database:
What the Durable Outbox Showed You
Your handler gained a transaction and one extra write. The system gained a guarantee:
The Outbox turns two systems into one transaction. Your data and the message announcing it are written to the same database on the same connection, so the database's own atomicity is what keeps them agreeing. No distributed transaction, no two-phase commit, no compensating logic.
The Sweeper decouples sending from committing. Once the row is in the Outbox the message will go out — on the next tick, or after a restart, or when the broker comes back. The sending process no longer has to survive for the message to be delivered.
What you get is at-least-once, not exactly-once. You watched
dispatchedstay null until after the message went to RabbitMQ — so a process that dies in that gap leaves a row the next sweep will send again. Combined with rung 2's redelivery-on-failure, that is two reasons your handlers should tolerate seeing the same message twice.Brighter owns one table and you own the other. Provisioning creates the Outbox;
Greetingwas yours to create, and your schema stays yours.
The next rung changes the transport rather than the guarantee: Streaming with Kafka, where messages are partitioned, consumers form a group, and ordering is something you get per key rather than per topic.
Further Reading
Your First Message Over a Broker — rung 2, if you skipped it
Box Provisioning — this page took Option A silently; if your database will not grant
CREATE TABLE, read the other oneUsing the PostgreSQL Outbox — the DDL, the configuration and the Entity Framework Core variant, in full
Brighter Outbox Support — every Outbox Brighter ships, the archiver, and how the Sweeper is configured beyond these two options
Outbox Pattern Support — why the pattern exists, without any configuration in the way
Transactional Messaging with the Outbox — the same idea taken to production, with an Inbox on the consuming side
Glossary — every term this page linked, and the rest
Last updated
Was this helpful?
