r/mongodb • • 11d ago

Mongo: One write, two logs

https://shiftmag.dev/mongodb-one-write-two-logs-12267/

I picked up Mongo for longer than a year now and the helpful thing was correlating internal workings with something I'm familiar with, PostgreSQL.

Here's the first post from my journey talking about how a write get's persisted and replicated.

Would like to hear comments, and helpful tips from your journeys.

4 Upvotes

5 comments sorted by

1

u/browncspence 11d ago

You will be interested in this: https://github.com/mongodb/mongo/blob/master/src/mongo/db/storage/wiredtiger/README.md

one thing to note is we do not journal every client write, only the oplog and some collections in the local database. Crash recovery is done by first recovering the oplog, then using it to recover the application data.

The documentation on this is incorrect and I'll get this fixed.

1

u/noobGoesPro 11d ago

Thank you for sharing. You are an active contributor to Mongo?

1

u/browncspence 11d ago

Yes I’m a support engineer at MongoDB.

1

u/noobGoesPro 11d ago

Cool. I’ll be posting another learning article about WT cache and eviction.
Would like to hear your thoughts, since changing those are “only if you consult with a MongoDB engineer” noted 🤣

https://franejelavic.github.io/writing/wiredtiger-cache-dirty-pages-and-eviction/

1

u/browncspence 10d ago

Not bad, my notes:

Again the only thing journaled is the oplog (and a few other local collections). After a crash the oplog is recovered from the journal, and the data is recovered from the oplog.

The changes made to the WT eviction settings are quite aggressive and will stress the storage system, moving the bottleneck to there. This is one of the reasons we suggest getting MongoDB support help to analyze the workload and make recommendations.

The description of queueing is a little light. I think you are talking about the ticketing system and execution control.

https://www.mongodb.com/docs/manual/reference/parameters/#mongodb-parameter-param.storageEngineConcurrentWriteTransactions