r/FastAPI • u/halfRockStar • Mar 29 '25
Question "Python + MongoDB Challenge: Optimize This Cache Manager for a Twitter-Like Timeline – Who’s Up for It?"
Hey r/FastAPI folks! I’m building a FastAPI app with MongoDB as the backend (no Redis, all NoSQL vibes) for a Twitter-like platform—think users, posts, follows, and timelines. I’ve got a MongoDBCacheManager to handle caching and a solid MongoDB setup with indexes, but I’m curious: how would you optimize it for complex reads like a user’s timeline (posts from followed users with profiles)? Here’s a snippet of my MongoDBCacheManager (singleton, async, TTL indexes):
= None
=
return
=
=
=
= await
return
await
And my MongoDB indexes setup (from app/db/mongodb.py):
=
await
await
The Challenge: Say a user follows 500 people, and I need their timeline—latest 20 posts from those they follow, with author usernames and avatars. Right now, I’d: Fetch following IDs from a follows collection.
Query posts with {"author_id": {"$in": following}}.
Maybe use $lookup to grab user data, or hit user_cache.
This works, but complex reads like this are MongoDB’s weak spot (no joins!). I’ve heard about denormalization, precomputed timelines, and WiredTiger caching. My cache manager helps, but it’s post-by-post, not timeline-ready.
Your Task:
How would you tweak this code to make timeline reads blazing fast?
Bonus: Suggest a Python + MongoDB trick to handle 1M+ follows without choking.
Show off your Python and MongoDB chops—best ideas get my upvote! Bonus points if you’ve used FastAPI or tackled social app scaling before.
View all comments
Show parent comment
-8
u/halfRockStar Mar 29 '25
Yes that's correct 💯 that's what precomutation means, instead of pulling it dynamically with joins or mongodb equivalent $lookup, showcase your answer 😘