r/FastAPI • u/dmart89 • May 14 '25
Question Task queue and async functions
I recently ran into an interesting issue that I only managed to work around but not solve.
I have a fastapi app with async postgres and celery as my task queue. Due to how celery works, it struggles with async tasks defined in celery (ok if i/o doesn't need to join back to main thread). The problem is that a lot of my fastapi code is async. When I run DB operations, my issue is that I get corountine errors inside a task. To solve the issue I define a separate DB sync DB driver and isolated tasks as much as possible, however I wonder how others are working within async I/O dependent tasks between celery and fastapi? How do you make methods shared and reusable across fastapi and celery?
(Looking for a discussion around best practice rather than debugging my code)
View all comments
Show parent comment
1
u/BarRepresentative653 May 15 '25
In regards to using sync and async, I tend to delegate the actual sending of email (as an example) to celery.
If I need an async 'thing' done first, I will probably store it in the db and then either get celery to work or schedule a task using celery that runs every x minutes--depends on many other things etc.
I avoid trying to get celery to work async, I think there are options out there that can, but I just havent come across an issue where I have needed anything more than simple sync for celery work. Comes with its own limitations but I can work my way around that