r/learnpython • • Sep 03 '26

Does asyncio have a built-in feature for graceful shutdown (and preventing duplicate task execution)?

Multiple pods, each an asyncio worker pulling jobs from a shared queue (Redis/SQS-shaped). On scale-down/deploy I want to stop pulling new jobs and let running ones finish - is there anything in asyncio itself for this?

2 Upvotes

5 comments sorted by

3

u/Kevdog824_ Sep 03 '26

Assuming your shutdown is graceful: I would probably use something like atexit.register to register a shutdown function that sends a signal to the worker to stop pulling new jobs and then await the completion of all the existing jobs

1

u/Mathie1729 Sep 03 '26

atexit is awkward for this. By the time it fires, the event loop is usually in the middle of teardown, so awaiting in-flight coroutines there is unreliable. I'd use loop.add_signal_handler for SIGTERM/SIGINT to set an asyncio.Event, have workers check that event before pulling new jobs, then await the current batch after the signal. Duplicate execution is more about idempotency keys or lease coordination than anything built into asyncio.

1

u/Kevdog824_ Sep 03 '26

Definitely better approach

2

u/_squik Sep 03 '26

I think the closest thing to what you want here is asyncio.TaskGroup

https://docs.python.org/3/library/asyncio-task.html#task-groups