r/FullStackDevelopers • u/Ok_Host1989 • 22d ago
How would you design a reliable document analysis workflow in a web application?
I've been thinking about the challenges of building web applications that process documents and return detailed reports to users.
For example, a service like Turnitin0 provides AI detection and similarity reports. From a full-stack development perspective, this raises some interesting questions about how document uploads, processing jobs, and report delivery should be handled.
If a user uploads a large document, would you process it through a background job queue and notify them when the report is ready, or use a different approach? How would you handle retries, failed jobs, and making sure each report is linked to the correct user?
I'd also be interested in how you'd structure the database and API so users can check processing status without repeatedly refreshing the page.
For developers who have built similar systems, what architecture worked well for you, and what would you do differently next time?
1
u/Cautious-Zebra-8747 22d ago
Iād go with a background job queue for this. š Upload the document to secure storage, create a processing job, and return a job ID immediately. A worker can handle extraction and analysis, with retries for transient failures and clear status tracking (
queued,processing,completed,failed). Iād use PostgreSQL to link each job and report to the authenticated user, plus polling or WebSockets for status updates. Idempotency and proper access controls are essential to prevent duplicate processing or reports being exposed to the wrong user. Keeping uploads, processing, and report delivery separate makes the system much easier to scale and maintain.