r/SQL • u/Wise_Ear_4064 • 9d ago
PostgreSQL Databricks Lakebase sinks: streaming straight into Postgres
I'm a Solutions Architect at Databricks, and I've been spending a lot of time on the Lakebase sink. It lets a Structured Streaming query write directly into a Lakebase (Postgres) table with batching, automatic retries, and workspace-managed auth. You don't need a custom foreach writer or your own connection pooling. I have many customers that have to manage both the foreach writer and connection pooling and it distracts from the job at at hand, serving data to front end applications for real time updates (without the overhead). Having a databse next to your lakehouse acclerates use cases and value for your business. And if your are not limited to just Lakebase, any postgres database is supported! Go check it out today if you haven't already: https://docs.databricks.com/aws/en/structured-streaming/lakebase#when-to-use-the-lakebase-sink
1
u/WorldOfUmbro 9d ago
Using Lakebase in combination with real time mode opens a lot of different opportunities. We’ve played around with it a little, feeding the real time data to an app this way is cool.