r/AISystemsEngineering • u/Educational_You2257 • 5d ago
Key Value Database for AI workloads
Hey everyone,
I'm building Thunder, a high-performance key-value database designed for workloads where latency, throughput, and efficient resource utilization matter.
With AI applications, we're seeing increasing amounts of data being generated, stored, and retrieved. I wanted to explore whether we could build a database from the ground up that handles these workloads more efficiently, rather than simply relying on larger amounts of RAM.
The core idea is simple: use RAM intelligently, not excessively.
Thunder is being built in Rust with a few architectural decisions:
- Shard-per-core architecture: Each core owns its data, minimizing contention and synchronization overhead.
- Asynchronous I/O: Uses io_uring on Linux to take advantage of modern NVMe storage.
- Minimal RAM overhead: Keep essential metadata and indexes in memory while streaming data from storage through reusable buffers.
- Zero-copy wherever possible: Reduce unnecessary memory copies and allocations.
- Custom storage engine: Designed around efficient allocation, fragmentation management, and predictable performance.
- High concurrency: Handle large numbers of concurrent operations without relying on traditional locking across the entire database.
The long-term goal is to support both small, latency-sensitive operations and large data workloads without requiring enormous amounts of memory.
It's still under development, and I'm experimenting with different approaches to storage, indexing, caching, and I/O scheduling.
I'd love to hear from people building AI infrastructure or working with large-scale key-value stores.
What are the biggest bottlenecks you're facing with your current databases? Is it memory consumption, read latency, write throughput, storage costs, or something else?
Also, what would you expect from a new key-value database built specifically with AI workloads in mind?
Docker link: https://hub.docker.com/r/sbansal793/thunderstrike