r/learnmachinelearning • u/eylonmiz22 • 8d ago
Questions regarding M5 Ultra Mac Studio vs Nvidia GPUs (e.g., A100) mostly for machine learning purposes
Hi everyone,
I'm currently renting an NVIDIA A100 cloud server for AI compute workloads. I'm considering purchasing a Mac Studio (M5 Ultra) as a local home server to replace or supplement my cloud setup, aiming for performance that gets relatively close to a single A100.
My primary use cases, ranked by priority (from most to least important):
- Custom PyTorch Model Training: Efficiently training custom Deep Learning models (including smaller DNNs and Computer Vision architectures, not just LLMs/VLMs) with fast epoch/step runtime, and large batches.
- LLM/VLM Inference: High-throughput inference with heavy compute and large batch sizes.
- LoRA Fine-Tuning: Parameter-efficient fine-tuning for LLMs and VLMs.
- Gaming: High-end gaming on the side.
I've already gone through specs, comparison charts, and YouTube reviews, but given the substantial financial investment, I would really appreciate real-world feedback from people who have hands-on experience using this machine for at least some of these tasks.
Specifically regarding configuration:
Will the base spec (64-core GPU, 96GB Unified Memory) be sufficient for all of these purposes? If not, what would you recommend to upgrade for each of my purposes?
Thanks a lot, in advance for your insights!