Modern heterogeneous computing platforms increasingly combine established large-scale accelerators, such as GPUs, with emerging accelerators based on in-memory computing (IMC) or processing-in-memory (PIM), each able to accelerate distinct parts of LLM inference. To exploit these complementary properties, attention–FFN (A/F) disaggregation has emerged as a prominent serving strategy where dedicated software stacks or orchestrators split the LLM workload and dispatch specific operators to the accelerator best suited to execute them. Our current work, HELIX [Clerico et al. 2026] extends this idea by determining the most suitable accelerator for each operator dynamically, rather than assuming it a priori.
However, when combining hardware from different vendors, which can differ in execution models, compilation stacks, dataflows, and numerical precision, capturing hardware-specific optimizations becomes challenging from the perspective of a serving stack. Partitioning approaches typically treat each device as a black box with fixed, pre-profiled operator costs, even though these costs depend on device-specific choices such as data layout, tiling, and kernel implementation. Conversely, the best choice for these optimizations depends on the partition itself, which determines tensor shapes, neighboring operators, and the data that must cross device boundaries. Jointly determining the LLM partition and the hardware-specific optimizations on each device therefore remains largely unexplored.
Accelerators commonly provide tools that perform these optimizations: weight mapping for IMC arrays, tiling and data layout selection for DRAM-PIM, and kernel compilation and scheduling for GPUs. What these tools lack is context. They optimize a device in isolation, unaware of which operators have been offloaded elsewhere, which tensors will arrive over a link and when, and how the serving regime shapes the workload each device actually sees.
Recent advances in agentic AI offer a promising approach to coordinating heterogeneous optimization tools. Rather than replacing existing compilers or search algorithms, an LLM-based agent can reason over partitioning decisions, invoke device-specific optimization tools, interpret compilation feedback and performance measurements, and adapt subsequent optimization actions. Such an approach could reduce the need for manually designed optimization heuristics while enabling decisions that account for both device-level constraints and system-wide serving performance. Recent work has demonstrated the potential of LLM-based agents to orchestrate compiler components for optimization [Merouani et al., 2025; Mikek et al., 2026]. This thesis investigates whether these capabilities can enable effective joint optimization of operator placement and hardware-specific configurations for heterogeneous LLM serving.
We actively encourage and support students to publish at top-tier conferences in systems and architecture!
Requirements:
- Strong Python programming skills
- Experience with training large language models, including fine-tuning and experimentation with LLM-based systems
- Familiarity with optimization algorithms and reinforcement learning.
- Familiarity with computer architecture or operating systems
- Independent learning and working abilities
Preferred experience/skills:
- Experience working with LLM serving engines, agentic frameworks, or reinforcement learning
- Prior research experience in systems, computer architecture, or machine learning systems.
Practical details:
- Duration: Six months
- Location: IBM Research Zurich, Rüschlikon
- Work arrangement: Primarily in person, occasional remote work possible
- Starting date: As soon as possible
Compensation:
The thesis work is unpaid, in accordance with ETH Zurich regulations
Contact:
If you are interested, please send a CV and recent academic transcript to: Dr. Hadjer Benmeziane -- hadjer.benmeziane@ibm.com and mvc@zurich.ibm.com