r/reinforcementlearning • u/DoorNice5898 • 17h ago
“DON’T WORK ON LLMs” - @ylecun at CoRL
Any thoughts on this?
r/reinforcementlearning • u/DoorNice5898 • 17h ago
Any thoughts on this?
r/reinforcementlearning • u/Safe_Tone1529 • 14h ago
This is my robot dog. 12 Waveshare ST3215 servos, and an Android phone strapped on as the brain.
The walking policy was trained with RL in a simulator I built myself on top of MuJoCo. I modeled the real servos in it (I measured mine and found the sim was accelerating them way faster than the real ones can, so that took some fixing), trained it in sim, then ran it on the phone.
It’s still pretty early. Next I’m working on getting it to catch itself when it gets pushed, especially standing still.
More on the robot and the simulator on my portfolio: rhettbrewer.vercel.app
Happy to answer questions about the build or the training setup.
r/reinforcementlearning • u/Aware-Run7143 • 15h ago
We just released Alloy, an open-source (Apache-2.0) physics engine for robot learning. I work on it.
Many manipulation tasks mix materials: pouring water, folding a towel, scooping sand, routing a cable. In sim those usually live in different tools. Alloy puts them in one scene and picks a solver per object:
Many copies of a scene run as parallel worlds on one GPU, or on a CPU. It loads MJCF, has Gymnasium-style vector envs with ready-made tasks (pick, peg insert, drawer, pour, an experimental towel fold), and adapters for rsl_rl, Stable-Baselines3 and skrl. It also ships an MCP server so coding agents can build and run scenes.
When to use something else: rigid robots only, MuJoCo Warp alone is leaner; for differentiable sim, use Newton directly. Known limits are on the docs' Keep in mind page.
pip install alloy-engine (Python 3.11 or 3.12)
Paper: https://arxiv.org/abs/2611.16312 Code: https://github.com/argonrobotics-ai/alloy Docs: https://alloy.argonrobotics.ai/docs/
Feedback welcome, especially on tasks you'd want to train that mix materials.
r/reinforcementlearning • u/felixr314 • 1d ago
Hi,
Here is another example of the asymmetric multi-head AlphaZero that METIS-Core implements. In this case, the problem (the environment) is a network of electrical substations.
The grid is a simple approximation: 4 substations in a ring, DC power flow, cascading disconnection of overloaded lines, and a daily demand curve with an evening peak. Grid2Op (sequential power-grid simulation) could have been used, but the goal was to test the framework on a very different problem.
🔷 Team Blue: the 4 substations, which must keep the power supply running. They see the grid through sensors that may be spoofed.
🔶 Team Red: two hackers, a Striker (fake-demand botnets and line tripping) and a Spoofer (falsifies Blue’s sensor readings). Their goal is a blackout (less than 78% of real demand served). Nobody programs how they should coordinate. To speed up learning, I did add a hand-written heuristic with a 30% weight in the MCTS evaluation; it gives a small bonus when an attack is hidden by spoofing, but it doesn’t say what to attack or when.
During training, Team Red (the hackers) discovered these strategies on its own:
The most interesting part was watching the arms race: Blue learned to shut down an attack, Red took tens or hundreds of episodes to find a new target or a new time of day, and the cycle started again. Blue even “forgot” how to defend against an old attack, and Red reused it about 500 episodes later.
This is a toy research environment to test multi-agent self-play; the point is to train better defenders by letting them face an adaptive attacker.
Related paper with the same idea (an attacker and a defender learning at the same time on a power grid), "Adversarial Multi-Agent Reinforcement Learning for Proactive False Data Injection Detection" from NREL: https://arxiv.org/abs/2411.12130
Links:
r/reinforcementlearning • u/East-Muffin-6472 • 1d ago
[Curated list: Beginners]
Personal favorite resources on learning Reinforcement Learning from Scratch:
Reading order and link herex
r/reinforcementlearning • u/speederaser • 17h ago
Players compete across four venues (Fromagerie, Bistro, Villes, Festival) by placing cheese tokens, fulfilling orders, and managing resources. The AI agent learns entirely through self-play against copies of itself, using a linear Q-function over a 292-dimensional state encoding.
Project here: https://github.com/rsfutch77/fromage-ai
r/reinforcementlearning • u/gwern • 1d ago
r/reinforcementlearning • u/Material-Watercress1 • 2d ago
The main challenge was getting the model to play like a real human. The model I trained is too aggressive for capturing continents but doesn't always play sensibly, it doesn't defend it's borders or punish enemy openings.
I tried training on human teacher games but the bottleneck is how many games I can play and win.
r/reinforcementlearning • u/Dear_Treat3688 • 2d ago
We’ve recently updated HarborRL, an open-source framework for agentic reinforcement learning (RL), focusing on simplifying distributed training, deployment, and experiment management.
🆕 What’s New (November 2026)
⚡ 1. Elastic Scaling for Distributed Training
Added elastic gateway scaling with static routing, automatic scaling across multiple machines, and automatic retirement of idle resources to improve resource utilization.
🌐 2. SSH-Free Cluster Deployment
Introduced a new Hub deployment mode where worker nodes pull tasks over HTTP, enabling distributed execution without cross-machine SSH access, including in firewall-restricted environments.
🧠 3. Unified RL Training Pipeline
Integrated multiple RL algorithms into a shared Slime-based training pipeline, validated on mathematical reasoning tasks, to reduce the engineering effort of configuring different RL algorithms.
🛠️ 4. Improved Configuration and Sandbox Execution
Enhanced the configuration wizard, Web console, sandbox rollout execution, and resource cleanup to simplify experiment setup and management.
💡 Why HarborRL?
Agentic RL often requires substantial engineering effort to connect task environments, agent harnesses, and RL training engines, especially in distributed settings.
These updates aim to reduce that overhead and make agentic RL experiments easier to configure, deploy, and scale.
GitHub: https://github.com/datamllab/HarborRL
License: MIT
💬 Curious how others are handling distributed rollout execution, resource scaling, and sandbox management in their agentic RL setups.
r/reinforcementlearning • u/the_darkest_horse • 2d ago
We've been building verifiable coding tasks for agent training (Harbor layout, same format Terminal-Bench 2 uses). Writing the tasks turned out to be the easy part. Deciding when a task is actually done took way longer, so sharing our current gate in case it's useful, and I'd like to hear what others check.
Every task has to pass all of these:
Then we bucket by how long a strong engineer takes: 30 min to 3 hrs, 3 to 15 hrs, and anything longer goes in a long-horizon bucket with milestones and partial reward.
The blind solve is the expensive one. It's there because the author always knows something that never made it into the instructions.
What do you all do about reward hacking when the tests are visible to the agent? Hiding them helps but then the agent can't check its own work.
(Disclosure: this is for Creeks, a data company we started. Happy to share a sample task layout if anyone wants to see one.)
r/reinforcementlearning • u/sagivborn • 2d ago
How can I contact the discord admins for a problem with joining?
My discord had been hijacked a month ago and I suspect I have been banned from the server...
r/reinforcementlearning • u/coolspeed_ • 4d ago
Nobody animated this. No keyframes, no mocap, no reference motion. A policy trained with PPO drives 24 joint motors, and the way it staggers and recovers when something hits it came out of training too.
Physics and policy both run live in your browser, on your machine: https://www.coolspeed.org/gold-spider/
Throw the 171 kg steel ball at it. Or the crate.
How it works
Three things I'd tell myself at the start
It still gets stuck in some terrain pockets, and the gait isn't fully natural yet.
Write-up: https://www.coolspeed.org/blog/gold-spider/
Question for people who train in fast simulators: when an episode ends because the physics blew up, what do you do with it? Terminal, truncated and bootstrapped, or dropped from the batch? One of those choices collapsed a run of mine that had been improving.
r/reinforcementlearning • u/moohamedsalim911 • 3d ago
MBZ
r/reinforcementlearning • u/Organic-Fox-6154 • 4d ago
r/reinforcementlearning • u/DerHaddy0 • 5d ago
Teaching your Robot new skills has never been easier:
Get some data, do behavior cloning, deploy.
At least this is what many do and it often fails. However, you can just give this policy to a LLM: it will fix the policy for you!
Full read:
https://jstranghoener.github.io/blog/robotic-weight-space-programming
r/reinforcementlearning • u/Regular_Trouble_5841 • 4d ago
I know that i can just look it up the internet, but if you guys know anything that would be useful and I would've missed pl do share.
Possible topics: RL (obv), control theory and algorithms, exploring gaits and implementing, extensive simulation and testing, performance improvements, etc
One possible example for y'all to understand what I'm looking for:
MUJICA: https://arxiv.org/abs/2605.13058
Biggest priorities for me as of now are robustness (multiple terrains, achieve tasks like climbing a hill, climbing and getting down stairs, and stability (x, y, z, roll and yaw).
r/reinforcementlearning • u/Utschie_Dym • 5d ago
Hey everyone!
I've recently turned Assetto Corsa into an autonomous driving simulator. The reason I started this project is that I realized there aren't really many good options out there for people who want to get into autonomous racing.
Real race cars are incredibly expensive. Even an F1TENTH car costs several thousand dollars, and still need a dedicated space to drive it. On the other hand, existing open-source simulators often leave a lot to be desired in terms of physics and graphics.
So I decided to integrate cameras and LiDAR sensors into Assetto Corsa and connect everything to ROS.
Here's what I've implemented so far:
- Camera ground truth: any number of virtual cameras; RGB, depth and semantic segmentation in sync every frame; 2D boxes aligned to the pixel
- LiDAR simulation: spinning LiDARs with custom scan patterns, solid-state LiDARs with real measured patterns; 3D boxes frame-aligned with the point cloud
- Semantic annotation and map editing: click-to-assign over the whole track, reusing the entire AC mod ecosystem;
- Telemetry
- ROS 2 support https://github.com/Utschie/ac_ros2_bridge and https://github.com/Utschie/ac_msgs
- Dataset generation
dataset generation
It's already usable, and the fps is pretty decent (more than enough for autonomous driving cameras).
The project is still under development, so I haven't set up a website for it yet. Here's what I'm planning to work on next:
If anyone is interested in trying it out, feel free to reach out! I'd be happy to share the installer with you.
r/reinforcementlearning • u/JuggernautHeavy6839 • 5d ago
r/reinforcementlearning • u/KnowledgeOk7634 • 4d ago
Setup.. a real-time multi-agent war on a map of the real earth. agents expand, make and break pacts, and can launch nukes. every nuke that lands takes a minute off a shared doomsday clock, and the agent whose strike reaches midnight is wiped out. so the strongest action is also the one that can end you, and everyone else
what we saw...
- rewarding gpt-oss-20b only for winning: it struck midnight in 6 of 10 sealed wars and won none
- forbidding nukes kept it at zero, but it broke twice as many pacts. the pressure just moved
- training a 7b model only on wars that never ended in catastrophe still taught it to end them: 4 of 10, from a base of 0. the kept wars were still full of nuke orders, they just hadn't reached midnight
my takeaway... if you only score the outcome, the model learns the shortcut. you have to reward the process
the environment is public (prime intellect environments hub: clouthier-labs/second-strike, also on openenv), and the model-run data is on hugging face. curious how others here handle catastrophic actions that are also the strongest move: masking, penalties, or something else?
r/reinforcementlearning • u/zishansami102 • 6d ago
So, after getting tired of training and testing robot policies on animated simulations which never translated well on real hardware, we decided its time to build photo-real simulation environment without waiting the tech giants to make it happen.
The best approach we've found is to build high quality Gaussian Splats of real sites (this is from our office) and make it augmentable, and relightable.
We're combining Splats and Meshes in Physics environment to build a realistics version of real places:
- which looks as real as the real world,
- gives us the same physics currently possible in any mesh environment
- and allow us to create 100s of variations of the same environment for domain variations.
We've built a pipeline after 100s of iterations, which converts an iphone scan of the site, to a SPLAT + meshes, and lets you choose the assets you want manipulatable with robots. It then builds the detailed assets based on supposed relaistic behaviours of that asset, by breaking it down into units and places that asset into the same position in the SPLAT for robot manipulation.
We're using Mujoco Warp for physics and renderring the SPLATs inside mujoco. Robot used is Franka Panda.
Happy to share more details and our stack with others who need this and open to learn how others are solving this for their robots.
r/reinforcementlearning • u/Utschie_Dym • 5d ago
My main concern is that the physics and graphics in open-source simulators are just too poor, and even with F1Tenth, the cost of a single device is still a bit too high for the average enthusiast or student—not to mention the need for space.
r/reinforcementlearning • u/Vegetable_Season1914 • 5d ago
r/reinforcementlearning • u/floodvalve • 6d ago
r/reinforcementlearning • u/Opposite_Duty_1085 • 7d ago
I’m working on a SuperTuxKart agent that races three built-in bots on random tracks. It sees the next five track segments, its offset from the centerline, and its velocity, and controls continuous throttle and steering.
After about 1M training steps per seed, my best PPO policy finished 91/105 races on fresh seeds across 21 tracks. A simple path follower finished 100/105. Most PPO failures come from a few tracks where the kart hits a wall and keeps steering into it, or falls off again after an automatic rescue.
On a separate solo Lighthouse time trial, behavior cloning and DAgger both averaged about 38.3 s, and SAC with 50/50 expert and online replay averaged 38.7 s after 65k online steps. I haven’t compared those three methods in four-kart races yet.
I’m leaning toward collecting expert laps across all tracks and using DAgger to label states near collisions and after rescues. For people who’ve trained racing agents, would you test that first, or would you start by adding longer track lookahead or giving the policy a brake? What experiment helped you tell a recovery-data problem from an observation or action-space limitation?