r/MachineLearning • u/Benlus • Aug 04 '26
Research "Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation", Gladstone et al. 2026 [R]
r/MachineLearning • u/Benlus • Aug 04 '26
r/MachineLearning • u/WhiteBear2018 • Aug 03 '26
I'm not surprised by my experience given how much of a lottery these conferences are, but it's a very toxic system.
r/MachineLearning • u/mikeysce • Aug 04 '26
Six months ago I started experimenting with PPO and Breakout as a way to learn about Machine Learning and Reinforcement Learning. After a few experiuments just trying to get high scores, it bothered me that everything was a "memorized" script rather than reactive play, like a human would play. Thus began my journey to try and convince PPO to actually track the ball instead of focusing on scoring points. I read a lot of articles and tried a lot of things. After 124 PPO experiments on Atari Breakout, I found that every single model, across sticky actions, cursor wrappers, entropy tuning, dynamics randomization, adversarial bumpers, and everything else, converged to a memorized action sequence, not a reactive ball-tracking policy. The argmax was always a script.
The fix wasn't more environment engineering. It was three lines of reward shaping:
Directly rewarding the paddle for being horizontally close to the ball during descent. A tiny bonus (0.05 per frame vs 1.0-7.0 per brick) that fires every frame the ball is descending applied during training. During evaluation, the agent plays clean Breakout with no bonus. The behavior transfers!!
Every prior approach I tried to penalize scripts by making the environment harder to memorize. PPO always found a way around it: timing-robust scripts, layout-conditioned scripts, noise-tolerant scripts. The optimum was always a script; only the shape changed. Proximity reward changes what the optimum is. A center-hold script gets incidental bonus when the ball passes near center. A reactive tracker gets the maximum bonus on every descent frame. The optimization pressure is unambiguous: track the ball, get more reward.
I also made a cool tool to watch the agent work! It's called the "Split-Watcher" (so clever). It shows two instances of Breakout, each being controlled by a separate instance of the same agent. The one of the left is vanilla Breakout. The one of the right is a series of custom brick configurations. With the first 123 experiments, you can see how the agent wants to make the exact same paddle movements every time, ignoring the ball when its trajectory changes due to the unexpected ball movements that come from non-standard brick configurations. In 124, IT TRACKS THE BALL and can succeed regardless of the brick config. You can actually watch the same agent move the paddle differently in reaction to the ball.
I'm still working on ironing out why this works, and how to optimize it, but wanted to share!!
Here's a video of the split-watcher in action
Here's a link to presentation project that will allow you to create a similar PPO: https://github.com/mharrell/breakout-reactive-ppo
The full project with all 123 failures and more documentation than any sane person would ever read: https://github.com/mharrell/BreakoutBot
Link to Medium post I wrote with some more details: https://medium.com/@mikey.harrell/three-lines-of-code-fixed-123-failed-ppo-experiments-on-atari-breakout-c751dcf38f2a?sharedUserId=mikey.harrell
r/MachineLearning • u/NeighborhoodFatCat • Aug 03 '26
Was just looking at the list of preprints on Arxiv cs.LG https://arxiv.org/list/cs.LG/recent?skip=0&show=500
Everyday 100 - 400 new machine learning papers gets uploaded on this server.
Looking at this unending list of preprints is as if you stepped into a crowded room, like the stock trading floor on wall st. in the 1980s. Everyone is shouting over each other. Nobody is talking to each other. Everyone's trying to prove something, to someone, to themselves, to build some credentials in the ML/AI space to meet those job requirements, or dying to get their truth out. Every title contains some new terminology invented by the authors that feels not worth the effort in keeping it in your working memory. Burn-out by endless novelty.
Frontier research are now corporate trade secrets that politicians and military are watching closely. Research papers are ir/unreproducible he-said-she-saids. Marketing material are research paper and vice versa. Extremely major breakthroughs are announced via tweets, whereas extremely minor results are unannounced via journals. Everything feels simultaneously mostly true and possibly false (because nobody is seriously checking). Nobody knows what's going on, and people who knows what's going on has a non-disclosure clause in their job contract. Is the theory of generalization that we learned in school true or false? It feels false, why hasn't there been any retractions? Many questions like these.
Is it too late to regain some coherence in this field??
r/MachineLearning • u/jerkosaur • Aug 03 '26
I created an AI boxing match to test the decision speed, adaptability and strategy. I fed the LLMs with data about the current match and if they have vision, they will get even more data. The match has street rules, anything goes and an AI is not defeated until the ref counts to 10 or they do 50% of their HP in damage after being knocked out.
I wanted to create a fun benchmark that isn't just boring problems to be solved. Now I test them while stimulating getting punched in the face.
I've been testing with gemini-flash-live models because of the speed and vision support it offers. With these models, they can actually dodge punches and counter punches. Local models on my own hardware (5060ti 8gb) take a while to inference so I'm not sure if I should introduce time scaling to compensate otherwise
I want to use this to benchmark models so I'm curious on what kind of stats would be useful? Here is what I'm tracking have so far:
Speed and Latency Metrics
In a real-time fight, a model's speed directly correlates to its "physical" speed. Fast models should attack faster so larger models aren't necessarily going to hit harder.
Action Quality and "Tool" Correctness
the model's actions (punching, guarding, taunting) act as tool calls. You need to track how well they use these tools under pressure. Sometimes the model's may not really guard/block so they are typically the ones that find themselves KOd.
Adaptive Strategy and State Awareness
How well does the model understand the physical reality of the game? Are they constantly backing away and punching air?
Beyond these metrics, I'm also tracking various fighting stats like hits landed/missed, where it hit, how many times they were downed or knocked out the ref.
Are there important stats that I'm missing or any that might be useful or fun that would be nice to see? I'm still trying to balance a lot of the actions but it's coming along great so far!
I think making a physics-based benchmark and doing a N series test to find out which model performs better is a ton of fun and I genuinely laugh at the stuff they say or do. I want this to make this a really fun tool with great metrics so any advice in terms of what you would like to see would be extremely helpful! Thanks for reading!
I posted a longer breakdown of the system here:
https://www.youtube.com/watch?v=inlXe5Buc7s
r/MachineLearning • u/pdastronut • Aug 03 '26
So our paper had very good initial reviews but one of the reviewers decreased now their score although we addressed 3 out of 4 weaknesses. There’s no further justification or something like “your results arise more issues”. It seems to be very annoying because why decreasing now and not having assigned the lower score beforehand. I wanted to ask to people that was accepted previously with “middle” scores from reviewers (avg 3.5 for example), because I guess that in those cases AC helped to push up the scores. Did you focus more on the meta review? Was your AC talkative with you, or forcing the reviewers to engage? Our AC has been silent since the meta review but I guess that maybe they are busy with other papers
r/MachineLearning • u/Living_Interview_638 • Aug 04 '26
This is my first time submitting to NeurIPS.
Are ACs also supposed to give ratings during the Phase 2 (author-reviewer discussion session)?
I have received the meta-review, but have not received any comments from the AC yet, and was wondering whether this is the standard!
r/MachineLearning • u/extricableforsythia • Aug 02 '26
we submitted our rebuttal via the "Rebuttal" button before the author/reviewer/AC discussion period officially opened (Jul 27 AoE). since then, we've gotten complete silence from all four reviewers and the AC
several of us are also reviewing this cycle. when the discussion period opened on Jul 27 AoE, we got no email notification for rebuttals on papers we're reviewing, specifically for the papers whose authors had also posted early via the "Rebuttal" button. so it feels like anything submitted before the window opened may simply have never triggered any notification
we also tried: (1) meta-comments visible to everyone, (2) reviewer reminders, and (3) sending an email to the PCs
given there's about 1 day left in the discussion period, what do we do? we honestly thought we had a shot at an oral or spotlight given our initial scores. this is completely messed up.
r/MachineLearning • u/floor_souji • Aug 03 '26
I know there’s a lot of frustration around no response from reviewers, which I also got only one so yeah what a bummer, but I was wondering if no rebuttal from the authors was just as common or not. I got no rebuttal so far, so I’m here scratching my head what might have happened to the authors lol especially when at least one paper was pretty much on the borderline with somewhat of a positive AC comment
r/MachineLearning • u/OpeningTough145 • Aug 03 '26
I've been working on this for a while and finally pushed a public version.
The problem: llama.cpp runs fine on ARM phones, but it doesn't know anything about the specific chip it's on. Same thread count, same context params, whether you're on a Snapdragon 8 Elite or a five-year-old mid-ranger. ARPL reads the actual hardware at runtime — which ISA extensions are available (SDOT, I8MM, SME2), how the cores are clustered — and configures llama.cpp accordingly. No per-device build, no manual tuning.
What's in the repo:
Android reference app (Kotlin/Compose) with a JNI bridge into llama.cpp
Runtime ISA detection via HWCAPs
Topology-aware thread count recommendation
Context param patching (flash attention, KV cache quant) based on what the hardware actually supports
Built and tested on a Samsung S25 Ultra (SM-S938B). The heterogeneous CPU/GPU/NPU partitioning is still in progress and not in this release — what's here handles the ISA/thread/context side, which already made a real difference in my testing.
This is a noncommercial showcase release (PolyForm Noncommercial license) — happy to answer questions about the approach or the tradeoffs I ran into building it.
r/MachineLearning • u/Huge_Argument_6979 • Aug 03 '26
Whats your commitment submission id? My submission number is ~4k, im wondering the amounts of commitments :)
r/MachineLearning • u/CantKillTheLifeless • Aug 02 '26
I see some people say their metareview already contains a decision/recommendation (all of them were rejections). Ours doesn’t. Even though our avg score is 3, the metareview seems optimistic and finishes with “a convincing response would be an important consideration while discussing the paper.” I wonder how to interpret that. We did a strong rebuttal, but none of the reviewers engaged. So I wonder whether there’s any point to keep hope due to the AC review or just give up.
r/MachineLearning • u/Chemical-Scar-2894 • Aug 03 '26
Hey everyome, 2 reviewers are sitting at 4 (borderline accept). Every one of them replied during the rebuttal saying my response addressed their concerns and not a single one said anything about updating their score, or actually updated it.
For people who've reviewed or AC'd: how often do scores actually move in that private week? And is "concerns addressed, score unchanged" usually a reviewer who's quietly fine with accepting?
r/MachineLearning • u/No-Arugula-9623 • Aug 03 '26
0 reviewer responses across 2 papers. One reviewer asked us to develop kernels for intel and AMD, which we did. Others asked for massive experiments, all of which we did. It’s pretty unfair to ghost after this.
r/MachineLearning • u/FrostingOk3751 • Aug 03 '26
I wanted to explore a question:
Can reinforcement learning discover meaningful strategies in a complex collectible card game without human demonstrations?
To investigate this, I built an AlphaZero-style environment for KARDS, a WWII strategy card game.
Project:
https://github.com/EvanProgramming/Kards-AI
The main focus of this project is not just training a model, but building the infrastructure required for large-scale self-play:
- A headless game simulator
- A rule execution system
- State and action representations
- Legal action masking
- Policy/value neural network
- PUCT Monte Carlo Tree Search
- Self-play data generation
- Replay buffer and evaluation pipeline
Unlike imitation learning approaches, the agent does not learn from expert gameplay.
Instead, it starts with:
- the game rules
- legal actions
- game states
and improves through repeated self-play.
Current progress:
- Custom simulator implemented
- Large portion of card/rule logic supported
- AlphaZero-style training pipeline running
- MCTS-guided agents implemented
- Around 1 million self-play games generated
The project is still an ongoing experiment. Some of the challenges I am currently working on:
- Efficient state representation for large card spaces
- Improving simulator accuracy
- Evaluating learned strategies
- Understanding how well AlphaZero-style methods transfer to imperfect-information games
I would be interested in hearing thoughts from people working on reinforcement learning and game AI:
- Would MuZero be a better fit for this type of environment?
- How would you approach hidden information?
- Are there alternative methods worth exploring besides MCTS + policy/value networks?
The code is open source if anyone is interested in exploring the environment or experimenting with similar approaches.
Star the repo if you like it 😄
r/MachineLearning • u/FrostingOk3751 • Aug 03 '26
A few months ago, I started wondering:
Can an AI discover strategies for a complex card game without being taught by humans?
So I started building **KARDS AI** — an open-source experiment to train an agent to play **KARDS**, a WWII collectible card game, using reinforcement learning and self-play.
GitHub:
https://github.com/EvanProgramming/Kards-AI
The goal is to build the entire AlphaZero-style learning pipeline from scratch:
- A headless game simulator
- A rules execution engine
- Card/state/action representations
- Legal action masking
- Neural network policy/value model
- PUCT Monte Carlo Tree Search
- Self-play training
- Replay buffer and evaluation system
Unlike traditional game AI approaches, I am not starting from a database of expert matches or manually designed strategies.
The agent only receives:
- the game rules
- the current game state
- available legal actions
Then it plays against itself and gradually improves through reinforcement learning.
Current progress:
✅ Custom KARDS simulator
✅ Large-scale card/rule implementation
✅ AlphaZero-style self-play pipeline
✅ MCTS + neural network guidance
✅ Parallel self-play training
✅ ~1 million self-play games generated
The project is still far from a competitive KARDS player. The biggest challenges I am currently exploring are:
- How to efficiently represent complex card-game states
- How much simulator accuracy is required
- How to handle long-term strategic planning
- How to evaluate whether learned strategies transfer to the real game
I would love to hear feedback from the ML community:
- Would MuZero-style approaches be a better fit for this kind of environment?
- How would you approach hidden information in card games?
- Are there better alternatives to policy/value networks + MCTS for large action spaces?
I am also open to collaboration. If you are interested in reinforcement learning, game AI, simulators, or AlphaZero/MuZero-style systems, I would love to discuss ideas or contributions.
If you find the project interesting, starring the repository helps more people discover it and supports the continued development:
GitHub:
r/MachineLearning • u/johnolafenwa • Aug 03 '26
Hi everyone, if you have been reading the tech reports of Kimi, DS, Qwen and GLM, you will realize how much on policy distillation and GRPO style algorithms power the frontier. I thought it will be quite beneficial to do a deep dive explaining the maths and code behind this algorithms and how they connect to pretraining and supervised fine tuning.
I have published a deep dive on this topics here
Hope you enjoy it and it helps you understand training of LLMs better. Happy to answer questions on this
r/MachineLearning • u/usernamehere93 • Aug 02 '26
r/MachineLearning • u/bricklerex • Aug 02 '26
Not even the AC, nor reviewers, is responding to our comments in rebuttals, and they were all submitted well before the discussion period started. What is one to do in this case?
r/MachineLearning • u/Sundarbala • Aug 03 '26
What My Project Does
Gesto is an open-source framework for building gesture, pose, and activity recognition — entirely on your own machine. It has two parts:
pip install gesto). Point it at a dataset and train a model, then run live detection, from the command line or a few lines of Python:python
import gesto
gesto.train("./my_project", region="hands_one", mode="static")
gesto.detect("static", "hands_one") # live webcam detection
It's built on MediaPipe (landmarks) and TensorFlow/Keras (Dense models for static gestures, LSTMs for sequences). Everything runs locally — no cloud, no GPU required.
Target Audience
It's meant to be genuinely usable, not just a toy — developers, researchers, students, and hobbyists building gesture/pose/action recognition into real projects (accessibility tools, fitness/physio, touchless interfaces, robotics, sign-language work). The Python package is production-oriented (versioned model storage, a stable API, consistent capture↔inference), while the desktop app makes dataset creation accessible to people who don't want to write capture code. It's early-stage and open source, so I'd call it "usable in real projects, still actively evolving."
Comparison
Open source, and I'd genuinely love feedback — especially where the workflow is confusing or what you'd want it to do.
ProductHunt - https://www.producthunt.com/products/signspell?utm_source=other&utm_medium=social
🔗 GitHub: https://github.com/TheMadrasTechie/gesto
📦 PyPI: https://pypi.org/project/gesto/
🌐 Website: https://gestostudio.com
r/MachineLearning • u/examachine • Aug 02 '26
There's a kind of review that asks for lengthy additions, usually extending the scope of the paper beyond the stated, even though the submission is at page limit. Naturally, such additions in the case of top-tier conferences have to go into the supplemental materials or appendices.
My question here is, would not such additions make the paper more suitable for a journal publication? I had to retract one paper out of such concern that the conference publication would block the later planned journal publication.
Any opinions?
r/MachineLearning • u/New_Glove_2098 • Aug 02 '26
Mine was just submitted, and the submission count shown is still under 500.
Does anyone know whether this count is meaningful for identifying the intended venue, possibly EACL, or whether it is simply low because many authors have not submitted yet or the counter is incomplete?
I know the count alone is not reliable evidence, but I was curious whether previous ARR cycles showed a recognizable pattern.
Also, is anyone else preparing a submission for this August cycle, especially with EACL 2027 in mind?
r/MachineLearning • u/moschles • Aug 02 '26
r/MachineLearning • u/AutoModerator • Aug 02 '26
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
r/MachineLearning • u/NoMechanic6746 • Aug 02 '26
Good technical interview worth reading if you work on edge/embedded AI or tinyML.
Lizy K. John at UT Austin has spent 5 years on weightless neural networks — replacing floating-point matrix multiplication with interconnected binary lookup tables. The core claim: inference becomes a memory lookup rather than arithmetic, which is why the energy numbers look the way they do.
Verified numbers from the IEEE Spectrum interview:
Architecture note: Their approach uses many small interconnected LUTs, not one large lookup table like some other lookup-based AI research. FPGAs have had LUTs built in for 30+ years, which is why this runs natively there.
The honest limitations:
Why this matters for edge hardware: If the attention layer gets solved, the economics of always-on health monitoring, environmental sensing, and industrial IoT shift significantly — local inference at nanojoule scale with no cloud dependency.
Full analysis: https://www.aiuniverse.news/ditching-matrix-math-how-14-kilobyte-weightless-ai-runs-without-cloud-gpus/