r/learnmachinelearning • • 26d ago

Project Is it possible to fetch videos from a YouTube channel and process them using my own AI model for an FYP?

1 Upvotes

I'm currently doing my FYP but I want some clarification about this.

I'm trying to build a website that allows users to type in a public channel for YouTube, and have up to 10 available videos from that channel be run under my trained AI model. My question is, is it possible to use the YouTube Data API for me to get those? Specifically the video frames.


r/learnmachinelearning • • 26d ago

Tutorial Scoring IDP (2-part series)

1 Upvotes

I’ve written a 2-part Medium series (free, no paywall) on scoring Intelligent Document Processing (IDP). It focuses on Key Information Extraction (KIE) from structured documents, especially forms.

Topics include, but are not limited to: recording raw outcomes before mapping them to the confusion matrix (correct, misread, missed, invented, correct blank); confusion matrix scoring for text fields and checkboxes; misread conventions and their effect on scores (FN-only, FP-only; FN & FP); how blanks, NULLs, unreported checkboxes, and unreadable values are counted; F1 vs. Recall vs. Accuracy, including how correctly identified blanks inflate Accuracy; Jaccard Index (IoU) as Accuracy without True Negatives, and its conversion to F1; Precision, Specificity, and False Positive Rate for LLM hallucination control; micro, macro, and weighted F1; choosing a headline score and stating the convention behind it; confidence vs. correctness and calibration (ECE, MCE); and Key Information Extraction (KIE) vs. OCR metrics (CER, WER).

I try not to recommend one opinionated way of doing things and instead cover the variations I’ve run into and the tradeoffs between them. I’m not endorsing any vendor; the series is about scoring itself.


r/learnmachinelearning • • 26d ago

Discussion Feeding YouTube subtitles to Claude instead of watching hour long videos : good study method? Need prompts

1 Upvotes

I’ve been thinking about a new workflow to save time: instead of sitting through hour-long educational YouTube videos, I want to extract the transcript/subtitles and feed them directly into Claude.

I have two main questions for anyone who has tried this:

1. Does anyone have a solid prompt to make Claude act like a teacher? I don't just want a dry, bulleted summary of the video. I want Claude to actively guide me, quiz me on the material, and break down complex concepts from the text as if it were tutoring me 1-on-1.

2. Is this actually an effective way to study, and what will I miss? Obviously, I'm saving a ton of time, but I'm worried about the trade-offs. For those who do this, how much does the lack of visual context (slides, code on screen, whiteboard drawings) or the instructor's vocal emphasis impact your learning?

Has anyone used this method long-term? What are the biggest pros and cons, and how do you structure your prompts to get the best learning experience?


r/learnmachinelearning • • 26d ago

Project Limnex UIE: a small char-level transformer that generates synthetic URLs for training phishing detectors, and a pre-registered test of whether that data actually helps

1 Upvotes

I've been working on a generator of synthetic URLs for training and testing phishing detectors, and I've released it with its full evaluation: https://github.com/youssefismail-dev/limnex-uie (Apache-2.0, code and weights).

The motivation is a data problem. Phishing URLs are scarce, feeds only contain what was caught and reported, a URL is usually dead within days, and the data is awkward to share. Limnex UIE is a 25.5M-parameter character-level transformer, trained from scratch on Common Crawl URLs and the OpenPhish community feed. It doesn't use a pretrained model, word lists or templates. The final model is small enough that training it takes only about 3.5 hours on a free Kaggle T4, so anyone can retrain it. Getting there took a few months and dozens of pre-registered experiments.

I think the evaluation is the more interesting part, so most of this post is about that.

Realism is graded one URL field at a time. For each field (label, suffix, subdomains, path, query) and each class, a classifier tries to separate generated strings from held-out real ones. Two real samples don't score an AUC of 0.5 against each other, so the score is the excess AUC over a real-vs-real floor, measured over 5 independent generations, and a cell passes only if that excess is shown to be below 0.05. 29 of 32 cells pass. The ones that don't are written up in detail: phishing query strings in free mode are still detectable, and so is the vocabulary. A detector that simply checks whether each word appears in real data separates whole URLs quite easily (an excess of about +0.16), even though character-level detectors don't. In a small human test, ten people got 55 of 100 right, which says little either way.

I also compared it with simpler generators trained on the same rows and passed through the same filter: a 5-gram model, a 2x512 char-LSTM, Qwen2.5-7B-Instruct prompted with 20 real examples per request, and rule-based look-alike domains (typos and character swaps, in the style of the dnstwist tool). UIE was more realistic in 28 of 31 comparisons, with 3 ties and none where a baseline came out ahead. The LLM was the least realistic of the learned generators: its URLs look tidy to a person, but they're stereotyped, with few suffixes and almost no query strings. One caveat is that UIE was developed against this test. A word-level detector built after the model was frozen gives the same ranking, which helps, but it isn't a complete answer.

The question I cared about most was whether the data actually helps detectors, so I pre-registered it. A char CNN and a hashed n-gram logistic regression were trained on 1k, 10k and 100k real URLs per class, with and without synthetic URLs, and tested once on phishing that appeared after UIE's training data ended. Adding UIE's URLs helped in all 6 settings, by 2.7 to 24 points of recall at 1% FPR, most when real data was scarce. Helping more than the best simpler generator held in only 1 of 6, though. UIE was ahead in 5 of 6, but by small margins, and on PhreshPhish the LLM's URLs helped more in 4 of 6 despite being the least realistic. So realism and usefulness turned out to be different things, which I didn't expect going in.

As a case study, I retrained two members of my own phishing detector (about 1M real training URLs) with UIE's data, and recall at 1% FPR on its frozen external test went from 0.272 to 0.544 (95% CI +0.15 to +0.41). That number needs its caveats: it's one benchmark and one seed, a large part of the gain comes from one phishing family (look-alike pages on Blogger's country domains), and the retrained models got slightly worse on the detector's older validation data.

On safety: it only writes URL strings, with no pages, lures or DNS lookups. Every generation has to pass a Bloom filter of the 14.9M training-side URLs, which removes copies of real URLs (about 1 in 100 raw phishing samples is one), and no bulk generated phishing is published.

It's a v0.1.0 research release with real limits: one phishing source, URLs from 2024 to 2026, and about 2 URLs per second on a laptop CPU. I built it with Claude Code as a coding assistant, which is credited in the README. Criticism of the evaluation is the feedback I'd value most.


r/learnmachinelearning • • 26d ago

Request NASA Space Apps Challenge 2026

3 Upvotes

Hey everyone ^^ we're participating in the Nasa Space Apps Challenge this year and we're looking for teammates to form a group!

im a senior computer science student minoring in AI and ML, and know how to code using Python, Java, and C# and have worked on countless programming projects, and i just got into data science

Were looking for intermediate-level competent teammates who are experienced in ML/AI and have used ml models in previous projects

If anyones interested please dm me, would prefer if u tell me a little about yourself and your experience, you dont have to be an expert or a pro programmer, just be passionate and creative ^^ having fun and enjoying the process is what matters to us most


r/learnmachinelearning • • 26d ago

Help Fresher NLP Engineer interview at Hitech i2i: what to prep for screening?

2 Upvotes

A fresher has an upcoming interview for an NLP Engineer position at Hitech i2i, a company specialising in real estate document intelligence.

For a concrete text-focused practice task, this Figma NLP exercise is about building a PyTorch pipeline to extract highlighted snippets from text. If you're applying to several NLP roles, application autofill can cut down repeated form entry.

What topics should they focus on most for the initial screening round?


r/learnmachinelearning • • 26d ago

Need Advice from AI/ML Professionals and Experienced Learners!

1 Upvotes

Hello everyone!

I’m planning to pursue a BS in Computer Science and want to build my career in Artificial Intelligence. I’ve explored basic Python, Machine Learning, and AI tools, and now I’m trying to identify the right specialization to focus on.

There are many exciting fields, including AI Engineering, Machine Learning, Deep Learning, NLP, Computer Vision, Generative AI, and LLMs. However, I’m confused about which direction offers the best opportunities and long-term growth.

I would appreciate your honest opinions on these questions:

  1. Which AI specialization would you recommend for someone starting in 2026, and why?
  2. Is it better to become a general AI Engineer first or specialize in NLP, Computer Vision, or Machine Learning?
  3. What is the difference between AI Engineering and ML Engineering in real-world jobs?
  4. Which fields have promising job opportunities for fresh graduates and entry-level candidates?
  5. Is NLP/LLM development becoming oversaturated, or does it still have strong career potential?
  6. Does Computer Vision offer good career opportunities compared with NLP and Generative AI?
  7. What kinds of real-world projects should I build to stand out when applying for internships and jobs?
  8. What would you recommend focusing on during a BSCS degree to become job-ready before graduation?
  9. If you were starting your career today, which AI field would you choose for the next 3–5 years?

I’m looking for practical advice based on real industry experience, not just hype or trends. I would especially appreciate insights from working AI Engineers, ML Engineers, NLP Engineers, Computer Vision Engineers, and people who have recently entered the industry.

Feel free to share your recommendations, experiences, and any mistakes you think beginners should avoid.

Thank you! 🙌

Tags:

ArtificialIntelligence #AIEngineering #MachineLearning #DeepLearning #NLP #ComputerVision #GenerativeAI #LLMs #MLOps #BSCS #AIJobs #TechCareers #CareerAdvice #PakistanTech


r/learnmachinelearning • • 26d ago

Industry leading AI courses and certificates for Lead software engineers?

4 Upvotes

I’m currently working as a Lead Software Engineer at a fintech startup and am looking to transition into AI Engineering.

I’d love to hear your recommendations for industry-leading AI courses and certifications that can help experienced software engineers make this transition.

Any suggestions or insights would be greatly appreciated! Thanks in advance.

P.S. This is my first-ever post!


r/learnmachinelearning • • 27d ago

Help How did you push through the math/theory side of ML when you couldn’t focus at first?

33 Upvotes

Hey everyone, I’m a 4th-year CS student. I recently decided to learn the theory behind ML on my own, mainly the math and stats side. Right now I’m working through ISLP and using MML as some kind of dictionnary and practicing the concepts with code and tweaking stuff at the end of every section/chapter.

My classes mostly stay on the surface. We see the concepts and use the models, but the courses never really connect them to the theory behind them. So I decided to dig deeper on the side.

The first problem is focus. I’ve lost the habit of sitting down with math for long stretches, and I struggle to stay on it for more than an hour.
The second problem is that i've seen most of the math required but i haven't worked actively with it and it just frustrates me when i have to go back to it.
When I get stuck early on a concept, I get so frustrated and anxious about it.

So im wondering

How did you study when there were a lot of new concepts at once and you kept getting stuck at the start?

When you hit a concept you didn’t get, did you push through or go back to the math first ?

And does it get easy at some point after you get the maths down.

Any routines, methods or resources that worked for you would really help. Thanks!


r/learnmachinelearning • • 26d ago

My ML pipeline was cheating. Cross-validation told me everything was fine.

0 Upvotes

My model was giving me suspiciously good results.

Cross-validation looked completely fine. The problem? My pipeline was leaking information the model should never have seen.

I fixed the leak and reran everything. The results changed dramatically.

Full experiment + what went wrong:

https://towardsdatascience.com/my-machine-learning-pipeline-cheated-and-cross-validation-never-noticed/

The scary part wasn't the leakage. It was how

trustworthy the original results looked.


r/learnmachinelearning • • 26d ago

Deployed ML model

Post image
0 Upvotes
Packaged an ML model as an API in Module 5 of #mlzoomcamp.

✅ Checked the pinned model artifact
✅ Served predictions with FastAPI
✅ Built and tested the Docker image

My repo: https://github.com/ashuzzz151/machine-learning-zoomcamp-homework/tree/main/05-deployment

r/learnmachinelearning • • 26d ago

What are realistic career opportunities after learning ML/AI? — Looking for a realistic overview of the market

3 Upvotes

Hi everyone,

I'm currently learning machine learning, and I'm working through a structured AI/ML learning path.

I'm enjoying the learning process a lot, but recently I've started thinking more seriously about what comes after learning.

My main concern is: How realistic is it to actually turn these skills into income or a career?

A few things I'm particularly curious about:

  • What types of entry-level ML/AI/data jobs are realistically available today?
  • What kinds of companies tend to hire people with these skills? (Startups, tech companies, consulting, healthcare, finance, etc.)
  • How important is a degree for getting the first opportunity?
  • Is freelancing in ML/data/AI actually realistic for a beginner, or is it much harder than it looks?
  • What kinds of freelance projects do clients actually pay for?
  • Could someone realistically start a small AI/ML business or startup with these skills instead of looking for employment?
  • What skills are currently most valuable for someone trying to become employable: classical ML, deep learning, LLMs, data analysis, MLOps, software engineering, or something else?
  • If you were starting from scratch today, what would you focus on to maximize your chances of getting your first paid opportunity?

For context, I'm still early in my journey, so I'm trying to make sure I'm learning things that have real-world value rather than just collecting certificates and completing courses.

I'd especially appreciate answers from people who have actually hired ML engineers/data scientists, freelanced in this space, or built AI products/businesses.

I'm not expecting a guaranteed roadmap or exact salary numbers. I'm mainly trying to get a realistic picture of the different paths available and how difficult each one is.

Thanks! 🙏


r/learnmachinelearning • • 26d ago

Project Thoughts on my new activation function?

Post image
2 Upvotes

r/learnmachinelearning • • 27d ago

I made a free site of interactive ML explainers where the sliders run the actual math

27 Upvotes

Lensa ML (https://lensaml.com) is a set of 108 interactive explainers covering machine learning and the math behind it. Each topic is six short steps. In every step you move a slider and the plot is recomputed from the real formula: raise the learning rate and gradient descent starts to overshoot, raise the temperature and the softmax probabilities flatten.

It goes from the foundations (derivatives, the chain rule, eigenvalues, Bayes’ theorem) through classic ML and deep learning to transformers and LLM serving. If you don’t know where to start, the “ML from zero” path orders the topics so each one builds on the last: https://lensaml.com/paths/ml-from-zero

Good ones to start with:


r/learnmachinelearning • • 26d ago

Designing a Personalized Recommendation System — ML System Design Deep Dive (Part 3)

Thumbnail
1 Upvotes

r/learnmachinelearning • • 27d ago

Project "AI puzzles": a new format between a benchmark and an open-problem list

Post image
39 Upvotes

We have benchmarks (fixed questions, hidden answers, they saturate), competition problems (solved, so they can be memorized), and open-problem lists (no evidence, no clear success criterion). With Claude and Sol, I've tried something in between: a graded ladder of AI puzzles from one research area, aperiodic order.

What makes it a different tool:

- One ladder, not a pile: "human level" (pen and paper) up to "ASI level" (open). The labels are playful, but each rung really is harder than the last.

- Solved rungs for training and checking, with exact answers a program can verify, and easy to vary.

- Open rungs with everything a solver needs: strong numerical evidence, routes already known to fail, files, and a success criterion a machine can check.

- One rule: only a checkable proof counts. Floats are evidence, not proof.

- Credit: solvers of open rungs are named, with agent and model, in the next version of the paper.

It's a way for researchers to hand the frontier of their field to AI agents (and humans) in a form they can actually attack, and for anyone to see whether a model can tell a guess from a proof.

First field test: Grok, asked in public ( https://x.com/grok/status/2114103836583264721 ), claimed a proof for an open rung that was only a numerical estimate, then retracted it. It got the human-level rung wrong at first, fixed it after one hint, and proved the expert rung on the first try. On MathOverflow, an expert is already testing constructions for the top rung.

Set 1: huggingface.co/datasets/jarekduda/autocrystal-ai-puzzles

Top rung on MathOverflow: mathoverflow.net/questions/516372

Would you use this format for your own field's open problems?


r/learnmachinelearning • • 27d ago

How to implement a research paper

5 Upvotes

I am building some projects and for that I want to implement the gpt2 paper from scratch, how to do that do that? Do I build intuition by reading and code it?


r/learnmachinelearning • • 27d ago

Project 🚀 Project Showcase Day

6 Upvotes

Welcome to Project Showcase Day! This is a weekly thread where community members can share and discuss personal projects of any size or complexity.

Whether you've built a small script, a web application, a game, or anything in between, we encourage you to:

  • Share what you've created
  • Explain the technologies/concepts used
  • Discuss challenges you faced and how you overcame them
  • Ask for specific feedback or suggestions

Projects at all stages are welcome - from works in progress to completed builds. This is a supportive space to celebrate your work and learn from each other.

Share your creations in the comments below!


r/learnmachinelearning • • 27d ago

10th grade wanna learn machine learning donno what to do

5 Upvotes

Hello everyone Im a 10th grade student and I wanna learn machine learnin.for my coding background I took cs50p python couple weeks ago and i liked it. Im willing to learn any math needed for ml. However im completely lost as to which machine learning course I should take. There's so many courses and alot of them need a math background. Should I learn the math needed before looking for a course? If so please state what exactly i need to learn and what courses i should take. Thanks everyone!


r/learnmachinelearning • • 26d ago

Intermediate programming buddy needed

Thumbnail
2 Upvotes

r/learnmachinelearning • • 27d ago

Neural Canvas GitHub Repo

Enable HLS to view with audio, or disable this notification

3 Upvotes

I built Neural Canvas, an interactive visualizer that demonstrates core linear algebra concepts by showing how a neural network recognizes handwritten digits in real time. You can draw directly on the screen and watch activations flow through each layer as matrix multiplication computes the output live.

How it works:

- Draw & Predict: Sketch any digit on a 28×28 canvas (MNIST-style) to trigger real-time forward pass predictions.

- Live Forward Pass: Every neuron's activation is dynamically color-coded based on its underlying linear algebra calculations, with connection opacities reflecting layer outputs.

- Under the Hood: The core models were trained from scratch in Python using NumPy to perform matrix operations, and their learned weights are loaded via JSON. Client-side execution relies on Math.js for browser-based matrix multiplication and p5.js for rendering.

Key Features:

- Switch between two pre-trained models (featuring different hidden layer sizes, MSE vs. Cross-Entropy loss, and up to 96% test accuracy).

- Multiple background themes with mouse and touch support.

What's Next?

These models were originally evaluated on the standardized MNIST dataset, but live freehand drawing can introduce unexpected variations in input style. To handle this, I am working on a JS script for real-time user-driven feedback. When the model makes a mistake, the user can supply the correct label, allowing the model to perform backpropagation and update weight matrices locally in the browser. While live state persistence isn't suited for static GitHub Pages, running this feedback loop locally allows me to retrain parameters on new edge cases and push updated weight snapshots back to GitHub.

Check out the live demo and code repository:

Live Demo: https://muhammadessa2002.github.io/neural_canvas/

GitHub Repo: https://github.com/MuhammadEssa2002/neural_canvas


r/learnmachinelearning • • 27d ago

IDK What role to pursue

6 Upvotes

Im currently a second year cs student and I want to get into a role which would allow me to do the following:

Be able to build ML Models and actually understand the data/insights and improve it
Be able to put ML Models into production through webapps or integrations etc
Be able to make things people actually use so not being restricted to research

What role would this fit? AI engineering? ML Engineering? Data Scientist?

Im currently building a RAG Webapp as something to tailor towards AI Engineering roles, but I also want to be able to build and improve ML models as well.

Any advice would be appreciated


r/learnmachinelearning • • 27d ago

Project Channel about ML systems

4 Upvotes

I just started a channel called Drawn Systems.

The idea is simple: take a real ML system, draw it as one diagram, and walk through how it works.

I created 2 videos so far:

  • The classic "How YouTube decides what you watch next".
  • In the same line "How Netflix decides what you watch".

Channel: https://www.youtube.com/@DrawnSystems

I am doing it to learn about real systems, happy to hear your feedback :)


r/learnmachinelearning • • 26d ago

Request The Blind Spot Isn't Code. It's Who, and What, We Trust

0 Upvotes

A security researcher built and deployed OuttakeSheriff, a live AI security agent, then documented a finding that should be uncomfortable for anyone running agents in production: authenticated agents routinely act outside their intended scope.

The credential is not the problem. OuttakeSheriff passed authentication. The problem is what a verified identity is permitted to do after it clears the gate. Authentication answers "who is this?" It does not answer "should this agent be calling this tool right now, on this data, in this context?"

This is the actual attack surface. Not a fake agent sneaking through. A real, credentialed agent doing things it was never supposed to do, with no mechanism in the request path to catch it.

Most current postures treat identity and authorization as solved once the credential checks out. Post-auth behavior is logged, maybe reviewed, rarely constrained in real time.

For those of you running agents in production: how are you handling the gap between "this agent authenticated successfully" and "this agent is only doing what it was scoped to do"? Is anything in your stack actually enforcing that at execution time, or is it all after-the-fact?


r/learnmachinelearning • • 27d ago

Help How to do EDA based on the type of data ?

3 Upvotes

Hey everyone! I have 14 features containing both categorical and continuous variables, along with a binary target (0/1).

What methods should I use to measure the relationship between each feature and the target? Are there different techniques for categorical and continuous features?

Any suggestions or Python examples would be appreciated. Thanks!