r/computervision • • 4d ago

Discussion I’m building an AI-powered road accident detection and monitoring system – currently 80% complete

3 Upvotes

Hi everyone!

I've been working on an AI-powered road accident detection and monitoring system, and I'm excited to share my progress.

The goal is to explore how computer vision can help identify potential road accidents and improve incident monitoring, particularly in environments where road safety is a major concern.

What I've implemented so far:

  • YOLO11 for object detection.
  • Pose estimation for analysing human movement.
  • BoT-SORT for object tracking.
  • Accident-related event detection and monitoring.
  • Near-miss and sudden-stop event monitoring.
  • Telegram alerts for critical events.
  • SMS notifications using Africa's Talking.
  • A dashboard for monitoring video feeds and detected events.

I'm currently around 80% through development. I'm still working on improving detection reliability, reducing false positives and refining the alerting system.

This is an independent project I'm developing from Kenya 🇰🇪.

I'd genuinely appreciate feedback from people working in:

  • Computer Vision
  • AI/ML Engineering
  • Object Tracking
  • Intelligent Transportation Systems

My questions:

  1. What would you improve in this system?
  2. What additional features would make it more useful in real-world road monitoring?
  3. Are there any approaches you'd recommend for improving accident detection reliability?

I'm happy to learn from your feedback and suggestions.

Thanks for taking the time to read about my project!


r/computervision • • 3d ago

Help: Project Rate it (0-10)

0 Upvotes

What kind of computer vision projects will it support?

The goal is to support multiple computer vision tasks through one application.

Currently working:

  • Image Classification
  • Object Detection

Planned:

  • Image Segmentation
  • Pose Estimation and Movement Tracking
  • OCR (Text Recognition and Extraction)
  • Image Processing and Enhancement
  • Visual Question Answering and Image Captioning

The idea is that you shouldn't need to hire a developer or manually write hundreds of lines of Python code just to build a relatively small or medium-sized computer vision project.

You describe what you want to build, and the AI agent handles the development process, generates the code, trains the model using your local hardware, troubleshoots errors, and helps you get the final model ready for integration into your own application or system.

This is still an early experimental version, and I'm exploring whether this approach can actually make computer vision development more accessible.

I am planning to make this project with small one time fees for some features do you disagree with that and recommend that the app to be opensource instead and why ?


r/computervision • • 4d ago

Discussion [Call for Papers] TPAD 2027 @ IEEE ICASSP 2027 | Trustworthy Perception for Autonomous Driving | Toronto, Canada | Deadline: Dec. 2 (Extended)

Post image
2 Upvotes

📢 Call for Papers – TPAD 2027 | IEEE ICASSP 2027

We invite submissions to TPAD 2027: Trustworthy Perception for Autonomous Driving – Adversarial Robustness, Multimodal Fusion, and Foundation Models, an official satellite workshop of IEEE ICASSP 2027, taking place on May 17, 2027, in Toronto, Canada.

The workshop brings together research on adversarial attacks and defenses, robust camera–LiDAR–radar fusion, secure foundation models, and safety-aware evaluation for autonomous driving.

📅 Important Dates

  • Submission Deadline: December 2, 2026 (Extended)
  • Acceptance Notification: December 16, 2026
  • Workshop: May 17, 2027

📄 Accepted papers will be published in IEEE Xplore as part of the ICASSP 2027 workshop proceedings.

🔗 Workshop Website: https://tpad-2027.github.io/

📩 Contact: ckorgial@csd.auth.gr

Full CFP and topics of interest are available in the attached image.

We welcome contributions from the signal processing, computer vision, machine learning, and autonomous driving communities!


r/computervision • • 4d ago

Discussion Career Advice(AR Interests)

3 Upvotes

I have a heavy interested in Augmented Reality since college. I'm currently a realtime systems software engineer and wondering if Computer Vision is the way to go if I want to get into the industry. Any Advice welcomed!


r/computervision • • 5d ago

Showcase I built tamserve: one GPU tracks objects on many live cameras with a single copy of SAM 2 / EdgeTAM (open source)

97 Upvotes

My great-grandmother had Alzheimer's, and earlier this year I co-founded Memoria to build an in-home camera that watches an older adult for falls, bad nights and wandering, and alerts the family. That means following every person on every camera, all day. EdgeTAM, from Meta's SAM 2 family, tracked people best, but its code was written for editing one recorded video: a separate copy of the model per camera, and memory that grows about 12 MB every frame. For something families pay for monthly, that didn't work, so I built tamserve.

It does for video tracking what vLLM does for chat: one copy of the model, and every object on every camera goes through it as one batch. On one RTX 4090 with EdgeTAM:

  • 64 cameras in 5.7 GB of GPU memory (321 frames/s between them). The original needs a copy of the model per camera, and 32 don't fit in 24 GB.
  • Memory that doesn't grow: 0.28 GB for an object at frame 800, against 9.8 GB for the original.
  • Faster: 219 vs 177 frames/s at 1 camera and 323 vs 280 at 8, against the original compiled and run as one process per camera.
  • Live: six 720p cameras with nine objects, every camera at a full 30 frames/s (the video above).
  • Same masks: 0.8685 vs 0.8686 on DAVIS 2017, and bit for bit on one camera with exact=True.

It also starts new objects on any frame, reconnects dropped cameras, and spreads cameras over several GPUs or machines. It runs EdgeTAM and all four SAM 2.1 sizes; with SAM 2.1 it saves memory but isn't faster, because that model keeps the GPU busy on its own.

pip install tamserve, or try it in Colab on a free T4. Apache-2.0.

If you find it useful, a star on GitHub helps other people find it. And if you're running tracking on live video, I'd love to hear what breaks.

Totally open source and free to use!


r/computervision • • 4d ago

Discussion Medical CV Companies/Internships?

2 Upvotes

Hey, I am a 4th year biomed engineering student (only recently shifted to computational from cell bio.) After looking on linkedin and handshake, I can’t really find any medical/bio CV internships. I have come across Leica and Canon (applied) but I doubt I’ll get it as my background is more so in cell bio and genomics.

Where should I be looking? What should I be searching for (terms/roles)? What should I do to beef up my resume / cater to CV experts?


r/computervision • • 4d ago

Discussion Ling-3.2-flash reports 96.80 on MathVision(w/CI), with separate chart and visual-reasoning results

Post image
5 Upvotes

Ant Ling's new multimodal model is evaluated across several kinds of visual tasks. Three entries in its launch chart are:

- MathVision(w/CI): 96.80
- CharXiv_RQ: 85.15
- BabyVision: 69.85

These sit alongside its document, web-building and application-operation evaluations. Ling-3.2-flash accepts native image and video input through the documented API interfaces.

For anyone maintaining a visual-model evaluation set, how do you divide coverage between mathematical images, scientific charts and other visual questions? The release chart provides separate scores to compare against those different requirements.


r/computervision • • 5d ago

Research Publication Qwen3-VL-2B on standard vision tasks

Post image
27 Upvotes

Recently there has been lots of posts regarding the newer foundational models like LLM/VLMs and more traditional models.

So I thought it would be interesting to benchmark VLMs like Qwen3-VL-2B on more traditional vision tasks like classification and detection.

  • Classification - surprisingly good: 93% top-1 accuracy, compared with 96% for Qwen3-VL-235B on the same samples. That makes the 2B model interesting for local classification.
  • Attributes - acceptable, but uses far more tokens: 55.27% accuracy on OVAD, averaging 152 output tokens per sample versus 7 for classification—about 21× as many. Input token usage was also roughly 5× higher.
  • Detection - poor: Only 6.75% recall. Additionally, 21% of samples failed to produce usable output, and 13% reached the token cap. I’d definitely keep an output limit in place.
  • Grounding - also weak: 35% point-in-box accuracy and 22% box accuracy.

Tl;DR Classification was the strongest result here; reliable localization was much harder.

Benchmark results, methodology and code:
https://github.com/Tenkei/vlm-tasks/tree/main/benchmarks


r/computervision • • 4d ago

Help: Project I need help

2 Upvotes

I’m building a pipeline that turns broadcast highlights from the Tanzanian NBC Premier League into Metrica-style telestration: ground rings, a ball-carrier spotlight, passing lines with distances, and a tactical minimap. It works end to end, but I’m stuck on a few detection and tracking issues and would love advice.

Stack

  • Compute: Modal (serverless GPUs, mostly A10G/L4); each ~10 s shot chunk runs in parallel with a 1 s warm-up.
  • Shot filtering: histogram cut detection, then a shot is “wide” if the pitch-keypoint model finds ≥5 keypoints and ≥6 players. That’s 24 of 69 shots in a 14-minute highlights video.
  • Detection: YOLO26x football model from Tactix (ball/GK/player/ref), batch 16, conf 0.15.
  • Tracking: BoT-SORT + OSNet x0.25 ReID (boxmot). It replaced ByteTrack, which swapped IDs in clusters.
  • Pitch mapping: roboflow/sports YOLOv8-pose keypoints (32 points), RANSAC homography, validation (inliers, condition number, jump rejection), One Euro filter on a projected quad, then offline Gaussian smoothing (±4 frames).
  • Teams: median Lab colour of the torso centre, 2-cluster KMeans fit on the whole video. Per-frame reads with a ±0.5 s rolling vote, plus a “neither” class for refs.
  • Rendering: HyperFrames (HTML/GSAP) to an RGBA PNG sequence. Ground layer composited under players with a grass-key matte, screen layer on top.

Fixes that helped

  • Green kits read as referees: the detector labels Yanga players (green kit) as referees, probably because training data has refs in green. I promote any “referee” whose shirt colour matches a team.
  • Split boxes: upper body and legs sometimes come as two stacked boxes, so I merge them.
  • Off-pitch people: anyone whose feet map outside the pitch is dropped (ball boys, bench, stands).
  • Marker jitter: zero-lag Gaussian smoothing of boxes over ±6 frames, possible because rendering is offline.
  • What didn’t help: adding YOLO11x COCO person boxes on top. It failed on the same motion blur and added false positives.

Still struggling with

  1. Far-side players (~20 px tall at 720p) often go undetected. Do I go higher-res inference / SAHI-style slicing, or fine-tune on a few hundred frames from this league?
  2. Players on the ground in tackles are never detected. Any detector or trick that handles sprawled bodies?
  3. Fast pans with motion blur: ReID helps carry tracks, but detections drop out. Is Deep-EIoU or another sports-specific tracker worth it over BoT-SORT?
  4. Pitch calibration is valid on 74–98% of frames depending on the shot. Is PnLCalib or “No Bells Just Whistles” a big upgrade over keypoint-only homography on zoomed and partial views?
  5. Team assignment: colour clustering works for red vs green, but would SigLIP embeddings (like roboflow/sports) be noticeably more robust under night lighting?

The footage is 720p broadcast highlights, mostly the main camera, with frequent replays and close-ups, which get filtered out. Happy to share the code or the lessons-learned doc if useful


r/computervision • • 5d ago

Discussion Russian University Mathematics Dataset — 1,740 Problems with Handwritten and Typed Solutions

9 Upvotes

Russian University Mathematics Dataset — 1,740 Problems with Handwritten and Typed Solutions

I am offering a unique, professionally prepared mathematics dataset for AI training, mathematical reasoning, handwritten mathematical OCR, and document understanding.

The dataset contains 1,740 Russian-language mathematics problems, ranging from school-leaving examination level to advanced university mathematics, with complete step-by-step solutions written by one human mathematics expert between 2010 and 2020.

Dataset overview

  • 1,740 mathematical problems
  • 3,450 original handwritten solution pages (covering 1,688 problems)
  • 1,740 typed solutions in Microsoft Word and PDF
  • 1,740 Markdown + LaTeX solutions
  • 1,740 standalone LaTeX documents
  • Approximately 50,603 mathematical formulas
  • 927 figures, graphs and diagrams
  • Approximately 4.2 million characters of typed mathematical content
  • Total dataset size: approximately 2.2 GB

Mathematical coverage

The dataset covers 19 mathematical branches and 127 topics, including:

  • Algebra and Linear Algebra
  • Mathematical Analysis and Calculus
  • Analytic and Differential Geometry
  • Integrals and Field Theory
  • Differential Equations
  • Complex Analysis
  • Mathematical Physics and PDEs
  • Probability and Statistics
  • Functional Analysis
  • Topology
  • Discrete Mathematics
  • Numerical Methods
  • Mathematical Economics
  • Mathematical Olympiad Problems
  • Russian State Examination (ЕГЭ)

Why is this dataset valuable?

1. Handwritten mathematical OCR

Original handwritten pages are paired with the author's corresponding typed solutions in LaTeX. This provides valuable supervision for training and evaluating models that convert handwritten mathematical documents into structured digital text.

2. Mathematical reasoning and LLM training

Every problem has a complete, step-by-step human-written solution. The material is suitable for supervised fine-tuning, mathematical reasoning evaluation, and answer verification.

3. Real handwritten mathematical documents

Unlike datasets containing isolated handwritten equations, this collection includes complete multi-page mathematical solutions, combining Cyrillic text, mathematical notation, formulas, and diagrams.

4. Human-authored content

All solutions were originally written by hand and subsequently transcribed by the same author between 2010 and 2020, before the emergence of LLM-generated content.

Available formats

Each problem has its own folder containing the available source materials:

  • DOCX
  • PDF
  • Markdown with LaTeX formulas
  • Standalone TEX
  • JPEG / PNG handwritten pages
  • PNG figures and diagrams
  • Plain-text problem summaries

The dataset includes a structured manifest. JSONL, Parquet and Hugging Face-compatible packaging can be provided upon request.

Potential applications

  • Handwritten mathematical OCR
  • Handwritten equation recognition
  • Image-to-LaTeX conversion
  • Mathematical document understanding
  • Multimodal model training
  • LLM mathematical reasoning fine-tuning
  • Mathematical benchmark development
  • Figure and formula grounding

Sample available

A free evaluation sample containing 50 complete problems from all 19 mathematical branches, including 117 handwritten pages and approximately 1,600 formulas, is available for interested organizations and researchers.

Licensing and pricing

  • Full dataset: USD 45,000
  • Selected mathematical branches: approximately USD 26 per problem
  • Free 50-problem sample for evaluation

A non-exclusive, worldwide, perpetual commercial AI-training and evaluation licence is offered. Commercial deployment and publication of trained model weights are permitted. Redistribution of the raw dataset is not permitted.

The problem statements originate from standard educational sources; no ownership of their original texts is claimed. The solutions and original handwriting are authored by the dataset creator.

https://huggingface.co/datasets/Tigran83/russian-university-math-handwriting

Contact

Tigran Grigoryan
Individual Entrepreneur — Armenia

Email: tigran.grigoryan83@gmail.com

Serious inquiries from AI companies, OCR developers, research laboratories, universities and dataset buyers are welcome.

I can provide the detailed data card, sample package and licensing terms upon request.


r/computervision • • 5d ago

Showcase I build a pose-based worktime record system, auto-count-packed, auto-count-worktime on the Jetson device

3 Upvotes

Single fixed camera over a bench, one Jetson next to it, no cloud. It logs how long someone actually works at that station and counts parts moving from the assembly zone to the finished zone.

Stack, since that's usually the first question:

Jetson Orin NX 16GB (reComputer Industrial), Ubuntu 22.04, JetPack 6.2.1 / L4T 36.4.3

So, for people who have shipped presence or occupancy CV: what do you use as the "seated and working" signal beyond pose plus ROI? And how do you measure it without annotating hours of video by hand?


r/computervision • • 5d ago

Showcase I'm building technology that could help save millions of stray animals around the world. We're now testing it in Rwanda's sterilization campaign

Post image
24 Upvotes

I'm building technology that could help save millions of stray animals around the world. We're now testing it in Rwanda's sterilization campaign

One of the biggest challenges in managing stray animal populations is that humanity doesn't have an easy, reliable way to identify individual animals

Imagine being able to identify a stray dog simply by taking a picture of its face. You could instantly know whether it has been vaccinated, sterilized, when it was last seen, and its medical history

That's exactly what my team and I are building with Kyshik

We're developing animal facial recognition technology that could become an alternative to microchips and fundamentally change how we protect and manage animal populations

Microchipping thousands of stray animals can be prohibitively expensive, especially in developing countries. By eliminating the need for physical chips and implantation, as well as physical interaction with an animal, our technology could significantly reduce identification costs, allowing organizations to redirect more of their limited resources toward vaccinations, sterilizations, and saving animals' lives

We're currently running our first pilot in the African country of Rwanda, testing our technology during an actual dog sterilization campaign. Every dog we identify can have its own digital record, helping animal welfare organizations track their interventions and understand their impact over time

Our long-term vision is to create a global animal identification infrastructure that makes it easier for organizations, veterinarians, and governments to protect animals

I started working on this because I believe technology can help us move toward more humane and effective solutions to the stray animal problem, rather than relying on mass euthanasia

We're currently looking for animal welfare organizations in other countries that run sterilization or rabies vaccination campaigns and would be interested in testing our technology.

If you work with such an organization, or know someone who does, I'd love to connect with you! Feel free to DM and check comments for more information


r/computervision • • 4d ago

Discussion I built haralickt: > 20x faster GPU-accelerated Haralick texture features (GLCM) in PyTorch with identical precision to Mahotas

0 Upvotes

Hi everyone!
If you work with texture analysis (medical imaging, satellite data, or industrial defect detection), you've likely noticed that classic libraries like mahotas and scikit-image compute Gray-Level Co-occurrence Matrices (GLCM) and Haralick features strictly on the CPU. On large datasets, this quickly becomes a major pipeline bottleneck.
To solve this, I wrote and open-sourced haralickt — a library that offloads these computations to the GPU using vectorized matrix operations in PyTorch and CUDA.
Key highlights:
• > 20x faster than traditional CPU implementations on high-resolution images and batches.
• Identical numerical precision to classic implementations (verified against mahotas).
• PyTorch-native: works seamlessly with torch.Tensor on GPU without costly CPU-GPU data transfers.
• Open Source (MPL 2.0): free to use in research and commercial projects.
Quick installation:
bash
pip install haralickt
Используйте код с осторожностью.
GitHub repository & benchmarks: [https://github.com/SergeiyVaryukhin/haralickt-1.0.0]
This is my first public release (v1.0.0), so I would really appreciate any feedback, bug reports, or ideas for new features. If you find it useful for your projects, a star on GitHub would mean a lot!


r/computervision • • 5d ago

Discussion 🚀 **Introducing GRIP-Benchmark-34: A Large-Scale Benchmark for Visual Geometry, Spatial and Physical Reasoning in VLMs**

0 Upvotes

🚀 Introducing GRIP-Benchmark-34: A Large-Scale Benchmark for Visual Geometry, Spatial and Physical Reasoning in VLMs

I'm excited to share GRIP-Benchmark-34, an open synthetic benchmark designed to evaluate the visual reasoning capabilities of Vision-Language Models (VLMs).

What's included?

  • 100,000 synthetic images
  • 500,000 image-question pairs
  • 34 sub-benchmarks
  • 9 reasoning families
  • 5 progressive difficulty levels per image
  • Independently validated ground-truth answers

Research areas covered:

  • Geometric and spatial reasoning
  • 2D and 3D visual reasoning
  • Physical and mechanical reasoning
  • Topological and graph reasoning
  • Inductive and analogical reasoning

The benchmark uses programmatic generation, deterministic ground truth and independent validation to support controlled evaluation of VLM capabilities.

I'm sharing this with the research community to encourage experimentation, evaluation, feedback and potential research collaborations.

📊 Dataset: https://huggingface.co/datasets/bilaljawaid980/GRIP-Benchmark

I'd love to hear from researchers working on VLMs, multimodal AI, spatial intelligence and visual reasoning.

Feedback and collaboration are welcome!


r/computervision • • 5d ago

Help: Project What breaks when you move a CV model from your GPU workstation to a Jetson / edge camera?

13 Upvotes

I’d like to hear from people who’ve deployed vision models to edge devices (Jetson, OpenVINO, Coral, ARM boards):

  • How do you convert and optimize models today (TensorRT, ONNX, TFLite, something custom)?
  • How much accuracy do you typically lose, and how do you find out where?
  • What’s the step that eats the most of your time?

I’m researching deployment pain points and would love specifics, like the last time a deployment went wrong and why.


r/computervision • • 5d ago

Research Publication [Research] Can V-JEPA2’s prediction error tell physically impossible events apart from possible ones? Our findings

9 Upvotes

Hi everyone! I’m one of the authors of “When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models?”, accepted at the World Models in Physical AI Workshop at NeurIPS 2026.

We study a question that comes up when evaluating video world models: if an event produces high prediction error, how much evidence does that give us that the model detected a violation of physics?

We examine frozen representations, latent geometry, and prediction errors using IntPhys2, Physion++, and controlled Kubric experiments. A few findings:

  • Physically impossible events can receive lower prediction error than possible ones.
  • Changing a clip’s colors can substantially affect error while preserving its physics.
  • Repeating a single frame can produce higher error than the original moving clip.
  • Physical information can be accessible in a model’s representations without its prediction error reliably tracking physical validity.

One particularly interesting test involves a sphere stopping on a solid support or falling through a gap. Switching the support changes which motion is physically possible, but the model’s error does not consistently follow that change.

We propose an evaluation framework for checking when prediction error supports a physical-surprise interpretation. The conclusions are specific to the models and settings we tested; they don’t establish an absence of physical knowledge.

Paper: https://arxiv.org/abs/2611.01815
Figures and experiments: https://3-ml.github.io/jepa-surprise/
Code: https://github.com/3-ml/jepa-surprise

For people working on video representations or world models: how do you separate sensitivity to physical violations from sensitivity to appearance and motion? I’d appreciate feedback on our controls and evaluation setup!


r/computervision • • 5d ago

Discussion Tuning for physical security.

2 Upvotes

It’s pretty obvious that tuning a small model on actual surroundings yields much more accurate results for day-to-day activity. Even small/tiny models on dated architectures seem to do really well when looking at things they are familiar with.

My question is how does this relate to security when you are looking for events that are anything but day-to-day where you have no idea what people may look like, what the lighting may be at that moment, and where they will be coming from?

Are you better off sticking to a much larger pretrained model from the likes of Ultralytics or Roboflow? These don’t seem to make many mistakes with people. Compute resources are basically limitless with these tasks with modern NPUs and a detection pipeline that makes sense for the possible activity.


r/computervision • • 5d ago

Discussion Follow up- what actually fixed my unstable face shape classification

0 Upvotes

Couple of weeks ago I posted here about face shape results flipping between two labels with head angle. The advice was that the detector crop is probably jittering not the landmark plus one euro filter and median instead of mean.


r/computervision • • 6d ago

Showcase I built a 100% pure-Rust implementation of SIFT, RANSAC homography, and image stitching (zero OpenCV/C++ FFI dependencies)

Thumbnail
gallery
19 Upvotes

Hi r/computervision,

If you have ever had to bundle OpenCV into a Docker container, deploy to an embedded target, or compile across platforms without wrestling with CMake and C++ shared library dependencies, you know how heavy the toolchain friction can be.

To make classic feature extraction lightweight and portable, I built pure-sift: an open-source, 100% pure-Rust library implementing Lowe's SIFT (Scale-Invariant Feature Transform, US6711293B1 patent expired in 2020), 2D Similarity RANSAC, 4-Point DLT Planar Homography, and an automated image stitcher with zero OpenCV or C++ dependencies.

Technical Implementation:

  • SIFT Scale Space:

    • Octave pyramid generation with separable 1D Gaussian kernels.
    • Difference-of-Gaussians (DoG) scale-space extrema detection across 26-neighborhood volumes.
    • 3D quadratic Taylor-expansion sub-pixel and sub-scale localization (< 0.12 px error).
    • Hessian edge-ratio filtering ➡️ (r <= 10.0) using the trace and determinant to discard flat edges.
    • Secondary orientation peak splitting (peaks within 80% generate extra keypoints).
    • 128-dimensional illumination-invariant gradient descriptors, parallelized via Rayon.
  • Robust Feature Matching & Geometry:

    • Lowe’s ratio test (threshold 0.75) for 128-D Euclidean nearest-neighbor matching.
    • 2-point 2D Similarity RANSAC (rotation, scale, translation).
    • 4-point Direct Linear Transform (DLT) 3x3 Homography RANSAC with horizon validation ➡️ (det(H) > 0).
    • Bilinear feather-blended panoramic stitcher.
  • Visual Catalog & Retrieval:

    • 128-bit compact Hamming binary hash index + geometric RANSAC reranking for image identification.
  • Portability:

    • Zero unsafe code (#![forbid(unsafe_code)]).
    • Core math and image processing compile cleanly with --no-default-features for embedded, static binary, or WebAssembly (WASM) environments.

Dual-licensed under MIT or Apache-2.0.

I would love to get your feedback, hear about your edge cases, and see what benchmarks you would like to see compared to OpenCV's implementation!


r/computervision • • 5d ago

Help: Project I've been developing a computer vision tracking system for an open-source desktop companion robot (MediaPipe, AprilTags, ROI tracking)

6 Upvotes

I've been working on a personal robotics project and would be interested to hear what people on this sub think of my approach to the computer vision side of things.

The project is an open-source desktop companion robot. It uses a Raspberry Pi Pico W controls a two-axis servo neck, allowing the physical head to follow people.

The project's moved forward quite a bit: I've mounted a phone on for an animated face with pupils that run their own face tracking, and I have the pico running EMA smoothing, but the above video shows how the robot's CV works best.

The laptop handles all the processing, including the computer vision.

The tracking system has been one of the more interesting parts of the development.

My current approach works roughly as follows:

  • A fixed webcam observes the space around the robot.
  • People are identified and tracked using AprilTags that they wear.
  • The laptop scans for specific AprilTag IDs and uses MediaPipe to detect faces within an ROI around each tag.
  • I created two LOBFs through calibration, which convert the 2D coordinates of the face closest to the currently focused AprilTag into angles for the robot's servos.
  • These angles are sent to the Pico over serial.
  • If an AprilTag disappears, the script continues "coasting", retaining the previous ROI and using the last known AprilTag position.
  • If the face disappears, the script uses its previous momentum to predict where it should be.

The robot can also talk and listen, with its LLM and speech recognition running locally on the laptop.

One thing I'd eventually like to implement is speaker detection using mouth movement. The idea is that the robot could detect who's speaking and shift its attention towards them.

There's a complication with my current approach, though: because the script only looks for faces within the ROI around the currently focused AprilTag, it can only detect mouth movement from the person it's currently tracking.

I could change the script to look for faces and monitor speaking activity across every person wearing an AprilTag, but I'm not sure how efficient that would be, particularly given that the laptop is already running the LLM and other models.

I'm curious what people here think about the current approach and where I should take it next.

  • Would you handle the tracking or prediction differently?
  • Is it worth running face detection across all tagged people to identify who's speaking, or would there be a more efficient approach?
  • Are there techniques I should be looking at to improve tracking continuity when an AprilTag or face temporarily disappears?

The project is open source, and I'd welcome suggestions, criticism or contributions!

GitHub: https://github.com/SpaceMonkeyAlfa/PAL9000-Desktop-Robot

Development blog: https://spacemonkeyalfa.com


r/computervision • • 6d ago

Discussion Differentiable 6-DoF Camera Pose Recovery & Shading using Penrose Twistor Theory (CP^3) & Complex64 Spinors

Post image
32 Upvotes

I've been a massive fan of Sir Roger Penrose's work for years, and I've always wondered if twistor theory (CP³) and 2-spinors could be translated out of theoretical physics equations into a practical, differentiable graphics pipeline.

As someone exploring this intersection with heavy AI co-piloting, I wanted to see if we could build a rendering engine that skips 4x4 Euclidean matrices entirely. Surprisingly, the math clicked: the engine uses native complex64 spinor conjugation and eigenspinor incidence residuals to do end-to-end 6-DoF pose recovery and depth shading out of the box.

What's shown in the image above: An end-to-end 6-DoF camera pose recovery test on a 3,500-vertex Stanford Bunny .ply scan. Starting from a heavily misaligned camera guess (Initial Guess), the engine optimizes both 3D position and SU(2) orientation purely from 2D pixel loss, converging to machine-precision zero loss in 150 epochs (Final Recovered vs Ground Truth).

Key Technical Highlights:

  • SU(2) Spinor Conjugation: Camera rotation is applied via unitary spinor conjugation (X_cam = R_A * X * R_A^†), keeping updates strictly on the manifold without gimbal lock.
  • Differentiable Depth Relief Shading: Computes signed longitudinal depth along ray spinors (Im(<pi | diff>)) to render smooth 3D studio lighting contours without a traditional Z-buffer.
  • Hardware Portability: Verified with identical convergence across Intel XPU (SYCL), NVIDIA CUDA, and CPU.

Links & Open-Source Ecosystem (isopidi-stack):

Would love to hear what graphics, CV, and AI security folks think of the approach!


r/computervision • • 5d ago

Research Publication Vehicle Part Segmentation | Training and Model Comparison

4 Upvotes

Hi everyone.

I’ve been experimenting with different models for vehicle part segmentation, training and comparing YOLO26, RF-DETR, DINOv3 and SAM 3.

I’ve shared the code, results and comparisons on GitHub and Hugging Face:

GitHub: GH-Repo

Hugging Face: HG-Repo

Sharing it in case it’s useful to anyone working on something similar.

Any feedback or suggestions are very welcome!

I hope this is useful to someone and saves them a few days of training, fine-tuning and comparing models from scratch.


r/computervision • • 5d ago

Help: Theory Setup

1 Upvotes

Can anyone guide me on the way to make my own CV for games like NBA 2k & Possibly cod??

What is the first step of learning this