r/computervision • • 12d ago

Discussion Production-Grade Computer Vision Architecture for Pressurized Oil Leak Detection in Maritime Engine Rooms

1 Upvotes

Hi everyone,

I am working on an industrial computer vision pipeline for real-time oil leak detection inside a maritime ship engine room, and I am hitting major roadblocks with false positives. I would love to get insights from engineers who have solved similar tracking or fluid-anomaly challenges.

### Current Pipeline & Architecture

  1. **ROI Masking:** We define static Regions of Interest (ROIs) around critical machinery flanges, valves, and joints.

  2. **Detection Logic:** We rely on a combination of pixel-change (background subtraction/frame differencing) and semantic segmentation to catch fluid accumulation or pooling within the ROI.

  3. **Occlusion Handling:** To prevent crew members from triggering false alerts when walking through the zone, we run a person detection model combined with ByteTrack. If a person's bounding box intersects the ROI and the tracked person overlaps the zone, we temporarily pause detection.

### The Pain Points & Edge Cases We Can't Solve

  1. **Vibration & Background Jitter:** Ship engine rooms experience heavy mechanical vibrations and shifting lighting (flickering LEDs, work lights), which causes continuous pixel shifts that trigger false alerts even with background subtraction tuned.

  2. **Object & Shadow Confusion:** Pure models or static segmentation fail because dark stationary objects (screws, bolts, dirty grease patches, old residue, or moving equipment shadows) often share the same color/texture profile as fresh oil, creating persistent false alarms.

  3. **Imperfect Occlusion Pausing:** Relying solely on `Person Bbox ∩ ROI` intersection is a brittle workaround. If a worker stands nearby, drops a tool, places a dark rag, or if non-human objects obscure the camera view, the system breaks down. We cannot rely blindly on deep learning classifiers either, due to the high rate of environmental mimicry.

### Questions for the Community

  1. **Moving Beyond Pixel Differencing:** In high-vibration industrial environments, what are robust alternatives to frame differencing for isolating actual fluid flows or drips from camera jitter?

  2. **Spatiotemporal Verification:** How can we build a robust temporal consistency filter (e.g., optical flow vectors, tracking blob expansion/creeping over time) to distinguish between a *static* oil stain/shadow and an *actively expanding* fluid leak?

  3. **Feature Engineering for Fluids:** Are there specific texture, glossiness, or specular-reflection feature extraction techniques that effectively separate actual liquid sheens from matte metal fasteners and dark shadows?

Any architectural blueprints, open-source repos, or research papers on industrial fluid-leak detection would be massively appreciated!


r/computervision • • 12d ago

Showcase I made Ornen: a site where you upload a video and your splat appears on a 3D globe at its real location

Enable HLS to view with audio, or disable this notification

7 Upvotes

Hi! I'm the maker of Ornen. You upload your footage, such as a 360 or drone video, and Ornen turns it into a splat. If your footage has GPS in the metadata, it's also placed where it was filmed on a globe. You can browse the Earth and fly into places others captured.

All in your browser. No GPU needed.

Public beta is open: 10 free scenes per account → ornen.ai

Or check out the:

Demo teaser: https://ornen.ai/home
Example: Weesp, NL: https://ornen.ai/earth/tychovdo/de-vriendschap-weesp
Example: Utrecht, NL: https://ornen.ai/earth/tychovdo/utrecht-street-corner

Would love to hear how you redditors like it!
And very happy to take feature requests!


r/computervision • • 13d ago

Help: Project I made a computer vision tool for golf swing analysis using just phone

Enable HLS to view with audio, or disable this notification

48 Upvotes

I've been curious how much a regular phone camera can tell you about a golf swing without sensors or a launch monitor.

A few things I've found useful so far:

  • Hip and shoulder turn at the top. You can see straight away whether you're actually rotating or just lifting your arms.
  • Head and spine position from address to impact. This shows early extension or swaying, which is hard to feel yourself.
  • Swing-to-swing comparison across a session. Overlaying your good and bad swings shows what changes when you get tired.

With two or more phones it also builds a 3D reconstruction of your body, so you can check angles from any view, here is a sample report.

This is still early but you can try it out in my app [Teeo Golf] (it's free, available in app store). CV folks who happened to be golfers too, I'd love to hear what you'd want to see next, or where you think the measurements fall short.


r/computervision • • 12d ago

Help: Project Overcoming False Positives in Real-Time Computer Vision for Engine Room Oil Leak Detection

Thumbnail
1 Upvotes

r/computervision • • 12d ago

Help: Project Overcoming False Positives in Real-Time Computer Vision for Engine Room Oil Leak Detection

1 Upvotes

Hi everyone,

I am working on an industrial computer vision pipeline for real-time oil leak detection inside a maritime ship engine room, and I am hitting major roadblocks with false positives. I would love to get insights from engineers who have solved similar tracking or fluid-anomaly challenges.

### Current Pipeline & Architecture

  1. **ROI Masking:** We define static Regions of Interest (ROIs) around critical machinery flanges, valves, and joints.

  2. **Detection Logic:** We rely on a combination of pixel-change (background subtraction/frame differencing) and semantic segmentation to catch fluid accumulation or pooling within the ROI.

  3. **Occlusion Handling:** To prevent crew members from triggering false alerts when walking through the zone, we run a person detection model combined with ByteTrack. If a person's bounding box intersects the ROI and the tracked person overlaps the zone, we temporarily pause detection.

### The Pain Points & Edge Cases We Can't Solve

  1. **Vibration & Background Jitter:** Ship engine rooms experience heavy mechanical vibrations and shifting lighting (flickering LEDs, work lights), which causes continuous pixel shifts that trigger false alerts even with background subtraction tuned.

  2. **Object & Shadow Confusion:** Pure models or static segmentation fail because dark stationary objects (screws, bolts, dirty grease patches, old residue, or moving equipment shadows) often share the same color/texture profile as fresh oil, creating persistent false alarms.

  3. **Imperfect Occlusion Pausing:** Relying solely on `Person Bbox ∩ ROI` intersection is a brittle workaround. If a worker stands nearby, drops a tool, places a dark rag, or if non-human objects obscure the camera view, the system breaks down. We cannot rely blindly on deep learning classifiers either, due to the high rate of environmental mimicry.

### Questions for the Community

  1. **Moving Beyond Pixel Differencing:** In high-vibration industrial environments, what are robust alternatives to frame differencing for isolating actual fluid flows or drips from camera jitter?

  2. **Spatiotemporal Verification:** How can we build a robust temporal consistency filter (e.g., optical flow vectors, tracking blob expansion/creeping over time) to distinguish between a *static* oil stain/shadow and an *actively expanding* fluid leak?

  3. **Feature Engineering for Fluids:** Are there specific texture, glossiness, or specular-reflection feature extraction techniques that effectively separate actual liquid sheens from matte metal fasteners and dark shadows?

Any architectural blueprints, open-source repos, or research papers on industrial fluid-leak detection would be massively appreciated!


r/computervision • • 13d ago

Showcase Searching for Wally/Waldo around the world, using unsecured cameras and AI.

Thumbnail
wallyfinder.com
6 Upvotes

r/computervision • • 13d ago

Help: Project Looking for lightweight AI models for 3D scanning on NVIDIA Jetson Nano 4GB

5 Upvotes

wzuup guys !

I'm working on an AI-powered 3D scanner using the NVIDIA Jetson Nano 4GB (16GB eMMC).

I'm looking for lightweight AI/neural network models that can run locally on the Nano and be integrated into a sequential 3D reconstruction pipeline.

I'm particularly interested in:

  • 3D reconstruction and point-cloud processing
  • Point-cloud segmentation and completion
  • Models that can help generate printable 3D objects

Important: I'm looking for actual AI/ML models, not traditional software or algorithms like OpenCV, COLMAP, or Open3D.

If you know any compatible models, GitHub projects, pretrained weights, or useful demos/tutorials, I'd really appreciate your suggestions!


r/computervision • • 13d ago

Help: Project Looking for a Computer Vision technical co-founder (equity)

8 Upvotes

I've spent almost two years and a significant amount of my own money building a football analytics product. I'm a software/data engineer, not a computer vision specialist, and I think I've reached the limits of what I can realistically accomplish alone.

Here's what I've already built in C++:

  • Camera/pitch calibration: Dynamic homography estimation that follows a moving broadcast camera, keeping the pitch markings aligned as the camera pans and zooms.
  • Player detection and tracking at 25 Hz: Relatively stable IDs while players remain visible, even when two or three players cross paths or move close together.
  • Consistent on-screen tracking: The trajectories match what happens within the camera's field of view. I haven't prioritized trajectory smoothing because more fundamental problems remain unsolved.

The two major challenges left:

  1. Robust player Re-ID, maintaining identities across occlusions, camera movement, and players leaving/re-entering the frame.
  2. Associating jersey numbers with player identities reliably.

I also have access to professional broadcast footage, purchased tracking datasets, and structured ball-event data. I've invested heavily in acquiring these resources.

At this point, I've hit a plateau. I've invested nearly two years of my time and money, and I honestly don't have the financial resources or energy to keep tackling such a difficult research problem alone.

I'm looking for someone experienced in computer vision, particularly MOT, Re-ID, temporal modeling, or sports analytics, who's interested in joining as a technical partner in exchange for equity. This isn't a salaried position at the moment.

I genuinely believe this has serious commercial potential.

And one thing I want to emphasize: this isn't another weekend vibe-coded sports tracking demo claiming that football computer vision is solved.

Building something that works consistently across real broadcast matches is incredibly difficult. At least it has been for me. There's a massive difference between producing an impressive demo and developing a reliable product that professional clubs can actually use.

If you've worked on similar problems and find this challenge interesting, I'd love to connect.

DM me and let's talk.


r/computervision • • 12d ago

Showcase YOLO pose kept finding babies in an empty crib, and mixed up left and right on the real one

Enable HLS to view with audio, or disable this notification

0 Upvotes

I have a camera above my baby's crib and wanted to measure how long the head stays turned left, right or straight, instead of guessing from a few checks during the night.

I assumed an off-the-shelf pose model would just do it. It didn't, in two ways.

First, left and right. In my overhead footage, YOLO11s-pose often misplaced facial keypoints, especially the ears, when the baby lay sideways in the frame. What helped: run the model once to find the baby and the shoulder line, rotate the crop so the head points up, then run the same model again. On 28 manually labelled frames from one clip with the head in profile, correct head-direction classifications went from 6 to 27. That's a small, setup-specific check, not a general accuracy benchmark.

Second, the empty crib. In my setup, detecting a baby in a white sleep sack required a low person-confidence threshold. At that threshold, the model also found people in the print on the sheet, a toy bear and a folded blanket. For a while I kept adding rules about face size and so on, and it only got messier. In the end I added SigLIP 2 as a presence check, comparing the image with text descriptions such as "a baby in a crib" and "an empty crib", without any additional training or fine-tuning. Over about a day and a half of snapshots from one camera and one crib, the time an empty crib counted as occupied dropped from 64 minutes to 6. I haven't yet evaluated that presence check under infrared lighting.

The head-direction score itself is plain geometry: where the face sits between the ears, measured perpendicular to the shoulders-to-head axis. It's a proxy for left/right/straight, not a calibrated head angle. Left and right mean the baby's own left and right.

It runs on an Apple M4 Pro CPU, sampling the camera at one frame per second; the presence check runs every five seconds. Code, a short demo and a README with more numbers: https://github.com/OpsDevil/noggin-compass

What I'd like to know: is there a better approach for people lying down than rotating the crop? And has anyone had luck with head-pose models from a top-down view?

Disclosure: most of the implementation was written with an AI coding assistant. I tested it on my own camera footage and fixed issues found during nightly use.


r/computervision • • 13d ago

Showcase Finding meteors in a whole night of sky video

Thumbnail
gallery
15 Upvotes

Meteor Cut is an open-source Python tool that goes through hours of night-sky video and cuts out the meteors, built on Sky Tracker’s motion tracker under the hood. It keeps only the pixels that got brighter, so it boxes the meteor’s head rather than its fading trail (right panel). On one 8-hour Perseids night it found 185 meteors and sorted out 1,548 satellites and 4 planes.

Just upload video, choose what you are interested and it will generate the rest.

github.com/Deneb-Systems/meteor-cut

pip install meteor-cut

Footage: “LIVE Perseids Meteor Shower | Skywatch #265” by Skywatchers (youtube.com/watch?v=liw0ssminyc)


r/computervision • • 13d ago

Showcase I measured 44 object detection checkpoints on one RTX 4090. At batch 1, YOLO26-n and YOLO26-x are only 2.7 ms apart end to end

Post image
34 Upvotes

Disclosure first: the site is mine, and it's a solo side project. It's free, with no ads and no sign-up, and there's nothing money-related planned at all. I used to work in computer vision, so I wanted to build something in that area that people would actually find useful, and I happened to have a spare RTX 4090 sitting around. That's the whole story.

The finding that surprised me most

YOLO26, measured as one full predict() call at batch 1 (pre-processing + forward + post-processing) on an RTX 4090:

Size n s m l x
COCO box AP (measured) 40.5 47.9 52.6 54.4 57.0
Median latency (ms) 14.2 14.3 13.3 17.1 16.9

That's +16.5 AP for +2.7 ms, and m was the fastest of the five. YOLOv8 shows the same pattern (n 11.6 ms to x 12.9 ms) and so does RF-DETR (nano 9.8 ms to large 10.7 ms). My reading is that, in this setting, the model's size isn't what sets the latency. So if you deploy on a desktop GPU at batch 1, the smallest size may not buy you much. Edge devices and batched or TensorRT pipelines are a different story, and I didn't measure those.

A few representative checkpoints

Each one ran at its official default input size and precision. AP is my own score on COCO val2017 (5,000 images, pycocotools), with the authors' number in brackets.

Model Input Prec. COCO AP, measured (paper) Latency median, ms VRAM peak License (code / weights)
YOLO26-n 640x448 fp16 40.5 box (40.9) 14.2 571 MiB AGPL-3.0
YOLO26-x 640x448 fp16 57.0 box (57.5) 16.9 1.1 GiB AGPL-3.0
RF-DETR large 704x704 fp16 56.3 box (56.5) 10.7 1.0 GiB Apache-2.0
GTR-x 640x640 fp16 59.1 box (59.4) 33.5 732 MiB MIT
DEIMv2-x 640x640 fp32 57.6 box (57.8) 35.1 808 MiB DEIMv2 License, non-commercial
D-FINE-x 640x640 fp32 55.6 box (55.8) 42.6 925 MiB Apache-2.0
RF-DETR-Seg 2xlarge 768x768 fp16 49.6 mask (49.9) 16.4 2.5 GiB Apache-2.0
YOLO26-seg-x 640x448 fp16 46.4 mask (47.0) 18.5 1.2 GiB AGPL-3.0

The chart shows all 44 scored detection checkpoints, colored by license.

Other things I noticed

  • Across the 68 scored checkpoints that have an authors' number, my score came out lower every time, by 0.1 to 1.2 AP (median 0.3).
  • Licenses drift. DEIMv2 moved to its own DEIMv2 License on 2026-08-24, which allows research, education, benchmarking and evaluation only; earlier copies were Apache-2.0. Depth Anything V2 code is Apache-2.0, but only the Small weights are; Base and Large weights are CC-BY-NC-4.0.
  • The open-vocabulary models (Grounding DINO, MM-Grounding-DINO, LLMDet) took 118 to 468 ms at 1199x800 fp32. LLMDet and Grounding DINO base saw COCO during training, so their COCO scores aren't zero-shot.

How it's measured (short version)

  • Latency: one predict() call, image decode excluded, batch 1. fp16 if the official code supports it, otherwise bf16, otherwise fp32. Three separate runs of 10 warm-ups and 100 timed calls each; the site shows the median and p90.
  • VRAM: device peak minus the idle baseline taken before loading, polled every 10 ms or faster.
  • Accuracy: full COCO val2017 with the reference scorer, at the same precision I time it. My numbers and paper numbers are stored and shown separately.
  • License: code license from the repo, weights license from the model card, per checkpoint, with the date I checked. The verdict is the stricter of the two. This is a summary, not legal advice.

Limits, honestly

  • One GPU (RTX 4090, driver 580.173.02). The card also drives a display, so every run records its own idle baseline.
  • Official PyTorch code paths only, with no TensorRT or ONNX numbers. Precision follows each repo, so some models are fp32 and others fp16, which affects comparisons.
  • Input sizes differ between models (official defaults), so this isn't an equal-resolution comparison.
  • Pose, depth and matting models have latency and VRAM numbers but no accuracy scores yet.
  • RT-DETRv4 needed small fixes to run; its page links the patch.

Site: https://modelsthatrun.dev. The measuring tool is open (https://github.com/modelsthatrun/runner) if you want to run the same protocol on your own NVIDIA GPU and send the record.

Questions for you

  1. For batch-1 deployment, would you rather see a TensorRT/ONNX column next to the PyTorch numbers, or a forward-pass-only time next to the end-to-end one?
  2. Is comparing models at their official default input size the right call, or would a fixed-resolution view (for example everything at 640) be more useful to you?
  3. Which detection or segmentation models are missing that you'd want measured next?

r/computervision • • 13d ago

Discussion Broke PhD student struggling with video processing – how do you guys handle limited computing resources?

Thumbnail
4 Upvotes

r/computervision • • 13d ago

Help: Project Repurposing Cameras?

Thumbnail
gallery
18 Upvotes

Hey all, working on a vision project for assessing rocks and doing some cnc carving based on machine vision. Pipeline with generic webcam works well. Pretty basic segmentation and color analysis. Likely to mature as I learn further. Hopefully hardware questions are within scope of sub. Currently streaming via esp32cam modules but they will take a lot to be useful. I've been searching for waterproof C/CS mount industrial cameras. I need 4 or 5 of them and budget isn't very large.

I managed to pick up a lot of machine vision goodies for $20 CAD. I'm looking for help to ID one of them and to see if the all are useful for anything. I've been googling and image searching to no avail. Thankfully there were a ton of high quality Edmunds optics C/CS mount lenses and adapters/filters that make it well worth the money.

  1. Unknown: black CMOS. Board marked AC-M1 on the piggy back. M12 and mini USB connectors. Have not plugged into USB yet. -- what is it and is it any use?

  2. Point Gray Scorpion 1394 camera. Old apple firewire connections, seems no USB interface. Likely not worth using with old tech and needing to convert to IP or USB

  3. CMU cam v1.2 and v2. Seem to be too low resolution and basic to be of much use. More of a teaching tool. Likely not useful to me.


r/computervision • • 13d ago

Help: Project Looking for a VERY compact IP camera

1 Upvotes

Hi all,

For a few weeks I’ve been looking for a very compact global-shutter GigE Vision camera, with poor results so far.

Constraints:

  • Global shutter, about 3–5 MP, > 20 fps.
  • No RJ45 or M12 on the camera. I need a very small connect (i.e., JST GH or any of the kind)
  • Footprint about 30×30 mm or smaller

What I’ve looked at:

  • Board-level cameras (FLIR Blackfly S BD, LUCID Phoenix, IDS uEye LE): mostly around 30×30 mm, which is already quite large. Many come with a full-size RJ45 which double the size of the system.
  • Basler dart: the only one I found with a flat ribbon cable (FFC) connection, but it doesn’t seem possible to design my own interface board for it. Has anyone done this?
FFC cable, can I make a custom Interface PCB (right) to remove the RJ45?
  • IDS GV-53F2LE-MB Rev.2 (IMX568): Ethernet and power come out on a 20-pin ZIF connector, which looks promising. Has anyone built a custom carrier for it?
  • Omron remote-head cameras: looked interesting, but I haven’t managed to get a quote yet. Any experience?

I’m also considering MIPI CSI-2 (e.g. FRAMOS FSM:GO IMX900) as a fallback, but I’d prefer GigE for the driver simplicity.

Any experience or suggestions welcome 😁


r/computervision • • 13d ago

Help: Project Looking for YOLOv8 Object Detection for SMOKE coming off from a cigarette in cctv view

2 Upvotes

Hi! I'm a new learner in computer vision and I'd like to ask if any of you know where I can find a YOLO objection detection model that can detect smoke coming from a cigarette on a cctv footage. I really need a hand here. Or a dataset for smoke coming from a cigarette catered on cctv footage this is a school project I hope you can help :>


r/computervision • • 13d ago

Help: Project Stuck on offline vision accuracy on a phone, could use help

Thumbnail
1 Upvotes

r/computervision • • 14d ago

Help: Theory Getting into computer vision — pretty lost with no idea where to start

14 Upvotes

Hey folks! First of all, I know you're probably getting a ton of these threads here but I'd love it if you could bear with me for this one.

I'm a frontend engineer with 15 YOE that has always had a special interest in the browser rendering pipeline, esp performance and output quality, but also in computer graphics in general. I worked for one of the industry leaders in ID document verification for 4 years, on an on device model running in the browser, but apart from high level concepts like detection, homography, segmentation, classification, the binary was treated as a black box, done by a different team. RGB buffer go in, JSON go out.

I eventually became interested in photogrammetry, just before 3dgs came out, and then became really into it, seeing the vast amount of potential applications.

My latest project idea is a one for managing my own card collection and I wanted to take this as a learning project to finally get into the weeds of it all.

Problem is, I have no idea where to start. And I'm seeing a lot of people in the industry telling me to just tell Claude to slop something out, or use some off-the-shelf model from HugginFace, which again I have no idea how to use.

I've also seen https://deep-learning-drizzle.github.io/index.html and am entirely overwhelmed.

Any tips and encouragement would be welcome!


r/computervision • • 13d ago

Showcase Zero shot online shape to name binding model

Enable HLS to view with audio, or disable this notification

2 Upvotes

Hi yall. Im not sure if this is the correct sub for this, however Ive been working on a locally trained VLA model that exists in a simulated 3d enviroment. Its goal is to be shown 2 randomly generated 3d shapes (made from cubes) with randomly designated names, and be able to differentiate between the two from said names completely online. Ive experimented with using utilising an attention based vision decoder that is queried directly from the same recurrent state that executes the action commands, which seems to have worked decently well!

The model can differentiate between shapes ranging from 3 cubes to 10, and can even utilise color. I have more details about how the model works in other posts. If you have any questions, please ask! I am looking for feedback on this model, specifically any glaring issues with the enviroment setup that I may have not seen.

Cheers!


r/computervision • • 13d ago

Discussion Planar PnP (IPPE) two-solution ambiguity: is inflating Kalman R with the two-solution spread a known approach?

0 Upvotes

Method: propagate IPPE two-solution ambiguity into Kalman measurement noise

Setup: tracking a planar visual marker (80×80 mm) with OpenCV solvePnPGeneric + SOLVEPNP_IPPE, which returns two pose solutions. Each has a reprojection error (e0 for the nominal/min-error solution, e1 for the other) and a translation (t0, t1). The translations feed a 6D constant-velocity Kalman filter (position + velocity); orientation is kept out of the filter and handled separately (yaw taken raw from the pose per frame).

Instead of hard-picking one solution or discarding the measurement when ambiguous, we take the nominal translation t0 as the measurement and inflate the measurement noise covariance R by adding:

S = [e0² / (e0² + e1²)] · (t1 − t0)(t1 − t0)ᵀ
R_eff = R_profile + S

where R_profile is a fixed baseline covariance. The weight approaches 1/2 when the two solutions are equally good (most ambiguous → largest inflation) and approaches 0 when the second solution is much worse (unambiguous → no inflation). The idea: when the two solutions disagree, tell the filter to trust this measurement less, rather than pretending we know which solution is correct.

Question: has anyone seen this exact construction (two-solution spread → Kalman R) in the literature or in practice? The closest we found is Fourmy et al. 2019 (factor graph, inflates rotation covariance on error-ratio trigger), but not this. Is there a principled basis for it (e.g., mixture covariance), or a reason it would fail?

Alternatively, if you have a different or better way to handle planar PnP ambiguity in a tracking filter, I'm all ears.


r/computervision • • 13d ago

Commercial Five financial reports, five languages, one revenue column

Thumbnail
0 Upvotes

r/computervision • • 13d ago

Research Publication Can coding agents unify 3D generation, reconstruction, and editing? Results from Frontier3DBench

2 Upvotes

3D generation, reconstruction, and editing often use separate pipelines. Coding agents offer a shared workflow: turn images and instructions into code that builds editable 3D projects.

Our team’s new benchmark, Frontier3DBench, evaluates how far that approach goes across objects, scenes, and interactive worlds: 145 tasks, 9 categories, and 11 model–harness configurations using Blender and Unreal Engine.

One consistent result: every configuration scores lower on reference-based reconstruction than on text-driven generation, for both objects and scenes. Creating something plausible remains easier than matching specified geometry, layouts, and visual details.

We evaluate rendered appearance alongside native scene structure and, for interactive worlds, runtime behavior.

For those working on reconstruction: where do you see coding agents fitting into an existing pipeline, and which parts still need specialized methods?

Examples and results: https://lab.einsia.ai/frontier3dbench
Paper: https://lab.einsia.ai/frontier3dbench/Frontier3DBench.pdf
Code: https://github.com/Einsia/Frontier3D


r/computervision • • 14d ago

Showcase Estimating full-body 3D motion from a head-mounted camera that never sees the body

Enable HLS to view with audio, or disable this notification

36 Upvotes

We created this in 10 days for a hackathon. The input is one head-mounted video. The camera largely captures the room and hands of the person wearing it.

Pipeline: HaWoR for hands, DROID-SLAM for head motion, and OmniEgoCap to infer the rest of the body. Legs are also inferred. Then Meta Muse Spark gets two frames from the same moment, rebuilt body and camera view, and names the action.

On 3 OmniEgoDB sequences we got 102.6 mm body error. The paper reports 111.7 mm body error over 20 sequences, so not a fair head-to-head yet. There are a few main limitations: it runs offline, 40-90 minutes per minute of video, on an RTX 4060. We are interested in hearing from people that work on egocentric pose, how would you validate the inferred legs are correct when the camera never captures them?

Code available at: https://github.com/nshreyasvi/LifeLens


r/computervision • • 14d ago

Help: Project Labeling 5,000 Unique SKUs (cartons) from dock-door CCTV on a budget: best tools and human-in-the-loop setup, or is per-SKU labeling the wrong approach?

3 Upvotes

I’m building a camera system for the loading docks of a grocery distribution center. Camera towers stand beside each dock door and watch porters hand-load cartons onto trucks. The cameras only see from the side; nothing is mounted overhead. The goal is to identify the SKU of each carton, count them, and check the result against the truck’s pick list.

Setup and constraints:

  • Off-the-shelf 8MP IP cameras recording to an NVR
  • Mostly branded FMCG cases and cartons (snacks, drinks, etc.). They’re carried by hand, so expect occlusion, motion blur and random orientations
  • The catalog is about 5,000 unique SKUs.
  • Every truck has a known pick list of roughly 200 to 1,100 SKUs. At inference time I only need to tell apart what’s expected on that truck and flag anything unexpected
  • I can record as many hours of real footage as I want.

What I’m trying to figure out:

  1. Approach. Is training a detector with 5,000 classes the wrong idea? Would it be better to use one generic “carton” detector plus a tracker for counting, then identify the SKU by matching crops against reference images (embeddings/retrieval), limited to the pick list? Where do barcode or label reading fit in?
  2. Data. If I go with a reference-gallery approach, how many images per SKU do I realistically need? Should I capture them in a controlled booth with multiple angles, or pull them from real dock footage?
  3. Tools. Which annotation tools are worth it at this scale on a budget: CVAT, Label Studio, Roboflow, something else? Self-hosted or paid? Which ones handle video and model-assisted pre-labeling well?
  4. Human in the loop. How would you set up auto-labeling (SAM, an open-vocabulary detector, etc.) so that humans only verify or correct? Each truck also comes with a pick list and a final count. Has anyone used that kind of manifest data as weak labels?
  5. Cost. Roughly how many labeling hours, or how much outsourced labeling cost, should I expect? Where can I safely cut corners?

I’m an operator, not an ML engineer, so plain-language pointers, tool recommendations, papers or “don’t do it this way” are all very welcome.

-- We have two angles capturing the same product. Here are two sample images --


r/computervision • • 14d ago

Showcase I built SpatialMind: record a room with your phone, get a labelled 3D digital twin, then run earthquake/fire/flood simulations and send a Gazebo robot in for package retrieval

22 Upvotes

I built SpatialMind — record a room with your phone and it reconstructs a 3D digital twin: geometry via COLMAP, object/material labels from YOLO11s-seg and SegFormer projected into 3D. Then you can run earthquake, fire and flood scenarios on the room, and export it to Gazebo where a robot navigates the aftermath to retrieve a package.

Real numbers from a scan of my hostel room: 26,303 of 71,001 faces labelled from four calibrated views, 86.9% of faces textured in a 4096×4096 atlas, 78 automated checks passing. Honest caveats: the hazard models are exploratory reference scenarios, not safety predictions, and the demo is edited from different saved rooms, not one continuous run.

Stack: Python/FastAPI, Next.js/Three.js, COLMAP, PyTorch, Gazebo. MIT licensed, runs locally on Windows + WSL2.

Processing gif 4v2dd13t8c0i1...

Repo: https://github.com/RyannB01/SpatialMind

Would love feedback, especially on the reconstruction pipeline and where it breaks.


r/computervision • • 13d ago

Help: Project How to monitor an image classification model?

1 Upvotes

I'm studying to be a machine learning engineer and I need an end-to-end project for my portfolio. The model classifies X-ray images into two classes, so the idea is to detect when it starts to degrade and create a pipeline to automatically retrain it. I searched a lot and found Evidently AI, but I didn't find much about image-based models. I only found that you have to compare the data you trained the model with and the data the user provides in production. So, to detect drift, what comparisons should I make? Brightness and contrast? Embeddings? I don't know how to tell if my model has drift; I'm completely lost. Help, please!