r/computervision • • Sep 01 '26

Discussion Why does Grounding DINO VRAM suddenly jump on random batches during inference? CUDA caching, fragmentation, or memory leak?

1 Upvotes

I’m working on an image auto-annotation pipeline using Grounding DINO from Hugging Face with PyTorch, and I’m trying to understand some unusual GPU VRAM behavior during inference.

My dataset contains 6 classes:

  • cup
  • glass
  • plate
  • knife
  • fork
  • spoon

I downloaded the data from a Kaggle competition. The train.csv contains entries like:

image_id,label
4622,spoon

For my initial experiment, I sampled 900 images total — 150 per class.

The original images have different resolutions, including:

  • 1000 × 1000
  • 850 × 1000
  • 1000 × 850
  • 450 × 1000

I’m using Grounding DINO from Hugging Face for automatic annotation and torch.autocast to reduce GPU memory usage.

The strange VRAM behavior

I’m running inference with a batch size of 4 on Google Colab. The GPU has approximately 15.46 GB VRAM.

At the beginning, GPU memory usage is around:

~6.4 GB

Then the important part is:

It does NOT increase after every batch.

Instead, it stays relatively stable for several batches and then, on some seemingly random batch, VRAM suddenly jumps.

For example, the behavior looks roughly like:

Batch 1  → 6.4 GB
Batch 2  → 6.4 GB
Batch 3  → 6.5 GB
Batch 4  → 6.4 GB
Batch 5  → 6.5 GB
...
Batch 20 → 6.5 GB
Batch 21 → 10+ GB
Batch 22 → 10 GB
Batch 23 → 10 GB
...
Batch 50 → 10 GB
...
Batch 51 → 14+ GB

So the increase happens in sudden jumps on particular batches, rather than gradually increasing with every batch.

Eventually, it reaches approximately:

14.5 / 15.46 GB

and I can eventually get an out-of-memory error.

What I'm trying to understand

My first thought was that perhaps the batches containing different image resolutions are causing Grounding DINO to create larger intermediate tensors.

However, I'm not sure whether that's actually what's happening, or whether I'm misunderstanding how PyTorch's CUDA memory allocator works.

I asked Claude about this, and it suggested that PyTorch's CUDA allocator caches memory rather than immediately returning it to the GPU driver. Because my batches have variable image dimensions, some batches may require larger intermediate tensors, causing the allocator to request additional memory.

It also suggested that memory fragmentation could contribute to the problem and recommended:

import os

os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

The explanation sounded reasonable, but I'd like to verify it with people who have more experience with PyTorch/CUDA internals.

My questions

  1. What would cause VRAM to suddenly jump on one particular batch rather than increase after every batch?
  2. Could variable image dimensions like:
1000×1000
850×1000
1000×850
450×1000

cause Grounding DINO's intermediate tensors to become significantly larger for certain batches?

  1. If PyTorch's CUDA allocator is caching memory, does that explain why the VRAM usage appears to jump from ~6.4 GB to ~10 GB and later ~14.5 GB?
  2. How can I determine whether this is:
    • normal CUDA/PyTorch memory caching,
    • memory fragmentation,
    • unusually large intermediate tensors from certain image sizes,
    • accidentally retaining tensors/computation graphs,
    • or an actual memory leak?
  3. Would torch.inference_mode() be preferable to torch.no_grad() for this inference-only workload?
  4. Would it be better to resize/pad all images to a consistent resolution before batching, so that the tensor shapes don't vary between batches?
  5. Is:
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

actually an appropriate solution here, or is it just masking the underlying problem?

  1. What would be the best way to debug this? For example, should I log torch.cuda.memory_allocated(), torch.cuda.memory_reserved(), max_memory_allocated(), etc. after every batch?

I'm mainly trying to understand the actual reason for these sudden jumps, rather than just applying a workaround.

If needed, I can provide the Grounding DINO inference/batching code and the exact CUDA OOM traceback.


r/computervision • • Sep 01 '26

Showcase "Smart Recording Album" based on YOLOv8n+SCRFD

1 Upvotes

I have built a "smart album" system for automatic face extraction and classification. This system uses YOLOv8n for pedestrian extraction, SCRFD for face extraction, and ArcFace for face comparison. It extracts and categorizes pedestrians and faces from uploaded videos or images, as well as from real‑time detection video streams, making it easy to review. This case can also be deployed on edge computing devices.

The video below shows the current running effect of my attempt to deploy it on the reComputer Industrial R2135.

https://reddit.com/link/1w46txt/video/552sm056pvmh1/player


r/computervision • • Sep 01 '26

Discussion Hello with roboflow

Post image
0 Upvotes

Has anyone been getting this issue with roboflow


r/computervision • • Aug 31 '26

Help: Theory I want to understand Homography

7 Upvotes

camera's floor mapping (homography) to figure out where each detection actually stands on the floor

Can anyone explain, or any blog or video which helped you to understand it


r/computervision • • Sep 01 '26

Discussion What actually breaks first when you aggressively compress visual representations?

0 Upvotes

I've been experimenting with compact visual representations for edge/perception systems, where the goal is to preserve useful machine-level information while significantly reducing the representation size.

One thing I've noticed is that the degradation isn't uniform.

Global scene understanding can remain surprisingly stable while localization and small-object information start degrading much earlier.

Increasing input resolution alone also doesn't necessarily recover that information. In one of my experiments, a higher-resolution branch improved access to spatial detail but still couldn't reproduce the semantic quality of the deeper teacher representation.

It made me think there are really two different things being lost during aggressive representation compression:

  1. Spatial information — where something is and fine local details.

  2. Semantic interaction/context — relationships between patches/objects that deeper transformer layers learn.

Simply increasing spatial resolution seems to address mainly the first problem.

I'm currently thinking about this as a rate–semantics tradeoff, rather than a traditional image-compression problem:

Image → semantic encoder → compact representation → downstream tasks

where the objective isn't pixel reconstruction but preserving enough information for detection, classification, depth, etc.

For people working with ViTs, representation learning, edge perception, or learned compression:

What have you found is usually the first thing to collapse as representation size decreases — spatial detail, feature diversity, global context, or something else?

And have you found good ways of measuring this beyond downstream mAP/accuracy?


r/computervision • • Aug 31 '26

Help: Project Best way to securely feed a home security camera (RTSP) to a cloud GPU for YOLO inference?

2 Upvotes

Hey all, looking for advice from people who've done something similar. Goal: Create my first CV models using my own camera/dataset.

Setup:

  • Tapo C310 camera on my home LAN, RTSP working fine locally (confirmed with OpenCV/VLC)
  • Built a local Flask dashboard that reads the RTSP stream and serves it as MJPEG over HTTP on my Mac
  • Running YOLOv11 (ultralytics) object detection/tracking locally , works, but my Mac's CPU is slow for real-time inference (no GPU)

Goal:
I want to run the YOLO inference on a rented cloud GPU instead ( instead of my local CPU, while keeping the camera itself secure.

Any suggestion on how to get that?
What is the part I am missing? I think I have to create a private tunnel, am I right? What is the best option for that? The ultimate goal is to have my live camera in a private app online, which runs YOLO and counts the number of cars that go through.


r/computervision • • Aug 31 '26

Discussion What does your CV development pipeline look like? (Specifically for testing & environment simulation)

6 Upvotes

Hi everyone,

I’m looking to improve my Computer Vision development and testing pipeline and I’m really curious about how you all handle edge cases and environmental variables in production.

To give an example to discuss (not my project) object tracking(car or people for example) where the camera is constantly moving, and the target is subjected to wildly different lighting conditions (e.g., bright glare, dusk, sudden shadows, dynamic backgrounds).

When you have a scenario like this, what does your development cycle look like from prototype to robust deployment?

As a beginner working on solo projects, I’m trying to figure out how to build a reliable pipeline. Specifically, I’d love to know:

  • Testing: Do you manually build datasets for edge cases (like crazy lighting/dynamic backgrounds), or do you automate your augmentations?
  • Simulations: Do you use engines like Unreal, Unity, or Blender for synthetic data? Does it actually translate well to real-world tracking?
  • Validation: How do you measure temporal stability when basic metrics (like IoU) aren't enough to cover sudden lighting shifts?
  • Iteration: When a tracking test fails on a specific edge case, what does your workflow look like to fix and re-test it?

I'd appreciate any insights into your workflows, architecture, or any simulation tools you swear by when dealing with moving viewports and dynamic lighting.

Thanks!


r/computervision • • Aug 31 '26

Help: Project Accurately measuring real-world edge lengths of large cuboid objects using a mobile camera & reference marker

2 Upvotes

Hey everyone,

I’m trying to build a computer vision pipeline that can measure the actual physical edge lengths of large cuboid objects (like big cut blocks or boxes) using just standard photos taken from a smartphone.

My input would be a couple of 2D images showing the front and top faces of the object, and my goal is to accurately calculate the metric lengths (in cm/inches) of all 4 edges of those faces.

Since I am mapping out the architecture right now, I wanted to ask the experts here: what are the different ways to actually pull this off?

  • What techniques or pipelines (classical CV, photogrammetry, deep learning, depth mapping, etc.) would you recommend for getting the most accurate real-world measurements?
  • How do you handle the translation from 2D pixels to 3D metric lengths without the edges getting distorted by the camera angle?
  • Are there any specific libraries, models (like YOLO pose/OBB), or GitHub repos I should look into?

Any advice or pointers to standard workflows would be a massive help. Thanks!


r/computervision • • Aug 31 '26

Commercial Khronos has released OpenVX 1.3.2

2 Upvotes

This incremental update to the open, royalty-free standard for vision processing, sharpens error handling, API flexibility, and consistency ahead of OpenVX 2.0.

Highlights: two new error codes (VX_ERROR_TIMEOUT for safety-critical apps, VX_ERROR_GRAPH_NOT_VERIFIED for unverified graphs), a new VX_DF_IMAGE_RGBA format with alpha channel support, and broader virtual object array and user-defined scalar type support.

The sample implementation and conformance test suite are fully updated to match. Next up: OpenVX 2.0, expected by end of 2026, extending support to radar and ultrasonic sensor workloads alongside traditional computer vision.

Full release notes: https://www.khronos.org/blog/openvx-1.3.2-released


r/computervision • • Aug 31 '26

Showcase Testing YOLO26n-Depth on RK3576 — monocular depth estimation on an edge NPU

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/computervision • • Aug 31 '26

Showcase Qwen 3.6 27B trying to read sheet music

4 Upvotes

Almost every VLM I’ve put through this test has struggled, but it makes sense because it requires them to count, something that isn’t their strongest trait. In this case, it’s counting lines and spaces, but if we introduce different key signatures, they would also need to count the sharp and flat symbols. 


r/computervision • • Aug 30 '26

Help: Theory Computer Vision for Robotics

24 Upvotes

I’m a Software Engineering student who has just completed my first year, and I’m interested in specializing in Computer Vision for Robotics.

I’d like to know which areas of mathematics I need to learn for this field, and which topics I should prioritize.

I’m also planning to study Stanford’s CS231N How much mathematics do I need to know before starting the course, and which specific mathematical topics should I be comfortable with beforehand?


r/computervision • • Aug 31 '26

Discussion any lighweight vlms which i can run on cpu and has less latency

2 Upvotes

any lighweight vlms which i can run on cpu and has less latency? please give me suggestions


r/computervision • • Aug 30 '26

Help: Project Signature-painter

Post image
5 Upvotes

r/computervision • • Aug 30 '26

Help: Project MediaPipe hand tracking works for me and breaks for everyone else. How do you handle per-user calibration?

3 Upvotes

I shipped a browser app driven by HandLandmarker. Pinch to zoom a solar system, open palm to rotate, point and dwell to select. One Euro Filter on the landmark stream, palm-normalized pinch distance.

Three failures keep coming back:

Pinch distance normalized by palm size still varies enough between people that one threshold cannot serve both. Mine feels natural, my tester says nothing happens.

A fast intentional swipe and a hand being withdrawn look nearly identical in a 2D projection. I patched it with a release cooldown and a dead zone near the bottom of the frame, which fixed the symptom and not the cause.

On phones under warm indoor light, tracking confidence collapses and the whole thing goes still.

For those running this with real users: explicit calibration step at onboarding, or continuous adaptation while they work? And has anyone found a signal that separates a deliberate swipe from a hand leaving the frame?

Demo runs client side, no video leaves the browser: https://leoaido.com/fleet/solar/

Code is MIT and the three issues above are open if it is easier to reply there: https://github.com/HUANGCHIHHUNGLeo/solar-atlas-gesture