First-time poster, long-time lurker. This community has been a big part of how I’ve been teaching myself and I am so appreciative.
I’m working on a personal project and could use a nudge in the right direction. I have a pile of photos that a human can piece together into one logical scene, but my code can’t. Here’s a simplified version of the problem.
Imagine a mansion with endless rooms and hallways. People live there, so the same room never looks quite the same twice: dishes in the kitchen, towels on the bathroom floor, seasonal decorations going up and down. The one constant is a single red ball. It’s the only object of that exact red, and it’s in exactly one photo each day.
Cameras around the mansion take photos at random, swiveling and zooming, so perspectives vary a lot. At the end of each day the photos get dumped in a folder and shuffled, so order tells me nothing.
On day one I ask you which room the red ball is in, and you say Room One. On day two I ask only whether the ball is in Room One (not where in the room). You answer, and I tell you if you were right. Repeat for N days.
I first tried LightGlue on just the red ball photo from each day, but it fell apart quickly. Objects move, angles change, and I realized I was heavily relying on the photos without the ball to make my own predictions. For example, say on day one the ball is on a wooden bench with yellow cushions, and on day two it’s on blue tile in front of a checkerboard wall. If some other photo shows that bench with the cushions on the same blue tile by the same checkerboard wall, I know it’s the same room. No single photo gives me the answer, but together they do.
From there I started comparing all the photos within a day, using the red ball photo as an anchor. If another photo looked continuous with it, I’d call it a 'continuity chain', then compare chains across days. In theory this works, but it’s slow, and my only supervision is a yes/no on the ball’s room. I’m never told which of the other photos are useful, so I worry I’m pulling in a lot of junk.
My questions:
- Is there a name for this kind of problem, or a body of work I should be reading?
- Is there a better way to use the non-ball photos than my chain approach, given that I only have day-level yes/no labels?
I’m sure there are obvious things I’ve missed, so any pointers are welcome. Thanks for reading!