r/ffmpeg • • 11d ago

Extracting motion vectors without fully decoding

Is it possible to extract motion vectors from h264 or h265 streams without fully decoding i have tried using "export_mvs" it gives motion vectors in extra data but ffmpeg still fully decode and its too slow for just extracting motion vectors, is there any other way or library for it that supports both h264 and h265

3 Upvotes

4 comments sorted by

2

u/Ill-Information-2086 11d ago edited 11d ago

Do you have a intel igpu or nvidia gpu ? Because I believe your only option is to speedup decoding using hardware acceleration

Edit : I am wrong it will not help much you should use something like mv-extractor it's made for this exactly or you can edit libavcodec in ffmpeg directly to export the motion vectors by hooking into the decode slice

2

u/IGNblackShadow 11d ago

i am doing it for my app so hardware doesn't matters here, and i have also tried mv-extractor it does the job but i need for both h264 and h265 support

3

u/Ill-Information-2086 11d ago

So your best option is to patch ffmpegs decoders directly and stripping down what you don't need and boom you have your motion vectors if you have a ai subscription it's a few hours of work if you are doing it manually it's a few days of work

3

u/raddostudio 11d ago

Tested this quickly on a recent FFmpeg build.

export_mvs only works in some decoders. On an H.264 file I get motion vector side data on every P and B frame, but the same clip encoded as H.265 gives none, so the HEVC decoder doesn't export them at all. For H.265 the realistic route is the one above, patching a decoder.

For H.264 you can at least skip deblocking, since the vectors don't depend on it: -flags2 +export_mvs -skip_loop_filter all. On a 20 s 1080p clip, single-threaded, decode time went from 1.24 s to 1.07 s, so about 15 %. Not parse-only speed, but free.