Repository navigation
How to seek to and cut from a frame in ffmpeg? #1216
Description
This is the age-old question that I have not yet found a definite answer to:
How does seeking (-ss) actually work in ffmpeg?
More specifically:
- How to seek to frame number N when cutting, so that the output begins at input's frame N? (lossy encoding)
- How to seek to a keyframe K when cutting, so that the output begins at (and includes) that keyframe K? (lossless
-c copy) - How to do this consistently across most of files supported by ffmpeg?
This has been discussed in #13 #1087 #126 #330 #1513 #1585 #1855. This also affects smart cut.
To complicate further:
- The HTML5 video player in LosslessCut sometimes renders the wrong frame at a particular timestamp. This also affect snapshot capture. Avidemux cuts more accurately than losslesscut #1786 Duplicate frame preview at keyframes #1616 Saving .jpeg with FFmpeg sometimes saves frame +1 #1487
- This can be worked around by converting to supported format (to use the FFmpeg-assisted playback).
- For some files, FFmpeg completely ignores certain keyframes when outputting to mkv, but it does not when outputting to mp4 Issue with repeating segments at start of each export & merge clip #2804
Concerns:
- Some files have variable FPS, and frame timestamps (DTS/PTS) could vary based on things like the clock of the recorder (?), but in such case you would think that the frame timestamps returned by
ffprobe show_entries packet=pts_time,flagsare correct. Or should we instead usebest_effort_timestamp_time? - How does seeking work when we have audio with "audio keyframes" (or audio+video) Cut at the nearest audio (key-) frame instead of the next one #1433
Options:
- For keyframes: seek the start cut-point to a few frames after the keyframe (sometimes 1, sometimes 2, sometimes 3, sometimes even more). this is now implemented as an export option
- Use a timecode that is in the "middle" of a frame's lifetime
- Ask the user to choose mp4 instead of mkv output
Activity
iopq commentedon Aug 4, 2022
when I used normal cut, but not keyframe cut if you cut it wrong you get black frames if you cut from the wrong place because the keyframe got cut out
Why do I have to use the normal cut? Because I want to cut off some trailing frames at the end, since that's not an issue (we have the previous keyframe, just cutting more frames off won't always be problematic)
So you may want to treat "beginning" segments and "ending" markers differently
DGrv commentedon Apr 6, 2023
Hi,
Thanks for your reply on #717. I deleted my post because I said something wrong.
Anyway I do not know where to post this but I guess this is the same topic.
I did a lot of test and the results are depending on a lot of parameters, especially the framerate and keyframes and certainly other thing that I have no clue.
My goal is to use the split method from losslesscut to cut exactly a video in 2. However this does not work well since it is not cutting exactly at the same place for both segments. Meaning that if you concat the 2 segments you will have at the trim position, duplicate frames, not always near each other.
I try to reproduce the behavior of losslesscut and try other things. I used smart cut.
In summary I found that :
Here a reproducible example with bash
All files and code example
Maybe this can be useful ...
in.mp4 is from a Gopro11, codec was converted to h264 (it was h265), this is the only thing I did with the original video.
Here the code and console output:
Lets take a random keyframe from in.mp4:
$$ kfn=20kf=$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -20 | tail -1 | perl -pe 's|frame,video,0,1,.?,(.?),.*|\1|')
$$ kf=3.803792Lets select keyframe at 3.803792. Here are the timestamps of the all the frames from in.mp4 around this keyframe.
$$ ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color 3.8037923.720375
3.737083
3.753750
3.770417
3.787125
3.803792 KF
3.820500
3.837167
3.853833
3.870542
3.887208
Lets compare 2 methods of split: actual losslesscut 3.53.0 and another one
Method 1 - Losslesscut
if I use the split method at timeframe 3.803792 it will run
$$ ffmpeg -stats -v error -i in.mp4 -t 3.803792 -map 0:0 -c:0 copy -map 0:1 -c:1 copy -map_metadata 0 -movflags use_metadata_tags -movflags +faststart -ignore_unknown -f mp4 -y in_seg1.mp4$$ ffmpeg -stats -v error -ss 3.803792 -i in.mp4 -avoid_negative_ts make_zero -map 0:0 -c:0 copy -map 0:1 -c:1 copy -map_metadata 0 -movflags use_metadata_tags -movflags +faststart -ignore_unknown -f mp4 -y in_seg2.mp4Timestamps of in_seg1.mp4:
$$ ffprobe -v error -select_streams v -show_frames -print_format csv in_seg1.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color 3.8037923.720375
3.737083
3.753750
3.770417
3.787125
3.803792 KF
we see that it is taking 1 or 2 frames too much,always at least the KF
Method 2
The only good solution and reproducible that I found it to set the -t to 2 frames before the keyframe wanted
kf2=$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | perl -pe 's|frame,video,0,.,.?,(.?),.*|\1|' | grep --color -B 2 3.803792 | head -1)
$$ kf2=3.770417Lets take then 3.770417 instead of 3.803792
$$ ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color 3.7704173.687000
3.703708
3.720375
3.737083
3.753750
3.770417
3.787125
3.803792 KF
3.820500
3.837167
3.853833
$$ ffmpeg -stats -v error -i in.mp4 -t 3.770417 -map 0:0 -c:0 copy -map 0:1 -c:1 copy -map_metadata 0 -movflags use_metadata_tags -movflags +faststart -ignore_unknown -f mp4 -y in_seg1b.mp4$$ ffprobe -v error -select_streams v -show_frames -print_format csv in_seg1b.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color 3.7704173.687000
3.703708
3.720375
3.737083
3.753750
3.770417
3.787125
now it it stopping just before the keyframe 3.803792.
Concat seg1 and seg2 from each method to compare.
$$ (echo file \'in_seg2.mp4\' & echo file \'in_seg1.mp4\' ) > list.txt$$ ffmpeg -stats -v error -safe 0 -f concat -i list.txt -c copy -y out.mp4$$ (echo file \'in_seg2.mp4\' & echo file \'in_seg1b.mp4\' ) > listb.txt$$ ffmpeg -stats -v error -safe 0 -f concat -i listb.txt -c copy -y outb.mp4$$ ffmpeg -stats -v error -i in.mp4 -i out.mp4 -filter_complex blend=all_mode=difference -c:a copy -y outdiff.mp4$$ ffmpeg -stats -v error -i in.mp4 -i outb.mp4 -filter_complex blend=all_mode=difference -c:a copy -y outdiffb.mp4Lets check the frame MD5 to see the difference between the 2 methods:
$$ ffmpeg -stats -v error -i in_seg1.mp4 -c copy -f framemd5 -y in_seg1.md5$$ ffmpeg -stats -v error -i in_seg1b.mp4 -c copy -f framemd5 -y in_seg1b.md5$$ ffmpeg -stats -v error -i in.mp4 -c copy -f framemd5 -y in.md5$$ ffmpeg -stats -v error -i out.mp4 -c copy -f framemd5 -y out.md5$$ ffmpeg -stats -v error -i outb.mp4 -c copy -f framemd5 -y outb.md5$$ lastframe=df14f6ef759edaf25ce112301ca804d9lastframe from in_seg1b.mp4 is df14f6ef759edaf25ce112301ca804d9
Lets find this frame in in_seg1.mp4, and in in.mp4
in_seg1.md5
$$ cat in_seg1.md5 | grep --color -B 5 -A 2 df14f6ef759edaf25ce112301ca804d91, 178176, 178176, 1024, 345, 6f543ce079298bd464e179b2f4ddd05a
0, 89289, 90891, 400, 21325, 11588ec4c0e943d8df6c1edb8de2d999
1, 179200, 179200, 1024, 344, 6260929978abe1af0deca162736bea8a
0, 89690, 90090, 400, 11299, c7d62f4b3cdbedff9a7a6391824facc6
0, 90090, 90490, 400, 13613, 8358c34183d7ab6f088ea56cd655f42b
1, 180224, 180224, 1024, 315, df14f6ef759edaf25ce112301ca804d9
0, 90490, 91291, 400, 284176, e9c6c1edea17faf8157fb42dd4b78f46
1, 181248, 181248, 1024, 306, 877029df74dbb5028ebdad7e6e664494
in_seg1b.md5
$$ cat in_seg1b.md5 | grep --color -B 5 -A 2 df14f6ef759edaf25ce112301ca804d91, 178176, 178176, 1024, 345, 6f543ce079298bd464e179b2f4ddd05a
0, 89289, 90891, 400, 21325, 11588ec4c0e943d8df6c1edb8de2d999
1, 179200, 179200, 1024, 344, 6260929978abe1af0deca162736bea8a
0, 89690, 90090, 400, 11299, c7d62f4b3cdbedff9a7a6391824facc6
0, 90090, 90490, 400, 13613, 8358c34183d7ab6f088ea56cd655f42b
1, 180224, 180224, 1024, 315, df14f6ef759edaf25ce112301ca804d9
in.md5
$$ cat in.md5 | grep --color -B 5 -A 5 df14f6ef759edaf25ce112301ca804d91, 178176, 178176, 1024, 345, 6f543ce079298bd464e179b2f4ddd05a
0, 89289, 90891, 400, 21325, 11588ec4c0e943d8df6c1edb8de2d999
1, 179200, 179200, 1024, 344, 6260929978abe1af0deca162736bea8a
0, 89690, 90090, 400, 11299, c7d62f4b3cdbedff9a7a6391824facc6
0, 90090, 90490, 400, 13613, 8358c34183d7ab6f088ea56cd655f42b
1, 180224, 180224, 1024, 315, df14f6ef759edaf25ce112301ca804d9
0, 90490, 91291, 400, 284176, e9c6c1edea17faf8157fb42dd4b78f46
1, 181248, 181248, 1024, 306, 877029df74dbb5028ebdad7e6e664494
0, 90891, 92893, 400, 53891, e47f5bf0d0bb4c82a10e2334f5ad6b09
1, 182272, 182272, 1024, 376, e0aeb413bf281fbd1fba0e866b08ba19
0, 91291, 92092, 400, 18068, aa6cf20b0cb7aee8265ae8acaf1b0d54
Lets check the next frame after lastframe in in.mp4 and find it again in the combine seg1 and seg2 in each method.
$$ lastframein=e9c6c1edea17faf8157fb42dd4b78f46lastframe from in.mp4 is e9c6c1edea17faf8157fb42dd4b78f46
out.md5
$$ cat out.md5 | grep --color -B 5 -A 5 e9c6c1edea17faf8157fb42dd4b78f46outb.md5
$$ cat outb.md5 | grep --color -B 5 -A 5 e9c6c1edea17faf8157fb42dd4b78f46This one is not found in either of both ...
Lets check a bit more in detail in.mp4
Duplicates in out.mp4
$$ cat out.md5 | awk '{print $6}' | grep --color -A 15 df14f6ef759edaf25ce112301ca804d9 | sort | uniq -d877029df74dbb5028ebdad7e6e664494
b2da0dda190820a8abb60b01faa2af39
df14f6ef759edaf25ce112301ca804d9
e0aeb413bf281fbd1fba0e866b08ba19
e47f5bf0d0bb4c82a10e2334f5ad6b09
Duplicates in outb.mp4
$$ cat outb.md5 | awk '{print $6}' | grep --color -A 15 df14f6ef759edaf25ce112301ca804d9 | sort | uniq -ddf14f6ef759edaf25ce112301ca804d9
[-]How to seek to a frame in ffmpeg?[/-][+]How to seek to and cut from a frame in ffmpeg?[/+]mifi commentedon Feb 14, 2024
next nightly build will have a new Export Option called "Shift all start times", it can be used to automatically shift all segment start times forward by one or more frames when cutting. This can be useful if the output video starts from the wrong (preceding) keyframe.
DGrv commentedon Dec 2, 2024
Is this already implemented ? I was looking everywhere in v3.64.0.0 but did not find it ? did I missed it ?
Thanks :)
mifi commentedon Dec 4, 2024
You need to enable the export cofnrim dialog. Then it should be under «advanced»
DGrv commentedon Dec 7, 2024
I tried the options and did not get what I wanted but I am actually not sure what you expect and which condition.
Is it supposed to fix this #717
decio commentedon Mar 11, 2026
I encountered this issue about the export file starting from a previous keyframe. Interestingly, even with a '+3 frame shift' in the export option, it still didn't solve the issue for me, but manually setting the marker one full keyframe ahead did. Perhaps a UI toggle to 'Offset Marker by +1 Keyframe' could be a future workaround for those of us with 'stubborn' files?
mifi commentedon Oct 18, 2026
found while reviewing #3109: for input formats without
AVFMT_SEEK_TO_PTS(e.g. matroska, but not mp4/mov) where a stream has b-frames, ffmpeg subtracts 3/23s (~0.13s) from the input-ssbefore seeking (fftools/ffmpeg_demux.c). so a start on a keyframe, or up to ~0.15s after it, lands on the previous keyframe, and with-c copy+-avoid_negative_ts make_zerothat whole extra GOP ends up in the output. this probably explains #2804 (mkv affected, mp4 not). a possible workaround:-copyts, input-ssat the cut start, and output-ssat the decode time of the last keyframe at or before the startCompensate ffmpeg's B-frame seek offset for Matroska keyframe cuts
mifi commentedon Oct 18, 2026
the mkv/webm case (b-frames, cut starting one keyframe too early) should be fixed by #3131: losslesscut now compensates for ffmpeg's 3/23s seek offset with
-ss+-itsoffset. it's in the nightly https://mifi.no/llc/nightly/ but only tested on synthetic test files so far, so feedback with real files is welcome. other formats without AVFMT_SEEK_TO_PTS (e.g. mpegts, avi) and cuts with external files added aren't covered yetSpaarpott commentedon Oct 19, 2026
I tested a Samsung A51 HEVC mp4 file, in which the nightly still takes the keyframe before.
I also don't see any change to 1487 Saving .jpeg with FFmpeg sometimes saves frame +1.
I tested Nightly 19-10-2026, 06:33:03 versus 3.69 with two AVC MP4 files from a Panasonic DC-TZ202.
Al exported pictures with both versions are taken from the same frames, sometimes correct, sometimes +1.
I noticed, the correct ones, are from when the preview shows the same frame, as the one before.
Sort a like 1616 Duplicate frame preview at keyframes, but with my files, it happens every few frames, and not at keyframes. So the keyframe-1,keyframe,keyframe+1 are all different!
But keyframe-2 and keyframe-1 have the same preview, and keyframe+1 and keyframe+2 have the same preview.
However, the ffmpeg saved pictures are all different. Maybe the first thing to do, is see if you can get the preview working, so all frames are different?
Keyframe cutting is already correct for these files and my Samsung A56, as in it doesn't take the keyframe before...
But I noticed now that in the cut from the TZ202, the first 4 frames are a copy of the "keyframe+1".
The fifth frame is the first frame changing. So frame 0-3 are the same. It is not happening when you cut-in from the start.
I went back in files as far as 2022 and it is already in those files, so this is happening for a long time already in previous versions also. Never noticed though.
This is only the case with the above camera files. With my HEVC Samsung A51 or A56 galaxy files, only the first two are duplicate preview.
I don't think there is an issue for this yet. It might be like #1973, but that one is closed due to no reply.
I've opened a new issue for this: #3189
I'll email to your gmail address, a link from which you can download the files for 6 days to reproduce and test.