How can we improve Premiere Pro?

← Premiere Pro

"I-frame first" fast seeking for inter frame compressed video files

When you are "seeking," (which means QUICKLY looking through a long video by clicking and dragging the playhead with the cursor,) there is sometimes very high "prefetch latency," meaning that frames will load slowly, resulting in very few frames being shown. This can make it very difficult to find what you're looking for.

But when seeking, perfect frame accuracy is not nearly as important as seeing what you're doing. Therefore, to reduce the "prefetch latency," I suggest that the nearest I-frame is loaded and displayed FIRST, (because I-frames are always the fastest to load) while the exact B or P frame is still being slowly decoded. As soon as it's decoded, it can be displayed, replacing the displayed I-frame.

If the user is seeking quickly, there might not be enough time to load the P and B frames before the user has already moved the playhead nearer to another I-frame. This is fine! The I-frames should always take priority.

In VLC, there is a similar feature called "fast seek."

Combined with my "ghost playhead" feature suggestion, it would be made clear to the user exactly what is going on. (Linked below)

However, it would probably be important for fast seeking to be optional, and perhaps even off by default, as it may confuse some users.

NOTE: This works best if only ONE video source is visible. With more than one video on the timeline, the I-frames will probably not be in the same spots. Nevertheless, it COULD still work, but there would have to be multiple ghost playheads. This might be too difficult to code, and confusing and frustrating for users, who won't understand why premiere is showing video frames out of lockstep, especially in cases where one video has high prefetch latency, and the other is low.

LINK TO FULL EXPLANATORY VIDEO: https://youtu.be/XeFezzmG22c

LINK TO GHOST PLAYHEAD USERVOICE ENTRY: https://adobe-video.uservoice.com/forums/911233-premiere-pro/suggestions/42186274-a-ghost-playhead-which-always-marks-the-timeline

164 votes

We're glad you're here

Please sign in to leave feedback
Signed in as (Sign out)
Close
Close

We’ll send you updates on this idea

TaranVH shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →
Under Review  ·  AdminPatrick Palmer (Principal Product Manager, Adobe DVA) responded  · 

Hi Taran and everyone subscribed to the thread,

first of all: thanks for the incredibly detailed suggestion and the great conversation in the comments. While making such a change for the playback engine has a ton of implications and therefore isn’t done ad hoc, we’re certainly going to create a POC for it and will evaluate if there’s a way to put this into a production build for a test drive in an upcoming beta version.

I’ll keep you posted.

Cheers,
Patrick

20 comments

We're glad you're here

Please sign in to leave feedback
Signed in as (Sign out)
Close
Close
Submitting...
An error occurred while saving the comment
  • Ariel Brener commented  ·   ·  Flag as inappropriate

    Wow! this is an amazing idea! it can be a real game-changer in many user's workflows.
    I have lots of footage that premiere can handle in simple playback (including sony alpha and DJI's hi compressed formats) but when I try to SKIM it or play it back in x8 and above it becomes unusable. - So I use PROXY workflow which has a lot of downsides.
    If this genius suggestion will be implemented it will be a huge step above other NLEs!

  • David Johnson commented  ·   ·  Flag as inappropriate

    SUCCESS. At least Taran ... at least via Patrick, Adobe is listening, and the olive branch is out there.

  • Pierre Louis Beranek commented  ·   ·  Flag as inappropriate

    @TaranVH
    I see what you're saying. I made an official FR for those 2 additional ideas in case it helps Adobe solve this issue.

    You can check it out and vote here: https://adobe-video.uservoice.com/forums/911233-premiere-pro/suggestions/42773999-improve-scrubbing-fast-forwarding-performance-with

    What I most often have to do is quickly scrub through lots footage since even though I put it in my Timeline, I don't know exactly where individual clips from my first rough cut are. Once I start assembling clips I select from my first rough cut, then I have a better idea of what is where in the final cut. ;)

  • TaranVH commented  ·   ·  Flag as inappropriate

    @Pierre Louis Beranek
    Those are some really interesting ideas that I had not considered. They could definitely help in some situations.
    But, they would not fix the problems that I pose in the explanatory video. In my example, the "where gaming begins.m4v" AMD video is in the source monitor. It has no effects on it at all. But it still has very high (slow) prefetch latency, because of the way it was encoded. The "downloading high sierra (mac).mp4" is even worse. Again, it had no effects, and was in the source monitor at 1/2 resolution.

    I don't really need to scrub through footage quickly to find stuff once it's already been placed onto the timeline. I already know where everything is on my timeline... I put it there!

    You might think that automatically lowering the quality to 1/4 or 1/8 or even 1/16 would help, but in my experience (all 8+ years of it) that only helps with all-I-frame video, like cineform, .r3d, and .braw. For H.264, I believe that a decoder still needs to load the entire frame BEFORE it can determine what it would look like at a lower resolution. Like, go look up "DCT zigzag pattern" and you'll see why. Each of those 8x8 blocks of pixels is a "macroblock." The decoder can't just SKIP 3/4 of the pixels. It needs to go through ALL of them or it'll get the values wrong.

    It is possible that only loading and displaying the luminance data would be faster than loading and displaying the luminance + chrominances, but I don't know enough about H.264, H.265, etc, to say for sure.

  • Pierre Louis Beranek commented  ·   ·  Flag as inappropriate

    Hi Taran,
    I did watch the video you link to in your post, but if your brilliant idea in this FR to display the I-frame first were applied, wouldn't it speed up video scrubbing to the point that displaying a ghost frame would no longer be necessary in most cases? If anything, I'd rather Pr do the following: anytime the user's system can't keep up because the footage has too many effects, Pr could show the footage without any (or without certain processing heavy) effects while scrubbing and then update to the frame under the playhead with all effects as soon as processing catches up. Whenever this happens, the playhead could change to another color, perhaps red. When the frame is properly rendered with all effects, the playhead would turn back to its current blue color. To best suit all users, this feature could be added to the Timeline's hamburger menu so users can turn it on/off.

    Another (additional or alternative) solution might be to provide adaptive quality playback while scrubbing. Pr already gives us options for Playback and Paused resolution, as well as a wrench menu on/off option for "High Quality Playback". What if there was an option for adaptive quality scrubbing (possibly also available when playing video back at 2x, 4x or faster), that would allow users to set the variable quality anywhere from Full only all the way down to 1/16th?

    Perhaps one or both of these solutions would take care of the delays you were seeing in your video?

    What do you think?

  • TaranVH commented  ·   ·  Flag as inappropriate

    @Pierre Louis Beranek
    You clearly did not watch the accompanying video. I recommend that you watch the whole thing.
    The delay from prefetch latency can be enormous -- multiple seconds, in some cases.
    https://youtu.be/XeFezzmG22c?t=378
    In those cases, it is impossible to know if the frame you're looking at, is /actually/ the frame that you have selected. Especially with screen capture footage. Thus, the ghost playhead would show which frame IS being shown. Premiere already knows which frame it's showing... it just has no way of telling the user.

  • Pierre Louis Beranek commented  ·   ·  Flag as inappropriate

    @Antoine Seeing the wrong frame for a fraction of a second while scrubbing wouldn't be a big deal. Probably no one would ever even notice the difference. As soon as the user stops scrubbing, the correct frame would show up perhaps 1/10th of a second later?

    Personally I think there's zero need for a 'ghost' playhead. Trying to program that into Pr would be utter nonsense, and the 'with and without fast seek.mp4' video the OP posted himself proves it. Just look at the delay when he stops scrubbing the VLC video between the last static image and the image just before it. The gap is so short, eyeballing it I'd say it's just 1 frame long (1/30th of a second). Why would anyone need to see a 'ghost' playhead for a 1-3 frame delay?

  • Antoine Autokroma commented  ·   ·  Flag as inappropriate

    Just to be clear Taran, you would be OK for Premiere Pro to show you the wrong frame ? Showing you the previous closest I frame instead of the real frame

  • Sander Skjegstad commented  ·   ·  Flag as inappropriate

    A simple solution for the multiple playheads issue would be to have the lines draw inside each media item. That way you automatically get one for each.

  • John Pooley commented  ·   ·  Flag as inappropriate

    Yes this is desired, but the FramePrefetchLatency bug is a bug and let's call it what it is. You shouldn't get multiple seconds of Prefetch lag before clearing the Media Cache, and 1ms afterwards. This wouldn't fix that entirely, just as hardware decoding didn't fix it. Adobe needs to get to the root of the problem.

  • Thomas Berglund commented  ·   ·  Flag as inappropriate

    @TaranVH Cool. Let us know if you notice a difference. I have tested this on both Windows 10 and macOS 10.15.7, with Premiere Pro 14.6, and even with a 2 hour long i-frame only based video the amount of frames displayed while scrubbing, when "Play audio while scrubbing" is turned off compared to on, is substantial.

    The Adobe Premiere team also changed the way Premiere Pro handles pre-roll for playback of audio, which dramatically speeds up start of playback for audio when using clip based audio effects. This changed was introduced in Premiere Pro 14.5.

    "Faster audio pre-roll in Premiere Pro provides responsive playback for large projects or projects that use a lot of audio effects. No more waiting for playback to start on macOS and Windows."

    This new feature can also be turned off/on under "Audio Preferences > Use legacy audio effect playback pre-roll".

  • Thomas Berglund commented  ·   ·  Flag as inappropriate

    @TaranVH Have you tried to turn off "Play audio while scrubbing" (shift+s, or Preferences > Audio)? This dramatically improves scrubbing performance overall. I have reported this to Adobe for many years, including screen recordings. It is especially noticeable with longer videos, and even more so with long highly compressed Long-GOP videos.

  • colin cameron commented  ·   ·  Flag as inappropriate

    yes just did an edit tonight where i had to find the start of a minute and a half segment in four and a half hours of footage and this feature would have been incredibly useful.

Premiere Pro: Playback

Categories

Feedback and Knowledge Base

(thinking…)