Generating Peak Files should be Multi-threaded
Generating peak files is one of the slowest aspects of Premiere. Can we please make this process faster?
Antoine Autokroma commented
Matt, why do you need to read the whole audiovideo stream (4 TB in your example) ? All it matters is the audio. It would be very helpful if you find a way to only demux the audio stream (and only read parts of the file that corresponds to this audio stream I mean)
Matt Stegner commented
The "Generate Audio Waveform" command has some interesting uses. If you execute it on a clip that is highlighted in the Project panel, the entire master clip waveform will be generated (same applies to a selection of clips). If you have an In and Out point in the timeline, only the in and out point will be generated (same allies to all clips in a timeline). The command also can reprioritize the waveform generation queue. If you have 100 files that need waveforms, you can select the first 10 and start scanning them. Then you realize you actually need to see files 11-20 before 1-10, you can select them and execute the command. Premiere will reprioritize 11-20 and scan them first, then go back and scan 1-10. This also applies to sequences. You can reprioritize a certain sequence before another. You need al the waveforms for sequence “A”, select all and execute that command. Oh, you realize you actually need to see “B”, select the clips in there and Generate Waveforms and it will be next.
If you work on a Network Storage, I would recommend that you try disabling the automatic peak file generation and try generating them at will and see if that helps. The “Generate Audio Waveforms” command was designed with the idea that a user would turn off the automatic generation, then assign a keyboard shortcut to “Generate Audio Waveforms” and quickly generate them at will.
Let me know if this helps or you have any more questions about this, I can try to help.
Matt Stegner commented
We've tried to make this faster and it isn't easy to make this significantly faster. I won’t answer the question about parallization, but can try to explain below how peak feel germination works and how you might be able to optimize the peak file generation.
Lemme explain some of the issue we face with generating peak files.
Generally peak file generation is bound by the read from disk, so parallelizing it will only bog down the entire disk I/O. To get generate the peak file Premiere needs to read the entire video stream from the disk (or network storage). This means your 4k video file will stream to the local disk, then the video frames thrown away, just to generate the peak file. Running a bunch of those in parallel could saturate the network quickly. You'd start to slow down the entire network when generating peak files. And it might not be much faster (sometimes it would and sometimes it wouldn't).
Lemme explain some of the way it does work. Not necessarily for you but for anyone coming across this post. I’m really trying to help here, please don’t misconstrue this as making excuses, but me trying to explain how a different workflow might help.
The peak files are small pictures of the waveform, which makes it easier to edit visually. By default, Premiere will scan a file on import and generate the peak file. It will then cache that file so it can be used again and agin in multiple projects. That is part of the function of the Media Cache database. You don't need the peak files to edit, it just makes it easier to see what is in your audio clips. Generating these files is effectively a background process. If you play, Premiere will stop generating the peak files and give that disk I/O bandwidth back to the playback engine, so you don't drop frames. You don't need to sit and wait fro Premiere to finish generating the peak files unless you need to see something in the waveform. And, since it is paused during playback, you wont lose any performance either. Effectively, if you wanted too, you could ignore that little progress bar in the corner.
Even in you do need to see something in the waveform, you can just scrub or play over that section of your clip and Premiere will automatically reprioritize the waveform generation of that section of the waveform, and draw it next. It pauses what ever file is being generated, and generates the current audio that is being touched. Basically, if you need to see a waveform right away but Premiere is busy generating a waveform for another clip, simple scrub or play over the section of the clip you need and the waveform will be generated on demand. Premiere has an optimization that it never need to generate the peak data twice, and by simply accessing a video stream (by playing or scrubbing) the waveform for that section will be drawn.
I talked about how generating peak files over the network, you will need to read the entire video stream. So if you import 4 TB of footage from the network, Premiere will try to, by default, read all 4 TB of footage and generate waveforms for the footage. Not super efficient, and any other editors you have on your network will be sharing that same bandwidth. You can, in the Premiere Preferences, turn off the automatic generation of waveforms, so media will not be needlessly streamed to your local computer just to generate the peak files. Great, but you now wont have any waveforms to see what is going on. You can selectively generate waveforms in this case, minimizing the bandwidth issue, by using the command "Clip>Generate Audio Waveform...". This command creates a queue and starts generating waveform right away. Also, remember you could simple play or scrub over the section of the clip that you need to see the waveform and premiere will generate it first.