fileexr/fileppm direct copy support
So, I was working on restoring direct copy support for cinelerra-cv and realized I can xcheck Cin-GG too. I think attached two patches make it possible to skip de/encoding for unchaged portions of timeline when working with image lists of exr and ppm types (in addition to png/tiff/tga/jpeg support already there) Please test?
Please test? I don't really know what test I could do. I rendered a video at 120 fps in an OpenEXR image sequence. I loaded the file in CinGG-appimage (without patches) and the playback goes at 18-23 fps, while moving the cursor up and down the timeline goes in real time, without freezing or slowing down. I loaded the same sequence of images into CinGG compiled with the 2 patches and the result is the same: playback at 17-23 fps and real-time timeline. I notice no differences Can you suggest other tests?
ср, 26 окт. 2022 г., 21:54 Andrea paz <[email protected]>:
Please test? I don't really know what test I could do. I rendered a video at 120 fps in an OpenEXR image sequence. I loaded the file in CinGG-appimage (without patches) and the playback goes at 18-23 fps, while moving the cursor up and down the timeline goes in real time, without freezing or slowing down. I loaded the same sequence of images into CinGG compiled with the 2 patches and the result is the same: playback at 17-23 fps and real-time timeline. I notice no differences Can you suggest other tests?
try to select part of timeline , add effect to it, and render into new sequence of same type. With patch you should see speedup.
try to select part of timeline , add effect to it, and render into new sequence of same type. With patch you should see speedup.
I added transitions and effects to the project. The rendering of the project (both whole and parts of it) is always the same between the test with appimage and with the build with patches. I notice no difference in both playback and rendering.
ср, 26 окт. 2022 г., 23:01 Andrea paz <[email protected]>:
try to select part of timeline , add effect to it, and render into new sequence of same type. With patch you should see speedup.
I added transitions and effects to the project. The rendering of the project (both whole and parts of it) is always the same between the test with appimage and with the build with patches. I notice no difference in both playback and rendering.
you rendered from EXR sequence to EXR sequence, right?
ср, 26 окт. 2022 г., 23:24 Andrea paz <[email protected]>:
you rendered from EXR sequence to EXR sequence, right?
Rendering the project from an exr sequence always leads to the same results with both appimage and build. Rendering is done with 19 fps (whereas from mp4 the render is done with 9 fps).
so i first tried to render exr/zip sequence from mjpeg file, it took around 16 sec. And then I just set different filename, load only my exr sequence and rendered it. Rendering was ..instant. Note, default exr compression 'none' can be brutal on storage Render::render_single: Session finished. ** rendered 9 frames in 8.092 secs, 1.112 fps Obt-Message: Failed to open an Input Method Render::render_single: Session finished. ** rendered 9 frames in 0.098 secs, 91.837 fps Render::render_single: Session finished. ** rendered 26 frames in 20.437 secs, 1.272 fps Render::render_single: Session finished. ** rendered 26 frames in 3.220 secs, 8.075 fps this was WITH patch BC_WindowBase::init_im: Could not open input method. tRender::render_single: Session finished. ** rendered 26 frames in 26.921 secs, 0.966 fps and THIS was without patch (exr/zip to exr/zip)
I am probably doing something wrong! A curiosity: doing renderings from both mp4 and exr sequence, the start of the rendering is always very fast but, coming towards 80%, it becomes very slow. It basically takes longer to do the final than the initial 80%. This happens with both appimage and patches.
чт, 27 окт. 2022 г., 16:26 Andrea paz <[email protected]>:
I am probably doing something wrong!
try to render first exr sequence (from any source) . Set EXR compression to some cpu intensive choice. Then load this sequence and in rendering dialog only change name of sequence, so it will create new set of images. Try last step with patched and unpatched cingg. A curiosity: doing renderings from both mp4 and exr sequence, the
start of the rendering is always very fast but, coming towards 80%, it becomes very slow. It basically takes longer to do the final than the initial 80%. This happens with both appimage and patches.
RAM flushed to relatively slow disk?
try to render first exr sequence (from any source) . Set EXR compression to some cpu intensive choice. Then load this sequence and in rendering dialog only change name of sequence, so it will create new set of images. Try last step with patched and unpatched cingg.
These are exactly the steps I took. I used "none" or "RLE" compression with the same results. I also tried rendering a tiff sequence still with the same results (21 fps).
RAM flushed to relatively slow disk?
I have 32 GB of RAM and nvme disk. No more than 3 GB is ever used during rendering and swap is not used. Today I no longer have the 80% slowdown! I wonder what had happened to my PC yesterday?
чт, 27 окт. 2022 г., 18:54 Andrea paz <[email protected]>:
try to render first exr sequence (from any source) . Set EXR compression to some cpu intensive choice. Then load this sequence and in rendering dialog only change name of sequence, so it will create new set of images. Try last step with patched and unpatched cingg.
These are exactly the steps I took. I used "none" or "RLE" compression with the same results. I also tried rendering a tiff sequence still with the same results (21 fps).
may be slow cpu on my tablet makes bigger difference? Note, I used ZIP compression. attached (blurred) screenrecording
RAM flushed to relatively slow disk?
I have 32 GB of RAM and nvme disk. No more than 3 GB is ever used during rendering and swap is not used. Today I no longer have the 80% slowdown! I wonder what had happened to my PC yesterday?
mystery!
Checked into GIT source, the 4 patched files of fileexr.C/.h and fileppm.C/.h after testing both. чт, 27 окт. 2022 г., 18:54 Andrea paz <[email protected]>:
try to render first exr sequence (from any source) . Set EXR compression to some cpu intensive choice. Then load this sequence and in rendering dialog only change name of sequence, so it will create new set of images. Try last step with patched and unpatched cingg.
These are exactly the steps I took. I used "none" or "RLE" compression with the same results. I also tried rendering a tiff sequence still with the same results (21 fps).
may be slow cpu on my tablet makes bigger difference? Note, I used ZIP compression.
There is a speed-up on the second render, to the point of almost
instantaneous when very minimal changes are made even on a relatively new laptop (only 4 years old). And a speed up when an effect is added to 1/4 of the video. Andrea, you can try these exact steps: 1) mkdir /tmp/ppm1 and mkdir /tmp/ppm2 and mkdir /tmp/ppm3 in a window 2) start cinelerra (with the 2 patchsets having been built in) then load with replace a video 3) render that video entire project using ppm sequence (render to /tmp/ *ppm1*/a.ppm) 4) load just the ppm sequence you created in step 3 (load with replace /tmp/ *ppm1*/a.ppm) 5) now just render this using ppm sequence and this time to /tmp/*ppm2* /a.ppm it should be really fast 6) for testing purposes, add an effect to a small section of the video loaded in step 4 and now render to /tmp/*ppm3.*a.ppm and it will still be faster than the original ppm1 but not as fast as in step 5 because it can only direct copy the unmodified portion of the video as opposed to where the effect is
Thanks Phyllis; I finally saw some difference: AppImage (no patch) render exr to exr: 176 sec, 19.491 fps render exr +plugin to exr: 183 sec, 18.717 fps Build (patch) render exr to exr: 172 sec, 19.970 fps render exr +plugin to exr: 173 sec, 19.8 fps
** rendered 2712 frames in 0.348 secs, 7793.103 fps ** rendered 2712 frames in 20.465 secs, 132.519 fps On Sat, Oct 29, 2022 at 9:22 AM Andrea paz <[email protected]> wrote:
Thanks Phyllis; I finally saw some difference:
AppImage (no patch)
render exr to exr: 176 sec, 19.491 fps render exr +plugin to exr: 183 sec, 18.717 fps
Build (patch)
render exr to exr: 172 sec, 19.970 fps render exr +plugin to exr: 173 sec, 19.8 fps
Not sure you are actually using Exr Sequence for the whole Project instead of 1 single frame ? Or maybe Arch just handles things much differently. Because there is not much variation on the speedup shown in your results. For a 1 1/2 minute video, this is what I am getting: *With patch* ** rendered 2712 frames in 0.348 secs, *7793.103 fps* No patch ** rendered 2712 frames in 20.465 secs, *132.519 fps*
But this worked for me to speed up the render (to me, it is incomprehensible as to why though) try to select part of timeline , add effect to it, and render into new
sequence of same type. With patch you should see speedup.
Doing the above made no difference for me in speed. BUT doing the below worked for me almost immediately to speed up the render (to me, it is incomprehensible as to why though). I have to do more tests until I make sure I am not making a mistake. try to render first exr sequence (from any source) . Set EXR compression to
some cpu intensive choice.
Then load this sequence and in rendering dialog only change name of sequence, so it will create new set of images.
чт, 27 окт. 2022 г., 22:21 Phyllis Smith <[email protected]>:
But this worked for me to speed up the render (to me, it is incomprehensible as to why though)
try to select part of timeline , add effect to it, and render into new
sequence of same type. With patch you should see speedup.
Doing the above made no difference for me in speed. BUT doing the below worked for me almost immediately to speed up the render (to me, it is incomprehensible as to why though). I have to do more tests until I make sure I am not making a mistake.
well, theory is rendering pipeline can pull already compressed input frame and pass it unchanged to output, saving on cpu if there was no processing, on per-frame basis..... I did little patch for cin-cv making this Performance preference checkbox, so hopefully next month I'll do the same for cin-gg. (I hope I'll able to put checkbox near 'cache transitions' one)
try to render first exr sequence (from any source) . Set EXR compression
to some cpu intensive choice.
Then load this sequence and in rendering dialog only change name of sequence, so it will create new set of images.
Il progetto che ho usato lo potete trovare sul mio Dropbox: https://www.dropbox.com/s/32gs99ulw2bpx78/project_exr.tar.gz?dl=0
participants (3)
-
Andrea paz -
Andrew Randrianasulu -
Phyllis Smith