Re: [Cin] Issues related to Motion Plugin
Georgy, I suppose it was not intentional, but in case this did not arrive to Cin ML (likely, it's not in the archive), I want to send this email, because I worked a couple of hours on it, and I mentioned it in another email today. I need it to show what issues I had. -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr my PGP-key: https://www.croatiafidelis.hr/FCF13245ED247DCE443855B7EA9884884FBAF0AE.asc I'm removing one layer of citation, for legibility, as if Georgy sent it, not me. ========================================================== On 220926-10:20+0700, Georgy Salnikov wrote: On Sun, 25 Sep 2022, Miroslav Rovis wrote:
Phyllis has such a very shaky example. May be, I have yet a cingg project to stabilize it using Motion plugin keyframes. Should I search for it? If you think so.
Miroslav, in attachment you find the archive. Unpack it, there are two files: the source video clip, and the cingg project file. This is a rather difficult example not only because of strong shaking. Moreover, the video is taken perhaps from a moving car, so the landscape itself in the viewport is changing. The example project demonstrates the possibility of the Motion plugin to switch from one key part of image to another as the former part goes off the viewport, becomes obscured or changes shape. This is not smoothing video, but freezing it. The result is not completely still, but additionally applying vidstab to it perhaps should improve the result once more. To test, do the following after unpacking: Open shaker2.xml (replace current project) Make sure: Motion and Time Average must be off Rewind to the beginning Uncheck 'Play track' Open Motion controls dialog Action: Do Nothing Calculation: Save coords to tracking file Scroll timeline forward, set cursor after the next keyframe Action: Do Nothing Calculation: Save coords to tracking file Repeat that for all the rest keyframes (total of 4) Close Motion controls dialog Rewind to the beginning Switch Motion on Check 'Play track' Play forward up to the end Video will be still very shaky. The file 'motion-shaker4' will appear in /tmp Uncheck 'Play track' Rewind to the beginning Open Motion controls dialog Action: Stabilize Subpixel Calculation: Load coords from tracking file Scroll timeline forward, set cursor after the next keyframe Action: Stabilize Subpixel Calculation: Load coords from tracking file Repeat that for all the rest keyframes (total of 4) Close Motion controls dialog Rewind to the beginning Switch Motion on Switch Time Average on Check 'Play track' Render project Now compare the result of rendering with the original one. ....... Some of my examples to apply ffmpeg's vid.stab is as follows: rm transforms.trf ffmpeg -i video-src.mp4 -c:v rawvideo -filter:v vidstabdetect -an -f null -y /dev/null ffmpeg -i video-src.mp4 -c:v libx264 -filter:v vidstabtransform,unsharp=5:5:0.8:3:3:0.4 -b:v 12000k -b:a 192k -ac 2 -ar 48000 -y video-dst.m4v Regards Georgy _______________________________________________________________________________ Georgy Salnikov NMR Group Novosibirsk Institute of Organic Chemistry Lavrentjeva, 9, 630090 Novosibirsk, Russia Phone +7-383-3307864 Email sge@nmr.nioch.nsc.ru _______________________________________________________________________________
On 220927-19:36+0200, Miroslav Rovis wrote:
Georgy, I suppose it was not intentional, but in case this did not arrive to Cin ML (likely, it's not in the archive), I want to send this email, because I worked a couple of hours on it, and I mentioned it in another email today. I need it to show what issues I had.
-- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr my PGP-key: https://www.croatiafidelis.hr/FCF13245ED247DCE443855B7EA9884884FBAF0AE.asc
I'm removing one layer of citation, for legibility, as if Georgy sent it, not me. ==========================================================
On 220926-10:20+0700, Georgy Salnikov wrote: On Sun, 25 Sep 2022, Miroslav Rovis wrote:
Phyllis has such a very shaky example. May be, I have yet a cingg project to stabilize it using Motion plugin keyframes. Should I search for it? If you think so.
Miroslav, in attachment you find the archive. Unpack it, there are two files: the source video clip, and the cingg project file. This is a rather difficult example not only because of strong shaking. Moreover, the video is taken perhaps from a moving car, so the landscape itself in the viewport is changing. The example project demonstrates the possibility of the Motion plugin to switch from one key part of image to another as the former part goes off the viewport, becomes obscured or changes shape. This is not smoothing video, but freezing it. The result is not completely still, but additionally applying vidstab to it perhaps should improve the result once more. To test, do the following after unpacking:
Open shaker2.xml (replace current project)
Make sure: Motion and Time Average must be off Rewind to the beginning Uncheck 'Play track'
Open Motion controls dialog Action: Do Nothing Calculation: Save coords to tracking file Scroll timeline forward, set cursor after the next keyframe Action: Do Nothing Calculation: Save coords to tracking file Repeat that for all the rest keyframes (total of 4) Close Motion controls dialog
Rewind to the beginning Switch Motion on Check 'Play track' Play forward up to the end
Video will be still very shaky. The file 'motion-shaker4' will appear in /tmp
Uncheck 'Play track' Rewind to the beginning
Open Motion controls dialog Action: Stabilize Subpixel Calculation: Load coords from tracking file Scroll timeline forward, set cursor after the next keyframe Action: Stabilize Subpixel Calculation: Load coords from tracking file Repeat that for all the rest keyframes (total of 4) Close Motion controls dialog
Rewind to the beginning Switch Motion on Switch Time Average on Check 'Play track' Render project
Now compare the result of rendering with the original one.
.......
Some of my examples to apply ffmpeg's vid.stab is as follows:
rm transforms.trf ffmpeg -i video-src.mp4 -c:v rawvideo -filter:v vidstabdetect -an -f null -y /dev/null ffmpeg -i video-src.mp4 -c:v libx264 -filter:v vidstabtransform,unsharp=5:5:0.8:3:3:0.4 -b:v 12000k -b:a 192k -ac 2 -ar 48000 -y video-dst.m4v
Regards Georgy _______________________________________________________________________________
Georgy Salnikov NMR Group Novosibirsk Institute of Organic Chemistry Lavrentjeva, 9, 630090 Novosibirsk, Russia Phone +7-383-3307864 Email sge@nmr.nioch.nsc.ru _______________________________________________________________________________
Hi Georgy, Phillis and the team! Here's how my tentative with the shaker2.xml and the video went: https://croatiafidelis.hr/foss/cap/cap-220925-cinelerra/pg-220926-motion.php The files necessary are there (as I got them in email that missed to be sent to Cin ML). Why it did not produce the expected result? (I haven't examined the XML-after yet, but it's available on the page.) And I can also try and see if I can do better with the files at: https://www.cinelerra-gg.org/media/demos/stabilize/ Regards! -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr my PGP-key: https://www.croatiafidelis.hr/FCF13245ED247DCE443855B7EA9884884FBAF0AE.asc
@Phyllis Smith
note to Andrea, I think we should make a footnote in the manual mentioning the vid.stab ffmpeg filter used with ffmpeg as a starting point for some motion stabilization beforehand. > It would probably save a lot of frustration on the part of users. What do you think?
I put the notes on the ffmpeg plugin 'vid_stab'. I added more of Georgy's tips (I didn't ask: is that okay with you Georgy?) and I also attach 'normal.png' which corrects the 'Subtract' image that was wrong (see other email).
Andrea, thanks for updating the manual. I moved the note about preprocessing with ffmpeg command line to the top of both Motion and Motion51 when stabilizing an entire video. That way they have an alternative before reading all of the fine details.
note to Andrea, I think we should make a footnote in the manual mentioning the vid.stab ffmpeg filter used with ffmpeg as a starting point for some motion stabilization beforehand. > It would probably save a lot of frustration on the part of users. What do you think?
I put the notes on the ffmpeg plugin 'vid_stab'. I added more of Georgy's tips (I didn't ask: is that okay with you Georgy?) and I also attach 'normal.png' which corrects the 'Subtract' image that was wrong (see other email).
On Tue, 27 Sep 2022, Miroslav Rovis wrote:
Here's how my tentative with the shaker2.xml and the video went: https://croatiafidelis.hr/foss/cap/cap-220925-cinelerra/pg-220926-motion.php The files necessary are there (as I got them in email that missed to be sent to Cin ML).
Why it did not produce the expected result? (I haven't examined the XML-after yet, but it's available on the page.)
Miroslav, you have tested everything right. Your result is the same as mine. OK, the result is in reality not perfect. By finer tuning the plugin parameters and keyframes placement achieving better results is in principle possible. But this particular example was only a testcase for me - to verify that the algorithm works. Getting visually perfect results was not my goal. _______________________________________________________________________________ Georgy Salnikov NMR Group Novosibirsk Institute of Organic Chemistry Lavrentjeva, 9, 630090 Novosibirsk, Russia Phone +7-383-3307864 Email sge@nmr.nioch.nsc.ru _______________________________________________________________________________
On 220928-13:48+0700, Georgy Salnikov wrote:
On Tue, 27 Sep 2022, Miroslav Rovis wrote:
Here's how my tentative with the shaker2.xml and the video went: https://croatiafidelis.hr/foss/cap/cap-220925-cinelerra/pg-220926-motion.php The files necessary are there (as I got them in email that missed to be sent to Cin ML).
Why it did not produce the expected result? (I haven't examined the XML-after yet, but it's available on the page.)
Miroslav, you have tested everything right. Your result is the same as mine.
OK, the result is in reality not perfect. By finer tuning the plugin parameters and keyframes placement achieving better results is in principle possible. But this particular example was only a testcase for me - to verify that the algorithm works. Getting visually perfect results was not my goal. _______________________________________________________________________________
Georgy Salnikov NMR Group Novosibirsk Institute of Organic Chemistry Lavrentjeva, 9, 630090 Novosibirsk, Russia Phone +7-383-3307864 Email sge@nmr.nioch.nsc.ru _______________________________________________________________________________
Well, that clears some of my doubts regarding motion plugin. Thanks! And there is one good thing this demostrates, if your result and my result are identical: consistency of the program, It may be long to go to get reproducibility [*]. I hope Cinelerra will be reproducible one day. Regards, kind developer! ---- [*] such as e.g. Debian is striving for: https://wiki.debian.org/ReproducibleBuilds . I for myself can state that when I compiled Getmail, I did get the exact same binaries out as the then current getmail package in Debian, as I reported in https://github.com/getmail6/getmail6/issues/123 However, long to go for Debian to be really reproducible as well. -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr my PGP-key: https://www.croatiafidelis.hr/FCF13245ED247DCE443855B7EA9884884FBAF0AE.asc
participants (4)
-
Andrea paz -
Georgy Salnikov -
Miroslav Rovis -
Phyllis Smith