[Cin] Uncompressed video to lossless compression FFV1/MKV

[Andrew Randrianasulu]

On Thursday, December 23, 2021, Terje J. Hanssen via Cin <
cin at lists.cinelerra-gg.org> wrote:

> Background:
>
> Digital video produces vast amounts of data. Take PAL-SD material (only
> image, no audio), stored as YUV422 with 8-bits per component (=16-bits per
> pixel) and a resolution of 720x576 pixels, for example.
> http://download.das-werkstatt.com/pb/mthk/info/video/compari
> son_video_codecs_containers.html#lossy_vs_lossless
>
>  * Applying no compression at all (=uncompressed
>    <http://download.das-werkstatt.com/pb/mthk/info/video/
> comparison_video_codecs_containers.html#codec_uncompressed>)
>    results in:
>  * 720 x 576 x 16 bits x 25 frames ≥ 1.159 GiB/Minute
>  *
>
>  * For YUV422, 10 bpc (10+5+5 = 20 bits per pixel), this would be:
>  * 720 x 576 x 20 bits x 25 frames ≥ 1.448 GiB/Minute
>
>
> So you would fit no more than 4 minutes of video on a whole 4.7 GiB DVD.
> Since these values are already uncomfortable (even for 422-subsampled PAL),
> imagine HD and beyond.
> Therefore, you will hardly find uncompressed video in the wild, except for
> temporary editing use cases.
>
> -----------
>
> The examples below show the differences in storage space required for
> uncompressed, lossless and lossy video codecs:
> https://www.scart.be/?q=en/content/short-guide-choosing-digi
> tal-format-video-archiving-masters
>
>  * uncompressed (e.g., v210) 10-bit -> approx. 100GB per hour of video;
>  * lossless compression (FFV1 and JPEG 2000) 10-bit -> approx. 45-50 GB
>    per hour of video;
>  * lossy compression;
>  * MPEG 2/DV50/IMX50 (50 Mbps) -> approx. 25 GB per hour of video;
>  * to compare with
>  * DV (DV25) -> approx. 12 GB per hour of video;
>  * MPEG 2 (DVD quality) -> approx. 3.6 GB per hour of video.
>
> -------------
>
> That is, if we manage to capture 422 uncompressed video via either an
> internal capture card or via an external SSD recorder, there is a need for
>
>  * mathematical lossless compression with typical intraframe FFV1/MKV
>    for preservation (archive masters)
>  * and/or possibly "visually lossless" compression to a lossy
>    intraframe intermediate/mezzanine format
>
>
> I had no uncompressed test file available, but tried this guide to
> generate a simple video (only) file:
> Compressing and uncompressing a preservation master video file
> https://bits.ashleyblewer.com/blog/2018/11/08/compressing-an
> d-uncompressing-a-preservation-master-video-file/
>
> ffmpeg -f lavfi -i mandelbrot=size=640x480:rate=25 -c:v v210 -t 5
> uncompressed.mov
>
> and next to turn it to a FFV1/MKV lossless compressed file
>
> ffmpeg -i uncompressed.mov -map 0 -dn -c:v ffv1 -level 3 -g 1 -slicecrc 1
> -slices 16 -c:a copy ffv1.mkv
>
> and if required a test to uncompress the FFV1/MKV file to uncompressed
> 10bit file again:
>
> ffmpeg -i ffv1.mkv -c:v v210 uncompressed-again.mov
>
> du -s *
> 105004    uncompressed.mov
> 22844    ffv1.mkv
> 105004    uncompressed-again.mov


i think there is special filter (framecrc?) in ffmpeg for checking
exactness of those frames (no example off hand..
http://underpop.online.fr/f/ffmpeg/help/framecrc.htm.gz  ? )

>
> In this simple case the lossless compressed ffv1.mkv size is schrinked to
> 22% of the uncompressed.mov file.
> In other practical tests I read the FFV1 file size was schrinked to 1/3 to
> 46% of the uncompressed video.
>
> (I tested for fun to transcode a "visual lossless" compressed ProResHQ.mov
> file to FFV1/MKV, but that extented the file size to 180% )
>
> I loaded the simple generated ffv1.mkv in Cinelerra, but also read about a
> problem in a previous thread
> [Cin] Single-frame step fwd/bak in viewer delayed?
> https://www.mail-archive.com/cin@lists.cinelerra-gg.org/msg01799.html
>
> Is this problem solved?
> And will the lossless compressed (preservation) format FFV1.MKV work as a
> editing format, or will there be a need beside to have another intermediate
> format MPEG 2/DV50/IMX50 (50 Mbps) for editing?


I think Andrea run into slowness/desyncro with some mkvs (unsolved)

ffv1 can be quite cpu intensive, so fludity during edition depends on host
cpu. Also, apparently there is two-level of  crc codes both in ffv1 and mkv
container. May be one can disable one level of checking for faster
de/compression (I will look into it)

>
> ----------
>
> Terje J. H
>
> --
> Cin mailing list
> Cin at lists.cinelerra-gg.org
> https://lists.cinelerra-gg.org/mailman/listinfo/cin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cinelerra-gg.org/pipermail/cin/attachments/20211223/c4eb0b9c/attachment.htm>