September 2024 - Cin - Cinelerra-GG Mailing Lists

Surprizes in YUV->RGB conversion
by Andrew Randrianasulu 15 Sep '24

15 Sep '24

I was using artificial test source from ffmpeg, yuvtestsrc. It sounds very simple ffmpeg -f lavfi -color_range 2 -i yuvtestsrc -color_range 2 -frames 2 /dev/shm/yuv-pc-yuv444.y4m but here lies dragons, at least when roundtrip to RGB is needed. This old BBC article explained to me for the first time that YUV as color space fundamentally bigger than RGB! https://web.archive.org/web/20200719051405/http://downloads.bbc.co.uk/rd/pu… By way stuff defined normal calculations can give you negative RGB values! An normally those are clipped to either min or max allowed channel value. But result is ...unintuitive, at least for me Here is online calculator: https://kdashg.github.io/misc/colors/from-coeffs.html if you enter RGB 601, narrow range Y=16, Cb=100 Cr=100 R=-44.68875 (minus 44!) G=33.73244 B=-56.4825 (minus 56!) so, if both negative channels will be clipped to 0 we will have visually purely green pixel ?! This probably explains some STRANGE results I was seeing in compositor and vectoroscope, because even if timeline set to YUV- for display values converted to RGB, and scope also works in RGB ? There was another patent paper from broadcom aimed at fixing this issue https://patents.google.com/patent/EP1560417A2/en "System and method for clipping values of pixels in one color space so not to exceed the limits of a second color space Abstract A system and method that detects when pixels from one color space would exceed the range of a second color space, and restrict the range of the pixels without distorting their brightness, while preserving their color. The method may provide the ability to restrict the range of pixels in the first color space that may ordinarily transform to "illegal" values with erroneous brightness in the second color space, to a sub-space containing the entire second color space, while preserving the brightness of the pixels. The parameters used to define the sub-space may be programmable. The first color space may be YCrCb and the second color space may be RGB." ===== so, while in theory this problem is combatable in practice I do not think we have any out of the box plugin for dealing with this? So .... back to hunting for relevant test image, may be rgbtestsrc stored as y4m will do?

1 0

Another book on digital video
by Andrew Randrianasulu 15 Sep '24

15 Sep '24

Digital video and HD (2012) https://wangwei1237.github.io/shares/Digital_Video_and_HD_Algorithms_and_In… nearly 40 mb, 753 pp! found via https://forum.blackmagicdesign.com/viewtopic.php?f=21&t=78837 not sure if I'll learn enough for stopping confusing color spaces vs primaries and so on ......

4 6

possible typo in documentation?
by Andrew Randrianasulu 15 Sep '24

15 Sep '24

https://cinelerra-gg.org/download/CinelerraGG_Manual/YUVShift.html === This effect is used for YUV input video from older cameras using 3 sensors. It is possible to have misalignment of the 3 sets of numbers: Y, which represents the luminance or brightness component, and for U and V representing the chrominance (color) components. If they were misaligned in the video, you can use YUVShift to realign. To move a specific component up/down, modify the dy value using the slider bar in the RGBShift window. === I think here specifically it must say YUVShift?

2 1

Hardware acceleration methods
by Terje J. Hanssen 15 Sep '24

15 Sep '24

The CinGG manual mentions VAAPI and VDPAU. https://cinelerra-gg.org/download/CinelerraGG_Manual/Hardware_video_acceler… https://cinelerra-gg.org/download/CinelerraGG_Manual/GPU_hardware_encoding.… My prebuild system FFmpeg lists ffmpeg -hide_banner -hwaccels Hardware acceleration methods: vdpau cuda vaapi qsv drm vulkan Among these, Vulkan has recently been discussed here. Is there something that prevent typical QSV to be prebuilt in the CinGG/FFmpeg rpm? References: Compiling FFmpeg with QSV on Linux Intel Platform: https://www.intel.com/content/www/us/en/content-details/728030/building-ffm… Arc: Intel QuickSync (QSV): https://wiki.archlinux.org/title/FFmpeg#Intel_QuickSync_(QSV) refer

3 6

cingg hevc_vaapi rendering on Arc A750 gpu
by Terje J. Hanssen 15 Sep '24

15 Sep '24

Two simple tests I got working Cinelerra Infinity - built: Sep 8 2024 10:37:25 (rpm installation) on Leap 15.6 on i7-12700KF + Arc A750 gpu As seen below, quite fast rendering: * 207 fps for hdv.m2t (yuv420p) input -> yuvj420p * 125 fps for proreshq.mov (yuv422p10le) -> yuvj420p (10-bit to 8-bit color depth) 1) hdv.m2t Stream #0:0[0x810]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 1440x1080 [SAR 4:3 DAR 16:9], 25000 kb/s, 25 fps, 25 tbr, 90k tbn Side data: cpb: bitrate max/min/avg: 25000000/0/0 buffer size: 7340032 vbv_delay: N/A Stream #0:1[0x814]: Audio: mp2 (mp3float) ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 384 kb/s -> hevc_vaapi.mp4 ** rendered 5972 frames in 28.721 secs, 207.931 fps audio0 pad 64 0 (64) tream #0:0[0x1](und): Video: hevc (Main) (hev1 / 0x31766568), yuvj420p(pc, bt709/unknown/unknown, top coded first (swapped)), 1440x1080 [SAR 4:3 DAR 16:9], 11716 kb/s, 25 fps, 25 tbr, 12800 tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 129 kb/s (default) 2) hd.mov Stream #0:0[0x1](eng): Video: prores (HQ) (apch / 0x68637061), yuv422p10le(bt709, top coded first (swapped)), 1920x1080, 182130 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 2500 tbn (default) Metadata: creation_time : 2016-02-23T23:49:21.000000Z handler_name : Apple Video Media Handler vendor_id : appl encoder : Apple ProRes 422 (HQ) Stream #0:1[0x2](eng): Audio: pcm_s24le (lpcm / 0x6D63706C), 48000 Hz, 16 channels, s32 (24 bit), 18432 kb/s (default) -> hevc_vaapi.mp4 ** rendered 1781 frames in 14.188 secs, 125.529 fps audio0 pad 128 0 (128) Stream #0:0[0x1](und): Video: hevc (Main) (hev1 / 0x31766568), yuvj420p(pc, bt709/unknown/unknown, top coded first (swapped)), 1920x1080 [SAR 1:1 DAR 16:9], 11427 kb/s, 25 fps, 25 tbr, 12800 tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 16 channels, fltp, 378 kb/s (default)

2 4

xvYCC
by Andrew Randrianasulu 15 Sep '24

15 Sep '24

https://en.m.wikipedia.org/wiki/XvYCC there is another interesting standartized hack for wider-gamut color space ==== The BT.709 YCbCr signal has unused code space, a limitation imposed for broadcasting purposes. In particular only 16-240 is used for the color Cb/Cr channels out of the 0-255 digital values available for 8 bit data encoding. xvYCC makes use of this portion of the signal to store extended gamut color data by using code values 1-15 and 241-254 in the Cb/Cr channels for gamut-extension. ==== not sure how (if at all) ffmpeg supports this, let alone players under Linux ..... https://fanrestore.com/thread-1961-post-85235.html#pid85235 some convolved way via avisynth and x264 (windows by default, but vapoursynth/zscale-enabled ffmpeg exist under linux) === copypasta ===== deleted user Wrote: <https://fanrestore.com/post-70888.html#pid70888>Ok for anyone who's wondering how to encode xvYCC, I might have found out. First, you need to start out with some color space that has a big gamut to begin with. Let's say Rec 2020. It doesn't have to be HDR, you can just do a normal Rec2020 Gamma 2.4. This ensures that your source has all the colors you want to preserve. So work in and/or render out in Rec2020. If you are working in After Effects for example, in 32 bit floating point mode, then your working color space can be Rec709, you just have to set Rec2020 for the output, since the gamut gets preserved thanks to the ability of the 32 bit floating point data to hold negative values. Otherwise you likely have to work immediately in Rec2020. Load your source in AVISynth and perform the following command: Code: z_ConvertFormat(colorspace_op="2020:709:2020:l=>709:xvycc:709:l",pixel_type="YUV444P16") Note that you have to adapt the part before "=>" to your own input. The above example would be for a YUV Rec2020 Gamma 2.4 file with Limited range. The reason I put "709" there is because that's the transfer characteristic for Gamma 2.4. It doesn't look elegant but it works. Now let's say you're encoding straight from a HDR source, then you'd do: Code: z_ConvertFormat(colorspace_op="2020ncl:st2084:2020:l=>709:xvycc:709:l",pixel_type="YUV444P16") This probably isn't a great idea though because converting straight from HDR without any tonemapping will lead to clipped highlights. An alternate approach that would allow for tonemapping: Code: z_ConvertFormat(colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f",pixel_type="RGBPS") // First convert to linear floating point Rec2020 // Do tonemapping here with whatever method you like. It needs to accept the "RGBPS" color space, which is a 32 bit floating point planar RGB color space. I believe the DGHable functions and such accept that. z_ConvertFormat(colorspace_op="rgb:linear:2020:f=>709:xvycc:709:l",pixel_type="YUV444P16") // Now finally to xvYCC, while preserving as wide of a gamut as possible For these commands to work, you need this plugin: http://avisynth.nl/index.php/Avsresize Now finally, I did a test loading such a script in VirtualDub and encoding to x264 with the internal x264 encoder. I loaded that back into AVISynth and I'm happy to say that the data was preserved. I tested it with some very saturated scenes. So now the data gets preserved, but the player still won't know that it's dealing with xvYCC data. I can't say with 100% certainty that the following will work, but it does produce the proper metadata in the MediaInfo. Simply use the following x264 parameters instead of the usual: Code: --transfer="iec61966-2-4" --colormatrix="bt709" --colorprim="bt709" To my knowledge, colorprim and colormatrix stay the same, but the transfer characteristic is different. When you encode it like this, the resulting file will show "xvYCC" as the Transfer characteristic. I think that should do the trick, but it would take someone with a compatible TV to try. I can definitely see a difference in MPC-HC, however I haven't attempted to verify that it looks "correct". If anyone wants a sample encode with 2 frames (one in xvycc and one in rec709), PM me. It's just 100KB. Note: This will also work with DCI-P3 as the source for example. The only requirement really is that z_ConvertFormat supports the source color space you are feeding into it and you give it the correct info about it. With that said .... now you have no more excuses for not having wide gamut colors in your projects. [image: Wink] These files are compatible with normal Rec709 as far as I know. ==== end of copypaste ====

2 5

Fwd: [FFmpeg-devel] is libswscale maintained?
by Andrew Randrianasulu 14 Sep '24

14 Sep '24

so, there is hope (and potential for new bugs!) ---------- Forwarded message --------- От: Niklas Haas <ffmpeg(a)haasn.xyz> Date: ср, 11 сент. 2024 г., 12:20 Subject: Re: [FFmpeg-devel] is libswscale maintained? To: FFmpeg development discussions and patches <ffmpeg-devel(a)ffmpeg.org> On Wed, 11 Sep 2024 02:45:59 +0300 Andrew Randrianasulu < randrianasulu(a)gmail.com> wrote: > I can't find entry about it in ffmpeg/MAINTAINERS > > bugs meanwhile exist > > https://trac.ffmpeg.org/ticket/3345 > https://trac.ffmpeg.org/ticket/7978 There is on ongoing effort to redesign swscale, as well as fixing all currently open bugs, throughout the rest of 2024. Stay tuned for this. > > I often saw suggestion to use -vf zscale but this is not default, and z.img > upstream seems to be in suspended animation: > > https://github.com/sekrit-twc/zimg/issues/207 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel(a)ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request(a)ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel(a)ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request(a)ffmpeg.org with subject "unsubscribe".

1 0

origin of those coefficients for rgb-yuv matrix
by Andrew Randrianasulu 13 Sep '24

13 Sep '24

Not sure if it was explained to me elsewhere before https://sites.google.com/view/ananyamukherjeehome/image-processing/color-sp… ==== UV coefficients from Y coefficients In a 3x3 RGB to YUV matrix, it is enough to know the RGB to Y conversion coefficients. The coefficients for U and V can be derived from them. [..] Often, x and y of red green and blue and a white point (eg D65 <https://www.google.com/url?q=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FIllumi…>) are specified in specs such as BT 709 <https://www.google.com/url?q=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FRec._7…>. Given the 6 primary chromaticity numbers and the 2 white point numbers, you can plug them in here <http://www.google.com/url?q=http%3A%2F%2Fwww.russellcottrell.com%2Fphoto%2F…> and get your RGB->XYZ matrix. The mathematical details of the conversion can be found here <http://www.google.com/url?q=http%3A%2F%2Fwww.brucelindbloom.com%2Findex.htm…>. Now the middle row of this matrix converts from RGB to Y, and this Y is the Y of YUV color space. So you can get the coefficients of Y from here given primary chromaticities ==== but of course it up to reader to figure out how limited/full range difference should be used in those calculations .... from example in mjpegtools-2.2.1/lavtools/jpeg2yuv.c /** Rescales the YUV values from the range 0..255 to the range 16..235 @param yp: buffer for Y plane of decoded JPEG @param up: buffer for U plane of decoded JPEG @param vp: buffer for V plane of decoded JPEG */ static void rescale_color_vals(int width, int height, uint8_t *yp, uint8_t *up, uint8_t *vp) { int x,y; for (y = 0; y < height; y++) for (x = 0; x < width; x++) yp[x+y*width] = (float)(yp[x+y*width]) * ((235.0 - 16.0)/255.0) + 16.0; for (y = 0; y < height/2; y++) for (x = 0; x < width/2; x++) { up[x+y*width/2] = (float)(up[x+y*width/2]) * ((240.0 - 16.0)/255.0) + 16.0; vp[x+y*width/2] = (float)(vp[x+y*width/2]) * ((240.0 - 16.0)/255.0) + 16.0; } } for 4:2:0 subsampled yuv arrays as far as I understand. so, this can be simplified to multiplication of chroma and luma values by their constants? Also matrix, but 2*1 ? And what to do with two differently sized matrixes? I am cornered myself .....

1 0

How to access a second Intel iGPU in a workstation with additional discrete Nvidia GPU?
by Terje J. Hanssen 10 Sep '24

10 Sep '24

I am stuck how to configure QSV access to an enabled iGPU onboard an Intel Core i7-6700K (Skylake). This is a legacy, rebuilt workstation with additional, discrete Nvidia GeForce GTX 960 GPU on a MSI-Z170-A Pro mobo. Any idea and code modification below? Details as follows: inxi -CGMSz System: Kernel: 6.10.5-1-default arch: x86_64 bits: 64 Desktop: GNOME v: 46.3.1 Distro: openSUSE Tumbleweed-Slowroll 20240803 Machine: Type: Desktop Mobo: MSI model: Z170-A PRO (MS-7971) v: 1.0 serial: <superuser required> UEFI: American Megatrends v: 1.K0 date: 07/10/2018 CPU: Info: quad core model: Intel Core i7-6700K bits: 64 type: MT MCP cache: L2: 1024 KiB Speed (MHz): avg: 800 min/max: 800/4200 cores: 1: 800 2: 800 3: 800 4: 800 5: 800 6: 800 7: 800 8: 800 Graphics: Device-1: Intel HD Graphics 530 driver: i915 v: kernel Device-2: NVIDIA GM206 [GeForce GTX 960] driver: nvidia v: 550.100 Display: wayland server: X.org v: 1.21.1.12 with: Xwayland v: 24.1.1 compositor: gnome-shell driver: X: loaded: modesetting unloaded: fbdev,vesa gpu: nvidia,nvidia-nvswitch resolution: 1920x1080~60Hz API: OpenGL v: 4.6.0 vendor: nvidia v: 550.100 renderer: NVIDIA GeForce GTX 960/PCIe/SSE2 API: Vulkan v: 1.3.290 drivers: N/A surfaces: xcb,xlib,wayland API: EGL Message: EGL data requires eglinfo. Check --recommends. lspci -k | grep -EA3 'VGA|3D|Display' 00:02.0 Display controller: Intel Corporation HD Graphics 530 (rev 06) DeviceName: Onboard IGD Subsystem: Micro-Star International Co., Ltd. [MSI] Device 7971 Kernel driver in use: i915 -- 01:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3201 Kernel driver in use: nvidia Kernel modules: nouveau, nvidia_drm, nvidia ===================================================================== Trying to test QSV via FFmpeg benchmark similar like this Vulkan code ffmpeg -init_hw_device "vulkan=vk:0" -hwaccel vulkan -hwaccel_output_format vulkan -i INPUT -f null - -benchmark https://trac.ffmpeg.org/wiki/HWAccelIntro#Vulkan cause an output error which reports: "device type qsv needed". Intel Skylake with HD 530 graphics gen 9, should be capable to decode and encode MPEG-2/H.264/H.265 according to https://trac.ffmpeg.org/wiki/Hardware/QuickSync#HardwareSupport ffmpeg -hide_banner -hwaccel qsv -hwaccel_output_format qsv -i h264_8bit_yuv420p.mp4 -f null - -benchmark [AVHWDeviceContext @ 0x557526873500] Error creating a MFX session: -9. Device creation failed: -1313558101. [vist#0:0/h264 @ 0x55752685c6c0] [dec:h264_qsv @ 0x55752685b300] No device available for decoder: device type qsv needed for codec h264_qsv. [vist#0:0/h264 @ 0x55752685c6c0] [dec:h264_qsv @ 0x55752685b300] Hardware device setup failed for decoder: Unknown error occurred [vost#0:0/wrapped_avframe @ 0x55752685a740] Error initializing a simple filtergraph Error opening output file -. Error opening output files: Unknown error occurred bench: maxrss=67452KiB I have tried to define and add a qsv device according to the last section here https://trac.ffmpeg.org/wiki/Hardware/QuickSync#Transcode |-qsv_device| is an qsv customized option can be used to specify a hardware device and avoid the default device initialization failure when multiple devices usable (eg: an Intel integrated GPU and an AMD/Nvidia discrete graphics card). One example on Linux (more details please see https://trac.ffmpeg.org/ticket/7649 <https://trac.ffmpeg.org/ticket/7649>) ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i input.mp4 -c:v h264_qsv output.mp4 and also had a look at this answer https://stackoverflow.com/questions/44747381/i-cant-use-hw-encoder-of-qsv-b… --------------------------------------------------- Other HWA verification info: vulkaninfo | egrep 'GPU|driver' VK_LUNARG_direct_driver_loading : extension revision 1 GPU id = 0 (Intel(R) HD Graphics 530 (SKL GT2)) GPU id = 1 (NVIDIA GeForce GTX 960) GPU id = 0 (Intel(R) HD Graphics 530 (SKL GT2)) GPU id = 1 (NVIDIA GeForce GTX 960) GPU id : 0 (Intel(R) HD Graphics 530 (SKL GT2)): GPU id : 1 (NVIDIA GeForce GTX 960): GPU id : 0 (Intel(R) HD Graphics 530 (SKL GT2)): GPU id : 1 (NVIDIA GeForce GTX 960): GPU0: driverVersion = 24.1.5 (100667397) deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU driverUUID = cc628f46-3d02-8624-0c8a-e98e0420c4f6 driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA driverName = Intel open-source Mesa driver driverInfo = Mesa 24.1.5 VK_KHR_driver_properties : extension revision 1 GPU1: driverVersion = 550.100.0.0 (2308505600) deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU driverUUID = 63390c76-60ef-5529-a6ac-99167eb42a9c driverID = DRIVER_ID_NVIDIA_PROPRIETARY driverName = NVIDIA driverInfo = 550.100 VK_KHR_driver_properties : extension revision 1 ----------- Verifying VA-API vainfo Trying display: wayland libva info: VA-API version 1.22.0 libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_21 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.22 (libva 2.22.0) vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 24.1.5 () vainfo: Supported profile and entrypoints VAProfileNone : VAEntrypointVideoProc VAProfileNone : VAEntrypointStats VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointFEI VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointFEI VAProfileH264High : VAEntrypointEncSliceLP VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointFEI VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointFEI ------- Verifying VDPAU vdpauinfo | egrep 'Driver|GPU' Information string: NVIDIA VDPAU Driver Shared Library 550.100 Thu Jun 27 18:11:42 UTC 2024 ===========================================

2 2

Understanding mailing list Moderation
by Phyllis Smith 09 Sep '24

09 Sep '24

Andrea asked the question in another thread: > I would like to ask how long moderation lasts for a new user. It > is now more than a week (with 7 posts) that Tarantas has been > registered... > It looks like it lasts forever until I uncheck the "Mod" box in the mailing list for that user. I think this started only when additional security was added in the last year or so and I did not realize it and had just been approving the messages. Today I looked into it and from now on when I see a new addition, I will know to uncheck the "Mod" box right away.

1 0