[Cin] 10bit VAAPI encoding test thread

[Andrew Randrianasulu]

сб, 26 окт. 2024 г., 00:58 Terje J. Hanssen <terjejhanssen at gmail.com>:

>
>
>
> Den 25.10.2024 21:48, skrev Andrew Randrianasulu:
>
> We branching!
>
> I looked at gpu-screen-recorder
>
> https://git.dec05eba.com/gpu-screen-recorder/tree/src/encoder/video/vaapi.c
>
> it sets
>
> AVHWFramesContext *hw_frame_context = (AVHWFramesContext*)frame_context->data;
>     hw_frame_context->width = video_codec_context->width;
>     hw_frame_context->height = video_codec_context->height;
>     hw_frame_context->sw_format = self->params.color_depth == GSR_COLOR_DEPTH_10_BITS ? AV_PIX_FMT_P010LE : AV_PIX_FMT_NV12;
>     hw_frame_context->format = video_codec_context->pix_fmt;
>     hw_frame_context->device_ctx = (AVHWDeviceContext*)self->device_ctx->data;
>
>   so may be we need to set AV_PIX_FMT_P010LE and not just P010?
>
>
> Sorry, It didn't render more than 8bit yuv420p
>


yeahh ... probably need more  debugging (seeing what pixel format goes into
ffmpeg vaapi interface at least).

I only have decoding on nouveau, and quite flaky at that - only 2 files so
far worked w,/o crashing, from around 2011/12.


I tried to change two instances of NV12 to P010LE and it  shows no visible
effect, so I assume it was supposed to  work with encoding ...?

it does not work even with qsv set to p010 AND patch ?

if so, I am afraid I only can put this item on shelf ... for now. Look at
mantis bug database, if there is no entry about 10bit vaapi
deciding/encoding - add it please ....

you can also test (at some point in time ) this gpu_screen_recorder, it
builds but does not work for me due to "nouveau" driver.



>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cinelerra-gg.org/pipermail/cin/attachments/20241026/ee55afa6/attachment-0001.htm>
-------------- next part --------------
diff --git a/cinelerra-5.1/cinelerra/ffmpeg.C b/cinelerra-5.1/cinelerra/ffmpeg.C
index 7521c0ba..26bf1bef 100644
--- a/cinelerra-5.1/cinelerra/ffmpeg.C
+++ b/cinelerra-5.1/cinelerra/ffmpeg.C
@@ -1229,7 +1229,7 @@ AVHWDeviceType FFVideoStream::encode_hw_activate(const char *hw_dev)
 	if( type != AV_HWDEVICE_TYPE_NONE ) {
 		AVHWFramesContext *frames_ctx = (AVHWFramesContext *)(hw_frames_ref->data);
 		frames_ctx->format = AV_PIX_FMT_VAAPI;
-		frames_ctx->sw_format = AV_PIX_FMT_NV12;
+		frames_ctx->sw_format = AV_PIX_FMT_P010LE;
 		frames_ctx->width = width;
 		frames_ctx->height = height;
 		frames_ctx->initial_pool_size = 0; // 200;
@@ -1391,7 +1391,7 @@ int FFVideoStream::init_frame(AVFrame *picture)
 {
 	switch( avctx->pix_fmt ) {
 	case AV_PIX_FMT_VAAPI:
-		picture->format = AV_PIX_FMT_NV12;
+		picture->format = AV_PIX_FMT_P010LE;
 		break;
 	default:
 		picture->format = avctx->pix_fmt;