Or we need to go and set

in->color_trc

out->color_trc

on avframes directly, like libavfilter/vf_scale.c does ?