The "Story of Dav1d" with Cinelerra builds (AV1 decoder)
As pointed out by Andrew (BT #214), using libdav1d as opposed to libaom for decoding (no encoding) is so attractive because it pretty much runs at full speed. So when FFmpeg 4.2 included the compile option for this, we were highly motivated to put this in as the default for AV1 media loading/editing. AV1 is yet another codec that seems to be newly relevant. However as mentioned in a previous email, dav1d inclusion was a big problem for the older system monthly builds. First, meson had to be at or above version 47 and only 5 out of 18 build systems were. Consequently, GG ended up eliminating the "meson" requirement and substituting the standard "make" instead. Then the "nasm" assembler on some of the older systems was not able to compile Dav1d so they had to be switched out for a newer/working version (11 out of 19 were OK). If we had known the problems that would be encountered in applying libdav1d, I wonder if we would have included it at this time? Also, as mentioned in the BT #214, a remapping for decoding with libdav1d being the preference is in the {cinelerra_path}/ffmpeg/decode.opts file. To revert to using libaom instead for decoding, you will need to comment out the line: remap_video_decoder libaom-av1=libdav1d every time you update Cinelerra. We have not tested what happens if there is either no libdav1d or libaom so if that creates a different problem, we will have to address that. gg/Phyllis *The following information is for the people who do their own builds.* The easy workaround, if you do not care about AV1 files, is to add to the configure line in your script: ./configure .... --disable-dav1d The better fix, is to upgrade nasm to at least 2.13 or higher. Here is the list of the problem distro systems: *libdav1d requires minimum nasm>=2.13 && meson>=0.47 * *In the chart below * designates compile works, x designates compile fails* *meson nasm os kernel* 0.51.2 * 2.14.02 * arch linux (5.2.9-arch1-1-ARCH) missing x 2.10.07 x centos 3.10.0-957.10.1.el7.x86_64 0.21.0 x 2.11.05 x debian8 3.16.0-10-amd64 0.37.1 x 2.12.01 x debian9 4.9.0-9-amd64 0.37.1 x 2.12.01 x debian9i 4.9.0-9-686-pae 0.49.2 * 2.14 * debian10 4.19.0-5-amd64 0.47.2 * 2.13.03 * fc28 5.0.16-100.fc28.x86_64 0.51.1 * 2.13.03 * fc29 5.2.7-100.fc29.x86_644 0.50.1 * 2.13.03 * fc30 5.2.9-200.fc30.x86_644 missing x 2.13.03 * su42.3 4.4.180-102-default 0.46.0 x 2.13.02 * leap05 4.12.14-lp151.28.13-default 0.40.1 x 2.11.08 x mint18 4.8.0-53-generic 0.45.1 x 2.13.02 * mint19 4.15.0-20-generic missing x 2.10.09 x ub14 3.13.0-170-generic 0.29.0 x 2.11.08 x ub16 4.4.0-159-generic 0.45.1 x 2.13.02 * 4.15.0-58-generic missing x 2.10.09 x ub14i 3.16.0-77-generic 0.46.1 x * slk32 0.46.1 x * slk34 *Summary:* meson 14 fail, 5 succeed nasm 8 fail, 11 succeed To get a working set of builds for the month, the way gg used was to: - generate a Makefile for x86 to replace meson (for now). - copy nasm "NASM version 2.13.03 compiled on Feb 2 2019" to: debian8 debian9 debian9i leap03 ub16 ub14 ubuntu mint18
В сообщении от Tuesday 03 September 2019 00:00:32 Phyllis Smith написал(а):
As pointed out by Andrew (BT #214), using libdav1d as opposed to libaom for decoding (no encoding) is so attractive because it pretty much runs at full speed. So when FFmpeg 4.2 included the compile option for this, we were highly motivated to put this in as the default for AV1 media loading/editing. AV1 is yet another codec that seems to be newly relevant.
However as mentioned in a previous email, dav1d inclusion was a big problem for the older system monthly builds. First, meson had to be at or above version 47 and only 5 out of 18 build systems were. Consequently, GG ended up eliminating the "meson" requirement and substituting the standard "make" instead. Then the "nasm" assembler on some of the older systems was not able to compile Dav1d so they had to be switched out for a newer/working version (11 out of 19 were OK).
If we had known the problems that would be encountered in applying libdav1d, I wonder if we would have included it at this time?
Sorry, for not testing all those combination, I tend to just add/update new software as some program ask for it. And being Slackware OS can be moved both ways (up/downgrade) equally easy. I think obvious fix for nasm problem will be testing nasm version in cinelerra-5.1/configure.ac (like lines 464/465 currnetly do, but for something specific for 2.13 and up) and disable libdav1d with message if nasm too old. nasm --version NASM version 2.13.03 compiled on Mar 11 2018 Or may be compile nasm as part of Cin-GG build system, but I'm afraid this way will lead to Cin OS :}
Also, as mentioned in the BT #214, a remapping for decoding with libdav1d being the preference is in the {cinelerra_path}/ffmpeg/decode.opts file. To revert to using libaom instead for decoding, you will need to comment out the line: remap_video_decoder libaom-av1=libdav1d every time you update Cinelerra. We have not tested what happens if there is either no libdav1d or libaom so if that creates a different problem, we will have to address that. gg/Phyllis
*The following information is for the people who do their own builds.*
The easy workaround, if you do not care about AV1 files, is to add to the configure line in your script: ./configure .... --disable-dav1d The better fix, is to upgrade nasm to at least 2.13 or higher.
Here is the list of the problem distro systems:
*libdav1d requires minimum nasm>=2.13 && meson>=0.47 * *In the chart below * designates compile works, x designates compile fails* *meson nasm os kernel* 0.51.2 * 2.14.02 * arch linux (5.2.9-arch1-1-ARCH) missing x 2.10.07 x centos 3.10.0-957.10.1.el7.x86_64 0.21.0 x 2.11.05 x debian8 3.16.0-10-amd64 0.37.1 x 2.12.01 x debian9 4.9.0-9-amd64 0.37.1 x 2.12.01 x debian9i 4.9.0-9-686-pae 0.49.2 * 2.14 * debian10 4.19.0-5-amd64 0.47.2 * 2.13.03 * fc28 5.0.16-100.fc28.x86_64 0.51.1 * 2.13.03 * fc29 5.2.7-100.fc29.x86_644 0.50.1 * 2.13.03 * fc30 5.2.9-200.fc30.x86_644 missing x 2.13.03 * su42.3 4.4.180-102-default 0.46.0 x 2.13.02 * leap05 4.12.14-lp151.28.13-default 0.40.1 x 2.11.08 x mint18 4.8.0-53-generic 0.45.1 x 2.13.02 * mint19 4.15.0-20-generic missing x 2.10.09 x ub14 3.13.0-170-generic 0.29.0 x 2.11.08 x ub16 4.4.0-159-generic 0.45.1 x 2.13.02 * 4.15.0-58-generic missing x 2.10.09 x ub14i 3.16.0-77-generic 0.46.1 x * slk32 0.46.1 x * slk34
*Summary:* meson 14 fail, 5 succeed nasm 8 fail, 11 succeed
To get a working set of builds for the month, the way gg used was to:
- generate a Makefile for x86 to replace meson (for now). - copy nasm "NASM version 2.13.03 compiled on Feb 2 2019" to: debian8 debian9 debian9i leap03 ub16 ub14 ubuntu mint18
Andrew,
Sorry, for not testing all those combination, I tend to just add/update new software as some program ask for it. And being Slackware OS can be moved both ways (up/downgrade) equally easy.
No problem -- testing of all the distros is most easily done here when we do the monthly builds. In fact we were quite surprised at the Slackware build working so well here.
I think obvious fix for nasm problem will be testing nasm version in cinelerra-5.1/configure.ac (like lines 464/465 currnetly do, but for something specific for 2.13 and up) and disable libdav1d with message if nasm too old.
nasm --version NASM version 2.13.03 compiled on Mar 11 2018
Or may be compile nasm as part of Cin-GG build system, but I'm afraid this way will lead to Cin OS :}
That is funny! a Cin OS! Also, FYI - we did contact dav1d software guru - JBKempf - and gave him our compliments and stated our concerns which he considered. Just wanted to let him know that one of our biggest concerns was the impediment other software packages may have incorporating libdav1d into their packages for the same reasons. But in the end, like most developers, he prefers to move forward with the latest. We still think it is worth putting into Cinelerra and hope that ffmpeg eventually adds an encode speed up too. Phyllis/GG
В сообщении от Tuesday 03 September 2019 17:33:32 Phyllis Smith написал(а):
Andrew,
Sorry, for not testing all those combination, I tend to just add/update new software as some program ask for it. And being Slackware OS can be moved both ways (up/downgrade) equally easy.
No problem -- testing of all the distros is most easily done here when we do the monthly builds. In fact we were quite surprised at the Slackware build working so well here.
I think obvious fix for nasm problem will be testing nasm version in cinelerra-5.1/configure.ac (like lines 464/465 currnetly do, but for something specific for 2.13 and up) and disable libdav1d with message if nasm too old.
nasm --version NASM version 2.13.03 compiled on Mar 11 2018
Or may be compile nasm as part of Cin-GG build system, but I'm afraid this way will lead to Cin OS :}
That is funny! a Cin OS!
Also, FYI - we did contact dav1d software guru - JBKempf - and gave him our compliments and stated our concerns which he considered. Just wanted to let him know that one of our biggest concerns was the impediment other software packages may have incorporating libdav1d into their packages for the same reasons. But in the end, like most developers, he prefers to move forward with the latest.
We still think it is worth putting into Cinelerra and hope that ffmpeg eventually adds an encode speed up too. Phyllis/GG
Aha, for static Makefile GG added ... +$(NASM_BLD)/%.obj: src/x86/%.asm + $(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@ replace elf64 with elf32 for 32-bit builds ..... (looking at my source folder for 32-bit libdav1d build via Meson/Ninja) actually there are two instances of this parameter, another for (libdav1d) testfiles down the file .... +$(TESTS_CHKASM_BLD)/x86/%.obj: $(TESTS_CHKASM)/x86/%.asm + @$(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
Andrea, gg is interested in this. Did you find a way to make dav1d work for 32-bit? Phyllis/GG
Aha, for static Makefile GG added ...
+$(NASM_BLD)/%.obj: src/x86/%.asm + $(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
replace elf64 with elf32 for 32-bit builds ..... (looking at my source folder for 32-bit libdav1d build via Meson/Ninja)
actually there are two instances of this parameter, another for (libdav1d) testfiles down the file ....
+$(TESTS_CHKASM_BLD)/x86/%.obj: $(TESTS_CHKASM)/x86/%.asm + @$(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
В сообщении от Thursday 05 September 2019 17:57:59 Phyllis Smith написал(а):
Andrea, gg is interested in this. Did you find a way to make dav1d work for 32-bit? Phyllis/GG
Not inside CinGG buildsystem (yet). I use FFMPEG_CFG override, so ffmpeg picks up system-wide installed copy of libav1d. And System-wide copy was done with Slackbuild, therefore using whatever Meson was cooked for my system. I'll try to convince CinGG to unpack/build its own copy of libdav1d, while I have sources unpacked.
Aha, for static Makefile GG added ...
+$(NASM_BLD)/%.obj: src/x86/%.asm + $(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
replace elf64 with elf32 for 32-bit builds ..... (looking at my source folder for 32-bit libdav1d build via Meson/Ninja)
actually there are two instances of this parameter, another for (libdav1d) testfiles down the file ....
+$(TESTS_CHKASM_BLD)/x86/%.obj: $(TESTS_CHKASM)/x86/%.asm + @$(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
В сообщении от Thursday 05 September 2019 17:57:59 Phyllis Smith написал(а):
Andrea, gg is interested in this. Did you find a way to make dav1d work for 32-bit? Phyllis/GG
Something like attached Makefile works, at least not errors out patch (diff -u) made from dav1d directory also attached I used this info as reference: https://stackoverflow.com/questions/9279765/how-to-detect-os-in-a-make-file
Aha, for static Makefile GG added ...
+$(NASM_BLD)/%.obj: src/x86/%.asm + $(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
replace elf64 with elf32 for 32-bit builds ..... (looking at my source folder for 32-bit libdav1d build via Meson/Ninja)
actually there are two instances of this parameter, another for (libdav1d) testfiles down the file ....
+$(TESTS_CHKASM_BLD)/x86/%.obj: $(TESTS_CHKASM)/x86/%.asm + @$(NASM) -f elf64 \ + -I src/ -I $(BUILD)/ -MQ $@ -MF $@.ndep \ + $< -o $@
participants (2)
-
Andrew Randrianasulu -
Phyllis Smith