Re: [Cin] libaom 3.2.0 patch1 fixed for arm/linux?
On Tuesday, January 11, 2022, <[email protected]> wrote:
On Mon, 10 Jan 2022 22:43:52 +0300 Andrew Randrianasulu <[email protected]> wrote:
<snip>
I am happy to report that the aarch64 version of CinGG builds properly here with libaom enabled. The change in the thirdparty Makefile compared to the git is now only the added creation of a dummy configure script through giflib.cfg_vars .
this is still with my fix for libaom patch, right?
Yep, I can make a diff if you want.
Also,.... does resulting cin binary works? (as in, show gui, allow you loading vids and images, edit them a bit and encode result? obviously on emulated aarch64 machine you want some small video, like 320x240... I think..)
Will test today, audio file loads and plays fine (but still no sound from the VM).
if software can do trick - why upgrade hw? :-)
The faster CPU (from 8 threads Zen to 16 threads Zen3) will speed up rendering too.
yeah... well, if hw upgrade is possibility fir you..
It might be worth investigating if cross-compile is an option, e.g. build an Linux aarch64 version on Linux 86_64. The tools exists, but I don't know if the configure scripts / cmake can handle that. Cross-compiling would limit the slow arm emulation to the testing of the build products.
supposedly proot can help here:
https://lists.nongnu.org/archive/html/qemu-devel/2012-03/msg04870.html
=== PRoot can also mix automatically the execution of host programs and the execution of guest programs emulated by QEMU user-mode. It's a convenient way to speed up build-time by using a cross-compiler instead of emulating the guest compiler. Even when mixing such applications, build-systems still believe they are running in a native guest environment, as a consequence most cross-compilation issues are avoided by design. For instance, with a typical "./configure" script (many lines were removed for readability purpose):
<linux-x86>$ proot -Q qemu-arm /path/to/any/arm-rootfs/
<linux-arm>$ ./configure CC=/host-rootfs/opt/devkit/armv7/bin/armv7-linux-gcc checking whether build environment is sane... yes checking build system type... armv7l-unknown-linux-gnueabi checking host system type... armv7l-unknown-linux-gnueabi checking for gcc... /host-rootfs/opt/devkit/armv7/bin/armv7-linux-gcc checking whether the C compiler works... yes checking whether we are cross compiling... no
I will look at your suggestion of using PRroot. However, that msg is 10 years old, and refers to qemu in user mode. I am just about to move to system mode, because file sharing will not work in user mode, and maybe that will fix the audio issue too.
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs. also, proot still active project (unlike scratchbox2 from same message). Just I thought you might try it as alternative. Bugs might prevent you/us from running it ..
Also, the msg says: ============== As you probably know, QEMU user-mode is mainly used with binfmt_misc, chroot, and sometimes "mount --bind". ============== This is not my understanding as it is used now. You just run a full distro in it, be it Linux or some other OS.
well, qemu's own docs not very useful at this moment https://www.qemu.org/docs/master/user/main.html but Debian's documentation looks better https://wiki.debian.org/QemuUserEmulation === This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. ./program). ==== Main feature from proot (aside from running as regular user - useful for me on non-rooted Android!) seems to be this accelerated cross-compiling - cross-compiler still runs from x86_64 host (not emulated at all - > fast!), yet interacts with headers, libraries and source files inside our (arm/aarch64) chroot! But I haven't tested this yet, due to difference in how clang (my host compiler in Termux) handles cross-compilation..
MatN
1. Attached is the thirdparty Makfile patch to fix the giflib invalid configure script. 2. I tested video rendering with a 24 seconds 1080p30 yuvj420 file wuth two channel audio. Loaded as new project. I used the default HD rendering, and put a radial blur on the video track, without changing its parameters. Rendering went OK, but slow: 4 seconds per frame. Not quite ready for prime time :-) The result played fine in Deb11/aarch64, Fedora35/x86_64 and Mint 19.2/x86_64 , no visible difference from the original.
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway. I will dig into this proot, maybe it is simpler than a --target option for all ./configure scripts.
well, qemu's own docs not very useful at this moment
https://www.qemu.org/docs/master/user/main.html
but Debian's documentation looks better https://wiki.debian.org/QemuUserEmulation === This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. ./program). ====
Almost all documentation I find about qemu/libvirt is outdated in some respects. Execution of a non-native target just like a native one is not so important I think, as long as the build of CinGG goes alright. I prefer each VM as a separate window. MatN
On Tuesday, January 11, 2022, <[email protected]> wrote:
1. Attached is the thirdparty Makfile patch to fix the giflib invalid configure script.
thanks!
2. I tested video rendering with a 24 seconds 1080p30 yuvj420 file wuth two channel audio. Loaded as new project. I used the default HD rendering, and put a radial blur on the video track, without changing its parameters. Rendering went OK, but slow: 4 seconds per frame. Not quite ready for prime time :-) The result played fine in Deb11/aarch64, Fedora35/x86_64 and Mint 19.2/x86_64 , no visible difference from the original.
at least it was not minutes per frame :-)
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I think your terminology on system/user modes a bit different from assumed by qemu? Can you try to explain what you mean by those two modes in your own words/experience?
I will dig into this proot, maybe it is simpler than a --target option for all ./configure scripts.
well, qemu's own docs not very useful at this moment
https://www.qemu.org/docs/master/user/main.html
but Debian's documentation looks better https://wiki.debian.org/QemuUserEmulation === This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. ./program). ====
Almost all documentation I find about qemu/libvirt is outdated in some respects. Execution of a non-native target just like a native one is not so important I think, as long as the build of CinGG goes alright.
I prefer each VM as a separate window.
MatN
On Tue, 11 Jan 2022 18:46:59 +0300 Andrew Randrianasulu <[email protected]> wrote: <snip>
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I think your terminology on system/user modes a bit different from assumed by qemu?
Can you try to explain what you mean by those two modes in your own words/experience?
In virt-manager you can have two kinds of VMs: QEMU/KVM and QEMU/KVM User Session. The first is referred to as "system", the second as "session". The default on Fedora_35 is (user) session, where qemu runs under the user's profile. If you use virsh to e.g. edit the VM's config you can type e.g. "virsh edit Debian11_aarch64" . If you want to use a VM under system, you have to type "virsh --connect qemu:///system edit Debian11_aarch64". The VMs have a different domain specified in the XML that defines a VM. user mode has domain "qemu", system mode domain "kvm". I noticed that whereas in user (session) mode you can define pretty much any hardware for the VM, in system mode some things are not available, like PCIe controllers. MatN
On Tuesday, January 11, 2022, <[email protected]> wrote:
On Tue, 11 Jan 2022 18:46:59 +0300 Andrew Randrianasulu <[email protected]> wrote:
<snip>
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I think your terminology on system/user modes a bit different from assumed by qemu?
Can you try to explain what you mean by those two modes in your own words/experience?
In virt-manager you can have two kinds of VMs: QEMU/KVM and QEMU/KVM User Session. The first is referred to as "system", the second as "session". The default on Fedora_35 is (user) session, where qemu runs under the user's profile. If you use virsh to e.g. edit the VM's config you can type e.g. "virsh edit Debian11_aarch64" . If you want to use a VM under system, you have to type "virsh --connect qemu:///system edit Debian11_aarch64". The VMs have a different domain specified in the XML that defines a VM. user mode has domain "qemu", system mode domain "kvm". I noticed that whereas in user (session) mode you can define pretty much any hardware for the VM, in system mode some things are not available, like PCIe controllers.
ah, thanks... so, this is virt-manager specific terminology.. I was referring to other method of qemu use, one where you call qemu-user-something and not qemu-system-something.
MatN
On Tuesday, January 11, 2022, Andrew Randrianasulu <[email protected]> wrote:
On Tuesday, January 11, 2022, <[email protected]> wrote:
On Tue, 11 Jan 2022 18:46:59 +0300 Andrew Randrianasulu <[email protected]> wrote:
<snip>
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I think your terminology on system/user modes a bit different from assumed by qemu?
Can you try to explain what you mean by those two modes in your own words/experience?
In virt-manager you can have two kinds of VMs: QEMU/KVM and QEMU/KVM User Session. The first is referred to as "system", the second as "session". The default on Fedora_35 is (user) session, where qemu runs under the user's profile. If you use virsh to e.g. edit the VM's config you can type e.g. "virsh edit Debian11_aarch64" . If you want to use a VM under system, you have to type "virsh --connect qemu:///system edit Debian11_aarch64". The VMs have a different domain specified in the XML that defines a VM. user mode has domain "qemu", system mode domain "kvm". I noticed that whereas in user (session) mode you can define pretty much any hardware for the VM, in system mode some things are not available, like PCIe controllers.
https://wiki.libvirt.org/page/FAQ#What_is_the_difference_between_qemu:.2F.2F... ==== What is the difference between qemu:///system and qemu:///session? Which one should I use? All 'system' URIs (be it qemu, lxc, uml, ...) connect to the libvirtd daemon running as root which is launched at system startup. Virtual machines created and run using 'system' are usually launched as root, unless configured otherwise (for example in /etc/libvirt/qemu.conf). All 'session' URIs launch a libvirtd instance as your local user, and all VMs are run with local user permissions. You will definitely want to use qemu:///system if your VMs are acting as servers. VM autostart on host boot only works for 'system', and the root libvirtd instance has necessary permissions to use proper networkings via bridges or virtual networks. qemu:///system is generally what tools like virt-manager default to. qemu:///session has a serious drawback: since the libvirtd instance does not have sufficient privileges, the only out of the box network option is qemu's usermode networking, which has nonobvious limitations, so its usage is discouraged. More info on qemu networking options: http://people.gnome.org/~markmc/qemu-networking.html The benefit of qemu:///session is that permission issues vanish: disk images can easily be stored in $HOME, serial PTYs are owned by the user, etc. ==== well... this is something virt-manager specific as I guessed.. all qemu modes likely run qemu-system-* under the hood (but uml is user-mode linux and lxc is container... mode. non-system, as far as guest kernel not needed for them... if I recall things correctly!). you can check with top while vm is running...
ah, thanks... so, this is virt-manager specific terminology..
I was referring to other method of qemu use, one where you call qemu-user-something and not qemu-system-something.
MatN
On Tue, 11 Jan 2022 22:26:51 +0300 Andrew Randrianasulu <[email protected]> wrote:
On Tuesday, January 11, 2022, Andrew Randrianasulu <[email protected]> wrote:
On Tuesday, January 11, 2022, <[email protected]> wrote:
On Tue, 11 Jan 2022 18:46:59 +0300 Andrew Randrianasulu <[email protected]> wrote:
<snip>
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I think your terminology on system/user modes a bit different from assumed by qemu?
Can you try to explain what you mean by those two modes in your own words/experience?
In virt-manager you can have two kinds of VMs: QEMU/KVM and QEMU/KVM User Session. The first is referred to as "system", the second as "session". The default on Fedora_35 is (user) session, where qemu runs under the user's profile. If you use virsh to e.g. edit the VM's config you can type e.g. "virsh edit Debian11_aarch64" . If you want to use a VM under system, you have to type "virsh --connect qemu:///system edit Debian11_aarch64". The VMs have a different domain specified in the XML that defines a VM. user mode has domain "qemu", system mode domain "kvm". I noticed that whereas in user (session) mode you can define pretty much any hardware for the VM, in system mode some things are not available, like PCIe controllers.
https://wiki.libvirt.org/page/FAQ#What_is_the_difference_between_qemu:.2F.2F...
==== What is the difference between qemu:///system and qemu:///session? Which one should I use? All 'system' URIs (be it qemu, lxc, uml, ...) connect to the libvirtd daemon running as root which is launched at system startup. Virtual machines created and run using 'system' are usually launched as root, unless configured otherwise (for example in /etc/libvirt/qemu.conf).
All 'session' URIs launch a libvirtd instance as your local user, and all VMs are run with local user permissions.
You will definitely want to use qemu:///system if your VMs are acting as servers. VM autostart on host boot only works for 'system', and the root libvirtd instance has necessary permissions to use proper networkings via bridges or virtual networks. qemu:///system is generally what tools like virt-manager default to.
Not anymore. virsh default is session (user), virt-manager shows whatever VMs are existing, in two list depending on the connection.
qemu:///session has a serious drawback: since the libvirtd instance does not have sufficient privileges, the only out of the box network option is qemu's usermode networking, which has nonobvious limitations, so its usage is discouraged. More info on qemu networking options: http://people.gnome.org/~markmc/qemu-networking.html
This networking info is out-of-date, at least for Fedora_35. The Fedora default setup already creates a bridge device, which is needed for user mode networking. It simply worked out of the box, even a network install where the booted system is at a minimum. On none of the three VMs I created so far was networking a problem. I don't remember if the first instance on install was NAT or bridge.
The benefit of qemu:///session is that permission issues vanish: disk images can easily be stored in $HOME, serial PTYs are owned by the user, etc.
What is a problem is file sharing, cut-and-paste, audio. Not of course for real hardware, but would be nice-to-have for the VMs. On Fedora_35, you get either frequent SELinux errors (some of which you can negate), or the solutions found on the web simple don't work, presumable because their qemu was much older.
====
well... this is something virt-manager specific as I guessed.. all qemu modes likely run qemu-system-* under the hood (but uml is user-mode linux and lxc is container... mode. non-system, as far as guest kernel not needed for them... if I recall things correctly!). you can check with top while vm is running...
virt-manager is a GUI front-end for libvirt, which is what really interfaces to a VM. Multiple VM types are supported by libvirt, like XEN and QEMU, but virt-manager is much more limited . You'll find many instructions to use virsh or cmd line to configure qemu. Very uncomfortable compared to virt-manager, which is more like VirtualBox in this respect, although I use virsh as well, but only if I cannot get a new VM install going using virt-manager. Indeed all qemu emulations are called qemu-system-xxxx . When a system emulation is installed (like aarch64) then virt-manager puts it in the list of options for the architecture of a new VM. I have X86_64, armhf, and aarch64 at the moment. My new CPU arrived, and I see a big increase in speed. 2400G versus 5700G: 4 minute 1080p30 video render with default settings: 32.5 fps -> 78 fps (no hardware support). But CPU was never 100%, 80% was peak, obviously there are some non-CPU limits there). It could be it is faster on Mint 20.2 where I can use vaapi for rendering (Fedora does not seem to support that). Compile CinGG after "make new" 12 minutes -> 4 minutes (just enough time to make myself a new coffee...) Boot Debian_11_aarch64 until login screen: 86 secs -> 55 secs. Login until desktop ready (CPU load drops stark): 68 secs -> 38 secs. If feels much more responsive too. Regrettable, I don't think building on aarch64 will be much faster (will time it): virt-manager does not allow more than 8 vCPUs to that VM, maybe it is architecture dependent. However, in many places there are single threaded pieces in the build, they will definitely speed up because the turbo is 30% higher, and there are more instructions per clock. MatN
On Wednesday, January 12, 2022, <[email protected]> wrote:
On Tue, 11 Jan 2022 22:26:51 +0300 Andrew Randrianasulu <[email protected]> wrote:
On Tuesday, January 11, 2022, Andrew Randrianasulu <[email protected]> wrote:
On Tuesday, January 11, 2022, <[email protected]> wrote:
On Tue, 11 Jan 2022 18:46:59 +0300 Andrew Randrianasulu <[email protected]> wrote:
<snip>
> I think you already using system mode (full system emulation > - so you can run NetBsd or MacOS or windows - they see > emulated/virtual machine to run on..) User-mode qemu run > Linux binaries on top of same kernel BUT they can belong to > another architecture! So overhead can be less.. (no mmu > emulation). You can edit files inside proot 'vm' from host - > no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I think your terminology on system/user modes a bit different from assumed by qemu?
Can you try to explain what you mean by those two modes in your own words/experience?
In virt-manager you can have two kinds of VMs: QEMU/KVM and QEMU/KVM User Session. The first is referred to as "system", the second as "session". The default on Fedora_35 is (user) session, where qemu runs under the user's profile. If you use virsh to e.g. edit the VM's config you can type e.g. "virsh edit Debian11_aarch64" . If you want to use a VM under system, you have to type "virsh --connect qemu:///system edit Debian11_aarch64". The VMs have a different domain specified in the XML that defines a VM. user mode has domain "qemu", system mode domain "kvm". I noticed that whereas in user (session) mode you can define pretty much any hardware for the VM, in system mode some things are not available, like PCIe controllers.
https://wiki.libvirt.org/page/FAQ#What_is_the_difference_ between_qemu:.2F.2F.2Fsystem_and_qemu:.2F.2F.2Fsession.3F_ Which_one_should_I_use.3F
==== What is the difference between qemu:///system and qemu:///session? Which one should I use? All 'system' URIs (be it qemu, lxc, uml, ...) connect to the libvirtd daemon running as root which is launched at system startup. Virtual machines created and run using 'system' are usually launched as root, unless configured otherwise (for example in /etc/libvirt/qemu.conf).
All 'session' URIs launch a libvirtd instance as your local user, and all VMs are run with local user permissions.
You will definitely want to use qemu:///system if your VMs are acting as servers. VM autostart on host boot only works for 'system', and the root libvirtd instance has necessary permissions to use proper networkings via bridges or virtual networks. qemu:///system is generally what tools like virt-manager default to.
Not anymore. virsh default is session (user), virt-manager shows whatever VMs are existing, in two list depending on the connection.
good to know, in case someone will need this info!
qemu:///session has a serious drawback: since the libvirtd instance does not have sufficient privileges, the only out of the box network option is qemu's usermode networking, which has nonobvious limitations, so its usage is discouraged. More info on qemu networking options: http://people.gnome.org/~markmc/qemu-networking.html
This networking info is out-of-date, at least for Fedora_35. The Fedora default setup already creates a bridge device, which is needed for user mode networking. It simply worked out of the box, even a network install where the booted system is at a minimum. On none of the three VMs I created so far was networking a problem. I don't remember if the first instance on install was NAT or bridge.
good!
The benefit of qemu:///session is that permission issues vanish: disk images can easily be stored in $HOME, serial PTYs are owned by the user, etc.
What is a problem is file sharing, cut-and-paste, audio. Not of course for real hardware, but would be nice-to-have for the VMs. On Fedora_35, you get either frequent SELinux errors (some of which you can negate), or the solutions found on the web simple don't work, presumable because their qemu was much older.
well, hopefully we can troubleshut them one by one. Just on new email thread.
====
well... this is something virt-manager specific as I guessed.. all qemu modes likely run qemu-system-* under the hood (but uml is user-mode linux and lxc is container... mode. non-system, as far as guest kernel not needed for them... if I recall things correctly!). you can check with top while vm is running...
virt-manager is a GUI front-end for libvirt, which is what really interfaces to a VM. Multiple VM types are supported by libvirt, like XEN and QEMU, but virt-manager is much more limited .
You'll find many instructions to use virsh or cmd line to configure qemu. Very uncomfortable compared to virt-manager, which is more like VirtualBox in this respect, although I use virsh as well, but only if I cannot get a new VM install going using virt-manager.
Indeed all qemu emulations are called qemu-system-xxxx . When a system emulation is installed (like aarch64) then virt-manager puts it in the list of options for the architecture of a new VM. I have X86_64, armhf, and aarch64 at the moment.
I found two papers/slideshows describing qemu-system vs qemu-user back in qemu-0.12.5 days proot + qemu-user was like 2x-4x faster than qemu-system (compilers tend to use integer arithmetic units, not fpu/simd). But since those days qemu in general improved and some devices now virtualized (less overhead) https://web.archive.org/web/20111119183403if_/http://adt.cs.upb.de:80/quf/qu... https://web.archive.org/web/20150301073841if_/http://adt.cs.upb.de:80/quf/qu... (see p. 48)
My new CPU arrived, and I see a big increase in speed. 2400G versus 5700G: 4 minute 1080p30 video render with default settings: 32.5 fps -> 78 fps (no hardware support). But CPU was never 100%, 80% was peak, obviously there are some non-CPU limits there). It could be it is faster on Mint 20.2 where I can use vaapi for rendering (Fedora does not seem to support that).
it should - one more item to troubleshoot.. Compile CinGG after "make new" 12 minutes -> 4 minutes (just enough
time to make myself a new coffee...)
3x faster! nice
Boot Debian_11_aarch64 until login screen: 86 secs -> 55 secs. Login until desktop ready (CPU load drops stark): 68 secs -> 38 secs. If feels much more responsive too.
cool! I was afraid new cpu might be delayed into infinity due to electroncs components shortages/delays...
Regrettable, I don't think building on aarch64 will be much faster (will time it): virt-manager does not allow more than 8 vCPUs to that VM, maybe it is architecture dependent. However, in many places there are single threaded pieces in the build, they will definitely speed up because the turbo is 30% higher, and there are more instructions per clock.
I hope cooling part (fan assembly) and PSU/motherboard will handle this well.. good luck with new machine!
MatN
MatN and Andrew, I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not? On Tue, Jan 11, 2022 at 8:23 AM mnieuw--- via Cin < [email protected]> wrote:
1. Attached is the thirdparty Makfile patch to fix the giflib invalid configure script.
2. I tested video rendering with a 24 seconds 1080p30 yuvj420 file wuth two channel audio. Loaded as new project. I used the default HD rendering, and put a radial blur on the video track, without changing its parameters. Rendering went OK, but slow: 4 seconds per frame. Not quite ready for prime time :-) The result played fine in Deb11/aarch64, Fedora35/x86_64 and Mint 19.2/x86_64 , no visible difference from the original.
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I will dig into this proot, maybe it is simpler than a --target option for all ./configure scripts.
well, qemu's own docs not very useful at this moment
https://www.qemu.org/docs/master/user/main.html
but Debian's documentation looks better https://wiki.debian.org/QemuUserEmulation === This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. ./program). ====
Almost all documentation I find about qemu/libvirt is outdated in some respects. Execution of a non-native target just like a native one is not so important I think, as long as the build of CinGG goes alright.
I prefer each VM as a separate window.
MatN -- Cin mailing list [email protected] https://lists.cinelerra-gg.org/mailman/listinfo/cin
On Wednesday, January 12, 2022, Phyllis Smith <[email protected]> wrote:
MatN and Andrew, I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not?
I can't comment much on libgif patch (Cingg worked dor me without it), but my patch only should have effect on Linux/arm, I do not have suitable chroot for testing, but from stackoverflow answers it seems __linux__ is right define, even if ANDROID and TERMUX are all-caps, and *BSD spelled with capitalization too, see example: https://www.boost.org/doc/libs/1_63_0/boost/config/platform/bsd.hpp)
On Tue, Jan 11, 2022 at 8:23 AM mnieuw--- via Cin < [email protected]> wrote:
1. Attached is the thirdparty Makfile patch to fix the giflib invalid configure script.
2. I tested video rendering with a 24 seconds 1080p30 yuvj420 file wuth two channel audio. Loaded as new project. I used the default HD rendering, and put a radial blur on the video track, without changing its parameters. Rendering went OK, but slow: 4 seconds per frame. Not quite ready for prime time :-) The result played fine in Deb11/aarch64, Fedora35/x86_64 and Mint 19.2/x86_64 , no visible difference from the original.
I think you already using system mode (full system emulation - so you can run NetBsd or MacOS or windows - they see emulated/virtual machine to run on..) User-mode qemu run Linux binaries on top of same kernel BUT they can belong to another architecture! So overhead can be less.. (no mmu emulation). You can edit files inside proot 'vm' from host - no need for samba/nfs.
I have macOS in user mode, it runs fine (but need to re-install). It also ran fine in system mode (since deleted). I have not checked if there is a speed difference between the two nodes, nothing very noticable anyway.
I will dig into this proot, maybe it is simpler than a --target option for all ./configure scripts.
well, qemu's own docs not very useful at this moment
https://www.qemu.org/docs/master/user/main.html
but Debian's documentation looks better https://wiki.debian.org/QemuUserEmulation === This page describes how to setup and use QEMU user emulation in a "transparent" fashion, allowing execution of non-native target executables just like native ones (i.e. ./program). ====
Almost all documentation I find about qemu/libvirt is outdated in some respects. Execution of a non-native target just like a native one is not so important I think, as long as the build of CinGG goes alright.
I prefer each VM as a separate window.
MatN -- Cin mailing list [email protected] https://lists.cinelerra-gg.org/mailman/listinfo/cin
On Thu, 2022-01-13 at 14:38 +0300, Andrew Randrianasulu wrote:
On Wednesday, January 12, 2022, Phyllis Smith <[email protected]> wrote:
MatN and Andrew, I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not?
I can't comment much on libgif patch (Cingg worked dor me without it), but my patch only should have effect on Linux/arm, I do not have suitable chroot for testing, but from stackoverflow answers it seems __linux__ is right define, even if ANDROID and TERMUX are all-caps, and *BSD spelled with capitalization too, see example: https://www.boost.org/doc/libs/1_63_0/boost/config/platform/bsd.hpp)
<snip> I tested both again to make sure. === giflib === Without the giflib change: Makefile: giflib.cfg_params=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && ./configure echo "exec true" > ./configure; chmod +x ./configure; and the configure file is 0 bytes. With the giflib change: Makefile: giflib.cfg_vars=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && echo "exec true" > ./configure; chmod +x ./configure; ./configure and the configure file is 10 bytes. So, it builds without the fix because the last cmd is a chmod which return OK. But it is by accident. So the fix should be in the git. There are two more with bad configure files, I will look at them. The Makefile always calls ./configure in the root of the unpacked source directory, so there should be a configure script, which in case of unused, should return true. == libaom patch==== The 20220108_Andrew patch for libaom, which changes __LINUX__ to __linux__ is not needed for X86_64 builds, it builds fine with either. However, it fails building on Debian_11/aarch64 compiling libaom_v3.2.0/aom_ports/arm_cpudetect.c . With the patch, which lowercases __linux__, it builds fine. So this fix should be in the git too. MatN
On Thursday, January 13, 2022, mat <[email protected]> wrote:
On Thu, 2022-01-13 at 14:38 +0300, Andrew Randrianasulu wrote:
On Wednesday, January 12, 2022, Phyllis Smith <[email protected]> wrote:
MatN and Andrew, I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not?
I can't comment much on libgif patch (Cingg worked dor me without it), but my patch only should have effect on Linux/arm, I do not have suitable chroot for testing, but from stackoverflow answers it seems __linux__ is right define, even if ANDROID and TERMUX are all-caps, and *BSD spelled with capitalization too, see example: https://www.boost.org/doc/ libs/1_63_0/boost/config/platform/bsd.hpp)
<snip> I tested both again to make sure. === giflib === Without the giflib change: Makefile: giflib.cfg_params=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && ./configure echo "exec true" > ./configure; chmod +x ./configure; and the configure file is 0 bytes.
With the giflib change: Makefile: giflib.cfg_vars=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && echo "exec true" > ./configure; chmod +x ./configure; ./configure and the configure file is 10 bytes.
So, it builds without the fix because the last cmd is a chmod which return OK. But it is by accident. So the fix should be in the git.
There are two more with bad configure files, I will look at them. The Makefile always calls ./configure in the root of the unpacked source directory, so there should be a configure script, which in case of unused, should return true.
== libaom patch==== The 20220108_Andrew patch for libaom, which changes __LINUX__ to __linux__ is not needed for X86_64 builds, it builds fine with either. However, it fails building on Debian_11/aarch64 compiling libaom_v3.2.0/aom_ports/arm_cpudetect.c .
With the patch, which lowercases __linux__, it builds fine.
So this fix should be in the git too.
thanks a lot for time-consuming aarch64 testing! and yeah, more consistent Makefile always better even for our own future reference!
MatN
On Thu, 2022-01-13 at 15:50 +0300, Andrew Randrianasulu wrote:
On Thursday, January 13, 2022, mat <[email protected]> wrote:
On Thu, 2022-01-13 at 14:38 +0300, Andrew Randrianasulu wrote:
On Wednesday, January 12, 2022, Phyllis Smith <[email protected]> wrote:
MatN and Andrew, I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not?
I can't comment much on libgif patch (Cingg worked dor me without it), but my patch only should have effect on Linux/arm, I do not have suitable chroot for testing, but from stackoverflow answers it seems __linux__ is right define, even if ANDROID and TERMUX are all-caps, and *BSD spelled with capitalization too, see example: https://www.boost.org/doc/libs/1_63_0/boost/config/platform/bsd.hpp )
<snip> I tested both again to make sure. === giflib === Without the giflib change: Makefile: giflib.cfg_params=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && ./configure echo "exec true" > ./configure; chmod +x ./configure; and the configure file is 0 bytes.
With the giflib change: Makefile: giflib.cfg_vars=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && echo "exec true" > ./configure; chmod +x ./configure; ./configure and the configure file is 10 bytes.
So, it builds without the fix because the last cmd is a chmod which return OK. But it is by accident. So the fix should be in the git.
There are two more with bad configure files, I will look at them. The Makefile always calls ./configure in the root of the unpacked source directory, so there should be a configure script, which in case of unused, should return true.
== libaom patch==== The 20220108_Andrew patch for libaom, which changes __LINUX__ to __linux__ is not needed for X86_64 builds, it builds fine with either. However, it fails building on Debian_11/aarch64 compiling libaom_v3.2.0/aom_ports/arm_cpudetect.c .
With the patch, which lowercases __linux__, it builds fine.
So this fix should be in the git too.
thanks a lot for time-consuming aarch64 testing!
and yeah, more consistent Makefile always better even for our own future reference!
I hope to get it to compile on macOS too, because one programmer wanted to do that and he has Apple only. Another programmer would be good. I am also trying to have migrate the VMs to the system environment, but is is a steep learning curve, and any web-found instructions are out of date. If it runs as system, it might actually be faster. Plus, it might expose host hardware features, like vaapi (not on arm, I think). MatN
On Thursday, January 13, 2022, mat <[email protected]> wrote:
On Thu, 2022-01-13 at 15:50 +0300, Andrew Randrianasulu wrote:
On Thursday, January 13, 2022, mat <[email protected]> wrote:
On Thu, 2022-01-13 at 14:38 +0300, Andrew Randrianasulu wrote:
On Wednesday, January 12, 2022, Phyllis Smith <[email protected]> wrote:
MatN and Andrew, I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not?
I can't comment much on libgif patch (Cingg worked dor me without it), but my patch only should have effect on Linux/arm, I do not have suitable chroot for testing, but from stackoverflow answers it seems __linux__ is right define, even if ANDROID and TERMUX are all-caps, and *BSD spelled with capitalization too, see example: https://www.boost.org/doc/libs /1_63_0/boost/config/platform/bsd.hpp)
<snip> I tested both again to make sure. === giflib === Without the giflib change: Makefile: giflib.cfg_params=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && ./configure echo "exec true" > ./configure; chmod +x ./configure; and the configure file is 0 bytes.
With the giflib change: Makefile: giflib.cfg_vars=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && echo "exec true" > ./configure; chmod +x ./configure; ./configure and the configure file is 10 bytes.
So, it builds without the fix because the last cmd is a chmod which return OK. But it is by accident. So the fix should be in the git.
There are two more with bad configure files, I will look at them. The Makefile always calls ./configure in the root of the unpacked source directory, so there should be a configure script, which in case of unused, should return true.
== libaom patch==== The 20220108_Andrew patch for libaom, which changes __LINUX__ to __linux__ is not needed for X86_64 builds, it builds fine with either. However, it fails building on Debian_11/aarch64 compiling libaom_v3.2.0/aom_ports/arm_cpudetect.c .
With the patch, which lowercases __linux__, it builds fine.
So this fix should be in the git too.
thanks a lot for time-consuming aarch64 testing!
and yeah, more consistent Makefile always better even for our own future reference!
I hope to get it to compile on macOS too, because one programmer wanted to do that and he has Apple only. Another programmer would be good.
I am also trying to have migrate the VMs to the system environment, but is is a steep learning curve, and any web-found instructions are out of date. If it runs as system, it might actually be faster. Plus, it might expose host hardware features, like vaapi (not on arm, I think).
there was patchset aimed at improved 3d/sound in qemu-user, back in 2016. But I guess it never landed fully https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg05334.html
MatN
I need clarification please. 1) Thirdparty-Makefile-giflib.patch that Mat attached is definitely needed and should be checked into GIT? probably yes? 2) libaom-v3.2.0.patch1 that Andrew provided on Jan. 08; is this needed? probably not? 3) have I missed some other changes? probably not?
<snip> I tested both again to make sure. === giflib === Without the giflib change: Makefile: giflib.cfg_params=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && ./configure echo "exec true" > ./configure; chmod +x ./configure; and the configure file is 0 bytes.
With the giflib change: Makefile: giflib.cfg_vars=echo "exec true" > ./configure; chmod +x ./configure; log: CONFIGURING giflib cd giflib* && echo "exec true" > ./configure; chmod +x ./configure; ./configure and the configure file is 10 bytes.
So, it builds without the fix because the last cmd is a chmod which return OK. But it is by accident. So the fix should be in the git.
OK, checked into GIT as stated after building on my laptop system.
There are two more with bad configure files, I will look at them. The Makefile always calls ./configure in the root of the unpacked source directory, so there should be a configure script, which in case of unused, should return true.
OK, will wait for your feedback if more needed.
== libaom patch==== The 20220108_Andrew patch for libaom, which changes __LINUX__ to __linux__ is not needed for X86_64 builds, it builds fine with either. However, it fails building on Debian_11/aarch64 compiling libaom_v3.2.0/aom_ports/arm_cpudetect.c .
With the patch, which lowercases __linux__, it builds fine.
So this fix should be in the git too.
OK, checked into GIT as stated after building on my laptop system.
Please find attached a thirdparty/Makefile patch, that corrects the two remaining invalid dummy configure scripts. Tested on Fedora_35/X86_64 and Debian_11/aarch64. This fix does not fix any problems, it only clears up the process. CinGG will build fine without it, but with this patch all thirdparty sources are treated the same. MatN
participants (4)
-
Andrew Randrianasulu -
mat -
mnieuw＠zap.a2000.nl
-
Phyllis Smith