The problem with having 2 libaom versions in thirdparty/src is that the build will always try to build the latest one even when configure.ac uses a previous version like in the following:

PKG_3RD([libaom],[auto],
  [libaom-v3.4.0],
  [ usr/local/lib*/libaom*.a ],
  [ usr/local/include ])

As Andrew already noted, v 3.8.0 does not build on Ubuntu 16 due to cmake.  So I tested the above and it fails too.  The problem is we would like for users who do their own builds for it to be automatic and to be able to build without having to delete stuff to do so.

The good news is that libaom3.8.0 DOES build on Debian 9.1-32 bit.  But I did not execute any render test yet.