Decided to do a fresh install for msys64. I am currently unable to compile with mingw64.exe due to an "MINGW32 is not set" error and using config64.bat under "C:\msys64\win32" doesn't work either. Compiling under win32env.bat still gives me the "out of memory" error.
There is very little information on this stuff on how to get a compile environment for Mame properly working. Is 16GB of ram not enough anymore?