Is LibRetro by the same guy who was trying to make MAMElib (or LibMAME?) many years ago? I recall that he had built MAME as a library with an API and he was using it in some kind of front end where the front end would start and stop independent instances of MAME and run them as virtual screens in a 3D cabinet or something like that. I need to try to find a link.
Conceptually, it didn't sound like a bad idea. He wanted to decouple the front end, which I think may have still been the Win32 MAMEUI, from the emulation engine itself. As long as the core API for MAME didn't change that much from version to version, you could just (in theory) drop in a new MAME version (DLL?) and keep the same UI setup.
Anyway, I remember him getting a hostile reception in the forums and I never heard about it again.