> Try deleting .cfg files from "mamedir\cfg\" folder... or rename the cfg folder > entirely as a backup so you can restore it later if that's not the problem.
Or just run mame with the -cc switch so it creates a default config file.
I used to have a lot of problems with MAME's video output too because I had really ancient config files that had stuff that was no longer supported. I just created a new config, ported over my folders and other settings, and things work fine now.