OK - So apparently the only limitation on filename characters with romcmp is the name of the zip file itself.
The files/roms internal to the zip file can contain extended UTF-8 characters in their file names. It's just that the zip file names that romcmp opens seem to have to be ASCII standard (or something similar).