I think this have something to do with Windows file caching system. First time files are accessed, it's very slow. Further access to same files are already in Windows file caching system and access to those files are instantaneous.
This always bugged me too. It's not just MAME, any file in any drive!