Compiles are generally HDD bound once you hit 4+ cores. If I do make clean / make / make clean / make on Fedora the second one takes about 20 seconds start to finish because the entire source tree's already in RAM. I'll time a completely cold run sometime though.
(I should note that on this system the compiler binaries and system headers and libs are all on an SSD, which also helps).