What's strange is that I have the same graphics card as you, only with 3GB. I updated to the latest graphic drivers from NVidia. I tried multiple games with different graphic backends. Tried using bgfx & hlsl & I couldn't get the games to crash. Every game ran just fine on the latest version of MAME
Windows 11 64 bit OS
Intel Core i7-10700
Nvidia GeForce RTX 2060 6GB
32GB DDR4 RAM