I’ll test it later if you haven’t worked it out by then, but my first thought is that the joystick inputs are active low. I at a quick glance, I didn’t see you accounting for that (either inverting the values when reading the ports, or inverting the values for the element states).