By upside down, you mean that the fan side is facing down, right?
If so, that's normal for all cards. Unless there's some wierd card out there, that's the only way cards come.
It's not unusual to have a few spaces between the other cards and the video card. It's really not a problem to have a card right next to the video card, but most people prefer not to do it that way, especially since they tend to overclock---which produces more heat.
Basically, the card positions don't sound abnormal and is probably what most people may do.
Can't you get a cheap AGP card (like a GeForce 2 MX or something) and try that? Also, can you remove the other cards in your system and uninstall the drivers for those devices (especially the sound card). Or if you don't want to do that, disable them in Device Manager and see if that fixes anything.