You definitely want ECC ram, and if you can afford it, dual power supplies. You'll want RAID disks. I recommend software raid1 or raid10. (There are complex drawbacks to raid that requires hardware support for good performance.) I use Fedora and RedHat. There is lots of community help on Fedora.
If you give up on the dual PS, I just bought a Dell Precision T3600 on Ebay for $63 (plus shipping). There are a lot at present, some corp must have got rid of a bunch. The T3600 has only a single PS, but supports ECC ram. You might have to upgrade the PS if you want more than 2 SSD drives internal (you can convert the CD/floppy bays to Sata drives for a total of 4).
You'll also want a UPS (of the $200+ type). On linux/bsd use nut to monitor the UPS. If the PS fails, you're outta luck - so be sure to use a journaled filesystem that handles that well. I've been very pleased with btrfs on Fedora. You can even skip the md driver for raid, and use the mirroring built in to btrfs. It checksums every block, so if mirrored drives disagree, it knows which one is correct.
The T3600 can be used in tower or desktop orientation. The noise level is reasonable, but not as quiet as the (much older) 400SC. It is quieter than my Poweredge T310.
