anything less than a 10Gbit network connection to the disk is not gonna be much fun, 5Gbit is adequate
Discussion
My cluster gets by with a 4gig lacp across 3 nodes, haven't really noticed any performance degradation for what I do. I guess caches make the most of it.