vCenter is showing between 2,000 to 4,000 ms on one server and 2,000 to 6,500 ms on another. The 100 Mb vmnic thing must have been a coincidence though. The frequency of the latency spikes correspond with the interval where 30 + vms are writing to disk all at the same time. I'm only using two out of four of my iscsi nics assuming that I had enough bandwidth, but I didn't think about latency. Perhaps adding more nics to the equation will solve my issue? Will try and let you guys know the result.
The spikes are short and so I don't lose access to datastores during this time.