NVLink is good if you are doing distributed training and sharing large amounts of gradients (very large models). If you're doing parallel experiments or your models are not huge, then PCI-e (16 GB/s) is generally good enough. AMD don't have anything close that i am aware of, but their new Ryzen board will have double the number of PCI lanes, which makes PCI-e competitive again - https://hackertimes.com/item?id=14450924