This most likely has to do with your DNS server. Try to change it to another one (e.g. Google's 8.8.8.8 or OpenDNS's 208.67.222.222) and see if it gets better.

Had the same problem popping up just after upgrading to Yosemite. Decided to do a clean install a couple of months ago as I saw other machines on my network didn't have this problem. After the clean install things are working out as expected again.

Not what I really wanted (checked quite a bit around for a solution without finding any), but at least I don't have this happening multiple times a day.