Had the same problem popping up just after upgrading to Yosemite. Decided to do a clean install a couple of months ago as I saw other machines on my network didn't have this problem. After the clean install things are working out as expected again.

Not what I really wanted (checked quite a bit around for a solution without finding any), but at least I don't have this happening multiple times a day.