Shameless plug: we have a workshop today @ CVPR 2017 that explores the classification task of ImageNet using crawled (and noisy) data: https://www.vision.ee.ethz.ch/webvision/workshop.html

In contrast to ImageNet, only the validation and test data were manually labeled. Impressively, on such data, an accuracy of ~95% (top5) is achieved by the winning solution this year.