If you took every image agreeing with your conception of a magpie and composed it into one image, would it look anything like a magpie? The Google inceptionism team did the same experiment but supplied the networks with natural image priors. Perhaps hand-picked, but those examples were actually quite convicining.