In addition to the other comments, I'd add that in the context of Chess, we have a contest in which there is a winner or a loser or a draw, and we can relatively easily verify that the system is functioning as designed with simple statistics. We also have a system that broadly speaking produces a fairly uniform ordering. Yes, at the very top of chess there's some question of who might win or whether styles have an effect, but broadly speaking in the population at large, a 50 point difference will have a certain meaning; at the macro level, we're measuring something with one dimension.

I would imagine many psychology studies are measuring something a great deal less well defined and more multidimensional in practice than "A will beat B at chess."