You'll probably find this talk [1] interesting. They control all the training data for small LLMs and then perform experiments (including reasoning experiments).

[1] Physics of LLMs: https://www.youtube.com/watch?v=yBL7J0kgldU&t=7s