I believe they did it all the time. Maybe it was not automated? But they boasted in news multiple times how many coreutils tests they are passing. I suspect that those tests are useless for security, they are more about compatibility or something like that.