> Same tools A, B and C, but minus tools D, E and F,

Why "minus D, E and F"? After all, once you have the harness set up, there's no additional work to add in new models, right?

The point being that there are always going to be more eyes, and more knowledge of available tools (i.e. including "D, E and F"), and more experience using them, with open source than with a single in-house dev team.

There's no more "eyes" though, it's all models, and they are all converging pretty damn fast.

If true then logically it will be sufficient to run this "master model" once before any code release for the level playing field to be restored. After all, even open-source software is private until it is released.

> If true then logically it will be sufficient to run this "master model" once before any code release for the level playing field to be restored.

I'm struggling to see how it is a level playing field:

1. Closed-source: defender runs llms to check the sources for vulns, runs llms on each PR, runs llm on deployment of the compiled output. Attacker runs llm only on compiled output.

2. Open-source: both attacker and defender runs llms on source, on PRs and on compiled output.

Fair enough