Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

Their methodology isn't published.

Its widely accepted[1] that it runs the same query through the model in parallel and then has a model that either selects the best answer or synthesizes an answer from the multiple ones generated.

I believe most people think it runs 6 sub-models, but I think that is based on the pricing.

It's a pity that OpenAI doesn't publish details like this.

[1]eg https://hackertimes.com/item?id=48799977



Basically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks.

Pro is quite limited on the web UI I reckon. This approach can be highly effective for reasonably verifiable task, for example, write comprehensive unit tests pointing out a tricky bug, get multiple agents to swarm at it.


It's been very successful at frontier math tasks - a bunch of the Erdos questions have been solved by it - more than any other model.

https://www.erdosproblems.com/


> Basically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks.

It's unclear how they would do this when there is no signal that provides an objective ground truth.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: