It was submitted by a seasoned user, who probably asked a frontier LLM. It still felt… wrong. I didn't understand it, and I wouldn't merge it without understanding it.
I also suspected it was wrong, in a way that would cause issues in the future.
So I reviewed it 4 different ways: (1) try to understand/improve it; (2) do it with better algorithms; (3) avoid it by fixing the issue upstream; (4) rewrite it from scratch probably just to match my brain.
I expected either (2) or (3) would be the answer. (2) didn't work, rather it's the correct answer but I need to redo the project from scratch to use it; (3) I wanted really bad to work, but didn't.
So I got to a blend of (1) and (4). I'm still not entirely convinced, but now I understand the issue/solution. I obviously think my approach is better.
Still, I still stripped both of comments, and asked my LLM to review.
The LLM came back and said the original one was clearly better. I explained why not, it then answered I was correct.
If I try it with comments, LLMs say the mine is better. Because I found a real issue (one that I pointed at in the original comment thread). But is it saying mine is better because I coerced it to say so?
It was submitted by a seasoned user, who probably asked a frontier LLM. It still felt… wrong. I didn't understand it, and I wouldn't merge it without understanding it.
I also suspected it was wrong, in a way that would cause issues in the future.
So I reviewed it 4 different ways: (1) try to understand/improve it; (2) do it with better algorithms; (3) avoid it by fixing the issue upstream; (4) rewrite it from scratch probably just to match my brain.
I expected either (2) or (3) would be the answer. (2) didn't work, rather it's the correct answer but I need to redo the project from scratch to use it; (3) I wanted really bad to work, but didn't.
So I got to a blend of (1) and (4). I'm still not entirely convinced, but now I understand the issue/solution. I obviously think my approach is better.
Still, I still stripped both of comments, and asked my LLM to review.
The LLM came back and said the original one was clearly better. I explained why not, it then answered I was correct.
If I try it with comments, LLMs say the mine is better. Because I found a real issue (one that I pointed at in the original comment thread). But is it saying mine is better because I coerced it to say so?