I'm replacing things that I used to delegate to juniors with generated code. Because it's quicker and better. And there's a category of stuff I used to not bother with at all that I'm also taking on. Because I can get it done in a reasonable time frame. It's more fun for me for sure and I definitely am more productive because of it.
My feeling is that this stuff is not bottle-necked on model quality but on UX. Chat is not that great of an interface. Copy pasting blobs of text back to an editor seems like it is a bit monkey work. And monkey work should be automated.
With AI interactions now being able to call functions, what we need is deeper integration with the tools we use. Refactor this, rename that. Move that function here. Etc. There's no need for it to imagine these things perfectly it just needs to use the tools that make that happen. IDEs have a large API surface but a machine readable description of that easily fits in a context window.
Recently chat gpt added the ability to connect applications. So, I can jump into a chat, connect Intellij to the chat and ask it a question about code in my open editor. Works great and is better than me just copy pasting that to a chat window. But why can't it make a modification for me? It still requires me to copy text back to the editor and then hope it will work.
Addressing that would be the next logical step. Do it such that I can review what it did and undo any damage. But it could be a huge time saver. And it would also save some tokens. Because a lot of code it generates is just echoing what I already had with only a few lines modification. I want it to modify those lines and not risk hallucinating introducing mistakes into the rest, which is a thing you have to worry about.
The other issue is that iterating on code gets progressively harder as there's more of it and it needs to regenerate more of it at every step. That's a UX problem as well. It stems from the context being an imperfect abstraction of my actual code. Applying a lot of small/simple changes to code would be much easier than re-imagining the entire thing from scratch every time. Most of my conversations the code under discussion diverges from what I have in my editor. At some point continuing the conversation becomes pointless and I just start a new one with the actual code. Which is tedious because now I'm dealing with ground hog day of having to explain the same context again. More monkey work. And if you do it wrong, you have to do it over and over again. It's amazing that it works but also quite tedious.
> this stuff is not bottle-necked on model quality but on UX. Chat is not that great of an interface. Copy pasting blobs of text back to an editor seems like it is a bit monkey work. And monkey work should be automated.
I agree wholeheartedly and that's why I recommend cursor to the point I'm being called a shill for them. I have no relationship to them, but they've shipped the first product that actually addresses this!
They have a "small model" that takes a suggestion in the chat mode (provided by claude3.5 usually but o1 / 4o also work) and "magic merges" it into your codebase at the click of a button. It feels like such an easy task, but I bet it's not and a lot of tinkering went into it and the small mdoel they use. But the UX results are great. You start a chat, frame the problem, get an answer, hit "apply" and see it go line by line and incorporate the changes into your existing code.
> The other issue is that iterating on code gets progressively harder as there's more of it and it needs to regenerate more of it at every step.
You might know this already, but if you're using the chatbot interfaces it helps quite a bit to prompt it with something along the lines of "only give me the bits that changed". There is nothing worse than fine-tuning a tiny bit of some code you didn't bother writing yourself only to have the bot give you an entire prompt's worth of code.
Check out continue.dev or (my favorite) Zed. It allows you to generate code in patch format and the editor will apply the changes to the various files itself. Copy pasting from ChatGPT is so 2023.
I don't want to replace my IDE (intellij) because I actually like it and use a lot of what it does most of which is not supported in other tools. I want AI models to work with my tools.
Tools like this are alright if your expectations of an IDE are low. E.g. if you are happy using just VS Code or whatever. Unfortunately I'm used to a bit more than that. Jetbrains has been doing some of their AI stuff. But I haven't really looked at it that much.
Don't get me wrong; I actually use VS Code for some stuff. But it's just not a replacement for intellij. Not even close. So, not really looking for IDEs with even less features just so I can have some AI.
My feeling is that this stuff is not bottle-necked on model quality but on UX. Chat is not that great of an interface. Copy pasting blobs of text back to an editor seems like it is a bit monkey work. And monkey work should be automated.
With AI interactions now being able to call functions, what we need is deeper integration with the tools we use. Refactor this, rename that. Move that function here. Etc. There's no need for it to imagine these things perfectly it just needs to use the tools that make that happen. IDEs have a large API surface but a machine readable description of that easily fits in a context window.
Recently chat gpt added the ability to connect applications. So, I can jump into a chat, connect Intellij to the chat and ask it a question about code in my open editor. Works great and is better than me just copy pasting that to a chat window. But why can't it make a modification for me? It still requires me to copy text back to the editor and then hope it will work.
Addressing that would be the next logical step. Do it such that I can review what it did and undo any damage. But it could be a huge time saver. And it would also save some tokens. Because a lot of code it generates is just echoing what I already had with only a few lines modification. I want it to modify those lines and not risk hallucinating introducing mistakes into the rest, which is a thing you have to worry about.
The other issue is that iterating on code gets progressively harder as there's more of it and it needs to regenerate more of it at every step. That's a UX problem as well. It stems from the context being an imperfect abstraction of my actual code. Applying a lot of small/simple changes to code would be much easier than re-imagining the entire thing from scratch every time. Most of my conversations the code under discussion diverges from what I have in my editor. At some point continuing the conversation becomes pointless and I just start a new one with the actual code. Which is tedious because now I'm dealing with ground hog day of having to explain the same context again. More monkey work. And if you do it wrong, you have to do it over and over again. It's amazing that it works but also quite tedious.