Decision fatigue may honestly be a learnt artifact from RLHF, which is discouraging.

Sure, but now we have to remodel whatever bias we want for our use case with every new release because the system prompt changes, whereas the underlying data does not.

Underlying data changes all the time, as do training methodologies / preferences.

You do realize that these LLMs are trained with a metric ton of synthetic examples? You describe the kind of examples / behavior you want, let it generate thousands of examples of this behavior (positive and negative), and you feed that to the training process.

So changing this type of data is cheap to change, and often not even stored (one LLM is generating examples while the other is training in real-time).

Here's a decent collection of papers on the topic: https://github.com/pengr/LLM-Synthetic-Data

Well, I'd say it's a reasonable expectation for the model to behave similarly across releases. Am I wrong to assume that?

I imagine the system prompt can correct some training artifacts and drive abnormal behavior to the mean in the dimensions that Anthropic deems fit. So it's either that they are responding to their brittle training process, or that they chose this direction deliberately for a different reason.

I agree with what you're saying, but I imagine products like this one aren't aimed at replacing this. LLMs are partially a dashboard business, and this is just one tool to aim at your boring business data

Do you have examples of "natural" UI that you like?

Most of the software we interact with is at the end of the day some db tables, queries to read/write, and some ui to read/write. There have been so many times I wished I could just do my own db joins on the underlying db to get the views I wanted. But I can’t - because the app has pre-defined ui/query paths.

With AI, I should be able to ask for things the product designers didn’t anticipate or left out and the system could query, create ui on the fly, etc…

Mind-reading brain implants. That's what he wants.

I get a similar feeling for when friends send me 2minute+ Instagram reels, it's as if my brain can't engage with the content. I'd much rather read a few paragraphs about the topic, and It'd probably take less time too.

I'd say autocomplete introduces a certain level of fuzziness into the code we work with, though to a lower degree. I used autocomplete for over a year, and initially it did feel like a productivity boost, yet when I later stopped using them, it never felt like my productivity decreased. I stopped because something about losing explicit intent of my code feels uncomfortable to me.