Yes. The presence of the correctly spelled name earlier in the context should've dominated the distribution so much that it should've been extremely unlikely to select an incorrectly spelled variable name in the place where the correct name should go.
What if the model got more training data? There’s no guarantee that the training data causes the model to converge on correctness. It’s not like this stuff is curated for specific problems.
People seem to have noticed that chatgpt has good days and bad days. we don't really know what openai is doing in the background, so it could be that some days you get assigned a really bad B in an A/B model deployment test, or maybe they're throttling the performance because they're at peak, or likely some other factor. But it's an observed phenomenon that it performs well and then doesn't.
From what I've heard, the reason it's doing that is to add an AI fingerprint. It has to pick less likely tokens to encode this information to the detriment of the output quality. Unfortunately it's only hearsay but it made sense to me so I thought I'd share.
It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.