This has been known for years now, and it's really depressing that nobody's been able to improve it.
Since you've looked at the code a lot more than me, I believe you when you say: "I doubt we can improve that much without breaking backward compatibility." -- although it doesn't make sense to me why partial lookup can't be cached at a given call site in a way that's perfectly backwards compat as well as fast (after first partial lookup from a given call-site).
But if true -- I wonder if it's time to consider backwards incompat? Perhaps in a way you can opt into, not just app-wide, but per-file/view/action or per-call, and the opt-in not just as a temporary deprecation situation, but planned for long-term support of being able to do it "both ways", the backwards compat way or the performant way.
But in the limited amount of time I have spent looking at the code, I quickly get lost, it has evolved into pretty convoluted code. If the problem is not really "there's no semantics for caching partial lookup that are both backwards compat and higher performance", but "the code is so convoluted we can't really figure out how to change it to do what we want without rewriting the whole thing in a way that's going to be hard to be backwards compat for all edge cases" -- that may be even harder.
> it's really depressing that nobody's been able to improve it.
That's quite unfair, several people improved this already...
> why partial lookup can't be cached at a given call site
Well first you have no state at the call site in which you could store that.
But even if we did, a single callsite can render different template based on request parameters, e.g. `render "foo"` may be either `foo.en.html.rb` or `foo.fr.html.rb` based on the request locale. But then you also have format, format variants, etc etc. So the number of possibilities is huge.
> I wonder if it's time to consider backwards incompat?
Of course we consider it, the explicit locals declaration is one step in that direction, but it's not like we can break people's code willy nilly.
Since you've looked at the code a lot more than me, I believe you when you say: "I doubt we can improve that much without breaking backward compatibility." -- although it doesn't make sense to me why partial lookup can't be cached at a given call site in a way that's perfectly backwards compat as well as fast (after first partial lookup from a given call-site).
But if true -- I wonder if it's time to consider backwards incompat? Perhaps in a way you can opt into, not just app-wide, but per-file/view/action or per-call, and the opt-in not just as a temporary deprecation situation, but planned for long-term support of being able to do it "both ways", the backwards compat way or the performant way.
But in the limited amount of time I have spent looking at the code, I quickly get lost, it has evolved into pretty convoluted code. If the problem is not really "there's no semantics for caching partial lookup that are both backwards compat and higher performance", but "the code is so convoluted we can't really figure out how to change it to do what we want without rewriting the whole thing in a way that's going to be hard to be backwards compat for all edge cases" -- that may be even harder.