This is actually very similar to some ideas I've been having for a while... that having a smaller entry model that knows enough about "expert" models that themselves are smaller to hand work over to could be better/faster/lighter in terms of working through real problems vs the megalith ones we currently use. Highly distilled experts and coordination with a fallback mode to a larger model option.
afaik there is some research at this area.
Also the new apple foundation model uses related idea. they process the whole prompt and based on prompt load required experts and use only these experts for generation. It doesn't require fitting full model into memory or per token ssd streaming
