>So, if software knows in advance which data chunk (expert) it'll need for the next token, it can load that in parallel with computing current token.

Yeah, I was thinking MADV_WILLNEED might work there but not sure