Inference throughout scales really well with larger batch sizes (at the cost of latency) due to rising arithmetic intensity and the fact that it's almost always memory BW limited.

Bedrock has a batch mode but only for claude 3.5 which is like one year old, which isn't very useful.

50% is my personal threshold for a discount going from not worth it to worth it.