Most of the time, the speed of these models are constrained by memory bandwidth.... | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

jboss10 26 days ago | parent | context | favorite | on: Bonsai 27B: A 27B-Class model that runs on a phone

Most of the time, the speed of these models are constrained by memory bandwidth. GPUs normally have much more memory bandwidth.

davedx 25 days ago [–]

I'd expect the memory bandwidth to be the same for the CPU and GPU under a unified memory architecture like Apple silicon uses?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact