I mean 800B tokens on a 3B model and 7B model is still way beyond the Chinchilla... | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

GaggiX on April 19, 2023 | parent | context | favorite | on: StableLM: A new open-source language model

I mean 800B tokens on a 3B model and 7B model is still way beyond the Chinchilla scale.

MacsHeadroom on April 19, 2023 [–]

They're going to 1.5T and possibly 3T. The 800B is just for the "Alpha" checkpoints released today. New checkpoints will be released later.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact