The tokens per second speed measurement is highly inflated nowadays because most... | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

		low_tech_punk 11 days ago \| parent \| context \| favorite \| on: Small Models Have Arrived The tokens per second speed measurement is highly inflated nowadays because most of the tokens went into thinking. I wonder if there is a more realistic measurement for "effective speed", which accounts for thinking efficiency.
		help

ak_t 11 days ago [–]

Many benchmarks now measure the total cost or energy usage per completed task.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact