Hacker Times
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
andy_xor_andrew
on April 19, 2023
|
parent
|
context
|
favorite
| on:
StableLM: A new open-source language model
> The context length for these models is 4096 tokens.
!!! And I was excited that llama gave us 2048!!
MacsHeadroom
on April 20, 2023
[–]
Rumor is RedPajama is going to have upwards of 60k token context by using Hyena:
https://arxiv.org/abs/2302.10866
But it's just a rumor. We'll see.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
!!! And I was excited that llama gave us 2048!!