Heykuki News
Top
New
Best
Ask
Show
Jobs
Toggle theme
Login
Top
New
Best
Ask
Show
Jobs
Code for the Byte Pair Encoding algorithm, commonly used in LLM tokenization
github.com/karpathy
81 points
magoghm
2 years ago
31 comments
Loading...
Code for the Byte Pair Encoding algorithm, commonly used in LLM tokenization | Heykuki News