Search: github.com/tokenium | Heykuki News

Heykuki News

Top New Best Ask Show Jobs

Top New Best Ask Show Jobs

1.

Tokenium: my new protocol to solve decentralized exchange flaws (github.com/tokenium)

1 point

9 years ago

2.

Ask HN: Anyone have experience with open-source alternatives to Parse?

1 point

14 years ago

3.

Show HN: HuggingFace – Fast tokenization library for deep-learning NLP pipelines (github.com/huggingface)

168 points

6 years ago

4.

Show HN: Convert a number to an approximated text expression (github.com/tokenmill)

114 points

6 years ago

5.

TokenMonster: Ungreedy tokenizer, outperforming tiktoken by 35% (github.com/alasdairforsythe)

10 points

3 years ago

6.

Show HN: Accelerated Text – open-source Natural Language Generation platform (github.com/tokenmill)

6 points

6 years ago

7.

superfly/tokenizer: HTTP proxy that injects 3rd party credentials into requests (github.com/superfly)

4 points

4 months ago

8.

Show HN: Email Sleuth – Find and Verify Emails (Rust CLI) (github.com/tokenizer-decode)

3 points

iLikeFunctional

a year ago

9.

EmailSleuth – Find and Verify Emails, Rust Based (SMTP, Chrome Drivers) (github.com/tokenizer-decode)

3 points

iLikeFunctional

a year ago

10.

Tokenization: A comprehensive deep dive into the world of tokens (github.com/SumanthRH)

3 points

3 years ago

11.

A curated list of resources dedicated to Natural Language Generation (github.com/tokenmill)

3 points

6 years ago

12.

A go library for tokenizing text (github.com/dannav)

2 points

10 years ago

13.

Show HN: Personal AI gateway for OpenClaw – tokenomics (github.com/rickcrawford)

2 points

3 months ago

14.

Show HN: New CLI tool to count tokens locally (github.com/zahidcakici)

2 points

4 months ago

15.

Tokenizations alignment library for Rust and Python, written in Rust (github.com/tamuhey)

2 points

6 years ago

16.

Show HN: Tokenusage – Rust CLI that tracks Claude Code/Codex tokens 214x faster (github.com/hanbu97)

1 point

3 months ago

17.

Free open source AI cost tracker – pip install tokenbudget (github.com/AIMasterLabs)

1 point

3 months ago

18.

Claude hack – Don't waste you token where it's not needed (github.com/justinjamesmathew)

1 point

12 days ago

19.

Tokenmonster: Determine tokens to optimally represents a dataset (github.com/alasdairforsythe)

1 point

3 years ago

20.

Atomic SWAR Token Buckets (github.com/jart)

1 point

4 years ago

21.

Supercharge your search with tokenization (github.com/ashubham)

1 point

8 years ago

22.

Show HN: Actual Claude Tokenizer (tokenizer.robkopel.me)

3 points

a month ago

23.

Show HN: My own tokenization method in Python, open to feedback (github.com/YasTouchedMySpaggett)

2 points

3 years ago

24.

Show HN: Stateless TikToken and Unix-Dictionary GitHub URL Shortener (selfdotsend.com)

1 point

6 months ago

25.

Show HN: ai-tokenizer – 5-7x faster than tiktoken with AI SDK support (coder.github.io)

1 point

8 months ago

26.

Show HN: LLaMA tokenizer that runs in browser (github.com/belladoreai)

115 points

3 years ago

27.

Approximation of the Claude 3 tokenizer by inspecting generation stream (github.com/javirandor)

9 points

2 years ago

28.

GPT-4-Tokenizer-Visualizer (github.com/functorism)

6 points

3 years ago

29.

Show HN: FlashTokenizer – 10x faster C++ tokenizer for Python (github.com/NLPOptimize)

5 points

a year ago

30.

Show HN: Byte-Pair Encoding tokenizer for training LLMs on large datasets (github.com/jmaczan)

5 points

2 years ago