Show HN: Jist now searches JSON 17x faster than jq

7 points

a year ago

Hi HN! Happy to share that jist can now search JSON 17x faster than jq for a provided input (string or a file). The major change is using the bleeding edge simdjson parser as the preferred way of parsing and querying the input.

Since simdjson can only work on files up to 4.2GB, jist falls back to its earlier implementation of scanning through input in chunks of 1MB which means it can still query files of virtually infinite size 3x faster than jq ever could while using ~10MB of memory.

The next major feature will be summarizing input JSON into a schema - and it will do this by scanning the whole file to get a complete picture. Stay tuned!

Any feedback (or code review) is much appreciated!