frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Building trade desk – Using agents to trade all possible markets

https://www.sayantan.sh/blog/hood-agent
1•sayantan317•1m ago•0 comments

The METR Graph, Updated

https://twitter.com/ben_j_todd/status/2122318704260403562/photo/1
1•simonebrunozzi•2m ago•0 comments

Joyce in Bloom (2011)

https://www.vanityfair.com/culture/2004/06/hitchens-200406
1•hamonrxe•5m ago•0 comments

Meta built TLX, a GPU compiler extension for Triton

https://engineering.fb.com/2026/11/10/open-source/tlx-a-hardware-native-gpu-compiler-for-producti...
1•fourfire•5m ago•0 comments

Redict Is Deprecated

https://redict.io/posts/2026-11-17-redict-deprecated/
1•progval•5m ago•0 comments

Show HN: AI Agent Infrastructure and Memory Compaction Specifications

https://github.com/grfdsasdvef/awesome-ai-agent-infrastructure
1•snowdensay•10m ago•0 comments

Run your FreeBSD services from one console

https://jailops.net/
1•root420•13m ago•0 comments

A Deep Dive into Building the Pylyzer-Zed Language Server Extension

https://medium.com/@therealcomtom/extending-zed-a-deep-dive-into-building-the-pylyzer-zed-languag...
1•comtom•13m ago•0 comments

Show HN: Lantern – desktop app that maps your repo and reviews AI-written PRs

https://github.com/MobileReality/lantern
1•gitsad•14m ago•0 comments

Chasing Ghosts in the Machine: Debugging a File Descriptor Leak in Exchangelib

https://medium.com/@therealcomtom/chasing-ghosts-in-the-machine-debugging-a-file-descriptor-leak-...
1•comtom•15m ago•0 comments

Detroit bus drivers get training to accommodate horses

https://apnews.com/video/city-of-detroit-says-drivers-will-get-additional-training-after-horses-w...
3•illidan1000•17m ago•1 comments

Autopilot: Use whichever AI model you want, with one coding-agent interface

https://github.com/directionally-local/autopilot
1•doppp•18m ago•0 comments

C++ Proposal: break label; and continue label;

https://eisenwave.github.io/cpp-proposals/break-continue-label.html
1•qtqtqt•18m ago•0 comments

Trump: I don't take any responsibility

https://bsky.app/profile/atrupar.com/post/3n2grgyw3u32e
1•o_-_o•18m ago•0 comments

What should an agent memory leaderboard measure beyond recall?

https://x.com/AgentMemoryL/
1•SndOfViento•20m ago•0 comments

DotShield – stop Claude Code and Cursor from reading your .env

https://github.com/dot-shield/dotshield-releases
1•gma692002•20m ago•0 comments

Agent Memory Leaderboard Cycle 2: What to Watch

https://twitter.com/AgentMemoryL/status/2122336386246484179
2•IreneAI•22m ago•0 comments

The Illusions Index

https://www.illusionsindex.org/
1•genxy•23m ago•0 comments

Ask HN: What was your weirdest or funniest job rejection?

1•cristaloleg•25m ago•1 comments

Contabo – Potential Security Incident

https://help.contabo.com/en/support/solutions/articles/103000426255-potential-security-incident-o...
2•ensocode•28m ago•0 comments

How Microsoft Vaporized a Trillion Dollars

https://isolveproblems.substack.com/p/how-microsoft-vaporized-a-trillion-b73
2•vismit2000•29m ago•0 comments

Show HN: A serene digital prayer journal.Log prayers,celebrate answered prayers

https://chromewebstore.google.com/detail/faithjournal-–-prayer-jou/pefifbegbbciegbaabkelegnaghl...
1•gsngsn•30m ago•0 comments

NASA's Voyager 1 is now one light-day away from Earth

https://science.nasa.gov/mission/voyager/voyager-1/shine-bright-for-voyagers-light-day/
3•hubraumhugo•30m ago•1 comments

We built a CX platform for people in founder mode

https://www.parlant.io/
1•ymarcov•37m ago•1 comments

Syria reportedly plans 100-year lease of northern border land to Türkiye

https://www.turkiyetoday.com/region/syria-reportedly-plans-100-year-lease-of-northern-border-land...
1•consumer451•37m ago•0 comments

Joint Statement from Department of Commerce and DOJ on Megaspeed Investigation

https://www.commerce.gov/news/press-releases/2026/11/joint-statement-department-commerce-and-depa...
2•lexandstuff•38m ago•0 comments

PyBot: Can Python Be a Chatbot?

https://github.com/huggingface/pybot
1•dsr12•42m ago•0 comments

Show HN: Northstack – Rust, React and SQLite in one binary, typed end to end

https://github.com/elitan/northstack
1•elitan•42m ago•0 comments

Warren Buffett donates $5.5B, says kindness 'compounds'

https://www.cnbc.com/2026/11/16/warren-buffett-donates-5point5-billion-thanksgiving.html
2•madihaa•42m ago•1 comments

CMU 11-768: AI Agents

https://www.cmu-agents.com/#/schedule
1•ijidak•46m ago•0 comments
Open in hackernews

Show HN: Fast and Quality Code Chunking with Chonkie

1•snyy•1y ago
Hi HN,

We’re Chonkie (https://github.com/chonkie-inc/chonkie) — we build open source tools that help split documents into meaningful chunks for use with AI models.

When you use LLMs over large documents or codebases, you often need to break them into smaller parts to fit the model’s context window. Our chunkers do this in a smart way: they preserve structure and meaning, so only the most relevant pieces are passed into the model. This reduces hallucinations, avoids confusion, and improves performance and accuracy.

Today we’re launching our Code Chunker — a fast, structure-aware way to break down source code into high-quality, token-aware chunks.

How it works:

(See the code: https://github.com/chonkie-inc/chonkie/blob/main/src/chonkie...)

Code Chunker uses tree-sitter (https://tree-sitter.github.io/tree-sitter/) to parse your code into an abstract syntax tree (AST). It then recursively merges and groups nodes in a way that respects both code structure and token limits.

It supports all languages that tree-sitter supports, and is designed to preserve formatting and semantics. Large functions or class definitions won’t be split in the middle of a block — instead, we dive recursively into the AST to produce clean, coherent chunks that fit your configured token budget.

What it’s useful for:

  - Embedding-based code search

  - RAG (retrieval-augmented generation) over codebases

  - Long-context analysis of code

  - Preparing repos for fine-tuning or pretraining
Try it out:

  - Open source package: https://docs.chonkie.ai/chunkers/code-chunker

  - Hosted playground (free with account): https://cloud.chonkie.ai
Happy Chonking!