frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

LIGO detects most massive black hole merger to date

https://www.caltech.edu/about/news/ligo-detects-most-massive-black-hole-merger-to-date
135•Eduard•3h ago•65 comments

Apple's MLX adding CUDA support

https://github.com/ml-explore/mlx/pull/1983
73•nsagent•1h ago•34 comments

RFC: PHP license update

https://wiki.php.net/rfc/php_license_update
94•josephwegner•2h ago•25 comments

DEWLine Museum – The Distant Early Warning Radar Line

https://dewlinemuseum.com/
11•reaperducer•1h ago•0 comments

Kiro: A new agentic IDE

https://kiro.dev/blog/introducing-kiro/
633•QuinnyPig•9h ago•278 comments

NeuralOS: An operating system powered by neural networks

https://neural-os.com/
61•yuntian•3h ago•20 comments

Show HN: Bedrock – An 8-bit computing system for running programs anywhere

https://benbridle.com/projects/bedrock.html
45•benbridle•4d ago•10 comments

Replicube: 3D shader puzzle game, online demo

https://replicube.xyz/staging/
70•inktype•3d ago•12 comments

Context Rot: How increasing input tokens impacts LLM performance

https://research.trychroma.com/context-rot
48•kellyhongsn•4h ago•9 comments

Cognition (Devin AI) to Acquire Windsurf

https://cognition.ai/blog/windsurf
320•alazsengul•5h ago•258 comments

Cidco MailStation as a Z80 Development Platform (2019)

https://jcs.org/2019/05/03/mailstation
41•robin_reala•5h ago•3 comments

Building Modular Rails Applications: A Deep Dive into Rails Engines

https://www.panasiti.me/blog/modular-rails-applications-rails-engines-active-storage-dashboard/
113•giovapanasiti•8h ago•26 comments

SQLite async connection pool for high-performance

https://github.com/slaily/aiosqlitepool
35•slaily•3d ago•20 comments

Anthropic, Google, OpenAI and XAI Granted Up to $200M from Defense Department

https://www.cnbc.com/2025/07/14/anthropic-google-openai-xai-granted-up-to-200-million-from-dod.html
86•ChrisArchitect•2h ago•60 comments

Embedding user-defined indexes in Apache Parquet

https://datafusion.apache.org/blog/2025/07/14/user-defined-parquet-indexes/
83•jasim•7h ago•10 comments

Strategies for Fast Lexers

https://xnacly.me/posts/2025/fast-lexer-strategies/
117•xnacly•8h ago•41 comments

Show HN: The HTML Maze – Escape an eerie labyrinth built with HTML pages

https://htmlmaze.com/
20•kyrylo•2h ago•2 comments

Japanese grandparents create life-size Totoro with bus stop for grandkids (2020)

https://mymodernmet.com/totoro-sculpture-bus-stop/
223•NaOH•7h ago•54 comments

Meticulous (YC S21) is hiring in UK to redefine software dev

https://tinyurl.com/join-meticulous
1•Gabriel_h•6h ago

Predicting Competitive Pokémon VGC Leads Using Latent Semantic Analysis

https://jgeekstudies.org/2025/07/11/predicting-competitive-pokemon-vgc-leads-using-latent-semantic-analysis-a-data-driven-approach-to-team-matchups/
3•zdw•2d ago•0 comments

Lightning Detector Circuits

https://techlib.com/electronics/lightningnew.htm
65•nateb2022•8h ago•35 comments

Data brokers are selling flight information to CBP and ICE

https://www.eff.org/deeplinks/2025/07/data-brokers-are-selling-your-flight-information-cbp-and-ice
387•exiguus•7h ago•186 comments

Tandy Corporation, Part 3 Becoming IBM Compatible

https://www.abortretry.fail/p/tandy-corporation-part-3
51•klelatti•3d ago•13 comments

East Asian aerosol cleanup has likely contributed to global warming

https://www.nature.com/articles/s43247-025-02527-3
144•defrost•14h ago•153 comments

Two guys hated using Comcast, so they built their own fiber ISP

https://arstechnica.com/tech-policy/2025/07/two-guys-hated-using-comcast-so-they-built-their-own-fiber-isp/
260•LorenDB•7h ago•169 comments

Impacts of adding PV solar system to internal combustion engine vehicles

https://www.jstor.org/stable/26169128
97•red369•12h ago•208 comments

The Corset X-Rays of Dr Ludovic O'Followell (1908)

https://publicdomainreview.org/collection/the-corset-x-rays-of-dr-ludovic-o-followell-1908/
22•healsdata•3d ago•1 comments

It took 45 years, but spreadsheet legend Mitch Kapor finally got his MIT degree

https://www.bostonglobe.com/2025/06/24/business/mitch-kapor-mit-degree-bill-aulet/
154•bookofjoe•3d ago•14 comments

Lossless Float Image Compression

https://aras-p.info/blog/2025/07/08/Lossless-Float-Image-Compression/
88•ingve•4d ago•10 comments

A Century of Quantum Mechanics

https://home.cern/news/news/physics/century-quantum-mechanics
101•bookofjoe•4d ago•77 comments
Open in hackernews

SQLite async connection pool for high-performance

https://github.com/slaily/aiosqlitepool
35•slaily•3d ago

Comments

slaily•3d ago
If you’re building Python async apps (FastAPI, background jobs, etc.) with SQLite, you’ll eventually hit two issues

- Opening/closing connections is fast, but not free—overhead adds up under load

- SQLite writes are globally locked

aiosqlitepool is a tiny library that adds connection pooling for any asyncio SQLite driver (like aiosqlite):

- It avoids repeated database connection setup (syscalls, memory allocation) and teardown (syscalls, deallocation) by reusing long-lived connections

- Long-lived connections keep SQLite's in-memory page cache "hot." This serves frequently requested data directly from memory, speeding up repetitive queries and reducing I/O operations

- Allows your application to process significantly more database queries per second under heavy load

Enjoy!

slashdev•2h ago
How does this help with the second issue, the write locks?
ncruces•2h ago
No idea if it applies, but one way would be to direct all writes (including any transaction that may eventually write) to a single connection.

Then writers queue up, while readers are unimpeded.

mostlysimilar•2h ago
Around what amount of load would you say the overhead of opening/closing becomes a problem?
manmal•22m ago
Doesn’t SQLite have its own in-memory cache? Is this about having more control re cache size?
d1l•2h ago
This is strange on so many levels.

SQLite does not even do network I/O.

How does sharing a connection (and transaction scope) in an asyncio environment even work? Won’t you still need a connection per asyncio context?

Does sqlite_open really take long compared to the inevitable contention for the write lock you’ll see when you have many concurrent contexts?

Does sqlite_open even register in comparison with the overhead of the python interpreter?

What is an asyncio SQLite connection anyways? Isn’t it just a regular one that gets hucked into a separate thread?

simonw•2h ago
If you're talking to a 100KB SQLite database file this kind of thing is likely unnecessary, just opening and closing a connection for each query is probably fine.

If you're querying a multi-GB SQLite database there are things like per-connection caches that may benefit from a connection pool.

> What is an asyncio SQLite connection anyways? Isn’t it just a regular one that gets hucked into a separate thread?

Basically yes - aiosqlite works by opening each connection in a dedicated thread and then sending async queries to it and waiting for a response that gets sent to a Future.

https://github.com/omnilib/aiosqlite/blob/895fd9183b43cecce8...

crazygringo•1h ago
> If you're querying a multi-GB SQLite database

In which case SQLite is probably the wrong tool for the job, and you should be using Postgres or MySQL that is actually designed from the ground up for lots of concurrent connections.

SQLite is amazing. I love SQLite. But I love it for single-user single-machine scenarios. Not multi-user. Not over a network.

simonw•1h ago
Multi-GB is tiny these days.

I didn't say anything about concurrent access. SQLite with WAL mode is fine for that these days for dozens of concurrent readers/writers (OK only one writer gets to write at a time, but if your writes queue for 1-2ms who cares?) - if you're dealing with hundreds or thousands over a network then yeah, use a server-based database engine.

brulard•1h ago
I always had troubles having multiple processes get write access to the sqlite file. For example if I have node.js backend work with that file, and I try to access the file with different tool (adminer for example) it fails (file in use or something like that). Should it work? I don't know if I'm doing something wrong, but this is my experience with multiple projects.
Groxx•1h ago
They can't write concurrently, but generally speaking yes, they can: https://sqlite.org/faq.html#q5

Your throughput will be much worse than a single process, but it's possible, and sometimes convenient. Maybe something in your stack is trying to hold open a writable connection in both processes?

cyanydeez•55m ago
PRAGMA journal_mode = WAL;
naasking•1h ago
> In which case SQLite is probably the wrong tool for the job

Why? If all it's missing is an async connection pool to make it a good tool for more jobs, what's the problem with just creating one?

d1l•1h ago
That's even crazier - so you're using asyncio because you have a ton of slow network-bound stuff - but for your database access you are running every sqlite connection in it's own thread and just managing those threads via the asyncio event loop?
quietbritishjim•1h ago
What is crazy about that?
lttlrck•1h ago
Of course I don't know what the parent is thinking, but my thought is: why can't it be entirely event loop driven? What are the threads adding here?

(I don't know anything about that project and this isn't meant as a criticism of its design or a challenge - cos I'd probably lose :-) )

maxbond•42m ago
Python's asyncio is single threaded. If you didn't send them into a different thread, the entire event loop would block, and it would degenerate to a fully synchronous single threaded program with additional overhead.
mayli•10m ago
Cause the sqlite-lib that python ships isn't async, and sqlite itself usually doesn't give an async API.
mayli•11m ago
FYI, I've once had few long-lived connection with wal, and wal file just goes exploded. Turns out sqlite won't truncate the wal if there are open connections.
bob1029•2m ago
I've been thinking about trying pre-serialization of SQLite commands to enable single-writer against a singleton SQLiteConnection using something like Channel<T> or other high performance MPSC abstraction. Most SQLite providers have an internal mutex that handles serialization, but if we can avoid all contention on this mutex things might go faster. Opening and closing SQLite connections is expensive. If we can re-use the same instance things go a lot faster.