frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Show HN: (bits) of a Libc, Optimized for Wasm

https://github.com/ncruces/go-sqlite3/tree/main/sqlite3/libc
78•ncruces•8mo ago
I make a no-CGO Go SQLite driver, by compiling the amalgamation to Wasm, then loading the result with wazero (a CGO-free Wasm runtime).

To compile SQLite, I use wasi-sdk, which uses wasi-libc, which is based on musl. It's been said that musl is slow(er than glibc), which is true, to a point.

musl uses SWAR on a size_t to implement various functions in string.h. This is fine, except size_t is just 32-bit on Wasm.

I found that implementing a few of those functions with Wasm SIMD128 can make them go around 4x faster.

Other functions don't even use SWAR; redoing those can make them 16x faster.

Smooth sort also has trouble pulling its own weight; a Shell sort seems both simpler and faster, while similarly avoiding recursion, allocations and the addressable stack.

I found that using SIMD intrinsics (rather than SWAR) makes it easier to avoid UB, but the code would definitely benefit from more eyeballs.

See this for some benchmarks on both x86-64 and Aarch64: https://github.com/ncruces/go-sqlite3/actions/runs/145169318...

Comments

phickey•8mo ago
This looks like a nice approach to making wasi-libc faster. Could you submit these changes upstream?
ncruces•8mo ago
I'd like to be a little more sure that I'm not totally messing things up before doing that, but yes, eventually, that would be a nice outcome.

I've also only really tested wazero. I can't know for sure that this is a straight improvement for other runtimes and architectures.

For instance, the code delays using wasm_i8x16_bitmask as much as possible, because on Aarch64 it can be slower than not using SIMD at all, whereas it's plenty fast on x86-64.

phickey•8mo ago
The maintainers of wasi-libc are some of the best people to review this, and I don’t think it would be wasting their time to ask them to look at a PR.
ncruces•8mo ago
A PR is a significant investment from me. I'd have to figure out where something like this is supposed to fit, how the build infra works, etc.

One of the nice things about Go is how much that's a solved issue out of the box, compared to almost everything else; certainly compared to C.

Pinging them in an issue: https://github.com/WebAssembly/wasi-libc/issues/580

nu11ptr•8mo ago
It is still a bit early, but I'm majorly bullish on WASM for multiple use cases:

1. Client side browser polyglot "applets" (Java applets were ahead of their time IMO)

2. Server side polyglot "servlets" (Node.js, embedded runtimes, etc.)

3. Language interop/FFI (Lang A -> WASM -> Lang B, like wasm2c)

Why is #3 so interesting? The hardest thing in language conversion is the library calls. WASI standardizes that, so all the proprietary libs will eventually compile down to WASI as a sort of POSIX/libc like layer. In addition, WASM standardizes calling convention. The resulting new source code may not look like much, but it will solve the FFI calling convention/marshalling/library issues nicely.

frumplestlatz•8mo ago
I’m not sure how it solves the FFI problem. Lowest common denominator calling conventions don’t make it any easier to bridge languages than it already is.

C calling conventions are already the standard for FFI in native code, and that means dropping down to what can be expressed in C if you want to cross that boundary.

ncruces•8mo ago
As far as Go is concerned, the Wasm sandbox makes the (addressable, C) stack explicit, which solves at least some of the issues CGO has to deal with.

It's not a panacea, though; it introduces other issues.

fuhsnn•8mo ago
Wasm intrinsics look neat as a higher-level fixed size SIMD abstraction. I wonder how good the compilers can do if using them for AOT targets with libraries like simd-everywhere.

string.h is missing strstr(), there's an algorithm of similar complexity you might consider: http://0x80.pl/notesen/2016-11-28-simd-strfind.html

ncruces•8mo ago
Yeah, so far I did exactly the ones (my build of) SQLite needed and not others.

If there's interest, the set of implemented functions can definitely be extended.

cedws•8mo ago
Would you consider writing some blog posts or other resources about WASM? I was experimenting recently with WIT, and ran into a mountain of issues. There's a lot of jargon that could do with some untangling.

It took me a lot longer than it should have to put together this basic module, and even then there's this shared library I had to download to build it, and I couldn't figure out why this requires a libc:

https://github.com/cedws/wasm-wit-test

ncruces•8mo ago
I'm not that great at long form writing to be honest, it's always a bit of a chore, and I'm never happy with the result.

To answer your question, it needs a libc because you're including stdlib.h, and exporting and allocator (even if you're not otherwise using it). You need a libc for malloc.

This is generally a good idea, if you need to send anything beyond numbers across the API (e.g. you need an allocator if you want to send strings as pointers).

I never used WIT, so I have no idea if this a requirement for WIT.

cedws•8mo ago
Ah ok. Thanks!
forrestthewoods•8mo ago
What is SWAR?
ncruces•8mo ago
SIMD within a register: https://en.wikipedia.org/wiki/SWAR

It's generally used for techniques that apply SIMD principles within general-purpose registers and instructions.

Assume you've loaded a 64-bit register (a uint64_t) with 8 bytes (unsigned char) of data. Can you answer the question “is any of these 8 bytes zero (the NUL terminator)?”

If you find a cheap way to do it, you can make strlen go faster by consuming 8 bytes at a time.

Et voilà:

   #define ONES ((uint64_t)-1/UCHAR_MAX)
   #define HIGHS (ONES \* (UCHAR_MAX/2+1))
   #define HASZERO(x) ((x)-ONES & ~(x) & HIGHS)
forrestthewoods•8mo ago
TIL, thanks!
tuananh•8mo ago
very cool project.

it's kinda frustrating to compile sqlite for wasm. can be done but quite troublesome.

Show HN: Netrinos – A keep it simple Mesh VPN for small teams

https://netrinos.com
67•pcarroll•2d ago•31 comments

Show HN: LLMKit – Compare LLMs side-by-side with real-time streaming

https://www.llmkit.cc/model-comparison/gpt-4-vs-claude-3-5-sonnet
2•chieund•21m ago•0 comments

Show HN: An easy way of broadcasting radio around you (looking for feedback)

https://github.com/dpipstudio/botwave
13•douxx•4d ago•0 comments

Show HN: Backlog – a public repository of real work problems

https://www.worldsbacklog.com/
96•anticlickwise•9h ago•22 comments

Show HN: Our vibe coded openrouter alternative just made over $40k last 30 days

https://trustmrr.com/startup/llm-gateway
2•smakosh•44m ago•0 comments

Show HN: CleanCloud – Read-only cloud hygiene checks for AWS and Azure

2•sureshcsdp•46m ago•0 comments

Show HN: Explore ArXiv/HN/LessWrong with SQL and Vector Algebra

https://exopriors.com/scry
2•Xyra•55m ago•0 comments

Show HN: Meds — High-performance firewall powered by NFQUEUE and Go

https://github.com/cnaize/meds
2•cnaize•1h ago•0 comments

Show HN: DeepSearch – a high-performance SMB directory scanner in Rust

https://github.com/dohuyhoang93/DeepSearch
3•dohuyhoangvn93•3h ago•1 comments

Show HN: Rust/WASM lighting data toolkit – parses legacy formats, generates SVGs

https://eulumdat.icu
42•holg•21h ago•0 comments

Show HN: Books mentioned on Hacker News in 2025

https://hackernews-readings-613604506318.us-west1.run.app
559•seinvak•1d ago•194 comments

Show HN: Jmail – Google Suite for Epstein files

https://www.jmail.world
1455•lukeigel•1d ago•337 comments

Show HN: Haystack Slop Detector – a lightweight barrier against unreviewed PRs

https://haystackeditor.com/slop-detector
2•akshaysg•57m ago•0 comments

Show HN: Skyler – AI email organizer, shut down due to OAuth compliance

https://skylerinbox.com/
3•sanjaykumar584•3h ago•0 comments

Show HN: Shittp – Volatile Dotfiles over SSH

https://github.com/FOBshippingpoint/shittp
130•sdovan1•1d ago•80 comments

Show HN: WalletWallet – create Apple passes from anything

https://walletwallet.alen.ro/
426•alentodorov•1d ago•107 comments

Show HN: HN Wrapped 2025 - an LLM reviews your year on HN

https://hn-wrapped.kadoa.com?year=2025
299•hubraumhugo•2d ago•150 comments

Show HN: RenderCV – Open-source CV/resume generator, YAML to PDF

https://github.com/rendercv/rendercv
90•sinaatalay•1d ago•40 comments

Show HN: Python Local Sandbox Code Execution (Podman and Uv)

https://github.com/portofcontext/pctx-py-sandbox
3•pmkelly4444•5h ago•0 comments

Show HN: The Official National Train Map Sucked, So I Made My Own

https://www.bdzmap.com/
75•Pavlinbg•1d ago•22 comments

Show HN: Make your own Danish Julehjerter (Braided hearts)

https://juleflet.dk
3•thomasahle•8h ago•0 comments

Show HN: I built a 1‑dollar feedback tool as a Sunday side project

https://onedollarfeedback.com/
16•jeremy0405•20h ago•4 comments

Show HN: Yapi – FOSS Terminal API Client for Power Users

https://yapi.run/blog/what-is-yapi
4•jamiepond•9h ago•1 comments

Show HN: Mushak – Zero config zero downtime Docker/Compose to server deployment

https://mushak.sh
27•hmontazeri•1d ago•15 comments

Show HN: AI-Augmented Memory for Groups

https://www.largemem.com/
10•vishal-ds•6d ago•4 comments

Show HN: Claude Code Plugin to play music when waiting on user input

https://github.com/Sevii/agent-marketplace/blob/main/plugins/elevator-music/README.md
54•Sevii•2d ago•14 comments

Show HN: Agent/Claude skill for creating ChatGPT Apps

https://github.com/BayramAnnakov/chatgpt-app-skill
2•Bayram•14h ago•1 comments

Show HN: Open-source Markdown research tool written in Rust – Ekphos

https://github.com/hanebox/ekphos
34•haneboxx•6d ago•15 comments

Show HN: TinyPDF – 3kb pdf library (70x smaller than jsPDF)

https://github.com/Lulzx/tinypdf
250•lulzx•3d ago•32 comments

Show HN: Open-source Markdown publishing framework for AI agents and developers

https://github.com/waynesutton/markdown-site
2•waynesutton•15h ago•0 comments