frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: Hashsync – Deduplicate SSH streams against the receiver's files

https://github.com/someplaceguy/hashsync
2•someplaceguy•1h ago
On a recent trip, while stuck on spotty hotel Wi-Fi and roaming 5G, I got fed up with how much bandwidth commands like rsync and nix-copy-closure waste.

rsync's delta transfer compares each file only with the file of the same name on the destination (or, with --fuzzy, a file in the same directory), so renamed, moved and copied files are sent in full.

nix-copy-closure can be worse: a rebuilt package gets a new store path, so the whole package is sent again, even when most of its contents haven't changed.

So I created hashsync, which runs a command over SSH and deduplicates the command's output, input, or both against an index of the receiving host's files.

As an example, for my last NixOS update, nix-copy-closure would have sent 1.7 GiB with --gzip, but hashsync took it down to 0.7 GiB instead. On some of my backups, I've seen up to 2x-10x less data and time to transfer compared to rsync with compression.

Two things make hashsync unique:

1. It searches at every byte offset of the stream, whatever its format, for blocks of the indexed files. In effect, it works like rsync's algorithm, but across every indexed file.

2. It's a transparent pipe, so it works with rsync, tar, or anything else you'd send through ssh.

See the repo's README for more info.

Feel free to try it out and let me know if you have any questions!

Show HN: Spinf-serve – inference server for bulk scoring and decisions

https://github.com/spinfinc/spinf-serve
7•giarec•1h ago•0 comments

Show HN: I built a multiplayer game where you're an aerial fighter cat

https://catcopter.com
36•pawelwentpawel•2h ago•18 comments

Show HN: Chess960v2 – Over 800 Rounds Played

https://chess960v2.com/en
2•lavren1974•58m ago•0 comments

Show HN: Deckdrill – Pimsleur-style audio drills from Anki decks, in your voice

https://www.deckdrill.app
2•pjacowski•1h ago•0 comments

Show HN: Hashsync – Deduplicate SSH streams against the receiver's files

https://github.com/someplaceguy/hashsync
2•someplaceguy•1h ago•0 comments

Show HN: I Turned DOOM into a Transformer

https://github.com/mottopanikeiku/doomformer
121•falperc•18h ago•35 comments

Show HN: Fraktur, a DJ mixing app in the browser

https://greenvitriol.com/posts/fraktur
12•janesconference•4h ago•0 comments

Show HN: Keyward – a TUI to manage SSH keys and audit your –/.ssh security

https://github.com/gateway-of-last-resort/keyward
2•weiqur•2h ago•0 comments

Show HN: Auto Dismiss Mac OS "Allow <app> to find devices on local networks?"

https://github.com/johncassil/dontallow
147•johncassil•22h ago•119 comments

Show HN: Sketch Stadium – watch your agent play Pictionary

https://sketch.wakeshift.ai/
3•ericlevine•2h ago•1 comments

Show HN: PicoPlayer – A tiny, open-source website for playing quality webgames

https://picoplayer.net/
4•popcar2•5h ago•6 comments

Show HN: Tinycap, a tiny Mac screen recorder that makes PR-sized GIFs and MP4s

https://soleo.github.io/tinycap/
4•soleo•3h ago•2 comments

Show HN: Every Permutation of a Deck of Cards

https://mrrc.cc/52!/
2•mrrccc•4h ago•1 comments

Show HN: A proof-of-concept formalization of the IRS federal tax forms

https://github.com/emmilco/us-tax-model
80•RGS1811•1d ago•13 comments

Show HN: Lasm – Write formally verified webapps in Lean for Node/serverless/edge

https://github.com/Millillion/lasm
33•lastmjs•16h ago•20 comments

Show HN: Showbuild – Hacker News but only for builders

https://showbuild.co/
5•ritik30•5h ago•1 comments

Show HN: Overtone – Transcribe, correct and cut captions for any screen shape

https://github.com/lonthq/overtone-captions
41•rvanderlinden•1d ago•3 comments

Show HN: Arken – Passwordless auth middleware using secp256k1 keys

https://github.com/zacharybenson/arken-protocol
10•zachbenson817•16h ago•0 comments

Show HN: Typebook.lol read books by typing the letters only

https://typebook.lol/
9•vegnus•22h ago•12 comments

Show HN: SlideBySide – Present slides without sharing your screen

https://slidebyside.com/
81•denismenace•2d ago•35 comments

Show HN: DistractFree-Chrome extension and web app to stop doomscrolling

https://distractfree.site/
3•ramthokane•13h ago•0 comments

Show HN: phpClaw – AI agent runtime for PHP with tools, memory and guards

https://github.com/phpclaw-php/phpclaw-monorepo
12•akashpatel7204•1d ago•4 comments

Show HN: Yavashark: I've Built a JavaScript Engine Every Day for 950 Days

https://yavashark.dev/blog/announcing-yavashark/
38•sharktheone•2d ago•3 comments

Show HN: Radix - A WebGPU-powered 3D Gaussian Splat renderer built on Three.js

https://sparkjs.dev/radix/
11•dmarcos•1d ago•0 comments

Show HN: OpenArcanum – An End-to-End Encrypted Alternative to Discord

https://github.com/hyperkubeorg/openarcanum/
5•SamInTheShell•21h ago•0 comments

Show HN: Fitness Functions for Dbt Projects

https://github.com/tjirab/tff
3•bartschuijt•22h ago•2 comments

Show HN: Tyrtulia, a headless forum with no passwords, made for the terminal

https://en.andros.dev/blog/b5c8c6aa/i-built-a-forum-with-no-interface-no-passwords-made-for-the-t...
3•andros•6h ago•0 comments

Show HN: Capo-S3 – a typed, async-first S3 client for Python

https://pypi.org/project/capo-s3/
16•karpetrosyan•1d ago•3 comments

Show HN: An open-source app for IELTS preparation

https://github.com/Luxshan2000/ielts-prep
5•luxshan_t•1d ago•0 comments

Show HN: Witnessory – a signed record of the web's move to post-quantum crypto

https://witnessory.org
6•parthod0x•1d ago•0 comments