frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: Hashsync – Deduplicate SSH streams against the receiver's files

https://github.com/someplaceguy/hashsync
2•someplaceguy•1h ago
On a recent trip, while stuck on spotty hotel Wi-Fi and roaming 5G, I got fed up with how much bandwidth commands like rsync and nix-copy-closure waste.

rsync's delta transfer compares each file only with the file of the same name on the destination (or, with --fuzzy, a file in the same directory), so renamed, moved and copied files are sent in full.

nix-copy-closure can be worse: a rebuilt package gets a new store path, so the whole package is sent again, even when most of its contents haven't changed.

So I created hashsync, which runs a command over SSH and deduplicates the command's output, input, or both against an index of the receiving host's files.

As an example, for my last NixOS update, nix-copy-closure would have sent 1.7 GiB with --gzip, but hashsync took it down to 0.7 GiB instead. On some of my backups, I've seen up to 2x-10x less data and time to transfer compared to rsync with compression.

Two things make hashsync unique:

1. It searches at every byte offset of the stream, whatever its format, for blocks of the indexed files. In effect, it works like rsync's algorithm, but across every indexed file.

2. It's a transparent pipe, so it works with rsync, tar, or anything else you'd send through ssh.

See the repo's README for more info.

Feel free to try it out and let me know if you have any questions!

GitHub Incident with Actions, Pages, and Webhooks

https://www.githubstatus.com/incidents/pwxsjgnvrk6b
2•damontal•1m ago•1 comments

Understanding Open-Ended Work from a Data Perspective

https://www.fig.inc/blog/understanding-open-ended-work-from-a-data-perspective/
1•joehabeebs•1m ago•0 comments

Show HN: Pollsight – a simple tool to aid ballot research

https://pollsight.napsty.com
1•gaieges•2m ago•0 comments

Show HN: Offense – Security tools for your agents

https://github.com/offense-research/offense
1•imranye•3m ago•0 comments

Truth, Bullshit and Reputation

https://blog.happyfellow.dev/truth-bullshit-and-reputation/
1•thehappyfellow•3m ago•0 comments

The Biggest Crypto Theft in History [video]

https://www.youtube.com/watch?v=m7hL3bVoUuQ
2•Bender•5m ago•0 comments

AI review of the research of Emmanuel Saez and Gabriel Zucman

https://saez-zucman-audit-replication.vercel.app/
1•sreeix•5m ago•0 comments

Trusted code is one bad update away from malware

https://photostructure.com/coding/malicious-editor-extensions/
1•mceachen•5m ago•0 comments

Altman says 'quite terrible' for your PCs to have RAM

https://www.tomshardware.com/tech-industry/data-centers/openai-ceo-says-its-quite-terrible-for-yo...
2•theanonymousone•6m ago•1 comments

Digital panopticon: 'When you work faster, you have to work even faster'

https://euobserver.com/245543/digital-panopticon-when-you-work-faster-you-have-to-work-even-faste...
1•robtherobber•7m ago•0 comments

MecAgent (AI agent for SolidWorks) scans drives for CAD files and uploads them

https://www.reddit.com/r/SolidWorks/comments/1xjhrvz/psa_mecagent_ai_agent_for_solidworks_scans_y...
2•ummonk•8m ago•0 comments

Sam Altman says the chip supply chain's invisible hand failing and DRAM is why

https://finance.biggo.com/news/c8f9be1b0cccfd2c
1•theanonymousone•8m ago•1 comments

Show HN: Newton Wars – Fight your friends and planetary gravity

https://newtonwars.wipgames.dev/
1•neuling•9m ago•1 comments

A Flutter game holds 60 FPS on a $100 Android phone

https://upsling.space/blog/60-fps-flutter-physics-on-a-100-dollar-phone/
1•bjornhoj•9m ago•0 comments

Show HN: Prompt – A New, Beautiful Terminal

https://github.com/paul-oms/prompt
4•albertgoeswoof•10m ago•1 comments

Alkaline batteries are obsolete – but not for the reason you'd think [video]

https://www.youtube.com/watch?v=I1rlK3BKmyA
1•tambourine_man•10m ago•0 comments

Claude subscription now usable in OpenCode (legit)

https://twitter.com/opencode/status/2116198602523513081
2•kandros•13m ago•0 comments

Singapore third-quarter layoffs at highest level since 2020

https://www.reuters.com/business/world-at-work/singapore-third-quarter-layoffs-highest-level-sinc...
1•frank_clover•13m ago•0 comments

Show HN: An easy-to-play, hard-to-complete card battler

https://cnaize.com/fawe-arena/en/
1•cnaize•14m ago•0 comments

Anthropic hacked to deliver remote access trojan to Claude developer laptops

https://doublepulsar.com/anthropic-hacked-to-deliver-remote-access-trojan-to-claude-developer-lap...
4•vrganj•15m ago•1 comments

Robert Caro – The master biographer reflects at 91

https://www.dwarkesh.com/p/robert-caro
2•swolpers•15m ago•0 comments

4 Real? Tracking exploitation and patching of eMudhra's weak keys

https://blog.cloudflare.com/emudhra-patching/
2•pure-orange•16m ago•0 comments

Muse wins a small claims case

https://twitter.com/philwinkle/status/2116180121447014434
1•CGMthrowaway•17m ago•0 comments

Alphabet Sheds Half a Trillion in Market Value in Stock's Worst Day

https://www.wsj.com/tech/ai/alphabet-sheds-half-a-trillion-in-market-value-in-stocks-worst-day-ev...
2•teachrdan•18m ago•1 comments

Libriak-C – A C11 Client for Riak KV with HTTP and Protocol Buffers

https://github.com/arthurtumanyan/libriak-c
1•arthurtumanyan•19m ago•0 comments

I miss the part of programming where I figure it out myself

https://medium.com/@behind-the-stack/ai-has-made-coding-boring-adb1826134b0
2•birdculture•19m ago•0 comments

ACM Article: from code scarcity to verification scarcity by Ahmed El-Deeb

https://dl.acm.org/doi/10.1145/3844691.3844693
1•qaprof•20m ago•0 comments

Anthropic is renaming their Max plans

https://claude.com/pricing
2•pretext•22m ago•1 comments

Prism (OpenAI) shutting down January 1, 2027

https://prism.openai.com/
1•armcat•23m ago•0 comments

Pop the AI Bubble

https://aibubble.bighea.com
1•cbg0•23m ago•0 comments