Ask HN: Should I implement my own integrity checks on my fileserver?

2•jacobwilliamroy•8mo ago

So I have an SFTP server [Ubuntu 24.04.2 LTS (GNU/Linux 6.8.0-55-generic x86_64)] and an SFTP client (Windows 11): the client downloads files from the SFTP server. I had an idea to implement integrity checks by hashing the file on server side and also hashing the client's copy of the file after it is downloaded, then comparing the hashes to ensure the data was downloaded correctly. My question is: should I spend time implementing this? Does the OS already do this? This seems like something an OS should do.

Comments

PaulHoule•8mo ago

It could be expensive for the OS to update a secure hash for a file every time you write() to it.

daveguy•8mo ago

SFTP protocol includes checksum hashing on each packet sent. So corruption from the network is very unlikely (as it is also encrypted).

On the write to disk side, you are probably best off using ZFS or btrfs as the filesystem. These contain the option for similar integrity checks / error correction on write.

What is your threat model? Are you concerned about adversarial changes to the data or just prevention of corruption? Either way an adversary would have to be deep in your system or mitm to get around the transfer protocol protections. And the transfer protocols used by SFTP should handle random network corruption.

jacobwilliamroy•8mo ago

I am only trying to ensure the data on the local system is the same as the data on the server. There is no adversary in the middle modifying data; so this is strictly about detecting corruption.

Near-Instantly Aborting the Worst Pain Imaginable with Psychedelics

Show HN: Nginx-defender – realtime abuse blocking for Nginx

The Super Sharp Blade

Smart Homes Are Terrible

What I haven't figured out

KPMG pressed its auditor to pass on AI cost savings

Open-source Claude skill that optimizes Hinge profiles. Pretty well.

First Proof

I squeezed a BERT sentiment analyzer into 1GB RAM on a $5 VPS

Kagi Translate

Building Interactive C/C++ workflows in Jupyter through Clang-REPL [video]

Tactical tornado is the new default

Full-Circle Test-Driven Firmware Development with OpenClaw

Automating Myself Out of My Job – Part 2

Google staff call for firm to cut ties with ICE

Dependency Resolution Methods

Crypto firm apologises for sending Bitcoin users $40B by mistake

Show HN: iPlotCSV: CSV Data, Visualized Beautifully for Free

There's no such thing as "tech" (Ten years later)

List of unproven and disproven cancer treatments

Me/CFS: The blind spot in proactive medicine (Open Letter)

Ask HN: What are the word games do you play everyday?

Show HN: Paper Arena – A social trading feed where only AI agents can post

TOSTracker – The AI Training Asymmetry

The Devil Inside GitHub

Show HN: Distill – Migrate LLM agents from expensive to cheap models

Show HN: Sigma Runtime – Maintaining 100% Fact Integrity over 120 LLM Cycles

Make a local open-source AI chatbot with access to Fedora documentation

Introduce the Vouch/Denouncement Contribution Model by Mitchellh

Software Factories and the Agentic Moment