frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

I got hacked: My Hetzner server started mining Monero

https://blog.jakesaunders.dev/my-server-started-mining-monero-this-morning/
145•jakelsaunders94•3h ago

Comments

guerrilla•2h ago
Whew, load average of 0 here.
V__•2h ago
> The Reddit post I’d seen earlier? That guy got completely owned because his container was running as root. The malware could: [...]

Is that the case, though? My understanding was, that even if I run a docker container as root and the container is 100% compromised, there still would need to be a vulnerability in docker for it to “attack” the host, or am I missing something?

Onavo•2h ago
Either docker or a kernel level exploit. With non-VM containers, you are sharing a kernel.
ronsor•2h ago
There would be, but a lot of docker containers are misconfigured or unnecessarily privileged, allowing for escape.

Also, if you've been compromised, you may have a rootkit that hides itself from the filesystem, so you can't be sure of a file's existence through a simple `ls` or `stat`.

miladyincontrol•36m ago
> but a lot of docker containers are misconfigured or unnecessarily privileged, allowing for escape

Honestly, citation needed. Very rare unless you're literally giving the container access to write to /usr/bin or other binaries the host is running, to reconfigure your entire /etc, access to sockets like docker's, or some other insane level of over reach I doubt even the least educated docker user would do.

While of course they should be scoped properly, people act like some elusive 0-day container escape will get used on their minecraft server or personal blog that has otherwise sane mounts, non-admin capabilities, etc. You arent that special.

TheRealPomax•2h ago
Docker containers with root have rootish rights on the host machine too because the userid will just be 0 for both. So if you have, say, a bind mount that you play fast and loose with, the docker user can create 0777 files outside the docker container, and now we're almost done. Even worse if "just to make it work" someone runs the container with --privileged and then makes the terminal mistake of exposing that container to the internet.
V__•2h ago
Can you explain this a bit further? Wouldn't that 0777 file outside docker be still executed inside the container and not on the host?
necovek•1h ago
I believe they meant you could create an executable that is accessible outside the container (maybe even as setuid root one), and depending on the path settings, it might be possible to get the user to run it on the host.

Imagine naming this executable "ls" or "echo" and someone having "." in their path (which is why you shouldn't): as long as you do "ls" in this directory, you've ran compromised code.

There are obviously other ways to get that executable to be run on the host, this just a simple example.

marwamc•1h ago
Another example is they would enumerate your directories and find the names of common scripts and then overwrite your script. Or to be even sneakier, they can append their malicious code to an existing script in your filesystem. Now each time you run your script, their code piggybacks.

OTH if I had written such a script for linux I'd be looking to grab the contents of $(hist) $(env) $(cat /etc/{group,passwd})... then enumerate /usr/bin/ /usr/local/bin/ and the XDG_{CACHE,CONFIG} dirs - some plaintext credentials are usually here.

The $HOME/.{aws,docker,claude,ssh}

Basically the attacker just needs to know their way around your OS. The script enumerating these directories is the 0777 script they were able to write from inside the root access container.

tracker1•1h ago
If your chosen development environment supports it, look into distroless or empty base containers, and run as --read-only if you can.

Go and Rust tend to lend themselves to these more restrictive environments a bit better than other options.

Havoc•2h ago
I think a root container can talk to docker daemon and launch additional containers...with volume mounts of additional parts of file system etc. Not particularly confident about that one though
minitech•2h ago
Unintentional vulnerabilities in Docker and the kernel aside, it can only do that if it has access to the Docker API (usually through a bind mount of the Unix socket). Having access to the Docker API is equivalent to having root on the host.
czbond•2h ago
Well $hit. I have been using Docker for installing NPM modules in interactive projects I was testing out. I believed Docker blocked access to the underlying host (my computer).

Thanks for mentioning it - but now... how does one deal with this?

minitech•2h ago
If you didn’t mount docker.sock or any directory above it (i.e. / or /run by default) or run your containers as --privileged, you’re probably fine with respect to this angle. I’d still recommend rootless containers under unprivileged users* or VMs for extra comfort. Qubes (https://www.qubes-os.org/) is good, even if it’s a little clunkier than it could be.

* but if you’re used to bind-mounting, they’ll be a hassle

Edit: This is by no means comprehensive, but I feel compelled to point it out specifically for some reason: remember not to mount .git writable, folks! Write access to .git is arbitrary code execution as whoever runs git.

3np•1h ago
As sibling mentioned, unless you or the runtime explicitly mount the docker socket, this particular scenario shouldn't affect you.

You might still want to tighten things up. Just adding on the "rootless" part - running the container runtime as an unprivileged user on the host instead of root - you also want to run npm/node as unprivileged user inside the container. I still see many defaulting to running as root inside the container since that's the default of most images. OP touches on this.

For rootless podman, this will run as a user with your current uid and map ownership of mounts/volumes:

    podman run -u$(id -u) --userns=keep-id
d4mi3n•2h ago
While this is true, the general security stance on this is: Docker is not a security boundary. You should not treat it like one. It will only give you _process level_ isolation. If you want something with better security guarantees, you can use a full VM (KVM/QEMU), something like gVisor[1] to limit the attack surface of a containerized process, or something like Firecracker[2] which is designed for multi-tenancy.

The core of the problem here is that process isolation doesn't save you from whole classes of attack vectors or misconfigurations that open you up to nasty surprises. Docker is great, just don't think of it as a sandbox to run untrusted code.

1. https://gvisor.dev/

2. https://firecracker-microvm.github.io/

socalgal2•1h ago
that's a really good point .. but, I think 99% of docker users believe it is a a sandbox and treat it as such.
freedomben•1h ago
And not without cause. We've been pitching docker as a security improvement for well over a decade now. And it is a security improvement, just not as much as many evangelists implied.
fragmede•1h ago
Must depend on who you've been talking to. Docker's not been pitched for security in the circles I run in, ever.
dist-epoch•1h ago
it is a sandbox against unintentional attacks and mistakes (sudo rm -rf /)

but will not stop serious malware

hsbauauvhabzb•1h ago
Virtual machines are treated as a security boundary despite the fact that with enough R&D they are not. Hosting minecraft servers in virtual machines is fine, but not a great idea if they’re cohosted on a machine that has billions of dollars in crypto or military secrets.

Docker is pretty much the same but supposedly more flimsy.

Both have non-obvious configuration weaknesses that can lead to escapes.

hoppp•57m ago
Yeah but why would somebody co-host military secrets or billions of dollars? Its a bit of a stretch
hsbauauvhabzb•53m ago
I think you’re missing the point, which was that high value targets adjacent to soft targets make escapes a legitimate target, but in low value scenarios vm escapes aren’t worth the R&D
michaelt•2h ago
Firstly, the attacker just wants to mine Monero with CPU, they can do that inside the container.

Second, even if your Docker container is configured properly, the attacker gets to call themselves root and talk to the kernel. It's a security boundary, sure, but it's not as battle-tested as the isolation of not being root, or the isolation between VMs.

Thirdly, in the stock configuration processes inside a docker container can use loads of RAM (causing random things to get swapped to disk or OOM killed), can consume lots of CPU, and can fill your disk up. If you consider denial-of-service an attack, there you are.

Fourthly, there are a bunch of settings that disable the security boundary, and a lot of guides online will tell you to use them. Doing something in Docker that needs to access hot-plugged webcams? Hmm, it's not working unless I set --privileged - oops, there goes the security boundary. Trying to attach a debugger while developing and you set CAP_SYS_PTRACE? Bypasses the security boundary. Things like that.

Nextgrid•1h ago
Container escapes exist. Now the question is whether the attacker has exploited it or not, and what the risk is.

Are you holding millions of dollars in crypto/sensitive data? Better assume the machine and data is compromised and plan accordingly.

Is this your toy server for some low-value things where nothing bad can happen besides a bit of embarrassment even if you do get hit by a container escape zero-day? You're probably fine.

This attack is just a large-scale automated attack designed to mine cryptocurrency; it's unlikely any human ever actually logged into your server. So cleaning up the container is most likely fine.

trhway•1h ago
>there still would need to be a vulnerability in docker for it to “attack” the host, or am I missing something?

non necessary vulnerability per. se. Bridged adapter for example lets you do a lot - few years ago there were a story of something like how a guy got a root in container and because the container used bridged adapter he was able to intercept traffic of an account info updates on GCP

minitech•2h ago
> Here’s the test. If /tmp/.XIN-unix/javae exists on my host, I’m fucked. If it doesn’t exist, then what I’m seeing is just Docker’s default behavior of showing container processes in the host’s ps output, but they’re actually isolated.

  /tmp/.XIN-unix/javae &
  rm /tmp/.XIN-unix/javae
This article’s LLM writing style is painful, and it’s full of misinformation (is Puppeteer even involved in the vulnerability?).
jakelsaunders94•2h ago
Yeah fair, I asked claude to help because honestly this was a little beyond my writing skills. I'm real though. Sorry. Will change
seafoamteal•2h ago
Hi Jake! Cool article, and it's something I'll keep in mind when I start giving my self-hosted setup a remodel soon. That said, I have to agree with the parent comment and say that the LLM writing style dulled what would otherwise have been a lovely sysadmin detective work article and didn't make me want to explore your site further.

I'm glad you're up to writing more of your own posts, though! I'm right there with you that writing is difficult, and I've definitely got some posts on similar topics up on my site that are overly long and meandering and not quite good, but that's fine because eventually once I write enough they'll hopefully get better.

Here's hoping I'll read more from you soon!

jakelsaunders94•2h ago
Thanks for the encouragement! I find it difficult to write articles beyond simply stating a series of facts.

I tried handwriting https://blog.jakesaunders.dev/schemaless-search-in-postgres/ bit I thought it came off as rambling.

Maybe I'll have a go at redrafting this tomorrow in non LLM-ese.

sincerely•2h ago
Just a data point - I would rather read bad human writing than LLM output
minitech•2h ago
Seconding what others have said about preferring to read bad human writing. And I don’t want to pick on you – this is a broadly applicable message prompted by a drop in the bucket – but please don’t publish articles beyond your ability to fact check. Just write what you actually know, and when you’re making a guess or you still have open questions at the end of your investigation, be honest about that. (People make mistakes all the time anyway, but we’re in an age where confident and detailed mistakes have become especially accessible.)
jakelsaunders94•2h ago
I fixed it, apologies for the misinformation.
Eduard•1h ago
I still see Puppeteer mentioned several times in your post and don't understand what that has to do with Umami, nextjs, and/or CVE-2025-66478.
3np•57m ago
It still says:

> IT NEVER ESCAPED.

You haven't confirmed this (at least from the contents of the article). You did some reasonable spot checks and confirmed/corrected your understanding of the setup. I'd agree that it looks likely that it did not escape or gain persistence on your host but in no way have you actually verified this. If it were me I'd still wipe the host and set up everything from scratch again[0].

Also your part about the container user not being root is still misinformed and/or misleading. The user inside the container, the container runtime user, and whether container is privileged are three different things that are being talked about as one.

Also, see my comment on firewall: https://news.ycombinator.com/item?id=46306974

[0]: Not necessarily drop-everything-you-do urgently but next time you get some downtime to do it calmly. Recovering like this is a good excercise anyway to make sure you can if you get a more critical situation in the future where you really need to. It will also be less time and work vs actually confirming that the host is uncontaminated.

jakelsaunders94•46m ago
I did see your comment on Firewall, and you're right about the escape. It seems safe enough for now. Between the hacking and accidentally hitting the front page of HN it's been a long day.

I'm going to sit down and rewrite the article and take a further look at the container tomorrow.

3np•16m ago
Hey, thanks for taking the time to share your learnings and engage. I'm sure there are HN readers who will be better off for it alongside you!

(And good to hear you're leaving the LLMs out of the writing next time <3)

mikaelmello•2h ago
This article is very interesting at first but I once again get disappointed after reading clear signs of AI like "Why this matters" and "The moment of truth", and then the whole thing gets tainted with signs all over the place.
dinkleberg•2h ago
Yeah personally I’d much rather read a poorly constructed article with actually interesting content than the same content put into the formulaic AI voice.
venturecruelty•1h ago
Article's been edited:

>Edit: A few people on HN have pointed out that this article sounds a little LLM generated. That’s because it’s largely a transcript of me panicking and talking to Claude. Sorry if it reads poorly, the incident really happened though!

For what it's worth, this is not an excuse, and I still don't appreciate being fed undisclosed slop. I'm not even reading it.

zamadatix•2h ago
I don't use Docker for my containers at home, but I take it by the concern that user namespacing is not the employed by them or something?
heavyset_go•2h ago
If you're root in a namespace and manage to escape, you can have root privileges outside of it.
zamadatix•45m ago
Are you referring to user namespaces and, if so, how does that kind of break out to host root work? I thought the whole point of user namespaces was your UID 0 inside the container is UID 100000 or whatever from the perspective of outside the container. Escaping the container shouldn't inherently grant you ability to change your actual UID in the host's main namespace in that kind of setup, but I'm not sure Docker actually leverages user namespaces or not.

E.g. on my systemd-nspawn setup with --private-users=pick (enables user namespacing) I created a container and gave it a bind mount. From the container it appears like files in the bind mount created by the container namespace's UID 0 are owned by UID 0 but from outside the container the same file looks owned by UID 100000. Inverted, files owned by the "real" UID 0 on the host look owned by 0 to the host but as owned by 65534 (i.e. "nobody") from the container's perspective. Breaking out of the container shouldn't inherently change the "actual" user of the process from 100000 to 0 any more than breaking out of the container as a non-0 UID in the first place - same as breaking out of any of the other namespaces doesn't make the "UID 0" user in the container turn into "UID 0" on the host.

heavyset_go•16m ago
Users in user namespaces are granted capabilities that root has, user namespaces themselves need to be locked down to prevent that, but if a user with root capabilities escapes the namespace, they have the capabilities on the host.

They also expose kernel interfaces that, if exploited, can lead to the same.

In the end, namespaces are just for partitioning resources, using them for sandboxes can work, but they aren't really sandboxes.

j45•2h ago
Never expose your server IP directly to the internet, vps or baremetal.
miramba•2h ago
Is there a way to do that and still be able to access the server?
Carrok•2h ago
Many ways. Using a "bastion host" is one option, with something like wireguard or tinc. Tailscale and similar services are another option. Tor is yet another option.
venturecruelty•1h ago
>Never expose your server IP directly to the internet, vps or baremetal.
cortesoft•1h ago
The bastion host is a server, though, and would be exposed to the internet.
sh3rl0ck•2h ago
Either via a VPN or a tunnel.
iLoveOncall•2h ago
Yes, CloudFlare ZeroTrust. It's entirely free, I use it for loads of containers on multiple hosts and it works perfectly.
m00x•2h ago
Yes, cloudflare tunnels do this, but I don't think it's really necessary for this.

I use them for self-hosting.

procaryote•2h ago
As in "always run a network firewall" or "keep the IP secret"? Because I've had people suggest both and one is silly.
mrkeen•2h ago
You're going to hate this thing called DNS
palata•2h ago
Unless you need it to be reachable from the Internet, at which point it has to be... reachable from the Internet.
sergsoares•1h ago
Not expose the server IP is one practice (obfuscation) in a list of several options.

But that alone would not solve the problem being a RCE from HTTP, that is why edge proxy provider like Cloudflare[0] and Fastfy[1] proactivily added protections in his WAF products.

Even cloudflare had an outage trying to protect his customers[3].

- [0] https://blog.cloudflare.com/waf-rules-react-vulnerability/ - [1] https://www.fastly.com/blog/fastlys-proactive-protection-cri... - [2] https://blog.cloudflare.com/5-december-2025-outage/

cortesoft•1h ago
Any server? How do you run a public website? Even if you put it behind a load balancer, the load balancer is still a “server exposed to the internet”
iLoveOncall•2h ago
> ls -la /tmp/.XIN-unix/javae

Unless ran as root this could return file not found because of missing permissions, and not just because the file doesn't actually exist, right?

> “I don’t use X” doesn’t mean your dependencies don’t use X

That is beyond obvious, and I don't understand how anyone would feel safe from reading about a CVE on a widely used technology when they run dozens of containers on their server. I have docker containers and as soon as I read the article I went and checked because I have no idea what technology most are built with.

> No more Umami. I’m salty. The CVE was disclosed, they patched it, but I’m not running Next.js-based analytics anymore.

Nonsensical reaction.

qingcharles•1h ago
Yeah, my Umami box was hit, but the time between the CVE disclosure and my box getting smacked was incredibly low. Umami patched it very quickly. And then patched it again a second time when the second CVE dropped right after.

Nothing is immune. What analytics are you going to run? If you roll your own you'll probably leave a hole somewhere.

Hackbraten•1h ago
> No more Umami. I’m salty.

But kudos for the word play!

grekowalski•2h ago
Recently, those Monero miners were installing themselves everywhere that had a vulnerable React 19. I had exactly the same problem.
qingcharles•2h ago
I had to nuke my Oracle Cloud box that runs my Umami server. It got hit. Was a good excuse to upgrade version and upgrade all my backup systems etc. Lost a few hours of data while it was returning 500 errors.
codegeek•2h ago
tl:dr: He got hacked but the damage was only restricted to one docker container runn ing Umami (that is built on top of NextJS). Thankfully, he was running the docker container as a non privileged non-root user which saved him big time considering the fact that the attack surface was limited only within the container and could not access the entire host/filesystem.

Is there ever a reason someone should run a docker container as root ?

d4mi3n•2h ago
If you're using the container to manage stuff on the host, it'll likely need to be a process running as root. I think the most common form of this is Docker-in-Docker style setups where a container is orchestrating other containers directly through the Docker socket.
wnevets•2h ago
Sure does seem like the primary outcome of cryptocurrencies being released onto the world has been criminals making money.
dylan604•1h ago
Is that really a surprise though?
venturecruelty•1h ago
Not for anyone who doesn't have a financial stake in said fraud, no.
nrhrjrjrjtntbt•1h ago
And fast malware detection.
BLKNSLVR•1h ago
Criminals and the porn industry are almost invariably early adopters of new technologies. For better or worse their use-cases are proof-of-concepts that get expanded and built on, if successful, by more legitimate industries.

Re: the Internet.

Re: Peer-to-peer.

Re: Video streaming.

Re: AI.

lapetitejort•27m ago
What is the average length of time for new tech to escape porn and crime and integrate into real applications? Longer than 15 years?
BLKNSLVR•12m ago
Some kind of function of how quickly regulation comes to the technology.
meisel•2h ago
Is mining via CPU even worthwhile for the hackers? I thought ASICs dominated mining
jsheard•2h ago
ASICs do dominate Bitcoin mining but Monero's POW algorithm is supposed to be ASIC resistant. Besides, who cares if it's efficient when it's someone else's server?
minitech•2h ago
Hard for it not to be worthwhile, since it’s free for them. Same automated exploit run across the entire internet.
Bender•2h ago
Optimal hardware costs money. Easy to hack machines are free and in nearly unlimited numbers.
justinsaccount•2h ago
If the effectiveness of mining is represented as profit divided by the cost of running the infrastructure, then a CPU that someone else is paying for is worth it as long as the profit is greater than zero.
heavyset_go•2h ago
Yes, for Monero it is the only real viable option. I'd also assume that the OP's instance is one of many other victims whose total mining might add up to a significant amount of crypto.
edm0nd•1h ago
Its easily worth it as they are not spending any money on compute or power.

If they can enslave 100s or even 1000s of machine mining XMR for them, easy money if you set aside the legality of it.

tgtweak•1h ago
Monero's proof of work (RandomX) is very asic-resistant and although it generates a very small amount of earnings, if you exploit a vulnerability like this with thousands or tens of thousands of nodes, it can add up (8 modern cores 24/7 on Monero would be in the 10-20c/day per node range). OPs Vps probably generated about $1 for those script kiddies.
pixl97•1h ago
Hit 1000 servers and it starts adding up. Especially if you live somewhere with a low cost of living.
rnhmjoj•1h ago
This is the PoW scheme that Monero currently uses:

> RandomX utilizes a virtual machine that executes programs in a special instruction set that consists of integer math, floating point math and branches. > These programs can be translated into the CPU's native machine code on the fly (example: program.asm). > At the end, the outputs of the executed programs are consolidated into a 256-bit result using a cryptographic hashing function (Blake2b).

I doubt that you anyone managed to create an ASIC that does this more efficiently and cost effective than a basic CPU. So, no, probably no one is mining Monero using an ASIC.

tolerance•2h ago
Was dad notified of the security breach? If not he may want to consider switching hosting providers. Dad deserves a proper LLM-free post mortem.
jakelsaunders94•2h ago
Hahaha, I did tell him this afternoon. This is the bloke who has the same password for all his banking apps despite me buying him 1password though. The imminent threat from RCE's just didn't land.
dylan604•1h ago
Buying someone 1Pass, or the like, and calling it good is not enough. People using password managers forget how long it takes to visit all of the sites you use to create that site's record, then update the password to a secure one, and then log out and log back in with the new password to test it is good. For a lot of people having a password manager bought for them is going to be over it after the second site. Just think about how many videos on TikTok they could have been watching instead
venturecruelty•1h ago
Yeah, mom and I sat down one afternoon and we changed all of her passwords to long, secure ones, generated by 1Password. It was a nice time! It also helped her remember all of the different services she needs to access, and now they're all safely stored with strong passwords. And it was a nice way to connect and spend some time together. :)
heavyset_go•2h ago
I wouldn't trust that boot image or storage again, I'd nuke it for peace of mind.

That said, do you have an image of the box or a container image? I'm curious about it.

jakelsaunders94•2h ago
Yeah I did consider just killing it, I'm going to keep an eye on it for a few days with a gun to it just in case.

I was lucky in that my DB backups were working so all my persistence wax backed up to S3. I think I could stand up another one in an hour.

Unfortunately I didn't keep an image no. I almost didn't have the foresight to investigate before yeeting the whole box into the sun!

qingcharles•1h ago
As an aside, if you're using a Hetzner VPS for Umami you might be over-specced. I just cut my Hetzner bill by $4/mo by moving my Umami box to one of the free Oracle Cloud VPS after someone on here pointed out the option to me. Depends whether this is a hobby thing or something more serious, but that option is there.
tgtweak•1h ago
The manageability of having everything on one host is kind of nice at that scale, but yeah you can stack free tiers on various providers for less.
ianschmitz•1h ago
I would pay $4/mo to stay as far away from Oracle as possible
spiderfarmer•1h ago
I pay for Hetzner because it’s an EU based, sane company without a power hungry CEO.
angulardragon03•1h ago
All fine and well, but oracle will threaten to turn off your instance if you don’t maintain a reasonable average CPU usage on the free hosts, and will eventually do so abruptly.

This became enough of a hassle that I stopped using them.

treesknees•1h ago
Do you mean if it’s idle, or if it’s maxed out? I’ve had a few relatively idle free-tier VMs with Oracle and I’ve not received any threats of shutoff over the last 3 years I’ve had them online.
jakelsaunders94•1h ago
I've got a whole Hetzner EX41 bare metal server, as opposed to a VPS. It's gotr like 20 services on it.

But yeah it is massively overspecced. Makes me feel cool load testing my go backend at 8000 requests per second though!

tgtweak•1h ago
Just a note - you can very much limit cpu usage on the docker containers by setting --cpus="0.5" (or cpus:0.5 in docker compose) if you expect it to be a very lightweight container, this isolation can help prevent one roudy container from hitting the rest of the system regardless of whether it's crypto-mining malware, a ddos attempt or a misbehaving service/software.
freedomben•1h ago
This is true, but it's also easy to set at one point and then later introduce a bursty endpoint that ends up throttled unnecessarily. Always a good idea to be familiar with your app's performance profile but it can be easy to let that get away from you.
jakelsaunders94•1h ago
This is a great shout actually. Thanks for pointing it out!
fragmede•1h ago
The other thing to note is that docker is for the most part, stateless. So if you're running something that has to deal with questionable user input (images and video or more importantly PDFs), is to stick it on its own VM and then cycle the docker container every hour and the VM every 12, and then still be worried about it getting hacked and leaking secrets.
tracker1•1h ago
Another is running containers in read-only mode, assuming they support this configuration... will minimize a lot of potential attack surface.
miladyincontrol•41m ago
Soft and hard memory limits are worth considering too, regardless of container method.
venturecruelty•1h ago
I still can't believe that there are so many people out here popping boxen and all they do is solve drug sudokus with the hardware. Hacks are so lame now.
ryanto•1h ago
Sorry to hear you got hacked.

I know we aren't supposed to rely on containers as a security boundary, but it sure is great hearing stories like this where the hack doesn't escape the container. The more obstacles the better I guess.

pigbearpig•1h ago
You might want to harden that those outbound firewall rules as another step. Did the Umami container need the ability to initiate connections? If not, that would eliminate the ability to do the outbound scans.

Also could prevent something to exfiltrate sensitive data.

danparsonson•1h ago
No firewall! Wow that's brave. Hetzner will let you configure one that runs outside of the box so you might want to add that too, as part of your defense in depth - that will cover you if you make a mistake with ufw. Personally I keep SSH firewalled only to my home address in this way; if I'm out and about and need access, I can just log into Hetzner's website and change it temporarily.
Nextgrid•1h ago
But the firewall wouldn't have saved them if they're running a public web service or need to interact with external services.

I guess you can have the appserver fully firewalled and have another bastion host acting as an HTTP proxy, both for inbound as well as outbound connections. But it's not trivial to set up especially for the outbound scenario.

danparsonson•1h ago
No you're right, I didn't mean the firewall would have saved them, but just as a general point of advice. And yes a second VPS running opnSense or similar makes a nice cheap proxy and then you can firewall off the main server completely. Although that wouldn't have saved them either - they'd still need to forward HTTP/S to the main box.
Nextgrid•1h ago
A firewall blocking outgoing connections (except those whitelisted through the proxy) would’ve likely prevented the download of the malware (as it’s usually done by using the RCE to call a curl/wget command rather than uploading the binary through the RCE) and/or its connection to the mining server.
denkmoon•40m ago
How many people do proper egress filtering though, even when running a firewall
jwrallie•25m ago
Password auth being enabled is also very brave. I don’t think fail2ban is necessary personally, but it’s popular enough that it always come up.
tete•11m ago
Firewalls in the majority of cases don't get you much. Yes it's a last line of defense if you do something really stupid and don't even know where or what you configure your services to listen on, but if you don't the difference between running firewalls and not is minuscule.

There are way more important things like actually knowing that you are running software with widely known RCE that don't even use established mechanisms to sandbox themselves it seems.

The way the author describes docker being the savior appears to be sheer luck.

OutOfHere•1h ago
You're lucky that Hetzner didn't delete your server and terminate your account.
nodesocket•1h ago
I also run Umami, but patched once the CVE patch was released. Also, I only expose the tracking js endpoint and /api/send via Caddy publically (though, /api/send might be enough to exploit the vul). To actually interact with Umami UI I use Twingate (similar to Tailscale) to tunnel into the VPC locally.
Computer0•1h ago
Still confused what I am supposed to do to avoid all this.
movedx•1h ago
Learning to manage an operating system in full, and having a healthy amount of paranoia, is a good first step.
seymon•1h ago
What's considered nowadays the best practice (in terms of security) for running selfhosted workloads with containers? Daemon less, unprivileged podman containers?

And maybe updating container images with a mechanism similar to renovate with "minimumReleaseTime=7days" or something similar!?

movedx•1h ago
You’ll set yourself up for success if you check the dependencies of anything you run, regardless of it being containerised. Use something like Snyk to scan containers and repositories for known exploits and see if anything stands out.

Then you need to run things with as least privilege as possible. Sadly, Docker and containers in general are an anti-pattern here because they’re about convenience first, security second. So the OP should have run the contains as read-only with tight resource limits and ideally IP restrictions on access if it’s not a public service.

Another thing you can do is use Tailscale, or something like it, to keep things being a zero trust, encrypted, access model. Not suitable for public services of course.

And a whole host of other things.

3np•1h ago
> I also enabled UFW (which I should have done ages ago)

I disrecommend UFW.

firewalld is a much better pick in current year and will not grow unmaintainable the way UFW rules can.

    firewall-cmd --persistent --set-default-zone=block
    firewall-cmd --persistent --zone=block --add-service=ssh
    firewall-cmd --persistent --zone=block --add-service=https
    firewall-cmd --persistent --zone=block --add-port=80/tcp
    firewall-cmd --reload
Configuration is backed by xml files in /etc/firewalld and /usr/lib/firewalld instead of the brittle pile of sticks that is the ufw rules files. Use the nftables backend unless you have your own reasons for needing legacy iptables.

Specifically for docker it is a very common gotcha that the container runtime can and will bypass firewall rules and open ports anyway. Depending on your configuration, those firewall rules in OP may not actually do anything to prevent docker from opening incoming ports.

Newer versions of firewalld gives an easy way to configure this via StrictForwardPorts=yes in /etc/firewalld/firewalld.conf.

exceptione•44m ago

  > Specifically for docker it is a very common gotcha that the container runtime can and will bypass firewall rules and open ports anyway. 
Like I said in another comment, drop Docker, install podman.
3np•32m ago
This affects podman too.
jsheard•11m ago
Not if you run it in rootless mode, which is more of a first class citizen in Podman compared to Docker.
3np•8m ago
Same as for docker, yes?

https://docs.docker.com/engine/security/rootless/

denkmoon•43m ago
I’ll just mention Foomuuri here. Its bit of a spiritual successor to shorewall and has firewalld emulation to work with tools compatible with firewalld
3np•21m ago
Thanks! Would be cool to have it packaged for alpine since firewalld requires D-Bus. There is awall but that's still on iptables and IMO at bit clunky to set up.
lloydatkinson•37m ago
> Specifically for docker it is a very common gotcha that the container runtime can and will bypass firewall rules and open ports anyway. Depending on your configuration, those firewall rules in OP may not actually do anything to prevent docker from opening incoming ports.

This sounds like great news. I followed some of the open issues about this on GitHub and it never really got a satisfactory fix. I found some previous threads on this "StrictForwardPorts": https://news.ycombinator.com/item?id=42603136.

marwamc•1h ago
Hahaha OP could be in deep trouble depending on what types of creds/data they had in that container. I had replied to a child comment but I figure best to reply to OP.

From the root container, depending on volume mounts and capabilities granted to the container, they would enumerate the host directories and find the names of common scripts and then overwrite one such script. Or to be even sneakier, they can append their malicious code to an existing script in the host filesystem. Now each time you run your script, their code piggybacks.

OTOH if I had written such a script for linux I'd be looking to grab the contents of $(hist) $(env) $(cat /etc/{group,passwd})... then enumerate /usr/bin/ /usr/local/bin/ and the XDG_{CACHE,CONFIG} dirs - some plaintext credentials are usually here. The $HOME/.{aws,docker,claude,ssh} Basically the attacker just needs to know their way around your OS. The script enumerating these directories is the 0777 script they were able to write from inside the root access container.

jakelsaunders94•49m ago
Nothing in that container luckily, just what Umami needed to run, so no creds at all. Thanks for the info though!
hoppp•59m ago
This nextjs vulnerability is gonna be exploited everywhere because its so easy. This is just the start
exceptione•46m ago
The first step I would take is running podman instead of Docker to prevent container escapes. Podman can be run truly rootless and doesn't mess with your firewall. Next I would drop all caps if possible.
doodlesdev•43m ago
What's the difference between running Podman and running Docker in rootless mode? (Other than Docker messing with the firewall, which apparently OP doesn't know about… yet). I understand Podman doesn't require a daemon, but is that all there is to it, or is there something I'm missing?
exceptione•26m ago
The runtime has been designed from the ground up to be run daemonless and rootless. They also have a K8s runtime, that has an extremely small surface, just enough to be K8s compliant.

But podman has also great integration with systemd. With that you could use a socket activated systemd unit, and stick the socket inside the container, instead of giving the container any network at all. And even if you want networking in the container, the podman folks developed slirp4netns, which is user space networking, and now something even better: passt/pasta.

hughw•14m ago
You can run Docker Scout on one repo for free, and that would alert you that something was using Next.js and had that CVE. AWS ECR has pretty affordable scanning too: 9 cents/image and 1 cent/rescan. Continuous scanning even for these home projects might be worth it.

[*] https://aws.amazon.com/inspector/pricing/

xp84•11m ago
I wonder in a case like this how hard it would be to "steal" the crypto that you've paid to mine. But I assume these people are probably smart enough to where everything is instantly forwarded to their C&C server to prevent that.

Gemini 3 Flash: Frontier intelligence built for speed

https://blog.google/products/gemini/gemini-3-flash/
717•meetpateltech•7h ago•364 comments

Developers can now submit apps to ChatGPT

https://openai.com/index/developers-can-now-submit-apps-to-chatgpt/
44•tananaev•2h ago•35 comments

Coursera to combine with Udemy

https://investor.coursera.com/news/news-details/2025/Coursera-to-Combine-with-Udemy-to-Empower-th...
400•throwaway019254•11h ago•227 comments

Inside PostHog: SSRF, ClickHouse SQL Escape and Default Postgres Creds to RCE

https://mdisec.com/inside-posthog-how-ssrf-a-clickhouse-sql-escaping-0day-and-default-postgresql-...
65•arwt•3h ago•15 comments

OBS Studio Gets a New Renderer

https://obsproject.com/blog/obs-studio-gets-a-new-renderer
53•aizk•3h ago•16 comments

I got hacked: My Hetzner server started mining Monero

https://blog.jakesaunders.dev/my-server-started-mining-monero-this-morning/
145•jakelsaunders94•3h ago•135 comments

AWS CEO says replacing junior devs with AI is 'one of the dumbest ideas'

https://www.finalroundai.com/blog/aws-ceo-ai-cannot-replace-junior-developers
700•birdculture•7h ago•393 comments

Show HN: High-Performance Wavelet Matrix for Python, Implemented in Rust

https://pypi.org/project/wavelet-matrix/
53•math-hiyoko•4h ago•0 comments

A Safer Container Ecosystem with Docker: Free Docker Hardened Images

https://www.docker.com/blog/docker-hardened-images-for-every-developer/
265•anttiharju•7h ago•55 comments

Cloudflare Radar 2025 Year in Review

https://radar.cloudflare.com/year-in-review/2025
38•ksec•2h ago•13 comments

Tell HN: HN was down

453•uyzstvqs•7h ago•271 comments

Fast Sequence Iteration in Common Lisp

https://world-playground-deceit.net/blog/2025/12/fast-sequence-iteration-in-common-lisp.html
24•BoingBoomTschak•4d ago•4 comments

How SQLite is tested

https://sqlite.org/testing.html
211•whatisabcdefgh•6h ago•51 comments

Zmij: Faster floating point double-to-string conversion

https://vitaut.net/posts/2025/faster-dtoa/
81•fanf2•3d ago•8 comments

The Number That Turned Sideways

https://zuriby.github.io/math.github.io/the-number-that-turned-sideways.html
9•tzury•4d ago•3 comments

Launch HN: Kenobi (YC W22) – Personalize your website for every visitor

26•sarreph•7h ago•49 comments

Venezuela's Navy Begins Escorting Ships as U.S. Threatens Blockade

https://www.nytimes.com/live/2025/12/17/us/trump-news
32•belter•1h ago•7 comments

Speed matters: Why working quickly is more important than it seems

https://jsomers.net/blog/speed-matters
24•bschne•2d ago•13 comments

Pornhub extorted after hackers steal Premium member activity data

https://www.bleepingcomputer.com/news/security/pornhub-extorted-after-hackers-steal-premium-membe...
80•coloneltcb•4h ago•27 comments

Flick (YC F25) Is Hiring Founding Engineer to Build Figma for AI Filmmaking

https://www.ycombinator.com/companies/flick/jobs/Tdu6FH6-founding-frontend-engineer
1•rayruiwang•7h ago

I couldn't find a logging library that worked for my library, so I made one

https://hackers.pub/@hongminhee/2025/logtape-fedify-case-study
24•todsacerdoti•5d ago•30 comments

No AI* Here – A Response to Mozilla's Next Chapter

https://www.waterfox.com/blog/no-ai-here-response-to-mozilla/
525•MrAlex94•1d ago•292 comments

The State of AI Coding Report 2025

https://www.greptile.com/state-of-ai-coding-2025
69•dakshgupta•7h ago•74 comments

Learning Fortran (2024)

https://uncenter.dev/posts/learning-fortran/
54•lioeters•11h ago•47 comments

I created a publishing system for step-by-step coding guides in Typst

https://press.knowledge.dev/p/new-150-pages-rust-guide-create-a
27•deniskolodin•4d ago•7 comments

Show HN: GitForms – Zero-cost contact forms using GitHub Issues as database

https://gitforms-landing.vercel.app/
15•lgreco•5h ago•7 comments

VRChat: “There are more Japanese creators than all other countries combined”

https://twitter.com/chyadosensei/status/2001356290531156159
65•numpad0•3h ago•42 comments

AI Isn't Just Spying on You. It's Tricking You into Spending More

https://newrepublic.com/article/204525/artificial-intelligence-consumers-data-dynamic-pricing
69•c420•3h ago•44 comments

Thin desires are eating life

https://www.joanwestenberg.com/thin-desires-are-eating-your-life/
745•mitchbob•1d ago•242 comments

Is Mozilla trying hard to kill itself?

https://infosec.press/brunomiguel/is-mozilla-trying-hard-to-kill-itself
804•pabs3•14h ago•720 comments