frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Fire destroys Korean government's cloud storage system, no backups available

https://koreajoongangdaily.joins.com/news/2025-10-01/national/socialAffairs/NIRS-fire-destroys-governments-cloud-storage-system-no-backups-available/2412936
201•ksec•2h ago

Comments

benoau•2h ago
> However, due to the system’s large-capacity, low-performance storage structure, no external backups were maintained — meaning all data has been permanently lost.

Yikes. You'd think they would at least have one redundant copy of it all.

> erasing work files saved individually by some 750,000 civil servants

> 30 gigabytes of storage per person

That's 22,500 terabytes, about 50 Backblaze storage pods.

Or even just mirrored locally.

yongjik•1h ago
It's even worse. According to other articles [1], the total data of "G drive" was 858 TB.

It's almost farcical to calculate, but AWS S3 has pricing of about $0.023/GB/month, which means the South Korean government could have reliable multi-storage backup of the whole data at about $20k/month. Or about $900/month if they opted for "Glacier deep archive" tier ($0.00099/GB/month).

They did have backup of the data ... in the same server room that burned down [2].

[1] https://www.hankyung.com/article/2025100115651

[2] https://www.hani.co.kr/arti/area/area_general/1221873.html

(both in Korean)

paleotrope•1h ago
That's unfortunate.
poly2it•39m ago
It's incompetent really.
lukan•34m ago
No. Fortuna had nothing to do with this, this is called bad planning.
BolexNOLA•50m ago
Couldn’t even be bothered to do a basic 3-2-1! Wow
sneak•4m ago
Did you expect government IT in a hierarchical respect-your-superiors-even-when-wrong society to be competent?
mouse_•2h ago
We will learn nothing
PeterStuer•1h ago
I'm sure they had dozens of process heavy cybersecurity committees producing hundreds if not thousands of powerpoints and word documents outlining procedures and best practices over the last decade.

There is this weird divide between the certified class of non-technical consultants and actual overworked and pushed to corner cut techs.

zaphar•1h ago
Ironically many of those documents for procedures probably lived on that drive...
ksec•1h ago
I dont know why but cant stop laughing. And the great thing is that they will get paid again to write the same thing.
comprev•16m ago
You jest, but I once had a client who's IaC provisioning code was - you guessed it - stored on the very infrastructure which got destroyed.
toast0•1h ago
The data seems secure. No cyberthreat actors can access it now. Effective access control: check.
pr337h4m•1h ago
Now imagine they had a CBDC.
glitchc•44m ago
I thought most liberal governments gave up on those.
Titan2189•1h ago
Surely there must be something that's missing in translation? This feels like it simply can't be right.
mrbluecoat•1h ago
I agree. No automated fire suppression system for critical infrastructure with no backup?
fredoralive•1h ago
That may not be a perfect answer. One issue with fire suppression systems and spinning rust drives is that the pressure change etc. from the system can also ‘suppress’ the glass platters in drives as well.
layer8•20m ago
It’s accurate: https://www.chosun.com/english/national-en/2025/10/02/FPWGFS...
blueflow•1h ago
no backup no sympathy
dvh•1h ago
Technically the data is still in the cloud
pestaa•1h ago
I've been putting off a cloud to cloud migration, but apparently it can be done in hours?
zigzag312•1h ago
You can use accelerants to speed up migration
VeninVidiaVicii•1h ago
The egress cost is gonna be a doozie though!
datadrivenangel•54m ago
one of many fires to fight in such a fast scenario
zigzag312•1h ago
The cloud has materialized
anonu•1h ago
Lossy upload though
_zoltan_•1h ago
Lossy download, no?
layer8•45m ago
No information is lost: https://en.wikipedia.org/wiki/No-hiding_theorem#:~:text=info...
pjc50•47m ago
Cloud of smoke, amirite.
higginsniggins•29m ago
Unfortunately, the algorithm to unhash it is written in smoke signals
BrandoElFollito•1h ago
> all documents be stored exclusively on G-Drive

Does G-Drive mean Google Drive, or "the drive you see as G:"?

If this is Google Drive, what they had locally were just pointers (for native Google Drive docs), or synchronized documents.

If this means the letter a network disk storage system was mapped to, this is a weird way of presenting the problem (I am typing on the black keyboard and the wooden table, so that you know)

lysace•1h ago
The name G-Drive is said to be derived from the word ‘government’.
indy•37m ago
It's now derived from the word 'gone'
ncr100•22m ago
'Gone' up in smoke
prmph•1h ago
G-drive was simply the name of the storage system
gnfargbl•1h ago
https://mastodon.social/@nixCraft/113524310004145896
cs702•38m ago
Brilliant.

This deserves its own HN submission. I submitted it but it was flagged due to the title.

Thank you for sharing it on HN.

kyrra•27m ago
Copy/paste:

7 things all kids need to hear

1 I love you

2 I'm proud of you

3 I'm sorry

4 I forgive you

5 I'm listening

6 RAID is not backup. Make offsite backups. Verify backup. Find out restore time. Otherwise, you got what we call Schrödinger backup

7 You've got what it takes

zer00eyz•1h ago
This is the reason the 3, 2, 1 rule for backing up exists.
dardeaup•1h ago
They might be singing this song now. (To the tune of 'Yesterday' from the Beatles).

    Yesterday,
    All those backups seemed a waste of pay.
    Now my database has gone away.
    Oh I believe in yesterday.

    Suddenly,
    There’s not half the files there used to be,
    And there’s a deadline
    hanging over me.
    The system crashed so suddenly.

    I pushed something wrong
    What it was I could not say.
    Now my data’s gone
    and I long for yesterday-ay-ay-ay.

    Yesterday,
    The need for back-ups seemed so far away.
    Thought all my data was here to stay,
    Now I believe in yesterday.
GLdRH•27m ago
For the German enjoyers among us I recommend also this old song: https://www.youtube.com/watch?v=jN5mICXIG9M
Zacharias030•7m ago
Thanks! mmd.
cramcgrab•1h ago
Well that works out doesn’t it? Saves them from discovery.
rolph•1h ago
repeat after me:

multiple copies; multiple locations; multiple formats.

bryanhogan•56m ago
Saw a few days ago that the application site for the GKS, the most important scholarship for international students in Korea, went offline for multiple days, surprising to hear that they really lost all of the data though. Great opportunity to build a better website now?

But yeah it's a big problem in Korea right now, lots of important information just vanished, many are talking about it.

Zacharias030•17m ago
Must have been a program without much trickle down into gov tech
MangoCoffee•2m ago
It's hard to believe this happened. South Korea has tech giants like Samsung, and yet this is how the government runs? Is the US government any better?
miohtama•56m ago
I thought clouds could not burn (:
johnnienaked•53m ago
Good example of a Technology trap
m3047•46m ago
Mindblowing. Took a walk. All I can say is that if business continues "as usual" and the economy and public services continue largely unaffected then either there were local copies of critical documents, or you can fire a lot of those workers; either one of those ways the "stress test" was a success.
layer8•42m ago
“Final reports and official records submitted to the government are also stored in OnNara, so this is not a total loss”.
BurningFrog•44m ago
"The day the cloud went up in smoke"
abujazar•41m ago
LOL
Havoc•40m ago
>the G-Drive’s structure did not allow for external backups.

ah the so called schrodingers drive. It's there unless you try to copy it

aio2•36m ago
Funny, because the same thing happened in Nepal a few weeks ago. Protestors/rioters burned some government buildings, along with the tech infrastructure within them, so now almost all electronic data is gone.
ahmgeek•29m ago
nice
727564797069706•28m ago
Meanwhile, Estonia has a "data embassy" in Luxembourg: https://e-estonia.com/solutions/e-governance/data-embassy/

TL;DR: Estonia operates a Tier 4 (highest security) data center in Luxembourg with diplomatic immunity. Can actively run critical government services in real-time, not just backups.

lostmsu•27m ago
This comment is in some way more interesting than the topic of the article.
nntwozz•23m ago
The Egyptians send their condolences.
gardnr•14m ago
Has there been a more recent event, or are you referring to Alexandria?
Zacharias030•13m ago
touché
layer8•21m ago
Some more details in this article: https://www.chosun.com/english/national-en/2025/10/02/FPWGFS...
shadowgovt•17m ago
Yikes. That is a nightmare scenario.
jopsen•13m ago
> The Interior Ministry explained that while most systems at the Daejeon data center are backed up daily to separate equipment within the same center and to a physically remote backup facility, the G-Drive’s structure did not allow for external backups.

This is why I don't really want to run my own cloud :)

Actually testing the backups is boring.

That said, ones the flames are out, they might actually be able to recover some of it.

MangoCoffee•13m ago
what's the point of a storage system with no back up?
WJW•4m ago
It works fine as long as it doesn't break, and it's cheaper to buy than an equivalently sized system that does have back ups.
thepill•5m ago
Watching Mr. Robot and seeing the burned batteries the same time...