frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Helldivers 2 devs slash install size from 154GB to 23GB

https://www.tomshardware.com/video-games/pc-gaming/helldivers-2-install-size-slashed-from-154gb-to-just-23gb-85-percent-reduction-accomplished-by-de-duplicating-game-data-an-optimization-for-older-mechanical-hard-drives
96•doener•1h ago

Comments

easyThrowaway•1h ago
Did the duplicated files were even used on pc? Like, do you even have such low access to the file system that you can deduce which duplicated instance has a faster access time on a mechanical hard drive?
arghwhat•1h ago
Not sure if this is what they did, but you can just put all the things you need together sequentially into a single file and rely on the filesystem to allocate contiguous blocks where possible (using the appropriate size hints to help). It's trivial unpack at loading time without any performance impact.

A filesystem is by itself just one big "file" acting like a file archive.

tehbeard•50m ago
It's not which duplicated instance....

Think of it as I have two packs for levels.

Creek.level and roboplanet.level

Both use the cyborg enemies, by duplicating the cyborg enemy model and texture data across both files, Only the level file needs to be opened to get all nessecary data for a match.

Because modern OS will allow you to preallocate contiguous segments and have auto defrag, you can have it read this level file at max speed, rather than having to stop and seek to go find cyborg.model file because it was referenced by the spawn pool. Engine limitations may prevent other optimisations you think up as a thought exercise after reading this.

It's similar to how crash bandicoot packed their level data to handle the slow speed of the ps1 disc drive.

As to why they had a HDD optimisation in 2024... Shrugs

rwmj•1h ago
23GB is supposed to be "slim"?!
bilekas•1h ago
In this day and age it's a gift to only be ~23GB.. I'm reminded of the old days when you literally didn't have the space so had to get creative, now any kind of space optimization isn't even considered.
onli•1h ago
Yes. High resolution textures take up a lot of space. Have a look at HD texture mods for skyrim for example. 23GB is more in line with a game from a few years ago, so this really is slim for a modern game with modern graphics.
mfro•59m ago
Have you played a big budget video game released in the last 10 years? It’s pretty standard to reach upwards of 60GB.
jwagenet•16m ago
GTAV had a 60GB install size over a decade ago.
phoronixrly•56m ago
I do love rich soundtracks with high quality compression, and textures that look crisp on 4k. And also games with 100+ hours of single-player campaign.
throw0101c•52m ago
Back in the day:

> 3-D Hardware Accelerator (with 16MB VRAM with full OpenGL® support; Pentium® II 400 Mhz processor or Athlon® processor; English version of Windows® 2000/XP Operating System; 128 MB RAM; 16-bit high color video mode; 800 MB of uncompressed hard disk space for game files (Minimum Install), plus 300 MB for the Windows swap file […]

* https://store.steampowered.com/app/9010/Return_to_Castle_Wol...

* https://en.wikipedia.org/wiki/Return_to_Castle_Wolfenstein

Even older games would be even smaller:

* https://www.oldgames.sk/en/game/ultima-vi-the-false-prophet/...

* https://en.wikipedia.org/wiki/Ultima_VI:_The_False_Prophet

filleduchaos•39m ago
I for one simply cannot believe that a game with 4K+ textures and high poly count models is bigger than a game that uses billboard sprites which aren't even HD. Whatever could be the reason? A complete mystery...
alias_neo•52m ago
I mean yes, it's a very nice looking game with fairly sizeable worlds and lots of different enemies, biomes, etc.

It's currently over 100GB because of duplicated assets, so this is a game-changer (pun intended).

EDIT: Just checked; 157GB on my SSD.

EDIT2: 26GB after updating with a 9.7GB download, 5.18GB of shaders before and after.

mghackerlady•16m ago
It can fit on a standard blu ray, so I'm inclined to say so
rincebrain•1h ago
I've been really curious precisely what changed, and what sort of optimization might have been involved here.

Because offhand, I know you could do things like cute optimizations of redundant data to minimize seek time on optical media, but with HDDs, you get no promises about layout to optimize around...

The only thing I can think of is if it was literally something as inane as checking the "store deduplicated by hash" option in the build, on a tree with copies of assets scattered everywhere, and it was just nobody had ever checked if the fear around the option was based on outcomes.

(I know they said in the original blog post that it was based around fears of client performance impact, but the whole reason I'm staring at that is that if it's just a deduplication table at storage time, the client shouldn't...care? It's not writing to the game data archives, it's just looking stuff up either way...)

alias_neo•53m ago
I'm not entirely clear what you're trying to say, but, my understanding is that they simply put lots of copies of files in lots of places like games have done for a long time, in the hopes it would lower seek times on HDDs for those players who use them.

They realised, after a lot of players asking, that it wasn't necessary, and probably had less of an impact than they thought.

They removed the duplicates, and drastically cut the install size. I updated last night, and the update alone was larger than the entire game after this deduplication run, so I'll be opting in to the Beta ASAP.

It's been almost a decade since I ran spinning rust in a desktop, and while I admire their efforts to support shitty hardware, who's playing this on a machine good enough to play but can't afford £60 for a basic SSD for their game storage?

eska•40m ago
HDDs also have a spinning medium and a read head , so the optimization is similar to optical media like CDs.

Let’s say you have UI textures that you always need, common player models and textures, the battle music, but world geometry and monsters change per stage. Create an archive file (pak, wad, …) for each stage, duplicating UI, player and battle music assets into each archive. This makes it so that you fully utilize HDD pages (some small config file won’t fill 4kb filesystem pages or even the smaller disk sectors). All the data of one stage will be read into disk cache in one fell swoop as well.

On optical media like CDs one would even put some data closer to the middle or on the outer edge of the disc because the reading speed is different due to the linear velocity.

This is an optimization for bandwidth at the cost of size (which often wasn’t a problem because the medium wasn’t filled anyway)

swiftcoder•22m ago
> HDDs also have a spinning medium and a read head , so the optimization is similar to optical media like CDs.

HDDs also have to real with fragmentation, I wonder what the odds that you get to write 150 GBs (and then regular updates in the 30GB range) without breaking it into fragments...

djmips•1h ago
I did similar work on a game a long time ago and it took over a month to slim it down to 1/4 of the size but in this case 'at runtime' - the producer wasn't impressed. It looked exactly the same. I wonder if they had any pushback.
geerlingguy•51m ago
Possibly a similar process to when you go into an AWS account, and find dozens of orphaned VMs, a few thousand orphaned disk volumes, etc., saving like $10k/month just deleting unused resources.
alias_neo•48m ago
We've all been there Jeff.

In this case, I don't think it was forgetfulness; unlike us, they have an excuse and they were trying to optimise for disk seek times.

Anyway, I've got a half-dozen cloud accounts I need to go check for unused resources waves.

snet0•50m ago
> With their latest data measurements specific to the game, the developers have confirmed the small number of players (11% last week) using mechanical hard drives will witness mission load times increase by only a few seconds in worst cases. Additionally, the post reads, “the majority of the loading time in Helldivers 2 is due to level-generation rather than asset loading. This level generation happens in parallel with loading assets from the disk and so is the main determining factor of the loading time.”

It seems bizarre to me that they'd have accepted such a high cost (150GB+ installation size!) without entirely verifying that it was necessary!

I expect it's a story that'll never get told in enough detail to satisfy curiosity, but it certainly seems strange from the outside for this optimisation to be both possible and acceptable.

jsheard•45m ago
From a business perspective the disk footprint is only a high cost if it results in fewer sales, which I doubt it does to any significant degree. It is wasteful, but can see why optimization efforts would get focused elsewhere.
Ekaros•43m ago
Also the cost is often offloaded to the "greedy" Valve... So there is less pressure to optimize their own CDN use.
jsheard•27m ago
Yeah, I don't think any of the stores charge developers in proportion to how much bandwidth they use. If that changed then the priorities could shift pretty quickly.

Publishers do have to care somewhat on the Switch since Nintendo does charge them more for higher capacity physical carts, but a lot of the time they just sidestep that by only putting part (or none) of the game on the cart and requiring the player to download the rest.

code_for_monkey•42m ago
I think certain games dont even bother to optimize the install size so that you cant fit other games on the hard drive, I think COD games are regularly hundreds of gigs
snet0•38m ago
I've often seen people mention that one reason for games like Call of Duty being so enormous is optimising for performance over storage. You'd rather decompress textures/audio files at install-time rather than during run-time, because you download/install so infrequently.
KeplerBoy•35m ago
Having a humongous game might be a competitive advantage in the era of live-service games.

Users might be more hesitant to switch to another game if it means uninstalling yours and reinstalling is a big pain in the backside due to long download times.

bombcar•25m ago
Given how many Steam games are bought but never even installed, it would seem not terribly worth optimizing for.

On phone, I bet you see some more effort.

georgeecollins•5m ago
Both things are sort of true. Its not sales where size can hurt you but retention, which is why it tended to matter more on phones. When you need space on your device the apps are listed from largest to smallest.

On both phones and PCs storage has just grown so its less of an issue. The one thing I have noticed is that Apple does its price windowing around memory so you pay an absurd amount for an extra 128 gb. The ultra competitive Chinese phone market crams high end phones with a ton of memory and battery. Si some popular Chinese phone games are huge compared to ones made for the iPhone.

PoignardAzur•43m ago
I don't find it surprising at all. A ton of developers do optimizations based on vibes and very rarely check if they're actually getting a real benefit from it.
bombcar•26m ago
This is the moral behind "premature optimization is the root of all evil" - you could say preconceived just as easily.
embedding-shape•8m ago
> you could say preconceived just as easily

Would have saved us from all the people who reject any sort of optimization work because for them it is always "too early" since some product team wanted their new feature in production yesterday, and users waiting 5 seconds for a page load isn't considered bad enough just yet.

nerdjon•37m ago
High cost to who though. We see the same thing when it comes to RAM and CPU usage, the developer is not the one paying for the hardware and many gamers have shown that they will spend money on hardware to play a game they want.

Sure they may loose some sales but I have never seen many numbers on how much it really impacted sales.

Also on the disk side, I can't say I have ever looked at how much space is required for a game before buying it. If I need to clear out some stuff I will. Especially with it not being uncommon for a game to be in the 100gb realm already.

That all being said, I am actually surprised by the 11% using mechanical hard drives. I figured that NVME would be a lower percentage and many are using SSD's... but I figured the percent with machines capable of running modern games in the first place with mechanical drives would be far lower.

I do wonder how long it will be until we see games just saying they are not compatible with mechanical drives.

onli•25m ago
That already happened :) Starfield claimed to not support HDDs and really ran bad with them. And I think I saw SSDs as requirement for a few other games now, in the requirement listings on steam.
embedding-shape•6m ago
> Starfield claimed to not support HDDs and really ran bad with them.

To be fair, at launch Starfield had pretty shit loading times even with blazing fast SSDs, and the game has a lot of loading screens, so makes sense they'll nip that one in the bud and just say it's unsupported with the slower type of disks.

literallywho•16m ago
Latest Ratchet and Clank game relies heavily on ps5’s nvme drive. Its PC port states that SSD is required. And IIRC, the experience on mechanical drives is quite terrible to the unplayable level.
aeve890•29m ago
>It seems bizarre to me that they'd have accepted such a high cost (150GB+ installation size!) without entirely verifying that it was necessary!

You should look at COD install sizes and almost weekly ridiculously huge "updates". 150gb for a first install is almost generous considering most AAA games.

deng•29m ago
11% still play HD2 with a spinning drive? I would've never guessed that. There's probably some vicious circle thing going on: because the install size is so big, people need to install it on their secondary, spinning drive...
amlib•19m ago
Even though I have two SSDs in my main machine I still use a hard drive as an overflow for games that I judge are not SSD worthy.

Because it's a recent 20TB HDD the read speeds approach 250MB/s and I've also specifically partitioned it at the beginning of the disk just for games so that it can sustain full transfer speeds without files falling into the slower tracks, the rest of the disk is then partitioned for media files that won't care much for the speed loss. It's honestly fine for the vast majority of games.

deng•8m ago
> It's honestly fine for the vast majority of games.

Yes, because they apparently still duplicate data so that the terrible IOPS of spinning disks does not factor as much. You people need to stop with this so that we can all have smaller games again! ;-)

afavour•12m ago
> It seems bizarre to me that they'd have accepted such a high cost

They’re not the ones bearing the cost. Customers are. And I’d wager very few check the hard disk requirements for a game before buying it. So the effect on their bottom line is negligible while the dev effort to fix it has a cost… so it remains unfixed until someone with pride in their work finally carves out the time to do it.

jeffwask•6m ago
Game companies these days barely optimize engine graphical performance before release never mind the package size or patching speed. They just stamp higher minimum system requirements on the package.
fleabitdev•35m ago
Back of the envelope, in the two years since the game was released, this single bug has wasted at least US$10,000,000 of hardware resources. That's a conservative estimate (20% of people who own the game keep it installed, the marginal cost of wasted SSD storage in a gaming PC is US$2.50 per TB per month, the install base grew linearly over time), so the true number is probably several times higher.

In other words, the game studio externalised an eight-figure hardware cost onto their users, to avoid a five-to-six-figure engineering cost on their side.

Data duplication can't just be banned by Steam, because it's a legitimate optimisation in some cases. The only safeguard against this sort of waste is a company culture which values software quality. I'm glad the developers fixed this bug, but it should never have been released to users in the first place.

ga2mer•16m ago
>Data duplication can't just be banned by Steam

Steam compresses games as much as possible, so in the case of Helldivers 2, you had to download between ~30 and ~40 GB, which was then unpacked to 150 GB (according to SteamDB[0])

[0] https://steamdb.info/app/553850/depots/

embedding-shape•6m ago
Makes sense, initial claim was that HD2 size was mainly because of duplicated assets, and any compression worth it's salt would de-duplicate things effectively.
fleabitdev•5m ago
[delayed]
CafeRacer•30m ago
In other news - "Call of Duty installer now takes additional 131GB of space on the disk"
miohtama•24m ago
AFAIK Helldivers 2 runs some really old engine which was discontinued many years ago. Not "state of the art."

It's also a title that shows you can have a really good game without the latest tech.

Congressional lawmakers 47% pts better at picking stocks

https://www.nber.org/papers/w34524
148•mhb•1h ago•117 comments

You Can't Fool the Optimizer

https://xania.org/202512/03-more-adding-integers
102•HeliumHydride•2h ago•46 comments

A Look at Rust from 2012

https://purplesyringa.moe/blog/a-look-at-rust-from-2012/
65•todsacerdoti•1w ago•3 comments

Anthropic acquires Bun

https://bun.com/blog/bun-joins-anthropic
1983•ryanvogel•20h ago•950 comments

Helldivers 2 devs slash install size from 154GB to 23GB

https://www.tomshardware.com/video-games/pc-gaming/helldivers-2-install-size-slashed-from-154gb-t...
98•doener•1h ago•50 comments

Zig quits GitHub, says Microsoft's AI obsession has ruined the service

https://www.theregister.com/2025/12/02/zig_quits_github_microsoft_ai_obsession/
568•Brajeshwar•6h ago•302 comments

IBM CEO says there is 'no way' spending on AI data centers will pay off

https://www.businessinsider.com/ibm-ceo-big-tech-ai-capex-data-center-spending-2025-12
676•nabla9•20h ago•770 comments

Interview with RollerCoaster Tycoon's Creator, Chris Sawyer (2024)

https://medium.com/atari-club/interview-with-rollercoaster-tycoons-creator-chris-sawyer-684a0efb0f13
187•areoform•10h ago•35 comments

The "Mad Men" in 4K on HBO Max Debacle

http://fxrant.blogspot.com/2025/12/the-mad-men-in-4k-on-hbo-max-debacle.html
182•tosh•3h ago•70 comments

Mission Critical Advanced Scheduling (ALAP/ASAP) System

https://github.com/rodmena-limited/scriptplan
19•rodmena•5d ago•0 comments

Super fast aggregations in PostgreSQL 19

https://www.cybertec-postgresql.com/en/super-fast-aggregations-in-postgresql-19/
137•jnord•1w ago•14 comments

AI agents break rules under everyday pressure

https://spectrum.ieee.org/ai-agents-safety
202•pseudolus•6d ago•105 comments

Researchers Find Microbe Capable of Producing Oxygen from Martian Soil

https://scienceclock.com/microbe-that-could-turn-martian-dust-into-oxygen/
61•ashishgupta2209•8h ago•25 comments

Paged Out

https://pagedout.institute
495•varjag•18h ago•53 comments

The Writing Is on the Wall for Handwriting Recognition

https://newsletter.dancohen.org/archive/the-writing-is-on-the-wall-for-handwriting-recognition/
92•speckx•6d ago•45 comments

OpenAI declares 'code red' as Google catches up in AI race

https://www.theverge.com/news/836212/openai-code-red-chatgpt
734•goplayoutside•23h ago•816 comments

I designed and printed a custom nose guard to help my dog with DLE

https://snoutcover.com/billie-story
560•ragswag•3d ago•67 comments

Optimizations in C++ compilers: a practical journey

https://queue.acm.org/detail.cfm?id=3372264
7•fanf2•4d ago•0 comments

Quad9 DOH HTTP/1.1 Retirement, December 15, 2025

https://quad9.net/news/blog/doh-http-1-1-retirement/
82•pickledoyster•8h ago•27 comments

Trying Out C++26 Executors

https://mropert.github.io/2025/11/21/trying_out_stdexec/
37•ingve•5d ago•22 comments

India scraps order to pre-install state-run cyber safety app on smartphones

https://www.bbc.com/news/articles/clydg2re4d1o
72•wolpoli•3h ago•8 comments

Learning music with Strudel

https://terryds.notion.site/Learning-Music-with-Strudel-2ac98431b24180deb890cc7de667ea92
526•terryds•1w ago•124 comments

Qwen3-VL can scan two-hour videos and pinpoint nearly every detail

https://the-decoder.com/qwen3-vl-can-scan-two-hour-videos-and-pinpoint-nearly-every-detail/
236•thm•3d ago•70 comments

Zig's new plan for asynchronous programs

https://lwn.net/SubscriberLink/1046084/4c048ee008e1c70e/
312•messe•1d ago•224 comments

Understanding ECDSA

https://avidthinker.github.io/2025/11/28/understanding-ecdsa/
82•avidthinker•10h ago•20 comments

Counter Galois Onion: Improved encryption for Tor circuit traffic

https://blog.torproject.org/introducing-cgo/
95•wrayjustin•1w ago•30 comments

All about automotive lidar

https://mainstreetautonomy.com/blog/2025-08-29-all-about-automotive-lidar/
187•dllu•1d ago•68 comments

What, if anything, is universal to music cognition? (2024)

https://www.nature.com/articles/s41562-023-01800-9
30•Hooke•1w ago•19 comments

Amazon launches Trainium3

https://techcrunch.com/2025/12/02/amazon-releases-an-impressive-new-ai-chip-and-teases-a-nvidia-f...
191•thnaks•19h ago•67 comments

Sending DMARC reports is somewhat hazardous

https://utcc.utoronto.ca/~cks/space/blog/spam/DMARCSendingReportsProblems
59•zdw•9h ago•23 comments