Good lord!
From that you can infer that there shouldn't be the need for liquid cooling in terms of getting the heat off the chip.
There still are overall system power dissipation problems, which might lead you to want to use liquid cooling, but not necessarily.
For example, Super Micro will sell you air cooled 1U servers that options up to 400W CPU options (https://www.supermicro.com/en/products/system/hyper/1u/as%20...)
PS. Having many cores doesn’t mean a lot more power. Multi core performance can be made very efficient by having many cores running at lower clock rate.
Blackwell 100+200 compression spin lock documentation.
i really wish the article would have spent 2 sec to write in parenthesis what 'ccd' is (its 'Core Complex Die' fyi)
If their goal was to appeal to more casual readers, then I agree.
Core Complex Die - an AMD term for a chiplet that contains the CPU cores and cache. It connects to an IOD (I/O die) that does memory, PCIe etc (≈southbridge?).
Aside: CCX is Core Complex - see Figure 1 of https://www.amd.com/content/dam/amd/en/documents/products/ep...
For any other older fogeys that CCD means something different.
northbridge
Its a switch that has a bunch of unified PHYs that can do many different tasks (non-primary PCI-E lanes, SATA ports, USB ports, etc), leveraging shared hardware to reduce silicon footprint while increasing utility, and connects to PCI-E lanes on the CPU.
The "northbridge" in modern Zen systems is the IO die, and in Zen 1/+, its the tiny fractional IO die that was colocated on each chip (which means a Zen 1/+ Epyc had the equivalent of 4 tiny northbridges).
However, they just embed the equivalent design of the chipsets into the IO Die SoC on Epycs.
Fun fact: For desktop, since Zen 1 (and AM4-compatible non-Zen CPUs) they included a micro-southbridge into the IO die. It gave you 2 SATA ports and 4 USB ports, usually the only "good" ones on the board. On Epyc, they just put the full sized one here instead of pairing it with an external one.
This also means, for example, if you have 4 USB3 10gbit ports, and its not handled by a third party add-on chip? Those are wired directly into the CPU, and aren't competing for the x4 that feeds the southbridge.
Also fun fact: The X, B, and A chips are all sibling designs, under the name of Promontory, made jointly with ASMedia. They're essentially all identical, only updated for PCI-E and USB versions as time went on, as well as adding more ports and shrinking die size.
The exception is the X570, its an AMD-produced variant of the Promontory that also contains the Zen 2/3 IO Die, as they're actually the same chip in this case. The chips that failed to become IO Dies had all their Promontory features enabled instead, and became chipset chips. The Zen 2/3 Epycs shipped their IO die, at least partly, as two X570s welded together, with even more DDR PHY thrown in, as some sort of cost saving.
I don't think that panned out, because the X/B/A 600 and 800 variants (Zen 4 and 5) went back to being straight Promontory again.
Wikipedia has some good charts for this: https://en.wikipedia.org/wiki/List_of_AMD_chipsets
Basically we are about to reach the scale where a single rack of these is a whole datacenter from the nineties or something like that
Or are they still building chips no one wants to use because cuda is the only thing that doesn’t suck balls
Even the slowest of All programming languages or framework with 1 request per second per vCPU, that is 2K Request per second.
Pure brute force hardware scaling.
mrlonglong•1d ago
jauntywundrkind•1d ago
It's a smaller denser core but still incredibly incredibly promising and so so neat.
jsheard•1d ago
zeusk•19h ago
iirc, in the 2016 a quadcore intel cpu ran the original crysis at ~15fps
mrlonglong•1d ago
CyberDildonics•1d ago
bri3d•1d ago
https://www.techpowerup.com/forums/threads/amd-zen-6-epyc-ve...
EDIT: actually, now that I think about it some more, my characterization of Zen-C cores as the same "idea" as Intel E-cores was pretty unfair too; they do serve the same market idea but the implementation is so much less silly that it's a bit daft to compare them. Intel E-Cores have different IPC, different tuning characteristics, and different feature support (ie, they are usually a different uarch) which makes them really annoying to deal with. Zen C cores are usually the same cores with less cache and sometimes fewer or narrower ports depending on the specific configuration.
eigenform•1d ago
eigenspace•1d ago
Fully agreed, they may be targetting a similar goal, but the execution is so different, and a Intel screwed up the idea so bad that it can really mislead people into assuming that dense Zen cores are the same junk as a Intel E-cores.
hypercube33•8h ago
AMD's approach was to basically trim the fat on Zen as far as they possibly could but keep the core fast and efficient and you end up with the C-Cores.
In practice (where I generally live with expertise) AMD's approach lets you move software between cores and it generally doesn't care or know, whereas Intel's method applications definetly do care and can have issues.
To me, the difference is moving a virtual machine between two similar CPUs and not having to reboot it (AMDs approach) and having to reboot for one reason or another (Compatibility) -- Intels method.
bri3d•7h ago
I've had the same experiences as you with Intel mixed-core desktop parts. They're incredibly difficult to optimize for due to the heterogeneous core mixture, whereas AMD mobile parts are generally more reasonable (you're on a slow core or a fast core, basically), and AMD never made a mixed-core desktop part.
However, Intel server parts several years ago switched to E-core only or P-core only, so all of the heterogeneous core mixture issues aren't a thing - you basically have two separate processor generations being sold at once, which isn't particularly surprising or uncommon.
With AMD server processor families (linked in my comment), depending on the part's density you get either "slow" or "fast" cores and either "wide" or "narrow" units, so you do still have to think about things a little bit there too.
Where Intel really screwed up in general, microarchitecture differences aside, is AVX512. That's the wrench that prevents the same compiled code from running across most Intel parts - they just couldn't decide what they wanted to do with it, whereas AMD just chose to support it and stick with it, even though the throughput for the wide instructions is wildly different between processors.
kijiki•1h ago
From the OS side, the change to support it is pretty simple. On the first #UD trap caused by an AVX512 instruction being missing, pin the process to just the P-cores and end the process's timeslice.
tester756•1d ago
eigenspace•1d ago
AMD's dense-cores are the same microarchitecture, have the same IPC, use all the same instruction sets. The only real difference between them and regular AMD cores is that their dense cores have less cache, and lower peak clocks.
tester756•23h ago
Do they?
I thought it caused very significant problems (when there's switch between E and P core) and they avoided it
But I cannot find anything about it
wmf•23h ago
eigenspace•22h ago
Yes?
adrian_b•10h ago
The Intel server CPUs with E-cores, both the current Sierra Forest CPUs with Crestmont cores and the future Clearwater Forest CPUs with Darkmont cores do not support AVX-512 and they are almost identical with the E-cores from Intel laptop/desktop CPUs.
Therefore, for demanding applications you cannot run the same programs on Intel servers with P-cores or E-cores, unless they use dynamical dispatch to select at run-time between AVX and AVX-512 libraries, as the gain from AVX-512 can be very substantial and on server applications not using it would lose money by lowering the throughput.
The Intel Darkmont cores of Panther Lake and Clearwater Forest are almost identical with the Skymont cores of Arrow Lake and Lunar Lake (the main difference is that the Skymont cores are made by TSMC, while the Darkmont cores are made by Intel in their new 18A CMOS process) and they are extremely similar in die size and in performance with the ARM Neoverse V3 cores from the newly launched AWS Graviton5 (which are known as Cortex-X4 in their smartphone variant).
Intel has said that they will eliminate this ISA difference between E-cores and P-cores, but a couple of years might pass until this will reach their server CPUs.
zozbot234•21h ago
saagarjha•16h ago
eigenspace•3h ago
That said, manually disabling AVX-512 on P-cores just so I can have E-cores is still a *bad* tradeoff as far as I'm concerned, but I get that my use-cases aren't everyone's use-cases.
bee_rider•19h ago
wmf•19h ago
fc417fc802•13h ago
I wonder how many of these you could cram into 1U? And what the maximum next gen kW/U figure looks like.
epistasis•1d ago
mort96•1d ago
Neywiny•1d ago
epistasis•5h ago
Here's analysis of a prior LTT video showing 1/3 of cores at 100%, 1/3 of cores at 50%, and 1/3 idle cores:
https://www.youtube.com/watch?v=XqSCRZJl7S0
In any case, CS2 can take advantage of far more cores than most games.
markhahn•1d ago
rbanffy•1d ago
Things like games, desktops, browsers, and such were designed for computers with a handful of cores, but the core count will only go up on these devices - a very pedestrian desktop these days has more than 8 cores.
If you want to make software that’ll run well enough 10 years from now, you’d better start using computers from 10 years from now. A 256 core chip might be just that.
markhahn•17h ago
the standard consumer computer of today has only a few cores that race-to-sleep, because there simply isn't that much to do. where do you imagine the parallel work will come from? even for games, will work shift off the GPU onto the host processor? seems unlikely.
future-proofing isn't about inflating your use of threads, but being smart about memory and IO. those have been the bottleneck for decades now.
rbanffy•8h ago
Because the cores will be there, regardless. At some point, machines will be able to do a lot of background activity, learn about what we are doing, so that local agentic models can act as better intelligent assistants. I don't know what will be the killer app for the kilocore desktop - nobody knows that, but when PARC made a workstation with bit-mapped graphics out of a semi custom built minicomputer that could easily serve a department of text terminals we got things like GUIs, WYSYWIG, Smalltalk, and a lot of other fancy things nobody imagined back then.
You can try to invent the future using current tech, or you might just try to see what's possible with tomorrow's tools and observe it first hand.
p12tic•1d ago
They scale very well for web and db servers as well. You just put lots of containers/VMs on a single server.
AMD EPYC has a separate architecture specifically for such workloads. It's a bit weaker, runs at lower frequency and power and takes less silicon area. This way AMD can put more such cores on a single CPU (192 vs 128 for Zen 5c vs 5). So it's the other way round - web servers love high core count CPUs.
markhahn•17h ago
tucnak•15h ago
Neywiny•1d ago
lifetimerubyist•22h ago
Neywiny•1d ago
bee_rider•19h ago
wmf•19h ago
markhahn•18h ago
bee_rider•17h ago
* It just seems like a lot of threads to wrangle without some hierarchy. Nested OpenMP is also possible…
* I’m wondering if explicit communication is better from one die to another in this sort of system.
fc417fc802•13h ago
The above doesn't even consider the possibility of multi-CPU systems. I suspect the existing programming models are quickly going to become insufficient for modeling these systems.
I also find myself wondering how atomic instruction performance will fare on these. GPU ISA and memory model on CPU when?
DiabloD3•11h ago
Before 8 cores per die (introduced in Zen 3, and retained in 4, 5 and 6), the Zen 1/+ and 2 series this would have been two sets of four cores instead of one set of eight (and a split L3 instead of a unified one). I can't remember if the split-CCX had its own NUMA layer in the tree or not, or if they were just iterated in pairs.
fc417fc802•11h ago
I realize that HPC code can be customized to the specific device it will be run on but more widely deployed software is going to want to abstract these increasingly complex relationships.
DiabloD3•11h ago
Ironically, a lot of people keep shooting themselves in the foot and blindly using MPI or OpenMP or any of the other popular industry supported frameworks, and then thinking that magically bails them out. It doesn't.
The most important thing you need, above all others: make sure the problem you're solving can be parallelized, and CPUs are the right way of doing it. Once you've answered this question, and the answer is yes, you can just write it pretty much normally.
Also, ironically, you can write Java that isn't shit and takes advantage of systems like these. Sun and the post-Sun community put a lot of work into the Hotspot JVM to make it scale alarmingly well on high core count machines. Java used correctly is performant.
bee_rider•11h ago
https://chipsandcheese.com/p/genoa-x-server-v-cache-round-2
fweimer•5h ago
Even today, I think very large enterprise systems (where a single kernel runs on a single system that spans multiple racks) are built like this, too.
m4rtink•1d ago
jsheard•1d ago
fooblaster•1d ago
jsheard•23h ago
fooblaster•23h ago
markhahn•18h ago
NVidia's use of "cores" is simply wrong. unless you think a core is a simple scalar ALU. but cores haven't been like that for decades.
or would you like to count cores in a current AMD or Intel CPU? each "core" has half a dozen ALUs/FP pipes, and don't forget to multiply by SIMD width.
phkahler•9h ago