Jon Blow’s Jai language famously added a feature to references that allowed you to easily experiment with moving data members between hot/cold arrays of structs.
https://halide-lang.org/ tackles a related problem. It decouples the math to be done from the access order so as to allow you to rapidly test looping over data in complicated ways to achieve cache-friendly access patterns for your specific hardware target without rewriting your whole core loop every time.
Halide is primarily an about image processing convolution kernels. I’m not sure how general purpose it can get.
The week after I first saw a Bloom Filter I imagined lots of places this would surely be great, in the decades since I probably have one bona fide use for a Bloom Filter per year, maybe less.
That said Java might be able to naturally decompose record types into SoA collections without a big lift. Same for C# struct.
You might even be able to access views types that still code like objects but point to the backing fields in the SoA collection.
One early result was the idea of storage combinators[3], and with those, AoS/SoA pretty much just falls out.
Storage Combinators are basically an interface for data. Any kind of data, so generalized a bit from the variations of "instances of a class" that you get in CLOS's MOP.
If you code against that interface, it doesn't matter how your data is actually stored: objects, dictionaries, computed on-demand, environment variables, files, http servers, whatever. And the "combinator" part means that these can be stacked/combined.
While you can do this using a library, and a lot is in fact just implemented in libraries, you need the language support so that things like objects/structs/variables can be accessed quickly using this mechanism.
SoA storage for tables has been on the list of things to do for a long time. I just started to work on some table ideas, so might actually get around to it Real Soon Now™.
Currently I am working on other aspects of the table abstraction, so for example just integrated query, so I ca write the following:
invoices[{Amount>30}]
[2] https://dl.acm.org/doi/10.1145/3689492.3690052
[3] https://2019.splashcon.org/details/splash-2019-Onward-papers...
define_aggregate(^^Pointers,
nsdms(^^T)
| std::views::transform([](std::meta::info member){
return data_member_spec(add_pointer(type_of(member)),
{.name = identifier_of(member)});
}));
† WG21 of SC22 of JTC1 between ISO and the IEC, "the C++ committee".
See P2996R11 for the latest draft of the work - https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p29...
At the same time, I don't completely understand how such a pivot would result in a speedup for random access. I suppose it would speed up sequential access, since the multiple array storage scheme might force many more cache line updates.
The explanation drifts into thinking of 2D byte arrays as 3D bit matrices, but in the end it was a 20-30x improvement in speed and binary size.
I was honestly surprised that C++ doesn't have anything built in for this, but at least it's trivial to write your own array type
It reminded me of a Haxe macro used by the Dune: Spice Wars devs to transform an AoS into a SoA at compile time to increase performance: https://youtu.be/pZcKyqLcjzc?t=941
The end result is quite cool, though those compile time type generation macros always look too magical to me. Makes me wonder if just getting values using an index wouldn't end up being more readable.
template<template<class> class G>
struct Point {
G<int> x;
G<int> y;
auto get() { return std::tie(x,y); }
};
template<template<template<class> class> class C>
struct SOA {
template<class T> using Id = T;
template<class T> using Ref = T&;
C<std::vector> vs;
void push_back(C<Id> x) {
std::apply([&] (auto&&... r) {
std::apply([&](auto&&... v){ ( (r.push_back(v)),...); }, x.get());
}, vs.get());
}
C<Ref> operator[](size_t i) {
return std::apply([&] (auto&&... r) { return C<Ref>{ r[i]...}; }, vs.get());
}
};
int main() {
SOA<Point> soa_point;
soa_point.push_back({1,2});
auto [x,y] = soa_point[0];
}The workflow is, I set up Vec3x8, and Quaternionx8, which wrap a simd instrinsic for each field (x: f32x8, y: f32x8...) etc.
For the GPU and general flattening, I just pack the args as Vecs, then the fn signature contains them like eps: &[f32], sigma: &[f32] etc. So, I'm having trouble mapping this SoA approach to the abstractions used in the article. Then the (C++-like CUDA) kernel sees these as *float3 params etc. It also feels like a complexity reverse of the Rust/Zig stereotypes...
Examples:
struct Vec3x8 {
x: f32x8,
y: f32x8,
z: f32x8
} // appropriate operator overloads...
struct Setup {
eps: Vec<f32>,
sigma: Vec<f32>,
}
I do like how directly accessing the fields individually (the whole reason you would do this) is a hypothetical presented as an after thought. Enjoyably absurd.
TheHideout•4h ago
throwawaymaths•4h ago
dgb23•3h ago
You can think about it as being a composition of fields, which are individually stored in their respective array.
(Slightly beside the point: Often they are also stored in pairs or larger, for example coordinates, slices and so on are almost always operated on in the same function or block.)
The benefit comes from execution. If you have functions that iterate over these structs, they only need to load the arrays that contain the fields they need.
Zig does some magic here based on comptime to make this happen automatically.
An ECS does something similar at a fundamental level. But usually there's a whole bunch of additional stuff on top, such as mapping ids to indices, registering individual components on the fly, selecting a components for entities and so on. So it can be a lot more complicated than what is presented here and more stuff happens at runtime. It is also a bit of a one size fits all kind of deal.
The article recommends watching Andrew Kelley's Talk on DoD, which inspired the post. I agree wholeheartedly, it's a very fun and interesting one.
One of the key takeaways for me is that he didn't just slap on a design pattern (like ECS), but went to each piece individually, thought about memory layout, execution, trade offs in storing information versus recalculating, doing measurements and back of the envelope calculations etc.
So the end result is a conglomerate of cleverly applied principles and learnings.
jayd16•2h ago