We've been thinking about it, but according to Show HN rules, blog posts are considered off-topic.

Assuming by undoing you mean splitting the cluster:

A linked list can be split in two in O(1). When it comes to updating the roots for all the removed nodes, there is no easy way out, but luckily:

- This process can be parallelized.

- It could be done just once for multiple clustering changes.

- This is a multi-level disjoint set, not all the levels or sub-clusters are usually affected. Upper level clustering, which is based on lower confidence level, can be rebuilt more easily.

If by undoing you mean reverting the changes, we don’t use a persistent data structure. When we need historical clustering, we use a patched forest with concurrent hash maps to track the changes, and then apply or throw them away.

We use a single instance for all clients, but when one CFD server processes new block data, it becomes fully blocked for read access. To solve this, we built a smart load balancer that redirects user requests to a secondary CFD server. This ensures there's always at least two servers running, and more if we need additional throughput.

First of all, at Caudena, we are not involved in crypto projects or investments ourselves. Our expertise lies in analyzing blockchains and providing deep technical insights into how various blockchains operate. We focus on tracking and understanding the flow of digital assets, often in support of government agencies investigating scams, fraud, and other illicit activities.

That said, we absolutely believe that blockchain and cryptocurrency will shape the future of the financial system. When you look beyond the noise of scam tokens, speculative NFTs, and high-profile scandals, there is significant and meaningful financial innovation happening. This extends beyond DeFi to include the tokenization of RWA, where major institutions like BlackRock and JPM Chase are actively exploring and implementing blockchain-based solutions. Numerous projects are driving real progress, and there’s a slow but steady movement toward a more decentralized and transparent financial ecosystem.

Who says that crypto is exclusively scams? There is that of course, but not only that. I do not find Bitcoin to be a scam.

We never targeted weakly-ordered architectures like ARM, only x86. We never used a wide variety of different processors. We are not developing the Linux kernel and are not into control dependencies, just relying on the fences and the memory model. There may be some CPU-dependent performance differences, like discrepancy because of NUMA or false sharing being noticeable on one processor, but not on another. RCU and hazard pointers are nothing new. For the disjoint sets we don't need them. For the forest patches and the tries we do. We are using TBB and OpenMP whenever possible and trying to keep things simple.

No.

Ohh, you're absolutely right!

barebone server is a thing fwiw: A product that comprises a motherboard installed in a case with PSU. Customer adds CPU, memory and storage devices to make a complete usable server. We typically buy servers in this way because figuring out what motherboard fits in which case is a pita, conversely buying complete servers is more expensive and potentially runs into inventory issues at the vendor. So possibly they are running bare metal servers that were also barebone.

What is wrong with C++?

With POSIX semaphores, mutexes, and shared pointers, it is very rare to hit upon a memory issue in modern C++.

Source: Writing code in C/C++ for 30 years.

Rust is the future of systems programming and will always be for the foreseeable future. The memory issue will mostly be addressed as needed, see from John Carmack yesterday[1], the C++ ecosystem advantage (a broad sense of how problems whether DS, Storage, OS, Networking, etc. have been solved) will be very hard to overcome for newer programming languages. I think it is ironic how modern C++ folks just keep chugging along releasing products while Rust folks are generally haranguing everyone about "memory safety" and generally leaving half finished projects (turns out writing Rust code is more fun than reading someone else, who would have guessed).

[1] https://x.com/ID_AA_Carmack/status/1935353905149341968

What's this Rust thing?

Because we are on the 'unsafe' territory. And Rust doesn't even have a defined memory model. Rust is a little bit immature. We have some other services written in Rust though.

Thanks for the questions! We don't currently have plans to open-source it. For anything else, feel free to reach out at pa@caudena.com - happy to discuss further there. We'd like to keep this thread focused on the technical side rather than product discussions :)

Are you from CA or CT? :)

FBI and Europol will work with the same forest (unless they are using on-premise setup), but with different "patches".

Yeah, we figured people would compare it to PRISM :)

There are many possible optimizations, but they’re all highly specific to the particular problems you’re trying to solve.

Can't it be related to that data field sizes are very fixed and never changes?