I've been working on extending CDC File Transfer to support Linux→Linux and macOS builds. This is an open-source project born from Google Stadia – fast file sync tools (cdc_rsync) and a streaming filesystem (cdc_stream) that use Content Defined Chunking (FastCDC) to transfer only the changed portions of files.
Originally Windows→Linux only, designed for game developers to iterate on 40+ GB builds. The CDC-based diffing is up to 30x faster than rsync (1500 MB/s vs 50 MB/s).
What I added (40 commits):
Linux Client Support:
Ported cdc_rsync client to run on Linux (was Windows-only)
Added process_linux.cc (~700 lines): Process management using fork/execve
Added port_manager_linux.cc: Port detection using POSIX shared memory
Now you can sync Linux→Linux, not just Windows→Linux
macOS Support:
Full build support for macOS (ARM64 and x86_64)
Fixed 17+ platform-specific issues:
/tmp, /var, /etc are symlinks to /private/* – breaks path comparisons
POSIX shared memory rounds ftruncate() up to 16KB, rejects shrinking
BSD fts_* API reports Bazel runfiles symlinks as FTS_SL not FTS_F
strerror(0) returns different strings per platform
stat64 doesn't exist on macOS (just use stat)
All 41 tests now pass on macOS
Infrastructure:
GitHub Actions CI for Linux and macOS (self-hosted runner for macOS)
Added comprehensive CLAUDE.md for AI-assisted development
The project uses Bazel with gRPC, protobuf, and FUSE. Dependencies via git submodules or http_archive.
vadiml•1w ago
Originally Windows→Linux only, designed for game developers to iterate on 40+ GB builds. The CDC-based diffing is up to 30x faster than rsync (1500 MB/s vs 50 MB/s).
What I added (40 commits):
Linux Client Support:
Ported cdc_rsync client to run on Linux (was Windows-only) Added process_linux.cc (~700 lines): Process management using fork/execve Added port_manager_linux.cc: Port detection using POSIX shared memory Now you can sync Linux→Linux, not just Windows→Linux macOS Support:
Full build support for macOS (ARM64 and x86_64) Fixed 17+ platform-specific issues: /tmp, /var, /etc are symlinks to /private/* – breaks path comparisons POSIX shared memory rounds ftruncate() up to 16KB, rejects shrinking BSD fts_* API reports Bazel runfiles symlinks as FTS_SL not FTS_F strerror(0) returns different strings per platform stat64 doesn't exist on macOS (just use stat) All 41 tests now pass on macOS Infrastructure:
GitHub Actions CI for Linux and macOS (self-hosted runner for macOS) Added comprehensive CLAUDE.md for AI-assisted development The project uses Bazel with gRPC, protobuf, and FUSE. Dependencies via git submodules or http_archive.
GitHub (original): https://github.com/google/cdc-file-transfer My fork: https://github.com/Vadiml1024/cdc-file-transfer
I've been using Claude Code (Anthropic's AI coding assistant) for most of this work. Happy to discuss the experience or the technical challenges.