For the past few months I've been managing over 1TB per month of media archives using custom Python scripts. It worked, but the operational debt kept growing-regex-heavy metadata pathing (eg: container/{file_created_date}/{file_created_datetime}:{filename}), async I/O quirks across OSes, and fragile long-running transfers became harder to reason about than the actual data movement.
Over the past Christmas break, I challenged myself to replace that setup with a state-aware, local-first workflow engine. The result is Blober: https://blober.io/
----
Architecture highlights
1. Local-first store
Blober isn't a CLI wrapper. It persists configurations, workflows, tasks, and state in a local SQLite DB so transfers can be resumed, inspected, and replayed deterministically.
2. Immutable jobs
Each run snapshots configuration into a Job object. Once started, the job doesn't change, even if the user edits the workflow later.
3. Background executor
A background loop (Electron-based) manages long-running transfers and prevents system sleep during multi-hour jobs. This was critical for my use case and a advantage over rclone.
4. Metadata-aware paths
Files are inspected at runtime (stat() or provider APIs), allowing variables like {file_created_date} or {file_size_mb} to be injected into destination paths without pre-scanning. This is the biggest advantage over rclone.
---
I built the desktop app in ~14 days using heavy AI assistance. What worked surprisingly well was forcing a planning gate before any code generation: having the model produce a phased implementation plan first reduced over-engineering and large hallucinated files.
That said, I'm looking to extend this to support all cloud providers and use cases, so any feedback is welcome.
I'd love brutal feedback, especially on:
- Simplicity of the application and the workflow model
- Whether this actually reduces "organizational debt" for large archives
- Where this breaks down compared to rclone or similar tools
mckabue•1mo ago
For the past few months I've been managing over 1TB per month of media archives using custom Python scripts. It worked, but the operational debt kept growing-regex-heavy metadata pathing (eg: container/{file_created_date}/{file_created_datetime}:{filename}), async I/O quirks across OSes, and fragile long-running transfers became harder to reason about than the actual data movement.
Over the past Christmas break, I challenged myself to replace that setup with a state-aware, local-first workflow engine. The result is Blober: https://blober.io/
----
Architecture highlights
1. Local-first store Blober isn't a CLI wrapper. It persists configurations, workflows, tasks, and state in a local SQLite DB so transfers can be resumed, inspected, and replayed deterministically.
2. Immutable jobs Each run snapshots configuration into a Job object. Once started, the job doesn't change, even if the user edits the workflow later.
3. Background executor A background loop (Electron-based) manages long-running transfers and prevents system sleep during multi-hour jobs. This was critical for my use case and a advantage over rclone.
4. Metadata-aware paths Files are inspected at runtime (stat() or provider APIs), allowing variables like {file_created_date} or {file_size_mb} to be injected into destination paths without pre-scanning. This is the biggest advantage over rclone.
---
I built the desktop app in ~14 days using heavy AI assistance. What worked surprisingly well was forcing a planning gate before any code generation: having the model produce a phased implementation plan first reduced over-engineering and large hallucinated files.
That said, I'm looking to extend this to support all cloud providers and use cases, so any feedback is welcome.
I'd love brutal feedback, especially on:
- Simplicity of the application and the workflow model
- Whether this actually reduces "organizational debt" for large archives
- Where this breaks down compared to rclone or similar tools
Blog post on the build process: https://toknow.ai/posts/vibe-coding-challenge/
App: https://blober.io/