The F.A.F. (Functional Architectural Flaw) is why schema drift happens silently. It's not a bug; it's a structural weakness that makes every single analytics model built on top of it fundamentally untrustworthy.
Until the data is validated at the point of entry, ETL tools are just transforming garbage. We've developed a containment protocol for this specific flaw. It requires low-level architectural intervention, not just another dashboard.
If your models keep breaking for "unknown reasons," F.A.F. is the answer.
sgt•3mo ago
vivekburman•3mo ago
From business managers point of view: 1. A solution that solves the problem 2. Has a management lifecycle 3. Allows productivity and team collab
sgt•3mo ago
vivekburman•3mo ago
dbt - code is done via VSCode and managed via git, Job Orchestration is done via Airflow or Dagster
Fivetran - its more of an ELT cloud hosted solution only, and not near real time cases.
Talend, Alteryx - Its a drag and drop first solution. Then comes IDE
My personal work experience - In my current job I was tasked to build dashboard platform which supports near real time sync. I've Postgres as main DB, Elastic search for dashboarding. Due to this near-real time requirement I had to choose a very niche library `pgsync`. Then immediately I rewrote the library again why because pgsync doesn't detect granular change and is tied to redis. Whereas I had rabbitmq as the middle layer for CDC.
sgt•3mo ago
vivekburman•3mo ago
sgt•3mo ago
The statement "dbt - code is done via VSCode" I found weird. I have used dbt but never VSCode.
vivekburman•3mo ago