Current ETL/ELT tools solve one problem, but seems lacking on E2E solution
2•vivekburman•3h ago
I looked onto Talend, Alteryx, dbt, fivetran... They are all in etl/elt space but seems to be solving one problem and lacking on another. If a business needs a streamlined solution, there seems to be a lack of it.
Comments
sgt•3h ago
What do you propose?
vivekburman•2h ago
A step back, looking at data engineers needs
1. Integrated Code IDE
2. Version Control, Permissions and stuff [For team collab]
3. Distributed Jobs management using remote agents
4. Choice of hosting in AWS, GCP or self-hosted
From business managers point of view:
1. A solution that solves the problem
2. Has a management lifecycle
3. Allows productivity and team collab
sgt•1h ago
But I mean all the commercial ETL solutions already have this. The details differ, but I think they all tick the boxes.
sgt•3h ago
vivekburman•2h ago
From business managers point of view: 1. A solution that solves the problem 2. Has a management lifecycle 3. Allows productivity and team collab
sgt•1h ago