Your arguments in "Keep data pipelines simple" with "...the starting point being the original data..." directly conflicts with your position in "Why not to E".

Asking source system engineers to create an API for the data guys basically means asking them to do part of the transformation job. In my experience this can lead to many misunderstandings and inefficiencies. But it also has its benefits as you pointed out.

In general I disagree with advocating ETL over ELT or vice versa. None of them is the cure for all. If you have a complex modularized data pipeline it may make…

Marko Petzold

Phd in Mathematics and data lover. CEO of Record Evolution, the creators of REPODS.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store