Didirtech
TREN

ETL pipelines run from a panel

A trimmed-down version of a project where pipelines moving data from external APIs into SQL Server were managed from a panel instead of code.

Demo · All data is made up and nothing is sent to a server.

This demo is not connected to real APIs or a database. Pipelines, counters and logs are simulated in your browser.

ETL · PIPELINE CONTROL
SIM · 14:31
VALIDATEMAP · TRANSFORMORDERS APIGET /v2/ordersFI10128rows/minSTOCK APIGET /v1/levelsFI10282rows/minFX APIGET /latestFI1030.4rows/minSHIPPING APIGET /v3/shipmentsFI10425rows/minAPI v3 · schema drifttracking_no → trackingNumberSQL SERVER · ETL_DBdbo.Orders1,284,112 rowsdbo.Stock48,920 rowsdbo.FxRates96,408 rowsdbo.Shipments902,377 rowsQUARANTINE · 0DRAWING ETL-01REV 14NOT TO SCALE13 Nov 2026

← scroll the diagram →

What to try

  • Watch data flow from the APIs into SQL Server tables on the diagram; close a valve or a switch to stop a pipeline, then open it again.
  • Let an API change: when the shipping API renames a field, records start landing in quarantine.
  • Open the pipeline settings, fix the broken field mapping and publish; the pipeline recovers without a deploy and the quarantined rows are reprocessed.
  • In the logs, see today’s runs on a timeline, filter by pipeline and level, and open the details of a run.
I want something like thisAll work

Contact

Tell me about your project. We’ll take it from there.

What do you need?

What you send is only used to get back to you. Privacy notice