Claude Code Tested Against Real Data Engineering Work

Robin Moffatt gave Claude Code a genuine data engineering task - building a full dbt project with DuckDB for UK flood monitoring data, including SCD Type 2 snapshots, incremental fact tables, and historical backfills from messy CSVs. It handled modeling, Jinja macros, documentation, and self-correcting dbt build errors impressively well. Then it silently fetched only 1,493 of 5,400+ stations because it didn't paginate an API call, dropped relevant columns without mentioning it, and left SCD check columns incomplete. None of these failures were obvious without domain knowledge. Moffatt's conclusion - "DE + AI > DE" - is the most honest framing of where coding agents actually sit for data work right now. The pattern matches what practitioners across dbt, Airflow, and Spark communities have been reporting: excellent at iteration, dangerous when trusted to make scoping decisions alone.