Examples

pydiverse.pipedag already has quite a rich set of features. So we like to provide some examples showing typical use cases.

• Quickstart example

• Simple pipeline

• Working with real database

• Imperative materialization / Materializing Subqueries

• Interactive development

• Grouping Tasks/Stages and visualization

• Stage validation before schema swap

• Slightly more realistic pipeline

• Introduction to vectorization principle with some example pipelines

• Multiple instances: full, mini, midi

• Raw SQL example

• Best practices / moving from Raw SQL over handwritten SELECT statements to programmatic SQL

• Best practices / multiple instances: full_fresh, full_stable, mini_stable, midi_stable

• Best practices / inline views, CTEs, and subqueries