Combine Dataform with Typescript for Improved Workflows
Dataform pipeline code that is fully typed with unit tests
Jan 29, 20258 min read268
Command Palette
Search for a command to run...
#data-engineering
Articles tagged with #data-engineering
Writing data quality tests in Dataform
Ensure your BigQuery tables have reliable data with Dataform
Nov 24, 20248 min read547
A Practical Introduction to Google Cloud Dataform
Building data pipelines in Dataform
Nov 4, 20249 min read877
Parquet format - A deep dive : Part 4
Demystifying how Parquet works internally
Jul 29, 20233 min read216
Parquet format - A deep dive : Part 3
Demystifying how Parquet works internally
Jul 23, 20235 min read614Automatically add partitions in Athena
Using S3 and Athena is great for data storage and retrieval using queries. But when I first started using it, one common problem that came up fairly quickly is: How can I add new partitions automatically? The issue was this: Partitioned data was gett...
Dec 18, 20225 min read1.9K