With this in-depth book, data engineers will learn powerful, real-world best practices for managing data big and small. Contributors from companies including Google, Microsoft, IBM, Facebook, Databricks, and GitHub share their experiences and lessons learned for cleaning, prepping, wrangling, storing, processing, and ingesting data.

Current and aspiring data engineers, data architects, data team managers, data scientists, machine learning engineers, and software engineers will get targeted advice for overcoming a variety of specific challenges from engineers at major companies.

Projects include:

• Building pipelines
• Stream processing
• Data privacy and security
• Data governance and lineage
• Data storage and architecture
• Ecosystem of modern tools
• Data team makeup and culture
• Career advice