Get up and running with Apache Airflow in this part one workshop of Track 6: Apache Airflow led by Arthur Wiedmer who will teach you how to install it on your local machine, build a simple example worfklow and test and run your workflow, before jumping into practical, hands-on tutorials in part two.
In this workshop, you will learn:
- Airflow: What is it and what kind of use cases it addresses.
- How to install it on you local machine.
- How to get started and a simple example workflow
- Testing and running your workflow
Skill Level:
Beginner - Intermediate
Prerequisites:
What you should know (or have pre-installed) to get the most value.
What you should know before starting :
- Some familiarity with Python. This is the core language used by Airflow, and you should feel comfortable with the basics.
- Installing packages on your machine/OS.
How to prepare your machine (I will try to provide more complete instructions before hand)
- Airflow is meant for a Unix environment. I would recommend having either a Linux, OSX laptop or a VM with Ubuntu (>= 14.04 LTS).
- [Metadata DB] In this environment, you will need SQLite to get started, but a somewhat recent MySQL (>= 5.6) will enable you to go faster.
- [Python] Airflow is compatible with both Python 2.7 (>=2.7.6 required and we recommend >=2.7.10) and 3.4+.
- [Python Environment] Upgrading pip using `pip install -U pip` is recommended.
Meet Your Instructor:
Arthur Wiedmer | Data Engineer | Airbnb
Arthur Wiedmer joined Airbnb as a Data Engineer in 2014. He builds frameworks and pipelines on top of Airflow and focuses on disseminating best practices for data pipelines and data security at Airbnb as part of the Data Platform Team. He is a committer on Apache Airflow (incubating). He is @awiedmer on Twitter and @artwr on Github.
