Engineer - ETL
Numerator | Engineering
We’re hiring a talented Data Engineer and Big Data enthusiast to work in our platform to help ensure that our data quality is flawless. As a company, we have millions of new data points every day that come into our system. You will be working with a passionate team of engineers to solve challenging problems and ensure that we can deliver the best data to our customers, on-time. You will be using the latest cloud data warehouse technology to build robust and reliable data pipelines.
- Develop expertise in the different upstream data stores and systems across Numerator.
- Design, develop and maintain data integration pipelines for Numerators growing data sets and product offerings.
- Build testing and QA plans for data pipelines.
- Build data validation testing frameworks to ensure high data quality and integrity.
- Write and maintain documentation on data pipelines and schemas
- BS or MS in Computer Science or related field of study
- 3 + years of experience in the data warehouse space
- Expert in SQL, including advanced analytical queries
- Proficiency in Python (data structures, algorithms, object oriented programming, using API’s)
- Experience working with a cloud data warehouse (Redshift, Snowflake, Vertica)
- Experience with a data pipeline scheduling framework (Airflow)
- Experience with schema design and data modeling
Exceptional candidates will have:
- Amazon Web Services (EC2, DMS, RDS) experience
- Terraform and/or ansible (or similar) for infrastructure deployment
- Airflow -- Experience building and monitoring DAGs, developing custom operators, using script templating solutions.
- Experience supporting production systems in an on-call environment