Data Pipeline Engineer
System1 is looking for a Data Pipeline Engineer with production data experience to join Data Engineering team. This team is the horizontal layer that supports business intelligence, optimization, machine learning, and external & internal reporting. We process and report on hundreds of millions of events and user attributes per day, gathered from an extremely heterogeneous set of data streams.
Our bread and butter is Python and PostgreSQL, but we also utilize a range of technologies including AWS Lambda, SNS, SQS, Redis, and Dynamodb for caching and mapping, as well as Redshift, Kinesis, and Spark for large dataset munging and ad hoc analysis.
The Role You Will Have
- Design and develop data processing infrastructure and reporting pipelines
- Collaborate with the data-science team to productionize ML pipelines
- Coordinate data models with other engineering teams
- Identify scale bottlenecks and successfully overcome them
- Contribute to technical specification documents for data architecture projects
- Contribute to ongoing maintenance of existing infrastructure and investigate issues and failures
What You Will Bring
- 5+ years of full-time Python development experience
- 3+ years PostgreSQL or Redshift experience. Understanding of relational data models is a must
- 3+ years experience working with Spark or other big data architectures (Hadoop, Apache Storm, etc…) in high-volume environments ( running the big data solutions in AWS is a plus)
- Extensive experience building and managing ETL pipelines on cloud based platforms from inception to production rollout
- Experience ensuring a high degree of reliability and data integrity for mission critical systems
- Experience with large migration projects is a plus
- Excellent communication skills and a collaborative mindset
What We Have to Offer
- Paid Parking or an ORCA Card
- Cross-team lunches and tech demos to foster learning
- Unlimited paid sick time, 17 days of PTO, and 10 paid company holidays
- Competitive Benefits package
- Weekly catered lunch and fully stocked kitchen
- Happy Hours at various bars, restaurants, and venues across Bellevue
- Weekly yoga and fitness classes with private trainer
- Company parties and outings
Related Searches: Sr. Data Pipeline Engineer, Senior Data Pipeline Engineer, Pipeline Engineer, ETL Engineer, Sr. ETL Engineer,Senior ETL Engineer, Data Engineer, Python Engineer, Sr. Python Engineer, Senior Python Engineer, Postgres, SQL, Data warehouse, ETL, Database Engineer