Big Data Solutions Engineer

It takes powerful technology to connect our brands and partners with an audience of hundreds of millions of people. Whether you’re looking to write mobile app code, engineer the servers behind our massive ad tech stacks, or develop algorithms to help us process trillions of data points a day, what you do here will have a huge impact on our business—and the world.

It takes powerful technology to connect our brands and partners with an audience of hundreds of millions of people. Whether you’re looking to write mobile app code, engineer the servers behind our massive ad tech stacks, or develop algorithms to help us process trillions of data points a day, what you do here will have a huge impact on our business—and the world.

Big Data Solutions Engineer

We build, improve, and maintain one of the highest scaling platforms in the world. Our amazing team of engineers works on next-generation big data platforms that transform how users connect every single day. Yahoo's Big Data Platform drives some of the most demanding applications in the industry. The system handles billions of requests a day and runs on some of the largest Hadoop clusters ever built! 50,000 nodes strong and several multi-thousand node clusters bring scalable computing to a whole new level. We work on problems that cover a wide spectrum - from web services to operating systems and networking layers. Our biggest challenges ahead are designing efficient cloud-native big data platforms. 

You will be part of the Big Data Solutions engineering team, contributing to the design and implementation of robust Big Data solutions on AWS, GCP, and Databricks platforms. You will be involved in the design, implementation, and testing of end-to-end solutions, ensuring sound development practices are followed to achieve well-tested technical solutions, ideally in a Unix environment. You will gain experience with containerization (ideally Kubernetes) and Big Data technologies, challenging security constraints, and myriad AWS/GCP services, with Python and Java as primary development languages.

A solid working understanding of cloud technologies is a must. Knowledge of developing clean APIs and integrating with existing web services is required, as well as a solid understanding of core architectural principles and a cloud-native solutions mindset. You will contribute to projects involving distributed systems, cloud technologies, Agentic AI workflows, and ML use cases. Experience with Spark, Airflow, Vertex AI, and Sagemaker is a plus. You will be highly motivated and able to work independently or as part of a cross-functional team on complex projects.

Responsibilities

• Big Data platform: implementing and scaling solutions using Big Data technologies like Hadoop, Spark, Airflow, ensuring high performance and scalability.

• Data Orchestration: Utilizing tools like Airflow to manage the scheduling, execution, and monitoring of data workflows across cloud platforms such as AWS and GCP.

• Integration and Support: Collaborating with different teams to integrate data processing, MLOps and AIOps.

• Feature Development: Implementing new features to support advanced processing capabilities, including cost and performance optimization..

• Must be self-motivated, proactive, driven and able to execute tasks both independently and within a team environment

• Ability to comprehend and communicate in a complex technical environment

• Understand and apply common design and architectural patterns

• Share knowledge and experience through thorough code review processes

• Display a history of creativity, innovative thought, awareness of current technical trends and high levels of motivation

Required Skills

• 3+ years of experience in software development

• Programming skills: Python/Java/Golang

• Excellent knowledge of Spark, Airflow, Terraform, AWS/GCP and cloud native services like EMR, Sagemaker, Dataproc, Bigquery, VertexAI, Airflow

• Familiarity with Kubernetes, Docker, and Containerization

• Operating systems: Mac/Linux

• Bachelor's degree in Computer Science, Software Engineering or a related field or relevant experience

The material job duties and responsibilities of this role include those listed above as well as adhering to Yahoo policies; exercising sound judgment; working effectively, safely and inclusively with others; exhibiting trustworthiness and meeting expectations; and safeguarding business operations and brand integrity.

At Yahoo, we offer flexible hybrid work options that our employees love! While most roles don’t require regular office attendance, you may occasionally be asked to attend in-person events or team sessions. You’ll always get notice to make arrangements. Your recruiter will let you know if a specific job requires regular attendance at a Yahoo office or facility. If you have any questions about how this applies to the role, just ask the recruiter!

Yahoo is proud to be an equal opportunity workplace. All qualified applicants will receive consideration for employment without regard to, and will not be discriminated against based on age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. Yahoo will consider for employment qualified applicants with criminal histories in a manner consistent with applicable law. Yahoo is dedicated to providing an accessible environment for all candidates during the application process and for employees during their employment. If you need accessibility assistance and/or a reasonable accommodation due to a disability, please submit a request via the Accommodation Request Form (www.yahooinc.com/careers/contact-us.html) or call +1.866.772.3182. Requests and calls received for non-disability related issues, such as following up on an application, will not receive a response.

We believe that a diverse and inclusive workplace strengthens Yahoo and deepens our relationships. When you support everyone to be their best selves, they spark discovery, innovation and creativity. Among other efforts, our 11 employee resource groups (ERGs) enhance a culture of belonging with programs, events and fellowship that help educate, support and create a workplace where all feel welcome.

The compensation for this position ranges from $88,500.00 - $184,375.00/yr and will vary depending on factors such as your location, skills and experience.The compensation package may also include incentive compensation opportunities in the form of discretionary annual bonus or commissions. Our comprehensive benefits include healthcare, a great 401k, backup childcare, education stipends and much (much) more.

Currently work for Yahoo? Please apply on our internal career site.