Spark is our main batch big data analytics query engine at Uber. It accounts for the majority of our data compute resources, powers most of our major ingestion, ETL, data science, and machine learning workloads. Spark at Uber is used by over 2.5K weekly active users, who run 3M+ applications every week.
As manager of the Spark team, you will lead the team to design, implement, optimize, and manage large scale batch compute infrastructure. You will work on problems like unification of stream and batch, batch ingestion for data lake, and minimum downtime support that will impact multiple business use cases at Uber scale. At the same time, you will also have the opportunity to collaborate with the open source community for Spark, Presto.
What the Candidate Will Need / Bonus Points
---- What the Candidate Will Do ----

• Manage a strong team of senior engineers and provide critical infrastructure for batch analytics at Uber.
• Leverage knowledge of spark internals to dramatically help improve customer's Spark job performance.
• Work on Spark internals so as to continue improving Spark's robustness and performance for Uber's unique use cases.
• Work with the community to upstream Uber's contributions to open source and also keep our internal fork up to date

---- Basic Qualifications ----

• Bachelor's degree in Computer Science or related field.
• 2+ years of experience managing backend data infra teams and 5+ years of industry experience.
• Solid understanding of Java for backend / batch processing systems
• Experience working with Large Scale distributed systems, HDFS / Yarn.

---- Preferred Qualifications ----

• MS / PhD in Computer Science or related field.
• Experience managing production systems with a strong availability SLA.
• Experience working with Apache Spark or similar analytics technologies.
• Experience working with large scale distributed systems, HDFS / Yarn.
• Experience working with SQL Compiler, SQL Plan / Runtime Optimization.

At Uber, we reimagine the way the world moves for the better. The idea was born on a snowy night in Paris in 2008, and ever since then, our DNA of reimagination and reinvention carries on. We've grown into a global platform moving people and things in ever-expanding ways, taking on big problems to help drivers, riders, delivery partners, and eaters make movement happen at the push of a button for everyone, everywhere.
We welcome people from all backgrounds who seek the opportunity to help build a future where everyone and everything can move independently. If you have the curiosity, passion, and collaborative spirit, work with us, and let's move the world forward, together.
Uber is proud to be an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you have a disability or special need that requires accommodation, please let us know by completing this form.
Offices continue to be central to collaboration and Uber's cultural identity. Unless formally approved to work fully remotely, Uber expects employees to spend at least half of their work time in their assigned office. For certain roles, such as those based at green-light hubs, employees are expected to be in-office for 100% of their time. Please speak with your recruiter to better understand in-office expectations for this role.
Uber is committed to a safe workplace. We have implemented COVID-19 safety protocols that meet or exceed local public health guidelines. Workplace safety remains our number one priority. As a result, Uber recommends all US- and Canada-based employees to be vaccinated in order to access any of our facilities; this is subject to change solely at the Company's discretion.
At Uber, we reimagine the way the world moves for the better. The idea was born on a snowy night in Paris in 2008, and ever since then, our DNA of reimagination and reinvention carries on. We've grown into a global platform moving people and things in ever-expanding ways, taking on big problems to help drivers, riders, delivery partners, and eaters make movement happen at the push of a button for everyone, everywhere.
We welcome people from all backgrounds who seek the opportunity to help build a future where everyone and everything can move independently. If you have the curiosity, passion, and collaborative spirit, work with us, and let's move the world forward, together.
Uber is proud to be an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you have a disability or special need that requires accommodation, please let us know by completing this form.
Offices continue to be central to collaboration and Uber's cultural identity. Unless formally approved to work fully remotely, Uber expects employees to spend at least half of their work time in their assigned office. For certain roles, such as those based at green-light hubs, employees are expected to be in-office for 100% of their time. Please speak with your recruiter to better understand in-office expectations for this role.
Uber is committed to a safe workplace. We have implemented COVID-19 safety protocols that meet or exceed local public health guidelines. Workplace safety remains our number one priority. As a result, Uber recommends all US- and Canada-based employees to be vaccinated in order to access any of our facilities; this is subject to change solely at the Company's discretion.