Truelogic Software Logo

Truelogic Software

Senior Data Engineer (Databricks) — Data & Analytics Platform

Posted 14 Days Ago
In-Office
Latham, NY
Senior level
In-Office
Latham, NY
Senior level
The Senior Data Engineer will design and optimize scalable ETL pipelines for data processing using PySpark and Databricks, focusing on performance and cost efficiency.
The summary above was generated by AI
About Truelogic

At Truelogic we are a leading provider of nearshore staff augmentation services headquartered in New York. For over two decades, we’ve been delivering top-tier technology solutions to companies of all sizes, from innovative startups to industry leaders, helping them achieve their digital transformation goals.

Our team of 600+ highly skilled tech professionals, based in Latin America, drives digital disruption by partnering with U.S. companies on their most impactful projects. Whether collaborating with Fortune 500 giants or scaling startups, we deliver results that make a difference.

By applying for this position, you’re taking the first step in joining a dynamic team that values your expertise and aspirations. We aim to align your skills with opportunities that foster exceptional career growth and success while contributing to transformative projects that shape the future.

Our Client

A technology-driven organization operating in the marketing and analytics space, focused on building advanced data platforms that power media mix modeling, scenario planning, and large-scale analytical decision-making.


Job Summary

The Data Engineer will play a key role in designing, building, and optimizing scalable data pipelines that support complex analytics and modeling use cases. This role is ideal for a hands-on engineer with strong experience in PySpark and Databricks who enjoys working on performance-critical data workflows and transforming large datasets efficiently.

The ideal candidate brings deep expertise in ETL development, data transformations, and pipeline optimization, along with the ability to collaborate effectively across technical and non-technical teams. You will contribute directly to the reliability, scalability, and cost-efficiency of a modern data platform.

Responsibilities
  • Design, build, and maintain scalable ETL pipelines for large-scale data processing.

  • Implement data transformations and workflows using PySpark at an intermediate to advanced level.

  • Work extensively with Databricks to develop, manage, and optimize data pipelines.

  • Optimize pipelines for performance, scalability, and cost efficiency across environments.

  • Troubleshoot, debug, and resolve data processing and pipeline issues.

  • Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions.

  • Apply best practices for data quality, governance, and performance tuning.

  • Document data workflows, design decisions, and operational processes to ensure maintainability.

Qualifications and Job Requirements
  • 5–7 years of professional experience in data engineering.

  • Strong hands-on proficiency with PySpark (intermediate to advanced level).

  • Solid experience working with Databricks, including Autoloader, Python-based workflows, and platform best practices.

  • Proven experience optimizing data pipelines for performance and cost efficiency.

  • Strong understanding of ETL processes and large-scale data transformations.

  • Excellent problem-solving skills with the ability to diagnose and resolve complex data issues.

  • Strong communication skills and the ability to collaborate effectively with both technical and non-technical stakeholders.

What We Offer
  • 100% Remote Work: Enjoy the freedom to work from the location that helps you thrive. All it takes is a laptop and a reliable internet connection.

  • Highly Competitive USD Pay: Earn an excellent, market-leading compensation in USD, that goes beyond typical market offerings.

  • Paid Time Off: We value your well-being. Our paid time off policies ensure you have the chance to unwind and recharge when needed.

  • Work with Autonomy: Enjoy the freedom to manage your time as long as the work gets done. Focus on results, not the clock.

  • Work with Top American Companies: Grow your expertise working on innovative, high-impact projects with Industry-Leading U.S. Companies.

Why You’ll Like Working Here
  • A Culture That Values You: We prioritize well-being and work-life balance, offering engagement activities and fostering dynamic teams to ensure you thrive both personally and professionally.

  • Diverse, Global Network: Connect with over 600 professionals in 25+ countries, expand your network, and collaborate with a multicultural team from Latin America.

  • Team Up with Skilled Professionals: Join forces with senior talent. All of our team members are seasoned experts, ensuring you're working with the best in your field.

Apply now!

Top Skills

Databricks
Pyspark

Similar Jobs

14 Days Ago
Easy Apply
In-Office or Remote
New York, NY, USA
Easy Apply
136K-200K Annually
Senior level
136K-200K Annually
Senior level
Consumer Web • Digital Media • eCommerce • Fintech • Financial Services
As a Senior Data Engineer, you'll optimize and maintain data platforms, develop integrations, and collaborate cross-functionally to enhance analytics and decision-making capabilities.
Top Skills: AirflowArgoAWSDbtElastic MapreduceFastapiFirehoseKinesisLambdaPandasPlotlyPythonRedshiftSnowflakeSQLTableau
59 Seconds Ago
Easy Apply
In-Office
New York, NY, USA
Easy Apply
160K-170K Annually
Senior level
160K-170K Annually
Senior level
Artificial Intelligence • Computer Vision • Machine Learning • Payments • Real Estate • PropTech
The Director of Business Development will drive sales within healthcare systems, managing relationships and negotiations to expand Metropolis's presence in the sector.
Top Skills: Salesforce
10 Minutes Ago
Remote or Hybrid
6 Locations
133K-235K Annually
Junior
133K-235K Annually
Junior
Artificial Intelligence • Cloud • Machine Learning • Mobile • Software • Virtual Reality • App development
Develop and optimize mobile and web applications, implement features, improve user interfaces, and contribute to the engineering culture at Snap.
Top Skills: AngularAWSCSSGoGraphQLHTMLJavaJavaScriptKubernetesMemcacheNode.jsNoSQLPythonReactRedisTypescriptVueWebgl

What you need to know about the Seattle Tech Scene

Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.

Key Facts About Seattle Tech

  • Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Amazon, Microsoft, Meta, Google
  • Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
  • Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Madrona, Fuse, Tola, Maveron
  • Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account