Principal Platform Engineer - Data Private Cloud (Kubernetes/OpenShift)

Principal Engineer -Data Platforms (Enterprise Data Platforms, Lakehouse, Multi-Tenant Architectures)
Core Role Expectation
This role is a hands-on Principal Engineer responsible for architecting, engineering, and enablement of large-scale, multi-tenant enterprise data platforms. The ideal candidate has deep experience building enterprise data lakes, lakehouses, or data warehouses that support data analytics, data engineering, AI/ML, and regulatory workloads across hundreds or thousands of users.
This isnot an application or pipeline-only role. The focus is on platform architecture, scalability, security, and operational excellence for shared enterprise data platforms.
Required Technical & Leadership Skillset
Enterprise Data Platform Architecture & Engineering

• Extensive experience designing, engineering, and operating enterprise-scale data platforms, including data lakes, lakehouses, or data warehouses
• Proven experience leading large, multi-tenant data platforms serving multiple lines of business with strict isolation, governance, and performance controls
• Deep understanding of data platform reference architectures, including:
  • Lakehouse patterns (compute/storage separation, open table formats)
  • Shared services vs. tenant-owned workloads
  • Platform-as-a-product operating models
• Demonstrated ownership of end-to-end platform lifecycle: architecture, build, migration, operations, and modernization

Multi-Tenancy, Scale & Performance

• Hands-on experience designing and enforcing multi-tenant isolation.
• Expertise in capacity planning, workload isolation, quota management, and performance optimization at enterprise scale
• Experience supporting mixed workloads (batch, interactive SQL, streaming, ML/AI) on shared platforms

Data Platform Technologies (Hands-On)

• Strong hands-on expertise with modern data platform ecosystems, such as:
  • Compute & Processing: Spark (including Spark at scale), distributed processing frameworks
  • Query & Analytics: Trino/Presto or similar distributed SQL engines
  • Table Formats & Storage: Iceberg (or similar), Iceberg Rest Catalogue, object storageand enterprise storage platforms
  • Metadata, Catalog & Governance: DataHub, Apache Atlas, Hive Metastore, or equivalent
• Experience designing and operating production-grade data services, not just proof-of-concepts

Platform Engineering & Automation

• Strong background in platform engineering principles applied to data platforms:
  • Infrastructure as Code (Terraform or equivalent)
  • Automated environment provisioning and repeatability
  • GitOps or declarative deployment models
• Experience standardizing and industrializing data platforms to support self-service consumption at scale

Security, Governance & Compliance

• Demonstrated experience building secure-by-design data platforms in regulated environments
• Hands-on knowledge of:
  • Authentication and authorization models (enterprise IAM integration)
  • Fine-grained access controls and data entitlements
  • Auditability, lineage, and compliance controls
• Proven ability to partner with Security, Risk, Compliance, and Audit teams to meet regulatory requirements (e.g., SOX, PCI, data privacy)

Technical Leadership & Influence

• Recognized technical leader capable of:
  • Setting data platform strategy and standards across the enterprise
  • Making architecture decisions that balance scalability, cost, risk, and time-to-market
  • Mentoring senior engineers and influencing platform adoption across teams
• Experience leading complex platform migrations or modernizations (e.g., legacy data platforms to modern lakehouse architectures)

Data Platform Components (Platform Enablement)
You provide leadership for the platform that runs these technologies, not the pipelines or applications built on them:

• Compute: Spark on K8s, Kyuubi, JupyterHub
• Query/Analytics: Trino, Superset
• Orchestration: Airflow on Kubernetes
• Catalog/Governance: Gravitino, DataHub, Ranger
• Storage: Iceberg, S3/NetApp, PostgreSQL
• Messaging/Search: Kafka, OpenSearch

Required Qualifications

• 7+ years of Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
• 5+ years of hands-on experience with Kubernetes in production environments (OpenShift Container Platform strongly preferred)
• Proven track record designing and operating large-scale data platforms in enterprise environments

Preferred / Differentiating Experience

• Experience in financial services or other highly regulated enterprises
• Prior ownership of enterprise data platform transformations
• Contributions to open-source data or platform ecosystems
• Background in platform product thinking or developer experience for data platforms
• Experience supporting AI/ML workloads on shared enterprise data platforms