Principal Data Engineering Lead - Services Special Project

Summary

At Apple, great ideas have a way of becoming phenomenal products, services, and customer experiences very quickly. Our team is building a massive, real-time platform that transforms continuous streams of multimodal data (including structured, image, and log data) into an intelligent, searchable foundation.

We are seeking a Principal Data Engineer to lead and drive not only our team's data processing systems, but also to partner at a larger scale, coordinating and synching strategically with other business groups and organizations within Apple.

Description

We are seeking a Principal Data Engineering Lead with deep expertise in ETL/ELT, data architecture, and applied ML pipelines to drive the design, build, and operations of this infrastructure. As a key member of our team, you will be responsible for driving critical decisions and operations across the entire system while aligning strategically across Apple.

Responsibilities

• Build and implement batch and streaming ETL/ELT pipelines that ingest, process, and model data from diverse sources, including unstructured media and real-time event streams, ensuring high reliability, performance, and scalability.
• Develop and maintain Kafka-based ingestion and processing pipelines, ensuring reliable data delivery across services and into the data lake.
• Build robust logical and physical data models with a focus on dimensional modeling, versioning, and storage patterns (e.g., Parquet, ORC) optimized for ingest, reporting, and operational use cases.
• Define and enforce data quality checks, SLAs, and observability standards to ensure data is accurate, timely, versioned, and trusted by stakeholders.
• Integrate and enrich raw signals with metadata and attribution to power downstream use cases such as analytics, billing, planning, and optimization.
• Implement standard methodologies for data lineage, metadata management, schema governance, versioning, and security in alignment with Apple's standards for data protection and privacy.
• Deliver solutions that include logging, anomaly detection, data validation, cleaning, and transformation, with strong emphasis on monitoring, debuggability, and continuous improvement.
• Work closely with ML engineers, data scientists, platform teams, and leadership to translate requirements into scalable, reliable data solutions.
• Help advance the team's data stack, including tooling, frameworks, and standards for development, testing, deployment, and operations.
• Align our team with other Apple teams strategically, participating in larger scale discussions and deliverables across our ecosystem.

Minimum Qualifications

• Masters Degree
• 12+ years of experience in data engineering, including building and maintaining large-scale ETL/ELT data pipelines
• Proficiency in data modeling, especially dimensional modeling, and designing schemas optimized for analytics and reporting
• Experience with leveraging databases including SQL/NoSQL Databases (including Postgres / Cassandra / Redis)
• Strong experience with distributed data processing frameworks including Apache Spark
• Strong experience with Parallel processing frameworks: BigTable/Hadoop
• Strong software engineering fundamentals and proven experience with Scala, Java
• Hands-on experience with Apache Kafka, Iceberg, and Flink.
• Experience with workflow orchestration tools including Apache Airflow and Beam
• Experience with AWS: e.g., S3, EMR, Lambda, Glue, Redshift, BigQuery, Kinesis, or similar services
• Experience with Analytics frameworks including Trino (Presto, BigQuery, Snowflake)
• Hands-on experience with big data lake architectures
• Experience with containerization and orchestration (Docker, Kubernetes/EKS) and CI/CD tooling including Jenkins
• Experience in Python and PySpark
• Familiarity with graph databases such as TigerGraph
• Experience building pipelines that process multimodal data (structured and image) and integrate ML model inference - including LLMs and embedding models - for data enrichment and transformation
• Hands-on experience deploying, serving, and optimizing LLMs or ML models directly in the production, inference runtimes/compilers (ONNX Runtime, TensorRT/TensorRT-LLM), and serving frameworks (Triton, vLLM, TorchServe or similar).
• Experience tuning batching, KV-cache, and GPU utilization for low-latency, high-throughput real-time inference in a data pipeline
• Knowledge of data governance principles, data security best practices, and data privacy regulations
• Proven experience delivering a consumer-oriented solution by participating at every stage of the development life-cycle.
• Excellent communication skills and a collaborative mindset with past experience presenting and partnering with VP and C level decision makers.

Preferred Qualifications

• Experience with data versioning tools and frameworks (e.g., DVC, Delta Lake)
• Experience storing/serving embeddings (e.g., pgvector, Milvus, FAISS)