Senior Data Architect

CSS Innovation Studios is seeking Senior Data Engineers/Architects with deep expertise in PySpark and data engineering to deliver strategic AI and data-transformation programs for Oracle’s largest enterprise customers.

In this senior, customer-facing role, you will design, build, modernize, and operationalize enterprise-scale data platforms and AI-driven data solutions on Oracle AI Data Platform (AIDP) and Oracle Cloud Infrastructure (OCI). You will work directly with CIOs, CDOs, CTOs, chief architects, engineering leaders, and Oracle product teams to accelerate AI adoption, modernize legacy data estates, and deliver secure, high-performing production solutions.

The role combines hands-on software and data engineering with technical consulting. You will develop scalable systems supporting generative AI, large language models, real-time analytics, and modern data workloads; lead complex platform implementations and migrations; and partner with architects, product managers, and engineers to apply and evolve Oracle product capabilities through real-world customer deployments.

• 10–12+ years of experience designing, architecting, and delivering enterprise-scale data platforms.
• Proven experience as a Senior Data Engineer, Data Architect, Principal Solution Architect, Consulting Engineer, or technical leader in professional services.
• Expert knowledge of one or more modern cloud data platforms like Databricks
• Extensive hands-on experience with Apache Spark, PySpark, Delta Lake, Python, SQL, Scala or Java, Kafka and modern data ecosystems.
• Exposure to building autonomous agents and orchestration pipelines.
• Deep expertise in Spark architecture, configuration, distributed systems, performance engineering, optimization, and tuning at scale.
• Demonstrated success modernizing enterprise analytics platforms and refactoring Spark and ETL workloads.
• Must have a solid understanding of networking concepts to design and optimize data transmission networks, configure network settings, and diagnose and resolve network-related issues.
• Strong understanding of DataOps, CI/CD, Infrastructure as Code, data security, governance, metadata, lineage, and observability.
• Outstanding executive communication, customer advisory, and stakeholder-management skills.

Responsibilities

Lead architecture and delivery of enterprise Oracle AI Data Platform transformations, including platform implementations and migrations.

• Design and build scalable, governed, AI-ready data architectures, software components, and services for data consolidation, integration, transformation, and data products.
• Develop Lakehouse, Medallion, streaming, and real-time analytics solutions on OCI using technologies such as Spark, PySpark, Delta Lake, Parquet, Flink, and distributed-systems patterns.
• Build AI/ML and Generative AI data pipelines, including feature engineering, feature extraction, MLOps workflows, and vector-ready datasets.
• Assess and modernize workloads by refactoring PySpark and ETL code and optimizing Spark configuration, cluster sizing, partitioning, memory, adaptive query execution, reliability, and performance at scale.
• Develop APIs, SDKs, web applications, and automation frameworks that simplify adoption of Oracle AI Data Platform and Generative AI services.
• Create reusable migration accelerators, reference architectures, automation frameworks, and delivery methodologies.
• Collaborate with engineering, product, and field teams to solve complex technical challenges; share customer insights to inform product roadmaps.
• Serve as a trusted technical advisor throughout customer engagements.
• Mentor senior engineers globally, promote engineering excellence across CSS, and continuously evaluate emerging AI, cloud, and data-engineering practices.