Principal BackEnd & Data Engineer — Deep Tech / GenAI Platform (Python)

The Role

We’re looking for a Principal Backend & Data Engineer to own and evolve the core data-intensive backend architecture of our AI platform. This is a senior individual contributor role with broad architectural responsibility and deep hands-on impact.

You will be a technical authority in building scalable, high-performance backend and data systems that power our AI research engine - working closely with AI scientists, platform engineers, and product leaders. Beyond implementation, you will define technical direction, set engineering standards, and influence how we build and scale data-driven AI systems across the organization.

What You’ll Do

Architecture & Technical Ownership

• Own the architecture of core backend and data systems that support large-scale, AI-driven computation and inference.
• Design and evolve high-performance analytical processing layers handling complex data workloads at scale.
• Make and drive long-term architectural decisions across backend, data, and AI integration domains.
• Define standards and best practices for performance, scalability, reliability, and data correctness.

Backend & Data Engineering

• Design, implement, and optimize data-intensive backend services using Python.
• Build and evolve analytical compute pipelines using technologies such as Polars, DuckDB, AirFlow, Iceberg, Spark and other ingestion and processing frameworks.
• Collaborate closely with AI scientists to productionize algorithms and embed advanced intelligence into the platform.

Platform & Infrastructure

• Architect and maintain containerized systems using Docker and Kubernetes.
• Ensure systems are observable, debuggable, and operable in production environments.
• Drive performance tuning and cost-efficiency across compute-heavy workloads.

Technical Leadership

• Act as a technical authority and thought leader within the engineering organization.
• Mentor senior engineers through design reviews, architectural guidance, and hands-on collaboration.
• Serve as a force multiplier by unblocking complex technical challenges and raising the overall engineering bar.

What We’re Looking For

• A passion for building cutting-edge AI and data platforms that solve real, high-impact business problems.
• Significant experience (8+ years) designing, building, and operating large-scale backend and data-intensive systems in production.
• Deep Hands-on expertise in Python and strong understanding of analytical data processing and distributed systems.
• Proven track record of owning and evolving complex system architectures over time.
• Strong experience with containerization and orchestration technologies (Docker, Kubernetes).
• Experience with Infrastructure-as-Code and modern delivery pipelines (e.g., Terraform, ArgoCD).
• Demonstrated technical leadership through architectural ownership, mentorship, and influence across teams.
• Comfort working at the intersection of backend engineering, data engineering, and AI systems.

Why Join SparkBeyond

• Work on deeply technical problems at the frontier of Generative AI and data-driven reasoning.
• Own and shape core platform architecture with real influence and autonomy.
• Collaborate with top-tier AI researchers and engineers in a fast-moving deep-tech environment.
• Build systems that operate at enterprise scale and drive meaningful business impact.