Distinguished Engineer - Data Architect

Job Title: Distinguished Engineer - Data Architect

Role Overview:

We are looking for a Distinguished Engineer- Data Architect to design and build the data foundation for our AI-driven security platform. In this role, you will own the architectural blueprint and core implementation of our data strategy, transforming massive streams of high-velocity network traffic logs, customer configurations, and security alerts into high-performance, ML-ready data fabrics.

As part of a lean, elite three-person Data & AI team, you will serve as the ultimate technical authority on data scale, streaming architecture, multi-tenant security, and cost-efficient storage systems.

Key Responsibilities:

• Architect the Core Data Platform: Design, implement, and evolve a unified blueprint for streaming and batch data processing capable of ingesting, normalizing, and storing petabyte-scale, high-velocity security telemetry.
• Enable AI & Advanced Analytics: Partner directly with the Lead AI Engineer to architect the underlying storage, retrieval systems, and feature stores—including optimal vector database structures and data lakehouse layers—required for low-latency AI inference and training.
• Enforce Zero-Trust Data Governance: Architect strict multi-tenant isolation, data privacy guardrails, and compliance controls directly into the data fabric to protect sensitive customer network configurations and logs.
• Optimize Economics & Performance: Drive infrastructure cost-efficiency and performance tuning across data processing engines, storage layers, and cloud infrastructure, balancing ultra-low latency requirements with predictable cost scaling.
• Lead by Example: Write clean, foundational, high-performance code for critical infrastructure components, setting the standard for engineering excellence within the team.

 

Required Experience & Skills:

• Experience: 12+ years of data engineering and architecture experience, with a proven track record of designing production systems handling massive, high-throughput streaming datasets (e.g., networking logs, telemetry, or real-time event streams).
• Distributed Systems Expertise: Deep, authoritative knowledge of distributed processing and streaming engines (e.g., Apache Flink, Spark, Kafka, Pulsar) and modern lakehouse architectures (e.g., Iceberg, Delta Lake).
• Hybrid Storage Architecture: Mastery of polyglot persistence, spanning traditional relational databases, NoSQL, Columnar engines (e.g., ClickHouse, DuckDB), and Vector databases (e.g., Milvus, Qdrant, Pinecone).
• Security & Infrastructure: Strong fundamentals in secure data architectures, network topography, encryption-at-rest/in-transit, containerization (Kubernetes), and cloud-native infrastructure (AWS/GCP/Azure).
• Mindset: A rare combination of visionary architecture and hands-on execution. You thrive in ultra-lean teams, enjoy writing core code, and possess the pragmatism to build for immediate needs while planning for massive scale.

#LI-DS9

#LI-Pune

#LI-Hybrid

#LI-DataArchitect

#LI-DataPlatform

#LI-Dataarchitecture