Data & AI Engineer

Sandeep Menon

I own the data path end to end, and I make it trustworthy at scale. From millions of events a day to the dashboards, models and AI features that depend on them.

Data contractsQuality gatesObservabilityGovernanceCost-awareAI-ready data
SCROLL
who

The Engineer

Five years. Billions of events.
No dropped records.
Sandeep Menon — Data Engineer
Data correctness is the product; not a post-launch concern.
I build pipelines that are IdempotentReplayableObservableGovernedCost-awareAI-assisted
0+
Years in production
since 2020
0B
Events / day at peak
across streaming systems
0%
Less effort with AI
25 pipelines migrated
0%
Lower cloud spend
Snowflake, within SLAs

Why I build, and what taught me to build it right.

I got into engineering to build things people actually need. At D.Y. Patil I got to do that again and again: a coordination app for a national-level event our college hosted, and an app that gave villagers one place to reach essential services.

At TCS, honestly, I made mistakes, and every one taught me how systems behave at scale: one bad load can quietly break every report downstream. That's where I understood data engineering isn't just a job. It's understanding how the whole system works, because a lot of people depend on it.

I went to USC to go deeper, into distributed systems, machine learning and where they meet. At Visa I owned several pipelines end to end, from ingestion to the features ML models trained on, and learned that the hard part isn't moving data fast, it's being able to rerun it safely. I built them to be idempotent and made sure data quality was never compromised. At Citi, I build Snowflake platforms and an AI agent that investigates data issues on its own.

All of that changed how I think. I don't just ask how to fix a problem today; I ask how the fix holds up a year from now, when the data has doubled and the people relying on it have changed. I use AI every day to move faster, but nothing ships until I've reviewed it, because if people are going to trust the data, I have to trust the code first.

timeline

The Pipeline

From CS fundamentals to real-time
pipelines — and now AI agents.
Citibank
Jan 2026 — Present
Remote, USA
Current role

Data Engineer · AML

Running automated Python, PySpark and Snowflake pipelines that process 8M+ records a day for AML analytics. Built a Snowflake Cortex AI agent that investigates data-quality anomalies on its own, cutting manual triage 30%; moved 25 on-prem pipelines to Snowflake with AI-assisted conversion (−40% effort); and trimmed Snowflake credit spend 18% without missing an SLA.

SnowflakeCortex AIdbtPySparkSQLAutoSys
Visa
Jul 2024 — Dec 2025
Bellevue, WA
Experience

Data Engineer · Payments & Fraud

Built idempotent Delta Lake pipelines on Azure Databricks for 10M+ card transactions a day from 8+ payment sources, plus Kafka-to-Delta streaming that feeds near-real-time fraud scoring. Moved legacy Hive jobs to PySpark (4 h → 90 min), published leak-free feature tables for fraud models, and added quality gates that cut data incidents 35%.

DatabricksDelta LakePySparkKafkaAirflowUnity CatalogMLflow
University of Southern California
Jan 2023 — Dec 2024
Los Angeles, California
Education

M.S. Computer Science

Advanced study in software systems and machine learning — efficient algorithms, scalable architectures, and ML techniques for complex, data-driven problems. Where data engineering and AI merged for me.

Database SystemsAnalysis of AlgorithmsMachine LearningDeep LearningInformation RetrievalSoftware Architecture
Tata Consultancy Services
Apr 2020 — Nov 2022
Pune, India
Experience

Data Engineer

Built PySpark ETL on AWS EMR joining batch claims, provider and member data with Kinesis-streamed eligibility events — 60M+ records a month into an encrypted S3 lake. Tuned Redshift for 28% faster dashboards, modeled SCD2 star schemas, and used Neo4j graphs to surface potential healthcare collusion.

PySparkAWS EMRKinesisS3RedshiftAthenaNeo4j
D.Y. Patil College of Engineering & Technology
Jun 2016 — May 2020
Kolhapur, Maharashtra, India
Education

B.E. Computer Science & Engineering

Built a strong foundation in CS fundamentals, with a focus on software development and systems engineering.

Data StructuresOperating SystemsSoftware EngineeringArtificial IntelligenceDistributed SystemsData AnalyticsWeb Technologies
projects

Things I build

Data platforms I design and build
end to end, on public or simulated data.
recognition

Recognition

Awards, leadership roles
and a few things off-résumé.
expertise

My Expertise

Stream-first, cloud-agnostic,
warehouse-native.
sandeep@pipeline: ~/stack — init --all
$ stack init --env production --all
Loading full engineering toolkit…
credentials

Certifications

Verified credentials &
completed coursework.
No.CredentialIssuedVerify
01
Microsoft Certified: Azure Databricks Data Engineer Associate
MicrosoftAzureDatabricksData Engineering
[Issued]
Verify credential ↗
02
AWS Certified Data Engineer – Associate
Amazon Web ServicesAWSData EngineeringCloud
[Issued]
Verify credential ↗
03
AWS APAC Solutions Architecture
ForageAWSArchitectureCloudSolutions
Jun 2025
Verify credential ↗
04
Python
HackerRankPythonAlgorithmsData Structures
Jul 2025
Verify credential ↗
05
SQL (Intermediate)
HackerRankSQLDatabaseQueries
Jul 2025
Verify credential ↗
notes

Writings

Hard-won lessons from
pipelines under pressure.
contact

Let's connect!

Tweaks ✕

Accent
Blueprint grid
Motion