Ayush Kumar

Ayush Kumar, smiling in sunglasses against a dark background hover to see what the model sees

(01) AI Engineer

LLM & Agentic Systems

Bengaluru, IN

Now · Applied AI, Ola Electric

I build LLM systems that hold up in production, and get a little better every day they run.

Scroll to explore
  1. 01 Noise

    Every system starts as noise.

  2. 02 Structure

    Then you give it structure: retrieval, agents, evals.

  3. 03 Signal

    Until it's something you'd trust with real money.

I'm an AI engineer out of IIT Guwahati. I like the part of LLM work that happens after the demo: evals, failure modes, feedback loops, and the slightly unglamorous plumbing that makes a model trustworthy enough to touch money. I'd rather own a hard problem end to end than a safe slice of an easy one.

  • FocusAgentic RAG, LLM evaluation, multi-agent orchestration
  • StackPython, LangGraph, MCP, FastAPI, PyTorch
  • EducationB.Tech CSE, minor in Data Science & AI · 8.92
  • BaseBengaluru, India
$1M

saved every month by an LLM pipeline that adjudicates warranty claims

67%

fewer daily claims, 1,500 down to 500, once evidence-based auto-rejection closed a fraud loop

50M+

daily autocomplete calls served by the NER model I trained for Ola Maps

4,000+

technicians onboarded onto a CRM we built in 10 days to replace Salesforce

A pipeline that learns from its own mistakes.

Technicians were claiming replacements for healthy parts and reselling the originals. Reviewing 1,500 claims a day by hand didn't scale and didn't catch it. So we built agents that read the evidence the way a senior reviewer would, and a loop that turns every disagreement with a human into a better rulebook.

  1. 01 / Claim

    A claim comes in

    A technician files a replacement with evidence: voltage readings, part diagnostics, SOP videos. The scooter's telematics come along too, often thousands of raw sensor readings.

  2. 02 / Agent

    One agent per part

    Motor, battery, VCU and BMS each get a dedicated LangGraph agent with that part's SOP and failure signatures. I compressed telematics with PAA into a few hundred segments, which cut token cost roughly in half without losing the signal.

  3. 03 / Audit

    Humans audit a sample

    Every day, reviewers check a slice of decisions and flag disagreements: wrong approvals, wrong rejections, and right answers reached for the wrong reason.

  4. 04 / Classify

    Separate signal from noise

    Each disagreement is structured and labelled as a missing rule, an ambiguous rule, two rules in conflict, or a one-off slip. A fix is only proposed once several similar misses cluster together, so the prompt doesn't rot into contradictions.

  5. 05 / Patch

    Patch at the anchor

    The policy lives in one large prompt with anchors marking where each piece of logic sits. Fixes land surgically at an anchor instead of rewriting the whole thing, which is also where a big chunk of the token savings came from.

  6. 06 / Verify

    Golden set, then shadow traffic

    Every patch runs against a golden set: the cases that triggered it plus nearby cases it used to get right. Then a day in shadow on live traffic. A false approval costs real money, so anything that could raise them waits for a human sign-off.

  7. 07 / Ship

    Promote, keep every version

    Passing patches go live, every version is kept for instant rollback, and the loop starts again tomorrow. Agreement with human reviewers went from 86% to 90%, and daily claims dropped from 1,500 to 500.

Ten days to replace Salesforce.

The Salesforce contract was running out and the service network couldn't stop for a migration. Five of us rebuilt the CRM in Spring Boot in ten days: tickets, inventory, billing, insurance and pan-India role-based access, with every historical job card carried over. I owned the warranty and insurance integrations with Reliance, Acko and Ola, each with its own eligibility rules and invoice format.

Day 00 / 10 5 engineers · Spring Boot
  • Tickets
  • Inventory
  • Billing
  • Insurance APIs
  • Pan-India RBAC
  • 5M+ job cards migrated
  • Go live
Then, launch week

So I gave it an on-call engineer.

A CRM built in ten days has a rough first week. Rather than triage every broken record by hand, I built a war-room agent on MCP that reads the codebase and the database, works out why a job card is stuck, fixes the data and raises the code fix as a PR. Anything destructive waits for a human yes in Slack.

0

job cards repaired by the agent

#safai-warroom
  1. alertJC-48213 stuck in INVOICE_PENDING
  2. agentsearch_code("InvoiceService.generate")
  3. agentquery_db("job_cards where status = 'INVOICE_PENDING'")
  4. agentcause: insurer code missing on migrated cards
  5. agentsame pattern on 212 cards
  6. holdwrite touches 212 rows. Asking a human in Slack
  7. humanapproved
  8. agentbackfill done, PR opened: null-safe insurer lookup
  9. done212 job cards moving again
Illustrative trace. Names and numbers simplified.
10days from blank repo to go-live
5M+historical job cards migrated
4,000+technicians onboarded
25Kparts moved through it daily

Agents that argue, then check each other's work.

Argument mining turns prose into a graph of claims, premises and the links between them. The usual answer is a fine-tuned model, which needs piles of labelled data and stumbles on unfamiliar text. With Prof. Ashish Anand, I made the opposite bet: no fine-tuning at all, just three agents and very carefully chosen examples.

ClaimWe must invest in solar energy.
PremiseIt significantly reduces carbon emissions.
  1. 01 / Remember

    Index meaning and shape

    Every training paragraph is stored twice: as an embedding in ChromaDB, and as a discourse tensor from a DMRST parser that captures how the argument is built. Spectral similarity links paragraphs with the same shape into a NetworkX graph, whatever they're about.

  2. 02 / Retrieve

    Find examples that think alike

    A new paragraph pulls its five nearest neighbours by meaning, expands two hops through the structure graph to about fifteen candidates, and keeps the five that score best on a blend of semantic and discourse similarity.

  3. 03 / Argue

    Extract, relate, critique

    One agent extracts claims and premises, one maps supports and attacks, and a reviewer checks the graph for logical holes. If it fails, the feedback goes into a shared state and the agents try again, up to three rounds.

Each change, measured Scores on the AAEC essays benchmark
    TANL
    DENIM

    Grey: GPT-OSS-20B. Orange: GPT-5 Mini. Dashed: fine-tuned state of the art.

    4 / 4metrics above fine-tuned SOTA, zero fine-tuning
    0.649component score, vs 0.621 for DENIM
    81 → 444paragraphs in the extended ARC policy corpus
    LREC2026 resource paper, submitted

    Fifty million searches a day, one address at a time.

    Indian addresses don't follow a format. People type a flat number, a landmark, half a locality and a pincode, in any order. As an intern on Ola Maps I worked on both ends of a ride: finding the place, then predicting when you'll get there.

    Example queries. The model labels each span before search ranks results.
    • Search

      Productionised a spaCy NER address parser for autocomplete and forward geocoding, after benchmarking it against BERT-style transformers. Live, it lifted top-3 search accuracy by 2%.

    • ETA

      Retuned the live historical ETA lookup with Bayesian optimisation in Optuna, on cyclic time features and geospatial ones. 60% of rides now land within 10% of the predicted ETA.

    • Thin data

      Tier-3 cities don't have enough rides to learn from, so I clustered location services and road segments into sub-city zones with DBSCAN. Error there dropped by about 3 minutes.

    • Highways

      Traced a systematic ETA overestimate on long highway segments back to the dataset logic and fixed it, and patched routing anomalies on newly built links.

    • EV pings

      Showed that electric-vehicle ping data was dense enough to feed routing models, and tested bucketing schemes for predicting segment popularity.

    50M+autocomplete calls a day
    +2%top-3 search accuracy
    60%of rides within 10% of ETA
    −3 minETA error in tier-3 cities

    More things I've shipped

    Ayush at IIT Guwahati convocation in a white and red ceremonial outfit, holding his degree

    Indian Institute of Technology Guwahati

    Class of '26

    B.Tech CSE · Minor in Data Science & AI · CGPA 8.92

    Where I've been

    1. Jun 2026–Now

      Ola Electric

      Graduate Engineer Trainee, Applied AI

      The self-improving warranty pipeline and the SAFAI war-room agent. Case studies 01 and 02

    2. Aug 2025–May 2026

      IIT Guwahati

      Undergraduate researcher, agentic argument mining

      Advised by Prof. Ashish Anand. Resource paper submitted to LREC 2026. Case study 03

    3. May–Jul 2025

      Ola Krutrim

      AI Engineer Intern, Ola Maps

      Address NER for autocomplete at 50M+ calls a day, and ETA prediction. Case study 04

    A few wins

    Gold

    Inter-IIT Tech Meet 12

    Rank 1 of 20 IITs on the Mphasis problem statement, with the quantum passenger-recovery engine.

    AIR 2

    EROH Hackathon, Naukri.com

    Out of 20,000+ students from IITs, NITs and IIITs. Won a scholarship.

    4th

    Inter-IIT Tech Meet 13

    Of 22 IITs on the Pathway GenAI problem.

    3rd

    OES ML Challenge

    Out of 5,000 participants. Prize winner.

    Before IIT

    1. 2022Mathematics Olympiad

      Cleared IOQM, selected for the national training camp (INMOTC).

    2. 2022Physics Olympiad

      State topper in IOQP, qualified for INPhO.

    3. 2022Chemistry & Astronomy

      Cleared IOQC and IOQA, qualified for INChO and INAO.

    4. 2022KVPY SX

      All India Rank 863.

    5. 2020CBSE Class X

      98.8%, third in the state.

    What I reach for

    LLMs & agents

    LangGraph, LangChain, MCP, RAG, tool calling, LLM evaluation, prompt optimisation, ChromaDB

    ML & backend

    PyTorch, spaCy, scikit-learn, pandas, NumPy, YOLOv8, FastAPI, REST, Redis, Docker

    Languages

    Python, C/C++, SQL

    I'd rather be on the hook for a harder problem with real autonomy than a safer one with none.