S">S"> Sivakumar Portfolio
Sivakumar Portfolio
Software Platform · Voice AI · Multi-Agent · Safety-Critical

Renault IQ Cockpit

Multi-agent AI platform powering the in-car voice experience across Renault's global fleet. Context-aware, learns from every trip, safety-constrained by design. Software orchestration problem deployed in a vehicle to not a hardware story.

Multi-Agent LangGraph HITL Safety Framework Mem0 Persistent Memory Voice-First 5 vehicle lines globally
Platform
Android Automotive OS
Framework
LangGraph · Mem0 · Cerence ASR/TTS
Agents
Supervisor · NAV · CLI · MED · COM
Delivery
FOTA · 5 vehicle lines globally
18.4
DCI baseline
Phase 1 start
7.1
DCI production
−61% from baseline
5.0
DCI Phase 3
target · beta trending
94%
Intent recognition
voice accuracy
87%
Session completion
no manual intervention
22%
Compound intents
P3 beta
North Star Metric
Driver Cognitive Impact (DCI)
Composite 0 to 10 score measuring cognitive burden per session. Weighted function of: interaction count per km, confirmation prompts triggered, voice round-trips per intent, override rate, and time-to-resolution. Why not CSAT? A driver who completes a dangerous interaction may still rate it 5/5. DCI is real-time, measurable, and tied directly to safety outcomes.
What I Own
  • HITL Framework to Designed the 3-tier escalation taxonomy, confirmation modality rules, timeout behaviour, and the code-level architectural boundary that prevents the LLM from approving its own escalation
  • Memory & Personalisation Layer to Mem0 architecture, the 3-layer memory model (structured / episodic / preference), GDPR compliance design, the 90-day rolling window spec
  • Agent Orchestration PRD to LangGraph supervisor graph design, agent capability boundaries, shared state bus spec, compound intent handling
  • Distraction Scorer (DCI) Specification to Metric definition, component weights, real-time calculation spec, threshold management
  • North Star Metric Definition to Chose DCI over CSAT, wrote the rationale, got Paris safety team alignment
Product Roadmap to 4 Phases
Phase 1 · Shipped · Production
HITL Safety Foundation
  • Designed 3-tier escalation taxonomy based on action consequence × reversibility
  • Always-escalate: phone calls, message send, door unlock, emergency SOS
  • Speed-conditional (above 60km/h): route deviation, climate zone change, window control
  • Never-escalate: volume, ETA reading, weather query
  • Confirmation is voice-only, 12-word max enforced, 6-second timeout above 30km/h then silent drop
  • Established DCI as north star metric replacing CSAT
Phase 2 · Shipped · Production
Mem0 Persistent Memory + Destination Prediction
  • Reno had no memory to every journey started from zero
  • Built Mem0 layer: structured memory (home/work/contacts), episodic (90-day rolling trips), preference (implicit learning)
  • On-device destination prediction: 81% top-1 accuracy after 15 trips, <5ms inference
  • Driver override rate: 14% to 86% acceptance confirms model quality
  • Result: DCI dropped from 18.4 → 7.1 (61% reduction)
Phase 3 · Active Beta
LangGraph Multi-Agent Coordination
  • Problem: NAV, CLI, MED, COM agents were powerful but isolated
  • A compound utterance ("navigate to hospital, call Dr. Sharma, lower AC") required 3 separate interactions
  • LangGraph supervisor graph: decomposes intent, dispatches agents in parallel, manages shared state
  • Single HITL confirmation for multi-action compound responses
  • Beta: 22% compound intent share, confirmation timeout rate 11% (vs 14% single-agent to compound reduces load)
  • Target DCI: 5.0
Phase 4 · Planned
Proactive Intelligence to Anticipate Before Prompted
  • Move from reactive (driver asks) to proactive (Reno anticipates)
  • Journey-start prediction with destination and route pre-loaded before driver speaks
  • Context-aware media + climate pre-configuration based on time, route, and passenger count
  • Predictive maintenance alerts surfaced at journey end, not mid-drive
  • Requires Phase 3 data maturity and memory layer confidence baseline above 0.85
System Architecture
Voice Interface
Cerence ASR · on-device + cloud fallbackCerence TTS12-word max enforcementWake word · local inference
Safety & Context Layer
DCI Scorer · real-time compositeSpeed sensor via vehicle APIHITL Decision EngineConfidence threshold · floor 0.72Escalation taxonomy · 3 tiers
LangGraph Orchestration · Phase 3
Supervisor Agent to intent decomposition + routing NAV Agent to navigation, routing, maps CLI Agent to climate, windows, ambient lighting MED Agent to media, music, entertainment COM Agent to calls, messages, contacts Shared state bus to inter-agent coordination
Memory & Personalisation
Mem0 · structured + episodic + preferenceOn-device destination prediction · <5msImplicit preference learning90-day rolling window · GDPR-compliant
Platform & Delivery
Android Automotive OS · AAOS APIsOpenR Link hardware layerFOTA delivery
HITL Framework to Escalation Taxonomy
Tier 1 to Always Escalate
Action required before execution

Triggers: Initiating a phone call, sending a message, unlocking doors, emergency SOS, navigation to a new destination that deviates from current route by >20 minutes.

Behaviour: System proposes. Voice confirmation requested. 12-word max prompt. 6-second timeout at speed >30km/h → silent drop (action does not execute). Driver can re-initiate when safe.

Why: These actions have real-world consequences that cannot be undone quickly. A wrong call initiated, a message sent to the wrong contact, or a door unlocked while moving to all require a gate.

Tier 2 to Speed-Conditional Escalation
Context-dependent gate

Triggers: Route deviation, climate zone change, window control, ambient lighting change.

Behaviour: Above 60km/h → confirmation required (same 12-word, 6-second rule). Below 60km/h → auto-execute. Speed threshold configurable per market.

Why: These actions are reversible but disruptive at speed. Below 60km/h, the cognitive cost of confirmation exceeds the risk of the action.

Tier 3 to Never Escalate
Auto-execute always

Triggers: Volume adjustment, ETA reading, weather query, fuel level check, music skip.

Behaviour: Immediate execution. No confirmation. No DCI impact from the gate (only from the interaction itself).

Why: These actions are fully reversible, low-consequence, and frequent. Confirmation on volume control would make the system unusable.

HITL Step-by-Step Flow
  1. Driver utterance received → ASR processes, intent classified by supervisor agent
  2. Confidence check → below 0.72 → clarification node triggers (one targeted voice question)
  3. Tier classification → action mapped to Always / Conditional / Never based on taxonomy
  4. Speed context read → vehicle API, determines whether Conditional tier requires gate
  5. Gate decision → if gate required: TTS plays ≤12-word confirmation prompt
  6. Timeout window → 6 seconds. Driver responds "yes/confirm" → action executes. No response → silent drop
  7. Execution → agent carries out action, logs to audit trail with: intent, tier, confidence, response, outcome
  8. DCI update → scorer updates composite metric in real time
The Architectural Constraint
The rule that cannot be broken

The LLM agent cannot approve its own HITL escalation. It proposes, waits, and executes only after a human responds to or after the safety engine applies its rule (timeout → silent drop). This is not policy. It is not a prompt instruction. It is a code-level architectural boundary.

The India engineering team implemented this as a hard constraint: the execution pathway for escalation-tier actions is architecturally separated from the LLM inference pathway. The LLM output can only generate a proposal to it has no write access to the execution function until a human confirmation event is received by the safety engine.

The Paris safety team cited this boundary specifically as the reason the HITL framework passed their review without modification. If it were policy or a prompt rule, it could be circumvented by a sufficiently confident model response. The code-level separation makes that impossible.

Architecture Decisions
DecisionChosenRejectedRationale
Confirmation modalityVoice-only · 12-word maxScreen popupScreen interaction is a distraction risk at speed. 12-word maximum enforced at code level.
Timeout at speedSilent dropRepeat promptRepetition at speed increases DCI. Silent drop is the safer default.
Memory storageMem0 · on-device + encrypted syncServer-side onlyOffline capability required. GDPR consent required per data category.
Multi-agent frameworkLangGraph supervisor graphSequential chainingCompound intents (22%) need parallel execution. Chaining = 3× DCI cost.
North star metricDCI · real-time compositeCSATCSAT is retrospective and gameable. DCI is real-time and correlated to safety.