Senior Software Engineer - IT
Agent Reliability Engineer
The Security Resiliency Organization is rebuilding fragmented, manually-run security processes into consolidated, AI-native workflows that run in production at scale. As the most senior IC on the applied AI team, you will set the engineering standard the rest of the team builds against and own the hardest implementation problems.
Join us as an Agent Reliability Engineer on our Security Resiliency team in Bangalore to do the best work of your career and make a profound social impact.
What you’ll achieve
- You will keep agentic security workflows operating reliably in production, restoring service quickly when performance degrades.
- You will build the observability and performance-measurement layer that shows, in real time, how every workflow is running and what it costs.
- You will make monitoring a routine step for every new workflow rather than a bespoke effort each time.
You will:
- Operate agentic security workflows in production, sustaining availability, completion rate, latency and cost within defined service levels, using performance-measuring tools such as OpenTelemetry, Grafana, Datadog or Splunk.
- Build and operate the observability layer: execution traces across agent steps and tool calls, structured logging, health signals and dashboards, backed by PostgreSQL and Elasticsearch for metrics and log storage.
- Measure token consumption and cost per agent, workflow and run using tools such as LangFuse, reporting actual cost against expectation so overruns are visible early.
- Detect agent-specific failure modes – stalled or looping executions, tool call failures, credential and upstream API changes, silent quality degradation – and respond to restore service on Kubernetes/Docker-based deployments.
- Triage production issues to root cause, escalate defects to the building team with traces and reproduction detail, and maintain runbooks so operational knowledge does not depend on one person.
Take the first step towards your dream career
Every Dell Technologies team member brings something unique to the table. Here’s what we are looking for with this role:
Essential Requirements
- 5+ years in site reliability engineering, production operations or platform engineering, with direct responsibility for systems in production.
- Explicit, hands-on knowledge of performance-measuring and observability tools such as OpenTelemetry, Grafana, Datadog or Splunk – this is mandatory, not preferred, for this role.
- Python (or equivalent) proficiency to build monitoring tooling, automate operational tasks and read the code of the systems being operated.
- Working knowledge of Kubernetes and Docker for container operations, with infrastructure-as-code familiarity, plus PostgreSQL and Elasticsearch for metrics/log storage and querying.
- Cost and consumption tracking for metered services, including LLM token accounting, usage attribution and forecasting against quota or budget.
Desirable Requirements
- Direct experience with LLM tracing tools such as LangFuse for agent-run performance measurement, in addition to general observability platforms.
- Experience with LLM provider rate limits, quota management and multi-provider fallback, and runbook automation or failure-injection testing.
Who we are
We believe that each of us has the power to make an impact. That’s why we put our team members at the center of everything we do. If you’re looking for an opportunity to grow your career with some of the best minds and most advanced tech in the industry, we’re looking for you.
Dell Technologies is a unique family of businesses that helps individuals and organizations transform how they work, live and play. Join us to build a future that works for everyone because Progress Takes All of Us.
Dell Technologies is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment. Read the full Equal Employment Opportunity Policy here.