Staff Software Engineer, Infrastructure

Toast•Published 1 hours ago•First seen 1 hours ago

About the Role

As a Staff Infrastructure Engineer on the Compute & Storage Platform team in Toast, you will design, build, and operate the foundational distributed systems powering the Toast platform as the company expands into further international markets and broadens its product offerings to encompass retail shopping.

You will bridge the gap between core infrastructure and cutting-edge product development—building low-latency state and storage systems, scalable compute clusters, and high-throughput streaming pipelines. Your work will directly enable Toast product delivery across our Product Development Life Cycle (PDLC).

Key Responsibilities

  • Online Databases and Storage: Architect, deploy, and manage low-latency, highly available storage solutions (RDS Postgres and Aurora Postgres; DynamoDB; OpenSearch; S3) to power applications across the Toast product ecosystem.
  • High Availability & Realtime Performance: Ensure sub-second end-user latency, near-zero-downtime reliability, and high throughput across storage engines to support Toast's products across POS, phone, computer, drive-thru, and kiosk interfaces.
  • Infrastructure for AI-Driven PDLC: Automate infrastructure provisioning and compute pipelines.
  • Cost Optimization & Performance Engineering: Right-size compute/storage resources, implement auto-scaling policies, and manage AWS infrastructure costs without compromising platform reliability or latency SLAs.

Required Qualifications

1. Core Infrastructure & Automation

  • Cloud & Infrastructure as Code: Hands-on experience building and managing scalable cloud infrastructure on AWS using Infrastructure as Code (Terraform, CloudFormation, etc.).
  • Software Development: Fluency in Python, Go, or a similar language for infrastructure automation, tooling, or backend development.

2. Storage & Compute Operations

  • Databases: Deep experience operating and tuning enterprise databases at scale. This should include at least one variety of relational database (such as Postgres, Oracle, or MySQL) and at least one type of nonrelational database (such as MongoDB, DynamoDB, Opensearch).
  • Containerization: Experience working with containerized environments and orchestration tools (e.g., Docker, ECS, Kubernetes).

3. Systems Reliability & Observability

  • Solid understanding of Site Reliability Engineering (SRE) principles, including monitoring, alerting, health checks, and performance troubleshooting in high-traffic environments.

Preferred Qualifications

  • Observability & Telemetry with Datadog, Prometheus
  • AI/ML Infrastructure support
  • Cloud Cost Optimization

Our Tech Stack

Toast’s commercial products run on a stack that ranges from guest and restaurant-facing Android tablets to backend microservices, guest- and restaurant-facing web apps, payment systems, event buses, and more.

The Infrastructure Engineering group manages Toast's infrastructure via the AWS API, using configuration management tools like Terraform, Ansible, and internally developed clients using Python and Go. We rely heavily on Datadog and Splunk for monitoring and observability and on FireHydrant for alerting and incident response.

Infrastructure Engineering provides the products and services to support, deploy, and monitor our product suite, which uses a microservice architecture written using JVM and Node.js apps. AWS products are also in heavy usage, ranging from S3 to RDS, DynamoDB, ECS, Lambda, and many others. We have our own platform for user management, service discovery/elevations, and load balancing. We use Apache Spark for large scale data workloads including query and batch processing. The main Toast POS application is an Android application written in Java and Kotlin. For data synchronization between tablets and our cloud platform we operate RabbitMQ and Apache Pulsar clusters as well as direct tablet communication to the applications' RESTful and GraphQL APIs.

Why Join Us?

You will be at the ground floor of building the data-storage systems that powers our flagship products. You'll solve complex, high-scale infrastructure challenges at the intersection of distributed storage and compute.

** This is a hybrid role requiring in-office presence two days per week ***

#LI-HYBRID
#BI-Hybrid

How Toast Uses AI in its Hiring Process

Throughout the hiring process, our goal is to get to know you. We use AI tools to support our recruiters and interviewers with tasks like note-taking, summarization, and documentation of interviews to ensure they can be fully focused on your conversation. All hiring decisions are made by people. We ask that you complete interviews and assessments on your own, without real-time AI tools, recording, or transcription bots, unless we tell you otherwise in advance. To learn more: https://careers.toasttab.com/ai-in-hiring

Our Approach to Hybrid Working

We embrace a hybrid work model that fosters in-person collaboration while valuing individual needs. Our goal is to build a strong culture of connection as we work together to empower the hospitality community, regardless of location. Please visit the Locations page on our career site to learn more about our in-office expectations by region: https://careers.toasttab.com/locations-toast

People Are the Secret Ingredient in Our Recipe for Success

At Toast, our employees are our secret ingredient—when they thrive, we thrive. The restaurant industry brings together people with a wide range of backgrounds, experiences, and perspectives, and we value that same breadth at Toast. We strive to build a culture grounded in authenticity, respect, and humility, where every Toaster has a real opportunity to grow, contribute, and do their best work—raising the bar in delivering exceptional experiences for our customers and each other.

Apply today!

Toast is committed to creating an accessible and inclusive hiring process. As part of this commitment, we strive to provide reasonable accommodations for persons with disabilities to enable them to access the hiring process. If you need an accommodation to access the job application or interview process, please contact candidateaccommodations@toasttab.com.

For roles in the United States, it is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.