Staff Software Engineer – Databases SRE

About

Grafana Labs is looking for a Staff Software Engineer to write code, automation, and product changes that enhance the reliability of multi-tenant SaaS databases across cloud providers.

Requirements

• 8+ years of software engineering experience building and owning distributed systems in production
• Strong production software skills, ideally in at least one object-oriented language
• Deep understanding of distributed systems performance, scaling, and failure modes
• Experience running multi-tenant systems at scale on Kubernetes in AWS, GCP, or Azure using Terraform, Helm, or Jsonnet
• Hands-on experience designing and implementing SLOs
• Experience being on call for production systems and conducting blameless post-incident reviews
• Staff-level technical leadership experience driving cross-team projects
• Working knowledge of Linux internals, networking, and cloud storage

Responsibilities

• Write production code in services, tooling, and Mimir, Loki, and Tempo codebases to make databases resilient and scalable
• Build automation and platform tooling to scale reliability practices across regions and cloud providers
• Author design docs and shape feature designs for scalability and operability
• Define and evolve per-tenant SLOs and reliability models
• Improve observability and alerting to mitigate issues proactively
• Serve on call for highest-SLA customers and lead customer-impacting incident responses
• Investigate customer escalations, follow up on incident actions, and tune capacity/configuration
• Mentor engineers and collaborate with engineering leadership on roadmaps

Benefits

• Equity and bonus (if applicable)
• Global annual leave policy of 30 days per annum, including 3 Grafana Shutdown Days
• 100% remote global work environment
• In-person onboarding