Mastering Self-Hosted Temporal Clusters: Achieving High Availability at Salesforce

Abstract

In this session, we will delve into the intricacies of operating self-hosted Temporal clusters, focusing on achieving high availability and cost-efficient scalability. Temporal, an open-source workflow orchestration engine, is renowned for its flexibility and robustness, but mastering its deployment and management in a self-hosted environment requires specialized knowledge and strategies. We would like to share Salesforce’s experience with managing and operating self-hosted Temporal clusters over the past 3 years and our learnings along the way.

We will explore key topics including:

• High Availability: Best practices for configuring Temporal clusters to ensure continuous operation and resilience against failures.
• Scalability: Techniques to efficiently scale your Temporal clusters to handle increasing workloads without compromising performance.
• Cost Efficiency: Strategies to optimize resource usage and minimize operational costs while maintaining high service levels.

Through real-world examples and our experience, attendees will gain a comprehensive understanding of how to effectively manage and optimize their self-hosted Temporal clusters. Whether you are new to Temporal or looking to enhance your current setup, this talk will equip you with the knowledge and tools to achieve a high-performing, secure, and cost-efficient orchestration environment.

About the Presenters

Varun Gupta, VP of Software Engineering at Salesforce, is a seasoned technology executive with 18 years of experience in designing, building, and managing complex distributed systems for modern applications. He is known for leading large engineering teams through transformational projects, focusing on scalability, reliability, and performance. Varun has developed robust distributed architectures, cloud-native solutions, and scalable microservices, earning him multiple patents for his work on large-scale automated systems.

Sahil Vazirani is a seasoned engineer currently serving as a Software Engineering Principal Architect at Salesforce. As the founding engineer who introduced durable execution and Temporal to Salesforce’s technology and product organization, Sahil has been instrumental in driving innovation within the company. With extensive experience in database management, application software development, and infrastructure engineering, he excels in working with multi-disciplinary teams to create robust solutions.