We’re thrilled to announce the launch of our migration tooling that enables seamless transitions from self-hosted Temporal instances to Temporal Cloud, now in Pre-release. With this new tool, we aim to make the migration process as smooth and secure as possible while ensuring minimal disruption to your Workflows. This tooling will let you more easily take advantage of the scalability, performance, and simplified operations that Temporal Cloud offers.
The migration tooling has been designed with the following objectives in mind:
- Zero-downtime migrations: Workflows, including long-running Workflows continue running during migration without downtime.
- End-to-end automation: Temporal handles the migration process using internal Workflows.
- No changes to application code: Your existing Workflows, signals, and queries will work without modification.
- Minimal effort on your side: Setup and coordination requirements are reduced, provided your self-hosted Temporal instance supports migration.
- Full control and customization: You decide when to initiate the migration, validate its success, and confirm or abort the process before completion.
It’s important to note that this tooling currently supports migrations from self-hosted Temporal to Temporal Cloud only. Reverse migrations (from Cloud back to self-hosted) are planned in our roadmap!
How It Works
Migration Server
A Temporal Service (aka Server) facilitates secure connections between your self-hosted instance and Temporal Cloud. It acts as the backbone of the migration process.
Migration Proxy
To enhance security, proxies are introduced between your self-hosted server and the migration server: Customer-side proxy: Installed on your infrastructure for secure communication. Cloud-side proxy: Managed by Temporal Cloud.
A step-by-step guide to migration
Here’s how the migration process will work:
- Preparation Phase
- Temporal coordinates with you to set up a migration server, which may take several hours.
- You install a migration proxy to establish secure connections between your self-hosted instance and the migration server.
- Initiating Migration
- Use the
StartMigrationRequestAPI to specify namespaces for migration along with endpoint details. - A corresponding namespace is created in Temporal Cloud with a “non-active” status. You can configure permissions and access controls during this phase.
- Monitoring Progress
- The
GetMigrationResponseAPI allows you to track replication progress, including workflows replicated, remaining workflows, and estimated time for completion.
- Validation and Handover
- Once replication is complete, you can use the
HandoverNamespaceRequestAPI to switch traffic between your source namespace (self-hosted) and target namespace (cloud). - This gives you time to validate that everything is functioning correctly in Temporal Cloud by switching worker traffic and workflow starters incrementally.
- Finalizing or Aborting Migration
- After validation, use
ConfirmMigrationRequestAPI to finalize the migration. -
If issues arise (e.g., incorrect namespace migrated or replication errors), you can abort using
AbortMigrationRequestAPI. Aborting will roll back changes without impacting your workflows. These APIs provide granular control over every step of the process, ensuring transparency and flexibility.
Limitations at Pre-release
While this tooling is robust, there are some limitations to keep in mind:
- Global namespaces are not supported unless specific configurations (e.g., FailoverVersionIncrement) are applied beforehand.
- History shard count must be a multiple of two.
- Payload encryption is not yet supported but is on our roadmap for future releases.
Conclusion
With this new migration tooling, migrating from self-hosted Temporal to Temporal Cloud has never been easier. By automating complex processes and minimizing customer effort, we’re empowering you to focus on building resilient applications while we handle the heavy lifting of infrastructure management.
Ready to migrate to Temporal Cloud? Contact us today to get started!