Orchestrating complex customer-defined DAGS with Temporal

Retool uses Temporal to orchestrate customers' "Workflows" – sequences of discrete JS/Python functions, external queries, and human inputs. Temporal allows us to abstract away many realities of infrastructure, like flaky APIs, and enables a rich feature set, like pausing execution mid-sequence and waiting on external triggers.

Our control-flow programming model and multi-tenant environment requires us to augment the Temporal orchestration layer with external intermediate blob storage and a sandboxing solution. Competing requirements for long-running durable Workflows (like waiting for human input) and low-latency API-like Workflows required us to explore the breakdown of latency overhead from Temporal, and design heuristics to optimize for either durability or latency.

In this talk, we will cover the technical problems and solutions implemented to support large payloads in a Temporal context, execute untrusted code, and support a broad range of use cases on the latency vs durability tradeoff curve. We will also cover our journey with common pitfalls with Temporal features – such as retry policies, queries, and timers – along with the opinionated and simplified interface we expose to our users.