Frank Shaw, Senior Principal Engineer at ZoomInfo, recently shared how adopting Temporal transformed their data processing capabilities, enabling them to handle hundreds of millions of operations monthly while dramatically reducing development time.
Shaw detailed how Temporal's workflow orchestration capabilities helped them evolve from a challenging homegrown solution to a scalable, maintainable system that powers critical business features.
The Challenge: Dynamic Audience Management at Scale
ZoomInfo’s account-based marketing product serves dynamic audiences based on continuously updating data. “We have 90 million companies and about 300 million contacts that we’re constantly gathering new data for and updating,” Shaw explained. The differentiator for their product is the dynamic nature of these audiences — as new companies or contacts enter their database matching audience criteria, they're automatically added, while others may be removed if they no longer fit the parameters.
The team faced significant challenges with their original homegrown solution. “We had built a homegrown solution where this whole process was kind of running in background threads, and then we had another kind of homegrown job scheduler,” Shaw recalled. They encountered throughput limitations and struggled with batch processing. “We were trying to run all of our audiences in one batch at one point, and it was really hard to figure out how to either spread that out throughout the day or really solve those throttling and throughput issues.”
While evaluating solutions including AWS Step Functions and GCP Workflows, Shaw’s team discovered Temporal. The platform’s cloud-agnostic nature particularly appealed to them as they were amid a cloud migration. “Selecting a tool that solves similar problems as these cloud native solutions but in a cloud agnostic manner was really helpful,” Shaw noted.
What set Temporal apart was its workflow-as-code approach. Shaw pointed out that other tools force developers to define workflow logic through complex configuration files. With Temporal, they define workflows using the same programming language they use for other parts of the application. “Seeing how Temporal designed workflow primitives as code, to me it was a no-brainer. It was a much better developer experience for all of our developers,” Shaw explained.
The built-in visibility tools also played a crucial role in their decision. “Out of the box, you get immediate observability into what your workflows are doing and what their current state is. If they fail at a specific step, you see the failure message directly in the workflow, so you can map that to the logs.”
Transformational Results
The impact was immediate and substantial. Before Temporal, the team faced weekly issues requiring extensive debugging. “When we were running on our old infrastructure, I would say probably once a week we had issues where a developer would have to spend almost an entire day debugging, going through logs, figuring out what went wrong,” Shaw shared. After implementing Temporal, these issues virtually disappeared. “Until we add any new audience types, this code is almost in maintenance mode now. It runs on itself.”
“With Temporal, we were able to complete all the necessary migrations — four or five audience types in one two-week sprint.”
The development process also became significantly more efficient. What previously took weeks could now be completed in two days. Shaw recalled a striking example: “We introduced a new audience type and it required us to speed up the migration of all the other audiences. With Temporal, we were able to complete all the necessary migrations — four or five audience types in one two-week sprint.”
“Based on what we’re using now, we’re actually going to be spending about a thousand dollars less a month moving to Temporal Cloud.”
Following the success of this effort, the team saw an opportunity to streamline operations by moving from a self-hosted deployment to Temporal Cloud. The migration decision was driven by both operational efficiency and cost considerations. “Based on what we’re using now, we’re actually going to be spending about a thousand dollars less a month moving to Temporal Cloud,” Shaw explained. “Having to tune that self-hosted cluster to match the usage and not over-provision, it’s really easy to over-provision. And then we would probably be paying more to host our cluster than we are paying to use Temporal Cloud.”
Scaling New Heights with Long-Running Workflows
The robustness of Temporal’s long-running workflows proved particularly valuable when launching their new Copilot product with Account AI feature. “We launched a new product earlier this year called Copilot, and as part of that was a feature called Account AI. We used Temporal to build generative AI–enabled account summaries for our customers,” Shaw explained. The system processes data from multiple sources including email, CRM calendar events, and Zoom meeting recordings.
The scalability has exceeded expectations. “Initially, we were going to launch this in a soft beta for the first six months to a year to about 200 customers,” Shaw said. “We’re about six months in, and we already have over 300 customers just because the product has worked. These pipelines have been able to handle the amount of data that we’re throwing at them.”
The numbers tell a compelling story. “In this new product that’s only been out for about six months, we’re already running 200 million actions per month in Temporal Cloud,” Shaw revealed. “We had to do nothing to the cluster. We’ve worked with our support team to adjust the rate limits and everything within our namespaces, but other than that it's just worked for us.”
ZoomInfo’s Advice to Others
For organizations considering Temporal, Shaw offers straightforward advice: “If you're thinking about it, try it. Just take a small project that you have and just try to run it in Temporal.” He emphasizes not to think of Temporal as just a workflow system. “I’ve even talked to developers at my company where they ask me, ‘Can I use Temporal here?’ And I’m like, ‘Of course you can.’ And they're like, ‘I need responses in less than a second.’”
Shaw points to real examples of Temporal’s versatility: “I pointed them to a place in the application where you click a button, we run a Temporal workflow that fetches data from Google Bigtable, processes all that data together, returns a count of the unique accounts, and it happens in 300 milliseconds. Anything you want to run, you can run it in Temporal if you really want that process to be durable and stable.”
The impact has extended beyond technical capabilities. “The second I see any sort of data pipeline or something, I start to think about how we would build this in Temporal,” Shaw notes. This shift in mindset has enabled his team to focus on innovation rather than infrastructure maintenance.
Shaw particularly appreciates Temporal’s developer experience: “With the local development tools that Temporal provides you, you don’t even have to set up a cloud instance, you don't have to set up a cluster. You can set up and run a Temporal workflow within a day.”
This interview was conducted at Replay 2024, Temporal’s annual conference, where industry leaders gather to share insights and advancements in workflow orchestration.
ZoomInfo’s experience is just one example of how Temporal Cloud can transform workflow orchestration. For teams interested in exploring Temporal’s full potential, $1000 in Temporal Cloud credits are available for a limited time.