Nordstrom: Orchestrating a Kafka Migration with Temporal

Abstract

Imagine orchestrating the migration of a streaming infrastructure that powers the heart of Nordstrom’s digital operations. We’re talking about a platform that supports 220+ engineering teams, thousands of applications, and a sprawling landscape of Kafka topics and streams.

This isn’t just a migration; it’s a transformational journey from Nordstrom’s self-managed Kafka platform to Confluent Cloud. And at the helm of this epic voyage? Temporal.

In this talk, we’ll dive into how Temporal’s durable execution framework became the linchpin for managing one of the most intricate migration projects at Nordstrom.

We’ll explore the creation of bespoke workflows, each designed to tackle specific facets of the migration - from assessing migration readiness and orchestrating data replication, to coordinating disparate application teams. We’ll share the challenges of coordinating across hundreds of teams, the strategies to ensure zero data loss, and the lessons learned in maintaining operational excellence.

You’ll leave with not just an understanding of Temporal’s capabilities, but a blueprint for leveraging this technology in your own systems. Because when it comes to managing complex, long-running processes, Temporal isn’t just a tool; it’s a game-changer.

About the Presenters

Ari Brown graduated Magna Cum Laude from the University of Illinois at Chicago in December 2019 with a Bachelor’s in Computer Science and a minor in Political Science. Currently, she works as a software engineer on Nordstrom’s Machine Learning Platform team, supporting data scientists in their ML projects. When not coding, she can be found hanging out with her dog and gardening in Seattle’s community gardens.

Jack Burns is a software engineer on Nordstrom’s data streaming platform. Over the past few years, he has worked on Nordstrom’s centralized analytics platform, designing and building data pipelines that enable data scientists, engineers, and analysts. Recently, he transitioned to Nordstrom’s data streaming platform to assist in migrating from Nordstrom’s Kafka platform to Confluent Cloud. When he's not programming and learning, Jack spends time at the boxing gym, drawing, listening to music, and wandering aimlessly around Seattle.