Temporal Schedules offer a major improvement in reliable and efficient scheduling systems. This feature is designed not only as an alternative to traditional Cron jobs but also as a replacement for scheduling systems like systemd timers and Celery.
More explicitly, Temporal Schedules:
- Allow developers to execute workflows at specific times or intervals, and enhance control and observability over these tasks. This approach eases management of task operations, such as starting, pausing, resuming, backfilling, deleting, describing, listing, triggering, and updating scheduled workflow executions.
- Implifies workflow management and provides a durable and more informed solution for task scheduling, treating tasks as independent entities with detailed specifications for each schedule.
Support systematic and efficient workflow management. From limiting runs and configuring overlap policies to enabling pause-on-failure.
The good news is, switching to Schedules is quite straightforward. If you are looking to convert your cron jobs into Schedules, check out this blog post.
Integration of Temporal Schedules with Non-Temporal Scripts
Integrating Temporal Schedules with existing non-Temporal scripts can be achieved by wrapping your current code in an Activity, which is then placed within a single-activity Workflow. This strategy allows for a straightforward transition to using Temporal as a Cron replacement without extensive code rewrites. It's essential to ensure the activity is idempotent or to disable the retry policy to prevent any potential issues, enabling a straightforward application of Temporal Schedules to your existing workflows.
Migration Preparation
The first step is to understand your existing scheduled jobs and how they are triggered.
- What is their schedule now? Is it per time interval, like day or hour, or does it run on a specific day of the week or month?
- Can your jobs overlap?
- Do they need to backfill if the schedule is interrupted?
- Should they pause the schedule if a job fails?
The second step is to write a Temporal Workflow that executes the existing batch job. You can start by wrapping an existing job task in a single-Activity Workflow as mentioned above. Alternatively, you can implement a job as a Temporal Workflow, to take advantage of Temporal Workflow capabilities, like retries for individual steps.
The final step is to deploy your workflow in a Temporal Worker that will allow it to execute on the schedule you specify. You’ll need the task queue name, Workflow ID, and Workflow Type (Name). One good practice is to deploy the job as a “shadow” to make sure it runs how you want. Implement a feature flag to have it log that it has started and run as expected on your schedule, and then when you are ready activate it and disable your old schedule.
Example: Migration from Kubernetes CronJobs to Temporal
Once you have your workflow deployed, you can schedule it as desired. Review your Kubernetes Cron schedule and implement it in Temporal. Let’s say you have a Kubernetes Cron job that takes a database backup every Friday at 12:15PM. Here’s a YAML snippet:
apiVersion: batch/v1
kind: CronJob
metadata:
name: rdb-backup
spec:
schedule: "15 12 * * 3"
It will run at 12:15 PM every Friday.
Temporal Schedules are much simpler to understand:
You can also create them using any of the SDKs or via the command line. Here is an example using the command line:
temporal schedule create \
--schedule-id 'friday-rdb-database-backup' \
--calendar '{"dayOfWeek":"Fri","hour":"12","minute":"15"}' \
--overlap-policy 'BufferAll' \
--workflow-id 'rdb-backup' \
--task-queue 'rdb-backup-task-queue' \
--workflow-type 'RDB-Backup'
Temporal Schedules are similar to Kubernetes Cron but far more powerful. With Schedules, you can do additional things, such as backfill, describe, list, pause, trigger, and update a Workflow Execution schedule with ease. You can also set an interval if you desire:
--interval '5h/15m' \
The interval parameter defines at what interval the Schedule runs. The only thing left to do is disable the Kubernetes Cron job once you have enabled the Schedule in Temporal. Then you can benefit from the power of Temporal Schedules and Workflows as mentioned above. Because Workflows are implemented in code, you can manage them like applications, with a full software development life cycle, versioning, testing, and change management, enabling strong DevOps patterns.
Example: Migration from systemd Timers to Temporal
As with Kubernetes Cron jobs, once you have your workflow deployed, you can schedule it in Temporal as desired. Review your systemd schedule and implement it in Temporal. Let’s say you have a systemd schedule that takes a database backup every Friday at 12:15PM:
$ sudo cat /etc/systemd/system/rdb-backup.timer
[Unit]
Description=Timer to run rdb-backup.service
[Timer]
Unit=rdb-backup.service
OnCalendar=Fri *-*-* 12:15:*
Persistent=True
[Install]
WantedBy=timers.target
It will run at 12:15 PM every Friday.
As above, you can implement this schedule in the Temporal UI or using the command line. systemd jobs run on specific hosts, and your Temporal Workflows can run on specific hosts, or across a fleet of hosts. It’s important to note that when implementing Temporal Workflows to replace systemd jobs, you must make sure that the Temporal Workflow has access to the host you want to run automation on. You can also run workers on the hosts that formerly ran the systemd jobs.
Once you have implemented the Temporal Schedule, you will have much better visibility for the automation - you can see every run in the Temporal UI, and you don’t have to be logged into the host to see automation logs.
The only thing left to do is disable the systemd job once you have enabled the Schedule in Temporal. At this point you can benefit from the power of Temporal Schedules and Workflows as mentioned above. Because Workflows are implemented in code, you can manage them like applications, with a full software development life cycle, versioning, testing, and change management, enabling strong DevOps patterns.
Example: Migration from Quartz to Temporal
Quartz schedules are implemented in Java code, such as with the simple schedule definition below:
Job Definition:
// A simple job
public class HelloJob implements Job {
public void execute(JobExecutionContext arg0) throws JobExecutionException {
System.out.println("Hello! It's been a few hours!");
}
}Group Definition:
// add job to a group
JobDetail job = JobBuilder.newJob(HelloJob.class)
.withIdentity("helloJob", "jobGroup1")
.build();Trigger Definition:
Trigger trigger = TriggerBuilder.newTrigger()
.withIdentity("myTrigger", "jobGroup1")
.startNow()
.withSchedule(SimpleScheduleBuilder.simpleSchedule()
.withIntervalInHours(10)
.repeatForever())
.build();
This job prints “Hello! It's been a few hours!” every ten hours, forever.
There are many other options for Quartz jobs, such as CronTriggers that map well to the calendar based jobs discussed above. There is also Quartz misfire configuration that maps in functionality to Temporal’s catchup window. For this blog post, we’ll work with this simple example. As mentioned above, once you have your workflow deployed, you can schedule it as desired. Schedules can be implemented in the Temporal UI or using the command line. Here is a command line example for this ten hour interval schedule:
temporal schedule create \
--schedule-id 'helloJob' \
--interval '10h' \
--overlap-policy 'BufferAll' \
--workflow-id 'helloJob' \
--task-queue 'hello-job-task-queue' \
--workflow-type 'HelloJob'
Here is an example for this ten hour interval schedule set up with Java:
Schedule schedule =
Schedule.newBuilder()
.setAction(
ScheduleActionStartWorkflow.newBuilder()
.setWorkflowType(HelloSchedules.GreetingWorkflow.class)
.setOptions(
WorkflowOptions.newBuilder()
.setWorkflowId("helloJob")
.setTaskQueue("hello-job-task-queue")
.build())
.build())
.setPolicy(SchedulePolicy.newBuilder()
.setOverlap(ScheduleOverlapPolicy.SCHEDULE_OVERLAP_POLICY_BUFFER_ALL)
.build())
// Run the schedule every 10 hours
.setIntervals(Collections.singletonList(new ScheduleIntervalSpec(Duration.ofHours(10))))
.build();
// Create a schedule on the server
ScheduleHandle handle =
scheduleClient.createSchedule("helloJob", schedule, ScheduleOptions.newBuilder().build());
To convert to Temporal Schedules, you can keep the core of the job in Java, or use any of the other languages supported by Temporal, such as Go or Python. The configuration of the Quartz job management can be removed from your Java code when you are ready to disable the Quartz version.
At this point you can benefit from the power of Temporal Schedules and Workflows as mentioned above. Conversion will be familiar in some ways, because Temporal Workers are applications, just like your Quartz job applications. The key difference is job configuration is external as part of the Temporal platform.
Example: Migration from Celery to Temporal
Celery beat is a scheduler for python. Here’s a sample job:
from celery import Celery
from celery.schedules import crontab
app = Celery()
@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
#calls hello every hour
sender.add_periodic_task(3600.0, hello.s('hello i am alive'), name='task every hour')
#calls hello every Tuesday 5:30 p.m.
sender.add_periodic_task(
crontab(hour=17, minute=30, day_of_week=2),
hello.s('Time for Tuesday Dinner!'),
)
@app.task
def hello(arg):
print(arg)These two jobs have different schedules - one periodically by time and one on a crontab schedule. They could be implemented as one Temporal Workflow with two different schedule configurations.
Once you have your workflow created and deployed, you can schedule it as desired. Schedules can be implemented in the Temporal UI or using the command line. Here is a command line example for the one hour interval schedule:
temporal schedule create \
--schedule-id 'helloHour' \
--interval '1h' \
--input '"hello i am alive"' \
--overlap-policy 'BufferAll' \
--workflow-id 'helloHour' \
--task-queue 'hello-job-task-queue' \
--workflow-type 'HelloJob'
Here is a command line example for the Tuesday schedule:
temporal schedule create \
--schedule-id 'helloTuesday' \
--calendar '{"dayOfWeek":"Tue","hour":"17","minute":"30"}' \
--input '"Time for Tuesday Dinner!"' \
--overlap-policy 'BufferAll' \
--workflow-id 'helloTuesday' \
--task-queue 'hello-job-task-queue' \
--workflow-type 'HelloJob'
Here is a Python SDK example for creating the every hour schedule:
#...
async def main():
client = await Client.connect("localhost:7233")
await client.create_schedule(
"helloHour",
Schedule(
action=ScheduleActionStartWorkflow(
HelloJob.run,
"hello i am alive",
id="helloHour",
task_queue="hello-job-task-queue",
),
spec=ScheduleSpec(
intervals=[ScheduleIntervalSpec(every=timedelta(hours=1))]
),
policy=SchedulePolicy(
overlap=ScheduleOverlapPolicy.BUFFER_ALL
)
),
)To convert python jobs to Temporal Schedules, you can keep the core of the job in python, or use any of the other languages supported by Temporal, such as Go or Java. The configuration of the Celery job management can be removed from your code when you are ready to disable the Celery version.
At this point you can benefit from using Temporal Schedules and Workflows as mentioned above. Conversion will be familiar in some ways, because Temporal Workers are applications, just like your Celery job workers might be. The key difference is job configuration is external as part of the Temporal platform.
Other Job Schedulers
Temporal Schedules provide a framework for many kinds of background processing. Conversion of your code to Temporal makes it more durable, and Temporal Schedules provide an easy way to execute repeating processes clearly. There are other job schedulers not mentioned here that Temporal could replace.
Temporal also can facilitate complex job streams, with many layers of nested job processing orchestration and complex dependencies. Temporal Workflows can be as complex as needed to process your work streams.
Learn more
Schedules Documentation Ask questions in the Temporal Community Slack Watch Schedules webinar: Simplifying and Scaling Cron Jobs with Temporal Schedules How Temporal works Subscribe to Temporal newsletter