| TL;DR |
|---|
| Temporal’s Release Engineering team built a shared report format and a Nexus Operation so teams’ test Workflows can send results to one reporting service over their existing Temporal connection. Test owners keep their tests and decide what results mean; the platform validates and stores reports so engineers review release evidence in one place. |
Our job on Temporal’s Release Engineering team is to get engineers’ local changes validated and rolled out to Temporal Cloud promptly. For release validation, we wanted one place to inspect test evidence while keeping the tests with their owners.
That evidence came from three kinds of tests run by different teams on different infrastructure. Nightly integration tests cover Temporal’s open source codebase, a second group exercises long-running Workflows, and cloud release validation is the third. Engineers had to find the relevant runs and correlate their results before deciding whether a release was ready.
We were building a new release platform, which gave us the flexibility to define how teams would report results.
The platform needed a common report format and a way to receive results from the teams’ test Workflows. Test owners would keep control of their implementations and decide what their results meant, while our service would validate and store their reports for a shared view.
Why we chose Nexus over a gRPC API for reporting test results#
We chose Nexus so a reporting Workflow could call our service through its existing Temporal connection. Callers use an Endpoint and a named Operation; Temporal routes the request to the Namespace and Task Queue where our Worker is running. Teams do not need the reporting service’s host address or details of its Worker deployment.
We also considered a direct gRPC reporting API called from an Activity. The Activity would still get Temporal’s retries, but callers would need a reachable service with authentication and access controls. Nexus lets producers use their existing Temporal authentication without separate reporting-service credentials. We explicitly allow each caller Namespace on the Endpoint, so existing credentials alone do not grant access. The Nexus security model describes how Temporal brokers the call.
Reading reports can use a different interface. Our terminal client retrieves them through a gRPC API, so an engineer inspecting evidence does not need to make a Nexus call. The same report contract and storage support Workflow reporting through Nexus and access from other clients.
How we built it#
A test owner’s Workflow reaches the reporting Worker through a Nexus Endpoint, and engineers read the stored reports through a separate API.
We define reports using shared protobuf messages. This shortened JSON example keeps the actual field names and uses illustrative values for a failed validation run:
{
"source": "validation",
"business_id": "validation/run-42",
"test_suite": "release-validation",
"verdict": "fail",
"results": [
{
"test_key": "history-rollout",
"verdict": "fail",
"summary": "Validation failed"
}
]
}
Each report and individual test result uses one of four predefined verdicts: pass, warn, fail, or skipped. The test owner’s validation logic determines the outcome, and the reporting code maps it to one of those verdicts. Our platform validates and stores the report so release engineers can review the evidence. The source and business ID identify the report; the test key identifies a result within it. The full format also supports tested components and links to source evidence. With those fields agreed, the platform can validate a report without knowing how the tests work.
A reporting Workflow sends that message to our Operation. This Go excerpt is adapted from our reporting code, with generated names shortened and setup omitted:
client := workflow.NewNexusClient(
"releaser-ingest", "releaser.test-reports",
)
var recorded *RecordTestReportOutput
err := client.ExecuteOperation(
ctx,
RecordTestReportOperation,
report,
workflow.NexusOperationOptions{
ScheduleToCloseTimeout: 2 * time.Minute,
},
).Get(ctx, &recorded)
if err != nil {
return err
}
Temporal routes the request through the Endpoint to the registered Worker. The caller waits for the stored report’s ID and handles the final error if the Operation cannot complete within its limits.
Our handler registers record-test-report as a synchronous Operation:
record := nexus.NewSyncOperation(
"record-test-report",
recordTestReportHandler(recorder),
)
It validates the report and passes its database parameters to the recorder. With mapping and logging omitted, the handler is:
if err := validateReport(input); err != nil {
return nil, err
}
id, err := recorder.RecordTestReport(ctx, params)
if err != nil {
return nil, nexus.NewHandlerErrorf(
nexus.HandlerErrorTypeInternal, "persist test report",
)
}
return &RecordTestReportOutput{TestReportId: uuidString(id)}, nil
The recorder inserts or updates the report using (source, business_id) as its key and replaces the associated result rows in the same transaction. It commits before returning the ID.
If the commit succeeds but the response is lost, the handler may run again: Nexus provides at-least-once execution. Our report key makes that repeated call return the same record ID. We define the key and replacement policy ourselves. Repeated submissions can change the contents, and the current write path does not reject an older payload based on its timestamp.
The read API returns stored reports to the terminal client. For the illustrative payload above, the report detail would contain:
| Report field | Value |
|---|---|
| Source | validation |
| Business ID | validation/run-42 |
| Suite | release-validation |
| Verdict | fail |
| Result | history-rollout: fail, “Validation failed” |
The client can display the same report format for each integration without implementing every team’s test-runner API. The test owner remains responsible for what the evidence means.
Temporal shows what happened to the reporting call. For this synchronous Operation, the caller’s Event History contains NexusOperationScheduled and, on success, NexusOperationCompleted. Pending Operations can expose retry state and the latest failure. The application’s report detail shows the test evidence.
The report comes from the platform-owned producer we used to test ingestion.
The selected test report appears in the shared client alongside other validation runs.
The matching reporting Operation completes and returns the stored report’s ID, while the test report’s verdict remains fail.
A report with verdict fail can have a successfully completed reporting Operation. Our direct open source reporting code also keeps ingestion status separate from the test outcome, so a reporting error does not change the original test verdict.
Lessons learned from connecting teams to a Nexus Endpoint#
We needed to test the reporting path without changing other teams’ Workflows and check which callers could reach the Endpoint.
A temporary producer we owned let us test the contract before asking another team to change its service. It read existing validation results and submitted reports through the same Nexus Operation. We could control the Endpoint tests while leaving the teams’ test Workflows unchanged, with the additional reporting executions running in our own Namespace.
Some test Workflows we wanted to connect were in a different Cloud Account. Temporal Cloud limits Nexus Endpoint access to callers in the same Account, with their Namespaces explicitly allowed. Changing the allowlist could not connect those Workflows, so we also had to consider where they would run. We would check that topology before treating a working handler as a complete integration. The Cloud security documentation describes the Account constraint.
The synchronous writer also puts a deadline on the database work. The full handler request, including its transaction, must finish within ten seconds. The caller’s two-minute overall timeout includes retries and does not extend an individual request. For database work with uncertain latency or availability, Temporal recommends an asynchronous Operation. We still own the handler design and repeated-write behavior when Temporal handles delivery.
One report format for cloud validation and open source tests#
The same report format supports cloud release validation and tests of Temporal’s open source codebase, including nightly integration tests and tests that exercise long-running Workflows. Our team owns cloud release validation; its reporting path reads existing results and submits them through Nexus. The open source test Workflows call the reporting Operation directly when they run in a Namespace configured for it. Their regular schedules still run in a different Namespace.
Both reporting paths use the same validation and storage code. The terminal client displays their reports using the common format.
The client can list reports from different suites, but assembling the evidence for a particular release also requires matching those reports to the software being released. Our release-specific view selects cloud validation evidence; the open source reports still need that matching step. The reports inform a release decision; they do not automatically mark a release ready.
For this platform, Nexus let us keep the reporting service’s deployment details with our team and reuse the producers’ existing Temporal connections. A new producer can call the same Operation once its Namespace has access and its payload meets the contract. Temporal handles delivery retries within the configured limits, while the application determines which reports to accept and how to store them.
Start with one report from a Workflow in an allowed Namespace. The Go Nexus quickstart walks through connecting it to a handler. Inspect the Operation’s outcome and the record returned by your application, then review them with the test owner before expanding the contract or adding more suites.

