Integrating Replicate Webhooks for Async AI Automation

by

Integrating Replicate Webhooks for Async AI Automation

GPU inference is slow relative to normal web requests. An image generation might take ten seconds; a fine-tuning job might take an hour. If your application sits in a loop asking “is it done yet?”, you’re burning worker capacity and adding latency to every downstream action. Webhooks flip the model: Replicate calls you the instant a job changes state, and your automation fires immediately.

This guide covers when polling is still the right tool, how to set up async prediction webhooks, building a production-shaped webhook listener in Python with FastAPI, triggering downstream automation, and securing the whole path.

[IMAGE: Flowchart of a Replicate webhook trigger automation catching a job completion]

Polling vs. Webhooks in the Replicate API

Replicate gives you two ways to learn a prediction’s fate:

Polling — create a prediction, then repeatedly fetch its status until it reaches a terminal state (succeeded, failed, canceled):

import time
import replicate

prediction = replicate.predictions.create(
    model="black-forest-labs/flux-schnell",
    input={"prompt": "a lighthouse in a thunderstorm"},
)

while prediction.status not in {"succeeded", "failed", "canceled"}:
    time.sleep(2)
    prediction.reload()

print(prediction.status, prediction.output)

Webhooks — pass a webhook URL when creating the prediction, and Replicate POSTs the prediction object to your endpoint as the job progresses and completes:

prediction = replicate.predictions.create(
    model="black-forest-labs/flux-schnell",
    input={"prompt": "a lighthouse in a thunderstorm"},
    webhook="https://api.yourapp.com/webhooks/replicate",
    webhook_events_filter=["completed"],
)
# Your code returns immediately; Replicate calls you back.

The trade-offs are straightforward: polling is simple and works anywhere (including scripts and notebooks with no public URL), but it holds a worker hostage, adds up to a full polling interval of extra latency, and scales badly. Webhooks require a publicly reachable HTTPS endpoint and idempotent handling, but they free your workers entirely and react within moments of completion. Tight polling loops also quietly waste your own infrastructure — see how idle polling increases your compute costs when workflows scale up.

When to use Prediction Status Polling

Polling remains the right choice when:

  • You’re in a script or notebook. No server, no public URL, no problem.
  • The job is short. For a 3-second model, a 2-second poll interval is perfectly fine.
  • You need a synchronous result inside one request. Sometimes the caller genuinely must wait.
  • You’re behind a firewall that can’t accept inbound traffic and can’t use a tunnel.

For everything long-running, high-volume, or automation-triggering, use webhooks.

Setting Up Async Prediction Webhooks

The async pattern uses replicate.predictions.create() (which returns immediately) instead of the blocking replicate.run(). Three parameters matter:

  • webhook — your HTTPS endpoint. It must be publicly reachable; during development, a tunneling tool (e.g., ngrok) exposes your local server.
  • webhook_events_filter — which events to receive: start, output, logs, and completed. For most automation, ["completed"] is all you need; subscribing to everything multiplies traffic without adding value.
  • Your own correlation ID — store prediction.id against your internal job record at creation time, so the callback can be matched to the work that spawned it.
import replicate

def submit_job(internal_job_id: str, prompt: str) -> str:
    prediction = replicate.predictions.create(
        model="black-forest-labs/flux-schnell",
        input={"prompt": prompt},
        webhook=f"https://api.yourapp.com/webhooks/replicate",
        webhook_events_filter=["completed"],
    )
    # Persist the mapping BEFORE returning — the webhook may arrive fast.
    save_mapping(prediction_id=prediction.id, job_id=internal_job_id)
    return prediction.id

That comment is not decorative: for fast models, the completion webhook can arrive before your own database write finishes if you defer it. Persist first.

Webhooks work for training jobs too — the same pattern lets you receive notifications when fine-tuning jobs finish rather than polling an hour-long run.

Job Complete Callback Integration in Python

When the prediction completes, Replicate POSTs a JSON body that mirrors the prediction object: id, status, input, output, error, logs, and metrics. Your listener’s contract is:

  1. Acknowledge fast. Return a 2xx immediately; do heavy work asynchronously. Slow handlers cause timeouts and redeliveries.
  2. Be idempotent. Webhooks can be delivered more than once. Processing the same id twice must not duplicate side effects.
  3. Check the status field. completed events fire for failures too — status may be failed, with details in error.

Creating a Webhook Listener (Flask or FastAPI)

Here’s a FastAPI listener implementing that contract:

from fastapi import BackgroundTasks, FastAPI, Request, Response

app = FastAPI()
processed_ids: set[str] = set()  # use Redis/DB in production


@app.post("/webhooks/replicate")
async def replicate_webhook(request: Request, background: BackgroundTasks):
    payload = await request.json()
    prediction_id = payload.get("id")
    status = payload.get("status")

    # Idempotency: skip anything we've already handled.
    if prediction_id in processed_ids:
        return Response(status_code=200)
    processed_ids.add(prediction_id)

    # Acknowledge immediately; do real work in the background.
    background.add_task(handle_prediction, prediction_id, status, payload)
    return Response(status_code=200)


def handle_prediction(prediction_id: str, status: str, payload: dict):
    job = lookup_job(prediction_id)  # your correlation mapping

    if status == "succeeded":
        output = payload.get("output")
        store_result(job, output)
        trigger_downstream(job, output)
    elif status == "failed":
        log_failure(job, payload.get("error"))
        maybe_retry(job)

The equivalent Flask version is nearly identical — a @app.route("/webhooks/replicate", methods=["POST"]) handler that enqueues work (e.g., to Celery or RQ) and returns 200. The framework matters far less than the acknowledge-fast, process-async, idempotent shape.

[IMAGE: Python FastAPI snippet for a Replicate async prediction webhook listener]

Triggering Downstream Automation

The webhook handler is where automation begins, not ends. Common downstream triggers:

  • Chaining the next model. The completion of stage N submits stage N+1. This is exactly how event-driven pipelines work — the webhook for a generated image is what starts triggering subsequent models in a chaining pipeline, keeping no worker blocked at any point.
  • Notifying users. Push a websocket event, send an email, or update a job-status record that your frontend polls (polling your own database is cheap).
  • Persisting artifacts. Download output files to your own storage promptly — treat Replicate’s output URLs as temporary and copy anything you need to keep.
  • Feeding queues. Publish a message to SQS/Pub/Sub/Rabbit so any number of consumers can react without coupling them to the webhook endpoint.

A useful discipline: keep the handler itself dumb. It validates, records, and emits an event. Business logic lives in consumers of that event, which you can test, retry, and evolve independently of the HTTP surface.

Security Best Practices for Webhooks

A webhook endpoint is a public door into your system. Lock it down:

  • Verify webhook signatures. Replicate signs webhook deliveries; validate the signature headers against your webhook signing secret (available via the API) before trusting any payload. Reject anything that fails verification. This is the single most important control — without it, anyone who discovers your URL can forge “completed” events.
  • HTTPS only. Never register a plain HTTP callback.
  • Treat the payload as untrusted input. Even with a valid signature, validate types and ranges before using values in queries or file paths.
  • Re-fetch when in doubt. For high-stakes actions, use the webhook as a hint and fetch the prediction by ID from the API as the source of truth.
  • Don’t leak through the URL. Avoid embedding secrets or user identifiers in the callback URL itself; use the prediction-ID mapping instead.
  • Rate-limit and alert. Unusual delivery volume to your endpoint should page someone — it’s either a bug or a probe.

Hardening beyond the endpoint — retries, dead-letter queues, and multi-instance listeners — is part of production readiness for webhook listeners.

Conclusion

Polling answers “is it done yet?”; webhooks answer “it’s done — act now.” For any Replicate workload that is long-running, high-volume, or feeds further automation, the async-create-plus-webhook pattern frees your workers, cuts reaction latency to near zero, and turns your ML workflow into a clean event-driven system. Build the listener with the three-part contract — acknowledge fast, process idempotently, verify signatures — and downstream automation becomes a set of small, testable event consumers rather than a tangle of sleep loops.

FAQ

How do I trigger automation when a Replicate job completes?

Create the prediction with replicate.predictions.create(), passing a webhook URL and webhook_events_filter=["completed"]. Replicate POSTs the prediction object to your endpoint when the job finishes, and your handler triggers the downstream action — chaining another model, notifying a user, or publishing to a queue.

Do Replicate webhooks work for fine-tuning jobs as well as predictions?

Yes. Training jobs support the same webhook mechanism, which is especially valuable because fine-tuning runs are long — polling an hour-long job wastes resources that a single callback replaces.

What happens if my webhook endpoint is down when Replicate calls it?

Design for missed deliveries: make handlers idempotent, and run a periodic reconciliation task that queries the API for any in-flight predictions whose callbacks never arrived. For critical workflows, treat the webhook as a latency optimization and reconciliation as the guarantee.

How do I test Replicate webhooks locally?

Expose your local server with a tunneling tool such as ngrok, register the tunnel’s HTTPS URL as the webhook, and run real predictions against it. Log full payloads during development so you understand the exact shape of the events you’ll receive.

Should I subscribe to all webhook events or just “completed”?

For most automation, ["completed"] is enough — it fires on success, failure, and cancellation with the final prediction state. Subscribe to start, output, or logs only if you’re building progress UIs or streaming partial output, since every extra event multiplies inbound traffic.

Leave a Comment