How to Fine-Tune and Deploy Custom Models on Replicate Using Python

by

How to Fine-Tune and Deploy Custom Models on Replicate Using Python

Fine-tuning a model used to mean renting GPUs, wrangling CUDA drivers, and babysitting training runs. Replicate compresses that entire workflow into a handful of API calls. In this tutorial, you’ll fine-tune a model on your own data, deploy it as a private endpoint, and call it from Python — all without touching a single piece of hardware.

This guide is written for developers who want a working pipeline, not a conceptual overview. Every step includes runnable code. By the end, you’ll have a custom model live on Replicate that you can query from any Python application.

[IMAGE: Python code snippet for a Replicate fine tuning tutorial setting up the environment]

Introduction to Replicate Fine-Tuning

Replicate hosts thousands of open-source models behind a unified API, and for a growing number of them — most notably image models like Flux — it also exposes a training API. That means you can submit a dataset, kick off a fine-tuning job, and receive back a new model version that belongs to your account.

The core workflow looks like this:

  1. Prepare a dataset (typically a zip of images, optionally with captions).
  2. Create a destination model on Replicate to hold your fine-tuned weights.
  3. Start a training job via the API, pointing at a trainer model version.
  4. Wait for completion (polling or webhooks).
  5. Run predictions against your new custom model version.

Why Fine-Tune Models via API?

You could fine-tune locally or on a cloud VM, so why use a hosted API?

  • No infrastructure management. No GPU provisioning, no driver mismatches, no out-of-memory debugging at 2 a.m.
  • Reproducibility. Every training run is versioned. You can pin your application to an exact model version hash.
  • Instant deployment. The output of a training job is a deployed endpoint. There’s no separate “serve the model” step.
  • Pay-per-use. You pay for training compute while the job runs and for inference compute when you call the model. Before committing to a training-heavy workflow, it’s worth taking time to understand Replicate API pricing and cold starts so your experiments don’t surprise you on the invoice.

The trade-off is less control over the training loop itself. Replicate’s trainers expose a curated set of hyperparameters rather than arbitrary training code. For most practical fine-tuning tasks — style adaptation, subject-specific LoRAs, domain vocabulary — that’s a feature, not a limitation.

Setting Up Your Python Environment

You need Python 3.8+ and the official Replicate client library.

pip install replicate

Next, grab an API token from your Replicate account settings and export it as an environment variable. Never hardcode tokens in source files.

export REPLICATE_API_TOKEN=r8_your_token_here

Verify your setup with a quick smoke test:

import replicate

# List a page of your account's models to confirm authentication works
for model in replicate.models.list():
    print(model.owner, model.name)
    break

If this runs without an authentication error, you’re ready to train.

Preparing Your Dataset for Fine-Tuning

Dataset quality determines fine-tuning quality. For image LoRA training (the most common Replicate fine-tuning use case), follow these guidelines:

  • Quantity: Start with roughly 10–30 images of your subject or style. More is not always better — consistency matters more than volume.
  • Variety: Include different angles, lighting conditions, and backgrounds so the model learns the subject rather than the scene.
  • Resolution: Use reasonably high-resolution images; avoid heavily compressed or watermarked files.
  • Captions (optional): Many trainers auto-caption your images, but you can supply your own by adding a .txt file with the same base name as each image.

Package everything into a single zip archive:

import zipfile
from pathlib import Path

dataset_dir = Path("training_images")
zip_path = Path("dataset.zip")

with zipfile.ZipFile(zip_path, "w") as zf:
    for file in dataset_dir.iterdir():
        if file.suffix.lower() in {".jpg", ".jpeg", ".png", ".txt"}:
            zf.write(file, arcname=file.name)

print(f"Packaged {zip_path.stat().st_size / 1024:.0f} KB dataset")

Host the zip somewhere the API can fetch it (an S3 presigned URL, a public bucket, or upload it directly as a file object with the Python client — the client accepts a local file handle for training inputs).

Step-by-Step Fine-Tuning Example (e.g., Flux LoRA)

We’ll fine-tune a Flux LoRA — a lightweight adapter that teaches the base image model a new subject or style, identified by a trigger word.

Step 1: Create the destination model. This is the container on Replicate that will hold your fine-tuned versions.

import replicate

model = replicate.models.create(
    owner="your-username",
    name="my-flux-lora",
    visibility="private",
    hardware="gpu-h100",  # hardware used for inference
    description="Flux LoRA fine-tuned on my product photos",
)
print(f"Created destination: {model.owner}/{model.name}")

Step 2: Start the training job. Point the training at the trainer version and your dataset.

training = replicate.trainings.create(
    # The trainer model version (check the trainer's page for the latest hash)
    version="ostris/flux-dev-lora-trainer:<version-hash>",
    input={
        "input_images": open("dataset.zip", "rb"),
        "trigger_word": "MYPRODUCT",
        "steps": 1000,
    },
    destination="your-username/my-flux-lora",
)
print(f"Training started: {training.id} — status: {training.status}")

Step 3: Monitor the run. You can poll the training object until it finishes:

import time

while training.status not in {"succeeded", "failed", "canceled"}:
    time.sleep(30)
    training.reload()
    print(f"Status: {training.status}")

if training.status == "succeeded":
    print(f"New version: {training.output['version']}")

Polling works fine for one-off runs, but for automated retraining pipelines you’ll want to set up async prediction webhooks so Replicate notifies your application the moment a job completes instead of you burning cycles checking status.

Deploying Your Custom Model on Replicate

Here’s the best part: there is no separate deployment step. When the training job succeeds, the resulting weights are published as a new version of your destination model, immediately callable via the API.

Run a prediction against your fine-tuned model:

output = replicate.run(
    "your-username/my-flux-lora:<new-version-hash>",
    input={
        "prompt": "a studio photo of MYPRODUCT on a marble table, soft lighting",
        "num_outputs": 1,
    },
)
print(output)

Note the trigger word MYPRODUCT in the prompt — that’s what activates your fine-tuned concept.

Managing API Keys and Model Visibility

A few operational details matter before you share this with a team:

  • Visibility: Models can be private (only your account/organization can run them) or public. Keep fine-tunes on proprietary data private.
  • Token scope: Use separate API tokens per environment (development, CI, production) so you can revoke one without breaking everything.
  • Secrets management: Load tokens from environment variables or a secrets manager. Rotate any token that has ever appeared in a log or notebook.
  • Version pinning: Always reference the full owner/name:version-hash string in production code. Referencing a model without a version hash can silently pick up new versions after retraining.

Testing the Deployed Model Endpoint

Before wiring the endpoint into an application, validate it systematically:

1. Smoke test with known-good inputs. Run the exact prompt style you trained for and confirm the output reflects your fine-tuned concept.

2. Test edge cases. Try prompts without the trigger word, unusually long prompts, and out-of-domain requests to understand failure modes.

3. Measure latency, including cold starts. The first request after a period of inactivity boots the model onto a GPU, which takes noticeably longer than warm requests:

import time

start = time.time()
output = replicate.run(
    "your-username/my-flux-lora:<version-hash>",
    input={"prompt": "MYPRODUCT product shot, white background"},
)
print(f"Round-trip: {time.time() - start:.1f}s")

Run this a few times in succession and compare the first (cold) call against subsequent (warm) calls.

4. Validate outputs programmatically. For image models, check that files download correctly and dimensions match expectations before passing outputs downstream.

Conclusion & Next Steps

You now have a complete fine-tuning workflow: dataset packaging in Python, training via replicate.trainings.create, automatic deployment to a private versioned endpoint, and structured endpoint testing.

Where to go from here:

  • Automate retraining. Wrap the training call in a script triggered by new data arriving in your storage bucket.
  • Chain your model with others. Feed your fine-tuned model’s output into an upscaler or captioning model in a multi-model pipeline.
  • Harden for production. When you’re ready to move beyond experiments, our guide to transitioning your fine-tuned model to production covers versioning discipline, rate limits, and monitoring.

[IMAGE: Terminal output showing a successful custom model deploy on Replicate via API]

FAQ

How long does fine-tuning take on Replicate?

It depends on the trainer, dataset size, and step count. Small LoRA runs (around 1,000 steps on 10–30 images) typically complete much faster than full fine-tunes. The training object’s status updates in real time, so poll it or attach a webhook rather than guessing.

Do I need my own GPU to fine-tune on Replicate?

No. Training runs entirely on Replicate’s infrastructure. Your local machine only needs Python and the replicate package to submit jobs and retrieve results.

Can I keep my fine-tuned model private?

Yes. Set visibility="private" when creating the destination model. Private models can only be run by your account or organization members with valid API tokens.

What happens if a training job fails?

The training status becomes failed and the object includes error details in its logs. Common causes are malformed zip archives, unsupported image formats, and invalid hyperparameter values. Fix the input and resubmit — you’re only billed for compute actually consumed.

How do I update my model with new data later?

Run a new training job against the same destination model. It produces a new version hash while old versions remain callable, so you can A/B test the retrained model before switching production traffic.

Leave a Comment