Building Specific Features with Ollama in Python: Text, Chat & Embeddings

by

Building Specific Features with Ollama in Python: Text, Chat & Embeddings

Modern software engineering with Large Language Models extends far beyond simple single-prompt text generation. Today’s production local AI architectures require multi-turn conversational agents, real-time context management, vector similarity search, and automated retrieval-augmented generation (RAG) pipelines.

When building features powered by local LLMs, understanding how to configure specific execution modes in Python is essential. This guide provides comprehensive, production-ready implementations for three foundational local AI tasks using Ollama:
1. Text Completion: Executing customized Ollama generate text Python tasks with fine-tuned hyperparameters.
2. Conversational Interfaces: Building interactive Ollama chat Python applications with active conversation memory.
3. Vector Embeddings: Creating dense numerical representations with Ollama embeddings Python pipelines for semantic retrieval.

How to Generate Text Using Ollama and Python

Single-turn text completion forms the core building block for tasks like content summarization, automated code translation, entity extraction, and sentiment analysis. Using Ollama, software developers can trigger generation requests programmatically with precise control over model parameters.

[IMAGE: Terminal output showing Ollama generate text Python results]

Basic Text Completion Parameters

When calling the generation endpoint via ollama.generate(), fine-tuning parameter options enables software teams to balance creativity, response length, and deterministic precision:

  • temperature: Controls randomness in token sampling. Lower values (e.g., 0.1–0.3) produce highly deterministic, focused outputs ideal for code generation or factual extraction. Higher values (e.g., 0.7–0.9) encourage creative variance.
  • top_p (Nucleus Sampling): Limits token choices to a cumulative probability threshold.
  • num_predict (Max Tokens): Controls the maximum number of tokens generated in the output response.
  • stop: Defines custom stopping sequences (e.g., ["\n\n", "User:"]) that immediately halt token generation.

Below is a complete Python script demonstrating how to configure advanced completion parameters for a text generation pipeline:

import sys
import ollama

def execute_custom_text_generation(
    prompt: str,
    model: str = "llama3",
    temperature: float = 0.2,
    max_tokens: int = 250
) -> str:
    """
    Executes a fine-tuned text completion request using Ollama generate text Python mechanics.
    """
    print(f"Executing text generation using model '{model}'...")

    try:
        response = ollama.generate(
            model=model,
            prompt=prompt,
            system="You are a precise technical writing assistant focused on concise explanations.",
            options={
                "temperature": temperature,
                "top_p": 0.9,
                "num_predict": max_tokens,
                "stop": ["\n\nTask Complete:"]
            }
        )
        return response.get('response', '')
    except Exception as error:
        print(f"Error during text generation: {error}", file=sys.stderr)
        return ""

if __name__ == "__main__":
    sample_prompt = "Summarize three core benefits of microservices architecture."
    result = execute_custom_text_generation(sample_prompt)
    print("\nGenerated Text Output:\n")
    print(result)

To review general client initializations or underlying connection settings, consult our comprehensive guide to the Ollama Python library.

How to Build a Chat Application with Ollama in Python

Unlike single-turn text completion, building interactive chat interfaces requires maintaining dialogue state across multiple conversational turns. The Ollama chat Python interface accepts structured message arrays, allowing developers to represent system, user, and assistant role histories effortlessly.

Managing Context in Ollama Chat via Python

An LLM is inherently stateless. To create the illusion of continuous conversation, your application code must accumulate each turn of the dialogue inside a structured history array and pass the full message list back to the model with every new request.

A standard message object follows this key-value structure:

{"role": "user", "content": "What is Python?"}
{"role": "assistant", "content": "Python is a high-level programming language..."}

Below is a production-ready, interactive command-line chat application implementation featuring multi-turn conversation memory and graceful exit handling:

from typing import List, Dict
import ollama

class LocalChatSession:
    def __init__(self, model_name: str = "llama3", system_instruction: str = None):
        self.model_name = model_name
        self.messages: List[Dict[str, str]] = []

        # Initialize conversation with optional system instruction
        if system_instruction:
            self.messages.append({
                "role": "system",
                "content": system_instruction
            })

    def send_message(self, user_input: str) -> str:
        """
        Appends user input to history, invokes Ollama chat, and records assistant response.
        """
        # Append user message turn
        self.messages.append({"role": "user", "content": user_input})

        try:
            # Trigger Ollama chat invocation with full message context
            response = ollama.chat(
                model=self.model_name,
                messages=self.messages
            )

            assistant_reply = response['message']['content']

            # Append assistant response turn to preserve chat context
            self.messages.append({"role": "assistant", "content": assistant_reply})
            return assistant_reply

        except Exception as err:
            error_msg = f"Chat failure: {err}"
            print(error_msg)
            return "Apologies, an internal error occurred while processing your message."

def run_interactive_cli():
    print("==================================================")
    print("  Ollama Local CLI Chat (Type 'exit' to quit)     ")
    print("==================================================")

    session = LocalChatSession(
        model_name="llama3",
        system_instruction="You are a helpful software tutor. Provide clear code examples."
    )

    while True:
        try:
            user_text = input("\nYou: ").strip()
            if not user_text:
                continue
            if user_text.lower() in ["exit", "quit"]:
                print("\nEnding chat session. Goodbye!")
                break

            print("Assistant: ", end="", flush=True)
            response_text = session.send_message(user_text)
            print(response_text)

        except (KeyboardInterrupt, EOFError):
            print("\nSession interrupted. Exiting...")
            break

if __name__ == "__main__":
    # To run the CLI chat app, uncomment the line below:
    # run_interactive_cli()
    pass

If you are exploring general environment setup steps before implementing full chat apps, review our walkthrough on running local LLMs in Python.

How to Use Ollama Embeddings in Python

Dense vector embeddings form the mathematical bedrock of semantic search, document clustering, duplicate detection, and Retrieval-Augmented Generation (RAG). An embedding converts arbitrary text strings into a fixed-length array of floating-point numbers (e.g., a vector of 768 or 1536 dimensions) where semantically similar texts cluster closely together in vector space.

Using Ollama embeddings Python functions, software teams can generate embeddings locally using open-weights embedding models like nomic-embed-text or all-minilm.

[IMAGE: Database diagram storing Ollama embeddings created via Python]

Generating and Storing Embeddings

Generating vector embeddings requires calling ollama.embeddings() with a designated embedding model name and target text string.

Step 1: Pull an Embedding Model

Execute the pull command in your terminal for a lightweight, specialized embedding model:

ollama pull nomic-embed-text

Step 2: Python Vector Generation and Cosine Similarity Calculation

Below is a complete Python script demonstrating how to generate embeddings for multiple document chunks, calculate cosine similarity between vectors, and prepare data for storage inside a vector database (such as ChromaDB, Qdrant, or PGVector):

import math
from typing import List
import ollama

def generate_text_embedding(text: str, model_name: str = "nomic-embed-text") -> List[float]:
    """
    Generates a dense vector embedding array for input text using Ollama.
    """
    response = ollama.embeddings(
        model=model_name,
        prompt=text
    )
    return response['embedding']

def calculate_cosine_similarity(vec1: List[float], vec2: List[float]) -> float:
    """
    Calculates cosine similarity score between two numerical vectors.
    """
    dot_product = sum(a * b for a, b in zip(vec1, vec2))
    magnitude_vec1 = math.sqrt(sum(a * a for a in vec1))
    magnitude_vec2 = math.sqrt(sum(b * b for b in vec2))

    if magnitude_vec1 == 0 or magnitude_vec2 == 0:
        return 0.0
    return dot_product / (magnitude_vec1 * magnitude_vec2)

if __name__ == "__main__":
    # Knowledge base documents
    doc_a = "Python is a popular programming language widely used in data science and web development."
    doc_b = "JavaScript is a lightweight script language used primarily for building interactive web browser interfaces."

    # Query string
    query = "Tell me about coding languages for data analysis."

    print("Generating vector embeddings locally...")
    emb_doc_a = generate_text_embedding(doc_a)
    emb_doc_b = generate_text_embedding(doc_b)
    emb_query = generate_text_embedding(query)

    print(f"Embedding Vector Dimensions: {len(emb_query)}")

    # Compute similarity metrics
    sim_a = calculate_cosine_similarity(emb_query, emb_doc_a)
    sim_b = calculate_cosine_similarity(emb_query, emb_doc_b)

    print("\n--- Vector Similarity Match Results ---")
    print(f"Document A ('Python data science'): Similarity = {sim_a:.4f}")
    print(f"Document B ('JavaScript web'):        Similarity = {sim_b:.4f}")

    if sim_a > sim_b:
        print("\nSemantic Search Result: Document A is the best match for the query!")
    else:
        print("\nSemantic Search Result: Document B is the best match for the query!")

For more hands-on code examples across basic and advanced API execution modes, explore our hands-on Python Ollama tutorials and examples.

Frequently Asked Questions (FAQ)

What is the difference between ollama.generate() and ollama.chat() in Python?

ollama.generate() is designed for single-prompt completion tasks where you pass a string prompt and receive a continuous text completion. ollama.chat() is designed for multi-turn conversational agents where you pass an array of role-structured message objects (system, user, assistant), allowing the model to process conversation history.

Which model is best for generating text embeddings locally?

Lightweight embedding models such as nomic-embed-text or all-minilm are optimized specifically for vector generation. They produce high-quality semantic representations with lower memory usage and significantly faster execution speeds than standard generative chat models.

How do I store Ollama embeddings in a database?

Ollama embedding functions return standard Python lists of floats. These vectors can be inserted directly into vector databases like ChromaDB, Qdrant, Milvus, or PostgreSQL (using the pgvector extension) alongside original document text payloads and metadata tags.

Can I run text generation, chat, and embeddings within the same Python application?

Yes. A single Python service can invoke ollama.embeddings() to index documents or user queries, ollama.chat() to maintain dialogue workflows, and ollama.generate() for standalone summarization tasks, all pointing to the same underlying Ollama server host.

Leave a Comment