Medgemma Agent

MedGemma does NOT support native Ollama tool calling. Instead, use prompt-based tool calling:

1. Tool definitions are embedded in the system prompt as Python function signatures with docstrings
2. MedGemma outputs ```tool_code blocks when it wants to call a function
3. The harness extracts and executes the code client-side
4. Results are sent back as ```tool_output blocks
5. MedGemma generates a final natural language response

Agent Harness Template

When creating or modifying the agent harness, follow this pattern:

import json, re, io, inspect, requests, sys
from contextlib import redirect_stdout
from typing import Callable

OLLAMA_URL = "http://localhost:11434"
MODEL = "hf.co/unsloth/medgemma-27b-it-GGUF:Q4_K_M"

# ── Tool Registry ──
TOOL_REGISTRY: dict[str, Callable] = {}

def register_tool(func: Callable) -> Callable:
    TOOL_REGISTRY[func.__name__] = func
    return func

def get_tool_descriptions() -> str:
    lines = []
    for name, func in TOOL_REGISTRY.items():
        source = inspect.getsource(func)
        source_lines = [l for l in source.split('\n') if '@register_tool' not in l]
        lines.append('\n'.join(source_lines))
    return '\n\n'.join(lines)

# ── Define Your Tools ──
@register_tool
def calculate_bmi(weight_kg: float, height_m: float) -> str:
    """Calculate Body Mass Index. Args: weight_kg, height_m"""
    bmi = weight_kg / (height_m ** 2)
    cat = "Underweight" if bmi < 18.5 else "Normal" if bmi < 25 else "Overweight" if bmi < 30 else "Obese"
    return f"BMI: {bmi:.1f} ({cat})"

@register_tool
def drug_interaction_check(drug_a: str, drug_b: str) -> str:
    """Check for known interactions between two drugs. Args: drug_a, drug_b"""
    interactions = {
        ("warfarin", "aspirin"): "HIGH RISK: Increased bleeding risk.",
        ("metformin", "contrast dye"): "MODERATE RISK: Hold metformin 48h before/after contrast.",
        ("lisinopril", "potassium"): "MODERATE RISK: Risk of hyperkalemia.",
    }
    key = (drug_a.lower(), drug_b.lower())
    rev = (drug_b.lower(), drug_a.lower())
    return interactions.get(key) or interactions.get(rev) or f"No known major interactions between {drug_a} and {drug_b}."

# ── System Prompt ──
def build_system_prompt() -> str:
    return f"""You are a helpful medical assistant with access to tools.
At each turn, if you decide to invoke any of the function(s), it should be wrapped with ```tool_code```.
The python methods described below are imported and available. You can only use defined methods.
The response to a method will be wrapped in ```tool_output``` — use it to generate a helpful response.

The following Python methods are available:
```python
{get_tool_descriptions()}
```"""

# ── Tool Executor ──
def extract_and_execute(text: str) -> str | None:
    match = re.search(r"```tool_code\s*(.*?)\s*```", text, re.DOTALL)
    if not match:
        return None
    code = match.group(1).strip()
    namespace = dict(TOOL_REGISTRY)
    try:
        f = io.StringIO()
        with redirect_stdout(f):
            result = eval(code, namespace)
        output = f.getvalue()
        return f"```tool_output\n{result if output == '' else output}\n```"
    except Exception as e:
        return f"```tool_output\nError: {e}\n```"

# ── Agent Loop ──
def run_agent(query: str, max_rounds: int = 3) -> str:
    messages = [
        {"role": "system", "content": build_system_prompt()},
        {"role": "user", "content": query},
    ]
    for _ in range(max_rounds):
        resp = requests.post(f"{OLLAMA_URL}/api/chat",
            json={"model": MODEL, "stream": False, "messages": messages})
        resp.raise_for_status()
        content = resp.json()["message"]["content"]
        tool_output = extract_and_execute(content)
        if tool_output is None:
            return content
        messages.append({"role": "assistant", "content": content})
        messages.append({"role": "user", "content": f"Results:\n{tool_output}"})
    return content

if __name__ == "__main__":
    query = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else sys.argv[0]
    print(run_agent(query))

Adding Custom Tools

To add a new tool, define a function with the @register_tool decorator:

@register_tool
def calculate_egfr(creatinine: float, age: int, is_female: bool) -> str:
    """Estimate glomerular filtration rate using CKD-EPI equation.
    Args: creatinine (mg/dL), age (years), is_female (bool)"""
    # CKD-EPI 2021 equation (race-free)
    k = 0.7 if is_female else 0.9
    alpha = -0.241 if is_female else -0.302
    scr_k = creatinine / k
    egfr = 142 * (min(scr_k, 1) ** alpha) * (max(scr_k, 1) ** -1.200) * (0.9938 ** age)
    if is_female:
        egfr *= 1.012
    stage = "Normal" if egfr >= 90 else "Mild" if egfr >= 60 else "Moderate" if egfr >= 30 else "Severe" if egfr >= 15 else "Kidney Failure"
    return f"eGFR: {egfr:.1f} mL/min/1.73m2 ({stage})"

The model will automatically see the new tool in the system prompt and can call it.

OpenAI Agents SDK Integration

To use this pattern with the OpenAI Agents SDK (v0.8.1+):

from agents import Agent, Runner, function_tool
from agents.models.openai_chatcompletions import OpenAIChatCompletionsModel
from openai import AsyncOpenAI

client = AsyncOpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
model = OpenAIChatCompletionsModel(model="hf.co/unsloth/medgemma-27b-it-GGUF:Q4_K_M", openai_client=client)

# Note: MedGemma does NOT support native tool calling via the SDK.
# The SDK passes tools to the API which returns HTTP 400.
# For SDK integration, use a basic Agent without tools and handle
# tool_code/tool_output parsing in a wrapper around Runner.run().

Important Notes

• Only the 27B model reliably handles tool calling (4/4 tests passed)
• The 4B model struggles with generating correct tool_code (2/4 tests passed)
• Tool execution uses eval() — only register trusted functions
• Ollama must be running on localhost:11434
• The agent loop runs max 3 rounds by default to prevent infinite loops