--- title: Non-LLM models description: Generate media or embeddings, transcribe audio, and rerank documents. type: guide summary: Use dedicated models for images, video, speech, embeddings, transcription, and reranking. --- # Non-LLM models Use `ai.ops` for model calls that do not produce a language-model message. These operations return an `ai.ops.Item` instead of adding output to message history. The built-in provider currently supports these operations through AI Gateway. Use an unprefixed model ID to route the request through Gateway. ## Generate images Pass an image model and a prompt string to `generate_image`: ```python title="generate_image.py" import asyncio import base64 import pathlib import ai async def main() -> None: model = ai.get_model("google/imagen-4.0-generate-001") result = await ai.ops.generate_image( model, "A watercolor cabin at sunset.", params=ai.ops.ImageParams(aspect_ratio="16:9"), ) image = result.value[0] data = ( image.data if isinstance(image.data, bytes) else base64.b64decode(image.data) ) pathlib.Path("cabin.png").write_bytes(data) if __name__ == "__main__": asyncio.run(main()) ``` Use `ImageParams` to set the number of images, size, aspect ratio, or seed. To edit an image, pass an `ImagePrompt` with the input images alongside the text. Input images accept a `FilePart`, raw bytes, or a URL or base64 string. Use `mask` for inpainting operations: ```python prompt = ai.ops.ImagePrompt( text="Make this photo look like a watercolor painting.", images=["https://example.com/photo.jpg"], ) result = await ai.ops.generate_image(model, prompt) ``` ## Generate video Use `generate_video` with a dedicated video model: ```python result = await ai.ops.generate_video( ai.get_model("google/veo-3.1-fast-generate-001"), "A paper boat drifting across a puddle in the rain.", params=ai.ops.VideoParams( aspect_ratio="16:9", duration=4, ), ) video = result.value[0] ``` `VideoParams` can also set the number of videos, resolution, frames per second, seed, and whether to generate audio alongside the video. For models that accept input media, pass a `VideoPrompt`: Start image First and last frame Reference media Extend a video ```python prompt = ai.ops.VideoPrompt( text="The boat sails away.", image="https://example.com/boat.jpg", ) result = await ai.ops.generate_video(model, prompt) ``` ```python prompt = ai.ops.VideoPrompt( text="The paper boat drifts from the puddle's edge to the drain.", frame_images=[ ai.ops.FrameImage( image=pathlib.Path("boat-start.png").read_bytes(), frame_type="first_frame", ), ai.ops.FrameImage( image=pathlib.Path("boat-end.png").read_bytes(), frame_type="last_frame", ), ], ) result = await ai.ops.generate_video(model, prompt) ``` ```python prompt = ai.ops.VideoPrompt( text="The same paper boat, now sailing through a storm.", references=[ "https://example.com/boat-closeup.jpg", "https://example.com/boat-side.jpg", ], ) result = await ai.ops.generate_video(model, prompt) ``` ```python prompt = ai.ops.VideoPrompt( text="The boat sails on into the storm.", references=["https://example.com/boat.mp4"], ) result = await ai.ops.generate_video(model, prompt) ``` Video inputs and parameters are provider-dependent. Check the model's documentation. ## Generate speech Use `generate_audio` to turn text into speech: ```python result = await ai.ops.generate_audio( ai.get_model("openai/tts-1"), "The robots have reached the mothership.", params=ai.ops.AudioParams( voice="alloy", output_format="mp3", ), ) audio = result.value[0] ``` `AudioParams` can also set speed and language. To control tone or emotion, pass an `AudioPrompt` with delivery instructions: ```python prompt = ai.ops.AudioPrompt( text="The robots have reached the mothership.", instructions="Speak in a calm, reassuring tone.", ) ``` Image, video, and speech operations return `FilePart` values. Their `data` field contains bytes or base64 text and can be saved in the same way as the image example. ## Embed text Use `embed` to create one vector for each input string: ```python values = [ "The robots reached the mothership.", "The machines arrived at the command ship.", ] result = await ai.ops.embed( ai.get_model("openai/text-embedding-3-small"), values, ) first_vector = result.value[0] ``` Vectors stay in the same order as the input strings. When the provider reports token usage, it is available on `result.usage`. ## Transcribe audio Pass audio bytes or an audio `FilePart` to `transcribe`: ```python import pathlib audio = pathlib.Path("meeting.mp3").read_bytes() result = await ai.ops.transcribe( ai.get_model("openai/whisper-1"), audio, ) print(result.value.text) for segment in result.value.segments: print(segment.start_second, segment.end_second, segment.text) ``` The result can also include the detected language and total duration when the provider reports them. ## Rerank documents Use `rerank` to order documents by their relevance to a query: ```python documents = [ "Reset your password from the sign-in page.", "The office opens at 9 AM.", "Contact support through the chat widget.", ] result = await ai.ops.rerank( ai.get_model("cohere/rerank-v3.5"), documents, "How do I change my password?", params=ai.ops.RerankParams(top_n=2), ) for ranked in result.value: print(ranked.score, documents[ranked.index]) ``` Each result contains the document's original index and relevance score. Results are ordered from the highest score to the lowest. ## Read operation metadata Every operation returns `ai.ops.Item`: ```python result.value result.usage result.warnings result.provider_metadata ``` `value` contains the operation output. `usage` is `None` when the provider does not report token usage. `warnings` lists provider warnings, such as an ignored parameter; it is empty when there are none. Provider-specific response data is available on `provider_metadata`. --- For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md) For an index of all available documentation, see [/llms.txt](/llms.txt) For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)