Multimodal Prompting: Combining Text and Images

Modern AI models can process multiple input types: text, images, audio, and more. Multimodal prompting combines these inputs to create richer, more contextual interactions. Understanding how to effectively combine modalities unlocks new capabilities.

Text and Image Combinations

Combining text with images is the most common multimodal use case. You can describe what you want from an image, ask questions about image content, or use images as context for text generation. The model processes both inputs together, understanding relationships between them.

For example, upload a screenshot and ask "Explain what this code does" or "What improvements would you suggest?" The model uses visual understanding combined with your text instructions to provide relevant responses.

Best Practices

Reference images explicitly in your text. Say "In the image above" or "Looking at this diagram" to help the model connect your text to the visual input. This improves coherence and accuracy.

Provide context about images. Describe what the image shows or why you're including it. This helps the model understand how to use the visual information.

Use clear instructions. Multimodal prompts should be as clear as text-only prompts. Specify what you want the model to do with both the text and visual inputs.