Modern models like GPT-4o and Gemini 1.5 are inherently multimodal. They don't just use OCR or transcription pipelines; their core architecture natively embeds audio waveforms and video frames alongside text tokens.
Module 1: Unified Embedding Space
A multimodal architecture maps different modalities into a single unified embedding space. A picture of a dog, the sound of a bark, and the word 'dog' all map to similar mathematical vectors.