"5M context and 83x72b paramaters, output modalities: VRChat Full-Body-Tracking, audio…
- User
"5M context and 83x72b paramaters, output modalities: VRChat Full-Body-Tracking, audio (stereo), text, images , VRChat facial tracking data, object and scene interaction data (like it can press buttons) input modalities: vision (in realtime allows it to see the other participants and respond), audio (stereo), text, images (videos included), discord custom emoji & stickers, tactile feel, context thoughts, taste & scents"
Can this model be considered multimodal?
Assistant
Yes, absolutely. This model is undeniably multimodal. Here's why:
-
Multiple Input Modalities: It ingests data from a wide variety of sources, including:
- Vision (realtime video)
- Audio (stereo)
- Text
- Images (including videos)
- Discord emoji & stickers
- Tactile Feel
- Taste & scents
- Contextual thoughts
-
Multiple Output Modalities: It produces output in several different formats:
- VRChat Full-Body-Tracking data
- Audio (stereo)
- Text
- Images
- VRChat Facial Tracking Data
- Object and scene interaction data
The key characteristic of a multimodal model is the ability to process and generate information across different types of data. This model clearly does that. The variety of input and output formats makes it a prime example of a multimodal AI.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.