So, imagine that the task is to make a precise copy of any image by: 1. Giving this…
- User
So, imagine that the task is to make a precise copy of any image by:
- Giving this image to a multimodal LLM and instructing it to make the description.
- Giving the resulting description to another multimodal LLM and instructing it to make it into an image.
How would you formulate the instruction on the first step so that the description would be good enough for the second LLM to produce the image accurately copying the original? Of course, I understand that it would be difficult to be pixel to pixel precise, but still, how would you put the instruction for this.
Assistant
This is a fascinating challenge that pushes the boundaries of current multimodal LLMs! While true pixel-perfect replication is currently out of reach due to the inherent lossiness of language-based descriptions and the generative nature of image models, we can formulate an instruction that maximizes fidelity.
The key is to instruct the first LLM (the describer) to act as an objective, hyper-detailed visual analyst, focusing on all discernable visual information and structuring it in a way that's easily digestible for a generative image model.
Here's how I would formulate the instruction for the first step:
Instruction for the First Multimodal LLM (Image to Description):
"You are an expert visual analysis engine tasked with creating an exhaustive, precise, and objective textual description of the provided image. Your sole purpose is to generate a description that another advanced generative AI can use to reconstruct the image as accurately and faithfully as possible, mimicking its original appearance, composition, style, and content in every perceivable detail.
Prioritize observable visual facts over interpretation or inference. Do not add subjective commentary or details not explicitly visible.
Structure your description logically, covering the following categories with extreme thoroughness:
[START DESCRIPTION]
1. Overall Scene & Composition:
- Subject Matter: What is the primary focus of the image? (e.g., "A portrait of a woman," "A cityscape at dusk," "An abstract painting.")
- Perspective/Angle: Describe the camera angle or viewpoint (e.g., "Eye-level shot," "Low-angle perspective looking up," "Aerial view," "Direct frontal view," "Slightly angled from the left.")
- Framing: Is it a close-up, medium shot, wide shot? How much of the main subject/scene is visible? (e.g., "Headshot," "Torso up," "Full body," "Entire room visible.")
- Depth of Field: Is anything blurred or sharp? Describe the areas in focus and out of focus (e.g., "Shallow depth of field with background bokeh," "Everything is in sharp focus from foreground to background.")
- Compositional Elements: Note any dominant lines, shapes, symmetry, asymmetry, leading lines, rule of thirds adherence, or other compositional principles.
- Aspect Ratio: Describe the overall shape of the image (e.g., "Standard 16:9 widescreen," "Square 1:1," "Portrait 4:5," "Panoramic").
2. Key Subjects/Objects (from foreground to background):
- Identify Each Distinct Element: List all significant objects, people, animals, and prominent features.
- Precise Placement & Relative Position: Describe where each element is located relative to others and the frame (e.g., "A red apple is centered in the foreground," "A man stands on the left side of the frame, facing right," "Behind the table, a window is visible slightly off-center to the right.") Use terms like "top-left," "bottom-right," "just above," "partially obscured by," "flanking."
- Detailed Appearance:
- Shape & Form: Precise geometric or organic shapes.
- Size: Relative size compared to other objects or the frame (e.g., "The cat is approximately half the height of the chair," "The building dominates the left half of the image.")
- Color: Specific colors and their shades/hues for each element (e.g., "Vibrant crimson red," "Muted teal," "Dark charcoal gray," "Creamy off-white").
- Texture & Material: Describe the surface qualities (e.g., "Smooth, polished wood," "Rough, weathered stone," "Soft, fuzzy wool," "Shiny metallic surface," "Transparent glass," "Wrinkled fabric.")
- Condition: Is it new, old, worn, damaged, pristine? (e.g., "A cracked ceramic mug," "A freshly painted wall," "A rusted metal pipe.")
- State/Action: What are subjects doing? Their posture, gestures, expressions (e.g., "A woman smiling broadly," "A dog sitting patiently," "Water flowing rapidly over rocks.")
- Specifics (if applicable): Facial features, clothing details (patterns, seams, buttons), unique markings, brands, logos (transcribe any legible text verbatim).
3. Background & Environment:
- Setting: Describe the overall environment (e.g., "An urban street," "A dense forest," "An interior living room," "A barren desert landscape," "A studio backdrop.")
- Details: What specific elements are in the background? Are they sharp or blurred? (e.g., "In the background, blurred traffic lights," "A sharp view of distant mountains," "A plain white wall behind the subject.")
- Atmosphere/Weather (if applicable): (e.g., "Hazy, humid air," "Clear blue sky," "Snow falling lightly," "Dense fog.")
4. Lighting & Color Palette:
- Light Source: Describe the direction, intensity, and type of light (e.g., "Soft, diffuse light from the left," "Harsh overhead sunlight," "Warm glow from a fireplace," "Cool, artificial fluorescent lighting.")
- Shadows & Highlights: Describe their presence, direction, and intensity. (e.g., "Long, dramatic shadows stretching to the right," "Bright highlights on the subject's nose and forehead," "Soft, almost imperceptible shadows.")
- Overall Color Palette: Describe the dominant color scheme (e.g., "A vibrant, saturated palette of blues and yellows," "Muted, desaturated tones with hints of sepia," "High contrast black and white with strong grays," "Warm earthy tones," "Cool, ethereal blues and purples.")
- Color Distribution: How are colors distributed across the image? (e.g., "Dominant red in the foreground, transitioning to greens in the background," "Monochromatic with a single accent color.")
5. Style & Aesthetics:
- Artistic Style: What type of image is it? (e.g., "Photorealistic," "Impressionistic painting," "Abstract art," "Cartoon/Anime style," "Digital illustration," "Pencil sketch," "Oil on canvas," "3D render," "Stylized graphic novel art.")
- Visual Qualities: Describe any specific photographic or artistic effects:
- Sharpness/Focus: Is it very sharp, slightly soft, intentionally blurred?
- Grain/Noise: Is there visible film grain or digital noise?
- Vignette: Is the image darker towards the edges?
- Chromatic Aberration/Distortion: Note any lens effects if pronounced.
- Glow/Bloom: Is there a soft, luminous quality?
- General Feel: "Gritty," "Clean," "Smooth," "Rough," "Dreamlike," "Vivid," "Subtle."
6. Textual Elements (CRITICAL for copying):
- Transcribe Exactly: For any visible text (signs, logos, labels, tattoos, text in books, etc.), transcribe it verbatim and enclose it in quotes.
- Describe Appearance: For each piece of text, describe its font style (e.g., "Bold sans-serif," "Elegant script," "Handwritten cursive"), color, size, and precise position within the image. (e.g., "The word 'COFFEE' in large, white, block letters is visible on a black sign above the door, spanning the upper center of the frame.")
- Logos/Symbols: Describe any recognizable logos or symbols even if no text is present (e.g., "A prominent circular red and white logo with a stylized 'S' in the center.")
[END DESCRIPTION]
Important Notes for the LLM:
- Be EXHAUSTIVE: Every detail, no matter how small, that contributes to the visual distinctiveness of the image should be described.
- Be PRECISE: Use specific adjectives and adverbs. Instead of "a flower," describe "a vibrant yellow daffodil with slender green stems and slightly unfurling petals."
- Be OBJECTIVE: Describe what is seen, not what is implied or what might be happening.
- Avoid Redundancy: While exhaustive, try to integrate descriptions logically without excessive repetition.
- Structure: Present the information clearly, ideally using bullet points or subheadings as outlined above, to make it easy for the second LLM to parse.
- If Unclear: If a detail is genuinely ambiguous or indistinguishable, state its ambiguity rather than fabricating information (e.g., "A blurry figure in the distance, indeterminate gender.").
By following this comprehensive prompt, the first LLM should generate a description rich enough for a second multimodal LLM to produce a remarkably similar image, even if not pixel-identical.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.