Skip to content
TheChatArchiveThe Chat Archive
o3-2025-04-16· OpenAIHugging Face LMArena Human Preference 140K

We are doing ai powered foundation shade matching through gpt. We aim to identify…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-opus-4-20250514’s answer.
  1. User

    We are doing ai powered foundation shade matching through gpt. We aim to identify undertone and skin tones for given pictures using gpt 4o. Is there another approach to accurately identify undertones apart from llm integration? Something that's cost effective too?

  2. Assistant

    Absolutely—Large-Language Models are great at describing a colour once it’s been measured, but they’re not the only (or even the cheapest) way to measure or classify undertone from an image.
    Below are a few alternative, budget-friendly routes you can explore. They range from “pure computer-vision + tiny ML” to simple hardware hacks, all doable without paying for GPT tokens on every call.


    1️⃣ Pure Computer-Vision Pipeline (No LLM)

    a. Capture & Calibrate

    1. In-app capture with guidance:
      • Ask the user to photograph a color-calibration patch (or even a white card) in the same lighting.
      • Use the patch to perform automatic white-balance & exposure correction (OpenCV’s cv::colorBalance, Grey-World, or Von Kries).
    2. Face/skin segmentation (Mediapipe FaceMesh, dlib, or a light MobileNet-based semantic-seg model).
    3. Mask out lips, eyes, hair to keep only “true skin”.

    b. Feature Extraction

    Convert the skin pixels to a perceptual colour space (CIELAB or CIELCH), then compute simple statistics:
    • Mean/median L, a*, b*
    • Hue angle (h° = atan2(b*, a*))
    • Chroma (C = √(a² + b²))

    c. Undertone Rule of Thumb

    • Warm: high +b* (yellow) and positive a* (red) → h° ~ 40–80°
    • Cool: lower b*, slightly negative a*, h° ~ 250–300°
    • Neutral/olive: moderate b*, low chroma, h° centred

    You can formalise this with a tiny classifier:

    if C < 8  ➜ Neutral  
    elif h° < 150 ➜ Warm  
    else         ➜ Cool
    

    d. Tiny-ML Refinement

    Feed [L, a*, b*, C, h°] into a logistic-regression model (5 weights!) or a small MLP (e.g. 1 hidden layer × 16 neurons). Train on a labelled set of ~2–5 k faces; deploy on-device (TensorFlow Lite ≈ 30 kB).

    Cost: Free open-source libs, no inference fees, runs in < 5 ms on a phone.


    2️⃣ Ultra-Cheap “Spectral” Trick

    Phone flash + coloured sticker filters (literally bits of theatre gel plastic) → capture the face under 2–3 narrow-band illuminations (e.g. amber & cyan).
    Compute the reflectance ratio R(amber)/R(cyan).
    • Ratio ↑ = warmer (more yellow/red)
    • Ratio ↓ = cooler (more blue)

    Just two exposures, some basic algebra, and you’ve approximated a 2-band spectrophotometer for cents.


    3️⃣ $15 Hardware Add-On

    Clip-on colorimeter sensors such as AMS TCS3472 or the NEW TCS34072 (RGB+IR) cost <$15 in volume.
    • I²C to the phone or microcontroller
    • Reads raw XYZ values independent of ambient light
    With a tiny decision tree, you get lighting-independent, repeatable undertone detection. No GPU, no cloud, no subscription.


    4️⃣ Questionnaire Hybrid (Zero ML)

    If development time is tight, you can combine:
    • A quick in-app “vein colour / jewellery / sunburn” quiz
    • Simple average skin b* from the selfie

    Blend the two signals with heuristic weights. Accuracy bumps surprisingly close to ML while being dirt cheap.


    5️⃣ Lightweight CNN Instead of GPT

    If you’d still like a “learned” solution but sans LLM fees:
    • Train a MobileNet-V3 Small (1.5 MB) end-to-end on labelled selfies (cool/neutral/warm).
    • Knowledge-distill to an even smaller 0.5 MB model.
    • Run with on-device NNAPI / CoreML.

    After initial training, inference cost is zero and speeds are real-time.


    Ball-Park Accuracy vs. Cost

    ApproachOne-time Dev CostRuntime $Typical F-score (if well-executed)
    Pure CV + rulesLow$00.70-0.80
    Tiny-ML (log-reg/MLP)Low-Med$00.80-0.90
    Clip-on sensorMed (HW)$00.90+ (lighting-independent)
    Questionnaire hybridTiny$00.70-0.85
    Small CNN on-deviceMed$00.88-0.93
    GPT/LLM visionLow devPay-per-call0.90-0.95 (depends on prompt)

    Putting It All Together (Suggested Flow)

    1. Guided capture + calibration patch for consistent data
    2. Skin mask with Mediapipe (free, fast)
    3. Convert to CIELAB, extract stats
    4. Feed to • logistic-reg model (on device)
      • fall-back rule set if the model confidence < 0.6
    5. Output undertone + nearest foundation shades from your SKU database

    Result: Entire pipeline runs offline, no token costs, and you can still bolt on GPT later for fancy product-copy or recommendations.


    Hope this helps you trim those API bills while keeping accuracy high!
    Let me know if you’d like code snippets, calibration chart suggestions, or a tiny-ML starter notebook. 🎨🤖

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.