We are doing ai powered foundation shade matching through gpt. We aim to identify…
- User
We are doing ai powered foundation shade matching through gpt. We aim to identify undertone and skin tones for given pictures using gpt 4o. Is there another approach to accurately identify undertones apart from llm integration? Something that's cost effective too?
- Assistant
Absolutely—Large-Language Models are great at describing a colour once it’s been measured, but they’re not the only (or even the cheapest) way to measure or classify undertone from an image.
Below are a few alternative, budget-friendly routes you can explore. They range from “pure computer-vision + tiny ML” to simple hardware hacks, all doable without paying for GPT tokens on every call.
1️⃣ Pure Computer-Vision Pipeline (No LLM)
a. Capture & Calibrate
- In-app capture with guidance:
• Ask the user to photograph a color-calibration patch (or even a white card) in the same lighting.
• Use the patch to perform automatic white-balance & exposure correction (OpenCV’scv::colorBalance, Grey-World, or Von Kries). - Face/skin segmentation (Mediapipe FaceMesh, dlib, or a light MobileNet-based semantic-seg model).
- Mask out lips, eyes, hair to keep only “true skin”.
b. Feature Extraction
Convert the skin pixels to a perceptual colour space (CIELAB or CIELCH), then compute simple statistics:
• Mean/median L, a*, b*
• Hue angle (h° = atan2(b*, a*))
• Chroma (C = √(a² + b²))c. Undertone Rule of Thumb
• Warm: high +b* (yellow) and positive a* (red) → h° ~ 40–80°
• Cool: lower b*, slightly negative a*, h° ~ 250–300°
• Neutral/olive: moderate b*, low chroma, h° centredYou can formalise this with a tiny classifier:
if C < 8 ➜ Neutral elif h° < 150 ➜ Warm else ➜ Coold. Tiny-ML Refinement
Feed
[L, a*, b*, C, h°]into a logistic-regression model (5 weights!) or a small MLP (e.g. 1 hidden layer × 16 neurons). Train on a labelled set of ~2–5 k faces; deploy on-device (TensorFlow Lite ≈ 30 kB).Cost: Free open-source libs, no inference fees, runs in < 5 ms on a phone.
2️⃣ Ultra-Cheap “Spectral” Trick
Phone flash + coloured sticker filters (literally bits of theatre gel plastic) → capture the face under 2–3 narrow-band illuminations (e.g. amber & cyan).
Compute the reflectance ratio R(amber)/R(cyan).
• Ratio ↑ = warmer (more yellow/red)
• Ratio ↓ = cooler (more blue)Just two exposures, some basic algebra, and you’ve approximated a 2-band spectrophotometer for cents.
3️⃣ $15 Hardware Add-On
Clip-on colorimeter sensors such as AMS TCS3472 or the NEW TCS34072 (RGB+IR) cost <$15 in volume.
• I²C to the phone or microcontroller
• Reads raw XYZ values independent of ambient light
With a tiny decision tree, you get lighting-independent, repeatable undertone detection. No GPU, no cloud, no subscription.
4️⃣ Questionnaire Hybrid (Zero ML)
If development time is tight, you can combine:
• A quick in-app “vein colour / jewellery / sunburn” quiz
• Simple average skin b* from the selfieBlend the two signals with heuristic weights. Accuracy bumps surprisingly close to ML while being dirt cheap.
5️⃣ Lightweight CNN Instead of GPT
If you’d still like a “learned” solution but sans LLM fees:
• Train a MobileNet-V3 Small (1.5 MB) end-to-end on labelled selfies (cool/neutral/warm).
• Knowledge-distill to an even smaller 0.5 MB model.
• Run with on-device NNAPI / CoreML.After initial training, inference cost is zero and speeds are real-time.
Ball-Park Accuracy vs. Cost
Approach One-time Dev Cost Runtime $ Typical F-score (if well-executed) Pure CV + rules Low $0 0.70-0.80 Tiny-ML (log-reg/MLP) Low-Med $0 0.80-0.90 Clip-on sensor Med (HW) $0 0.90+ (lighting-independent) Questionnaire hybrid Tiny $0 0.70-0.85 Small CNN on-device Med $0 0.88-0.93 GPT/LLM vision Low dev Pay-per-call 0.90-0.95 (depends on prompt)
Putting It All Together (Suggested Flow)
- Guided capture + calibration patch for consistent data
- Skin mask with Mediapipe (free, fast)
- Convert to CIELAB, extract stats
- Feed to • logistic-reg model (on device)
• fall-back rule set if the model confidence < 0.6 - Output undertone + nearest foundation shades from your SKU database
Result: Entire pipeline runs offline, no token costs, and you can still bolt on GPT later for fancy product-copy or recommendations.
Hope this helps you trim those API bills while keeping accuracy high!
Let me know if you’d like code snippets, calibration chart suggestions, or a tiny-ML starter notebook. 🎨🤖 - In-app capture with guidance:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.