Skip to content
TheChatArchiveThe Chat Archive
Qwenqwen3-235b-a22b-instruct-2507· Alibaba (Qwen)Hugging Face LMArena Human Preference 140K

I'm interested in evaluating and ranking the performance of geometrical measurement…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-opus-4-20250514’s answer.
  1. User

    I'm interested in evaluating and ranking the performance of geometrical measurement systems (CMMs). By performance I am interested in knowing the actual accuracy and repeatability of the various systems. I am interested in touch and non touch (non contact) systems

  2. Qwen Assistant

    To evaluate and rank Coordinate Measuring Machines (CMMs), both touch (contact) and non-touch (non-contact), based on accuracy and repeatability, follow this structured framework:


    1. Key Performance Metrics

    • Accuracy: Deviation from a known reference (traceable artifact).
    • Repeatability (Precision): Consistency of measurements under identical conditions.
    • Additional Factors:
      • Volumetric Accuracy: Performance across the entire measurement volume.
      • Form Accuracy: Ability to measure complex geometries (e.g., flatness, roundness).
      • Scanning Speed (for non-contact systems).
      • Surface Sensitivity: How surface finish/material affects measurements.
      • Environmental Robustness: Sensitivity to temperature, vibration, and humidity.

    2. Standards & Compliance

    • ISO 10360: The primary standard for CMM performance evaluation.
      • ISO 10360-2: Accuracy testing using calibrated artifacts (e.g., step gauges, spheres).
      • ASME B5.54/B5.57 (U.S. alternatives).
    • Traceability: Ensure measurements are traceable to national standards (e.g., NIST).

    3. Testing Methodology

    Artifacts for Testing

    • Contact Systems: Use calibrated spheres, gauge blocks, or step gauges.
    • Non-Contact Systems: Use glass scales, ceramic plates, or 3D-printed lattice structures with known features.
    • Common Artifacts:
      • Ball Plate: For volumetric accuracy.
      • Standard Sphere: For form accuracy.
      • Multi-Stylus Calibrators: For probe qualification (touch systems).

    Accuracy Testing

    1. Single-Point Measurements: Compare CMM readings to certified values of an artifact.
    2. Feature Measurement: Measure geometries (e.g., circles, planes) and compare to reference values.
    3. Volumetric Testing: Assess errors across X/Y/Z axes using a ball plate or laser tracker.

    Repeatability Testing

    • Multiple Measurements: Measure the same feature 25–50 times under identical conditions.
    • Statistics: Calculate standard deviation (σ) and 6σ spread (to estimate repeatability error).
    • Gage R&R Study: Assess operator and system variability (if applicable).

    Non-Contact Specific Tests

    • Point Cloud Density: Measure coverage and noise levels on flat/curved surfaces.
    • Edge Detection: Test sharp edge resolution (e.g., step height).
    • Surface Reflectivity: Evaluate performance on glossy, matte, or transparent surfaces.

    4. Considerations for Touch vs. Non-Contact Systems

    FactorTouch ProbesNon-Contact
    AccuracyHigh (sub-micron for high-end systems).Lower (micron-level, depends on optics).
    SpeedSlower (point-by-point).Faster (area scanning).
    Surface InteractionRisk of part deformation/wear.Non-invasive.
    Material SensitivityWorks on all materials.Struggles with transparent/reflective surfaces.
    MaintenanceProbe calibration/wear.Lens cleaning, optical alignment.

    5. Data Analysis & Ranking

    • Accuracy Score:
      • Compute the maximum deviation from the reference value.
      • Example: System A deviates ±1 µm; System B ±2 µm → A scores higher.
    • Repeatability Score:
      • Use 6σ spread; lower values rank higher.
      • Example: σ = 0.2 µm → 6σ = 1.2 µm (better than σ = 0.5 µm → 3 µm).
    • Weighted Ranking:
      • Define weights (e.g., 70% accuracy, 20% repeatability, 10% other factors).
      • Example:
        SystemAccuracy (µm)Repeatability (µm)Speed (pts)Final Score
        Touch A1.0 (70)1.2 (20)5/10 (10)95
        Laser B2.0 (60)1.5 (18)9/10 (10)88

    6. Reporting & Recommendations

    • Uncertainty Budget: Include contributions from the CMM, artifact, and environment.
    • Application-Specific Notes:
      • Prioritize touch probes for high-accuracy, small parts (e.g., gears).
      • Use non-contact for fragile, complex, or high-speed measurements (e.g., turbine blades).
    • Long-Term Factors: Calibration frequency, maintenance costs, and operator training.

    7. Tools & Equipment

    • Laser Interferometers: For volumetric accuracy (ISO 10360-6).
    • Calibrated Artifacts: Step gauges, ceramic spheres, and optical standards.
    • Software: Statistical analysis (e.g., Minitab) and CMM evaluation packages (e.g., PC-DMIS).

    By following this framework, you can systematically evaluate and rank CMMs based on their real-world performance. Adjust weights and tests to align with your specific application (e.g., aerospace vs. medical devices). Always validate manufacturer claims with hands-on testing.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.