Skip to content
TheChatArchiveThe Chat Archive
o3-2025-04-16· OpenAIHugging Face LMArena Human Preference 140K

how to annotate bounding boxes for object detection dataset

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Metallama-3.3-70b-instruct’s answer.
  1. User

    how to annotate bounding boxes for object detection dataset

  2. Assistant

    Below is a field-tested, end-to-end guide that will get you from “raw images” to a clean, machine-readable bounding-box dataset for object detection.


    1. Decide on a Labeling Strategy

    DecisionWhy it mattersTypical Options
    ClassesDrives model output sizee.g. person, car, dog…
    Box DefinitionTightness impacts IoUTight-fit vs. lenient; include occlusion margins?
    Dataset SplitPrevents leakageTrain / Val / Test (70 / 20 / 10 is common)
    Annotation FormatDictates tooling & conversion scriptsPascal VOC (XML), COCO (JSON), YOLO (txt), TFRecord

    Tip: Write a 1-2 page labeling guideline and pin it in your workspace. Small ambiguities snowball quickly.


    2. Pick a Tool

    Free & open source
    • LabelImg (desktop, YOLO/VOC)
    • CVAT (web, VOC/COCO/YOLO, videos)
    • LabelMe (web, polygons too)
    • VIA – VGG Image Annotator (single-html, super-light)

    Freemium / SaaS
    • Roboflow Annotate
    • Makesense.ai
    • SuperAnnotate, LabelBox, Scale, etc.

    If you’re solo or in a small team, LabelImg or CVAT are great starters.


    3. Install & Configure (LabelImg example)

    # 1. Get the app
    pip install labelImg          # or build from source
    # 2. Launch
    labelImg
    # 3. Set default save dir + format
    View ➜ Auto Save Mode ✓
    PascalVOC ⇄ YOLO toggle on the toolbar
    

    4. Annotate Like a Pro

    1. Open an image (hotkey: Ctrl + O).
    2. Press W (or click “Create Rect Box”), drag corner-to-corner.
    3. Enter class name → OK (LabelImg auto-creates the class list).
    4. Fine-tune with corner handles; keep the box tight but include the entire visible part.
    5. Use tags/flags if your tool supports them:
      • truncated – object extends outside frame
      • occluded – partially hidden
      • difficult/ignore – optional, for small/blurry objects
    6. Hit Ctrl + S (saves *.xml or *.txt).
    7. Next Image (d) → repeat.

    Batch shortcuts save hours: duplicate last box (Ctrl + D), zoom (Ctrl + wheel), etc.


    5. Annotation Formats in 30 sec

    YOLO v5/8 .txt (one file per image)

    class_id  x_center  y_center  width  height  (normalized 0-1)
    

    Pascal VOC .xml

    <object>
      <name>dog</name>
      <bndbox><xmin>48</xmin><ymin>240</ymin><xmax>195</xmax><ymax>371</ymax></bndbox>
    </object>
    

    COCO instances_train.json (snippet)

    {"images":[{"id":3,"file_name":"0003.jpg","height":480,"width":640}],
     "annotations":[{"id":17,"image_id":3,"category_id":2,
                     "bbox":[48,240,147,131], "iscrowd":0}]}
    

    Conversion ≠ scary. Python libraries: roboflow, coco-tools, or a tiny script:

    # Pascal VOC -> YOLO example
    import xml.etree.ElementTree as ET, pathlib, cv2, os, glob
    
    for xml_file in glob.glob("annotations/*.xml"):
        tree = ET.parse(xml_file); root = tree.getroot()
        img_w = int(root.find('size/width').text)
        img_h = int(root.find('size/height').text)
        yolo_lines = []
        for obj in root.iter('object'):
            cls = obj.find('name').text
            xmin = int(obj.find('bndbox/xmin').text)
            xmax = int(obj.find('bndbox/xmax').text)
            ymin = int(obj.find('bndbox/ymin').text)
            ymax = int(obj.find('bndbox/ymax').text)
            x_c = ((xmin + xmax) / 2) / img_w
            y_c = ((ymin + ymax) / 2) / img_h
            w   = (xmax - xmin) / img_w
            h   = (ymax - ymin) / img_h
            yolo_lines.append(f"{cls_map[cls]} {x_c:.6f} {y_c:.6f} {w:.6f} {h:.6f}")
        txt_path = pathlib.Path(xml_file).with_suffix('.txt')
        txt_path.write_text("\n".join(yolo_lines))
    

    6. Quality Assurance (QA)

    • Double-label ~5 % of images → compute IoU agreement.
    • Use built-in reviewer role (CVAT) or a simple checklist spreadsheet.
    • Spot-check edge cases: tiny objects, heavy occlusion, overlapping classes.
    • Keep a “hall of fame” for tricky examples; update guidelines.


    7. Speed Hacks & Semi-Auto Tricks

    🔹 Pre-annotate with a model: Run a generic detector (e.g., YOLOv8 pretrained → auto-import boxes → human adjust).
    🔹 Copy-Paste: Reuse boxes across video frames or similar images.
    🔹 Active Learning Loop: Train → find low-confidence detections → send those images back for labeling.


    8. Packaging the Dataset

    dataset/
     ├── train/
     │    ├── images/0001.jpg
     │    └── labels/0001.txt
     ├── val/
     └── test/
    annotations/            # if COCO/VOC
    class_list.txt          # if YOLO
    README.md               # cite guideline + license
    

    Zip it, checksum it (sha256sum), and version-control with DVC or Git-LFS.


    9. Common Gotchas & Fixes

    SymptomRoot CauseFix
    Model predicts off-center boxesLoose/imbalanced labelingEnforce tighter boxes, re-label outliers
    Class imbalanceOver-represented background imagesOversample/augment minority classes
    Can’t read dataset at train timeWrong path prefixes / newline encodingsVerify relative paths & LF/EOL, run a loader script

    10. Cheat-Sheet Summary

    1. Draft guidelines → pick format.
    2. Install tool → create class list.
    3. Draw tight boxes → save every image.
    4. Convert/export in target format.
    5. QA 5-10 % → iterate.
    6. Package dataset → version & release.

    Happy labeling! 🖍️🔲

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.