Skip to content
The Chat Archive
llama-4-scout-17b-16e-instruct· Meta LMArena Human Preference 140K

Show the parameters for the last stage of Swin-B

  1. User

    Show the parameters for the last stage of Swin-B

  2. Assistant

    Swin-B (Last Stage) Parameters

    For Swin-B, the architecture has 4 stages. Here's a breakdown of the parameters across stages:

    1. Stage 1:
      • d_model (C): 96
      • n_heads: 3
      • d_head: 32
      • n_ctx (per window): 7 * 7 = 49
    2. Stage 2:
      • d_model (C): 192 (2 * 96)
      • n_heads: 6 (2 * 3)
      • d_head: 32 (constant)
      • n_ctx (per window): 7 * 7 = 49 (window size doesn't change, but tokens are merged)
    3. Stage 3:
      • d_model (C): 384 (2 * 192)
      • n_heads: 12 (2 * 6)
      • d_head: 32 (constant)
      • n_ctx (per window): 7 * 7 = 49
    4. Stage 4 (Last Stage):
      • d_model (C): 768 (2 * 384)
      • n_heads: 24 (2 * 12)
      • d_head: 32 (constant)
      • n_ctx (per window): 7 * 7 = 49

    So, for the last stage of Swin-B:

    • d_model = 768
    • n_heads = 24
    • d_head = 32 (d_model / n_heads = 768 / 24 = 32)
    • n_ctx = 49 (per window, 7x7)

    Keep in mind that these values are based on the standard Swin-B configuration. Depending on specific modifications or applications, these parameters might vary.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.