llama-4-scout-17b-16e-instruct· Meta LMArena Human Preference 140K
Show the parameters for the last stage of Swin-B
- User
Show the parameters for the last stage of Swin-B
- Assistant
Swin-B (Last Stage) Parameters
For Swin-B, the architecture has 4 stages. Here's a breakdown of the parameters across stages:
- Stage 1:
d_model(C): 96n_heads: 3d_head: 32n_ctx(per window): 7 * 7 = 49
- Stage 2:
d_model(C): 192 (2 * 96)n_heads: 6 (2 * 3)d_head: 32 (constant)n_ctx(per window): 7 * 7 = 49 (window size doesn't change, but tokens are merged)
- Stage 3:
d_model(C): 384 (2 * 192)n_heads: 12 (2 * 6)d_head: 32 (constant)n_ctx(per window): 7 * 7 = 49
- Stage 4 (Last Stage):
d_model(C): 768 (2 * 384)n_heads: 24 (2 * 12)d_head: 32 (constant)n_ctx(per window): 7 * 7 = 49
So, for the last stage of Swin-B:
d_model= 768n_heads= 24d_head= 32 (d_model / n_heads= 768 / 24 = 32)n_ctx= 49 (per window, 7x7)
Keep in mind that these values are based on the standard Swin-B configuration. Depending on specific modifications or applications, these parameters might vary.
- Stage 1:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.