if i have an einsum: gamma = gamma.permute(0, 2, 1). cond = torch.einsum("bts,bsd->btd",…
- User
if i have an einsum:
gamma = gamma.permute(0, 2, 1). cond = torch.einsum("bts,bsd->btd", gamma, self.conditioner(txt_pd[:, 1:]))is that the same as:
cond = torch.einsum("bst,bsd->btd", gamma, self.conditioner(txt_pd[:, 1:]))?
Assistant
No, these are not the same. The key difference is in how the
gammatensor dimensions are interpreted in the einsum operation.In the first version:
gammais permuted from shape[b, s, t]to[b, t, s]- The einsum
"bts,bsd->btd"expectsgammato have dimensions[b, t, s] - This performs:
sum over sofgamma[b,t,s] * conditioner[b,s,d]
In the second version:
gammakeeps its original shape[b, s, t]- The einsum
"bst,bsd->btd"expectsgammato have dimensions[b, s, t] - This performs:
sum over sofgamma[b,s,t] * conditioner[b,s,d]
The mathematical operations are different:
- First version:
cond[b,t,d] = Σ_s gamma[b,t,s] * conditioner[b,s,d] - Second version:
cond[b,t,d] = Σ_s gamma[b,s,t] * conditioner[b,s,d]
Since
gamma[b,t,s] ≠ gamma[b,s,t]in general (unlessgammahappens to be symmetric in those dimensions), these will produce different results.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.