Is there a way to delete the for cycle, and do the same operatins with the same results…
- User
Is there a way to delete the for cycle, and do the same operatins with the same results ? def forward(self, inputs, **kwargs): users, items = inputs batch_size = users.shape[0]
# Get user embeddings and reshape for rules gu = self.Gu.weight[users] gu = gu.view(batch_size, self.n_rules, self.embed_k) # 3 dim: 1st_dim = user_id, 2nd_dim = rule_no, 3d_dim = embedding of user-rule # Ensure embeddings are in [0,1] # Get item embeddings gamma_i = self.Gi.weight[items] # Shape: [batch_size, embed_k] # Compute conjunctions for each rule and_scores = [] for r in range(self.n_rules): gamma_u_r = gu[:, r, :] # User embedding for rule r # score_and = self.conjunction_rule(u=gamma_u_r, i=gamma_i) # Shape: [batch_size] score_and = torch.sum(gamma_u_r * gamma_i, -1) and_scores.append(score_and.unsqueeze(1)) # Shape: [batch_size, 1] # Concatenate and compute disjunction and_scores_tensor = torch.nn.functional.sigmoid(torch.cat(and_scores, dim=1)) # Shape: [batch_size, n_rules] - Assistant
Yes, you can eliminate the for-loop by vectorizing the computation to perform the same operations with the same results. Here's how you can modify your code:
def forward(self, inputs, **kwargs): users, items = inputs batch_size = users.shape[0] # Get user embeddings and reshape for rules gu = self.Gu.weight[users] gu = gu.view(batch_size, self.n_rules, self.embed_k) # Shape: [batch_size, n_rules, embed_k] # Get item embeddings gamma_i = self.Gi.weight[items] # Shape: [batch_size, embed_k] # Compute conjunctions for each rule without a for-loop # Expand gamma_i to match the dimensions of gu gamma_i_expanded = gamma_i.unsqueeze(1) # Shape: [batch_size, 1, embed_k] # Element-wise multiplication and summation over the embedding dimension score_and = torch.sum(gu * gamma_i_expanded, dim=-1) # Shape: [batch_size, n_rules] # Apply the sigmoid function and_scores_tensor = torch.sigmoid(score_and) # Shape: [batch_size, n_rules]Explanation:
-
Broadcasting: By unsqueezing
gamma_ito shape[batch_size, 1, embed_k], it can be broadcasted across then_rulesdimension ofgu(which has shape[batch_size, n_rules, embed_k]). -
Element-wise Multiplication: The operation
gu * gamma_i_expandedresults in a tensor of shape[batch_size, n_rules, embed_k]where the item embeddings are multiplied with each rule's user embeddings. -
Summation over Embedding Dimension: Using
torch.sum(..., dim=-1)collapses theembed_kdimension, resulting in a tensor of shape[batch_size, n_rules], which contains the scores for each rule without looping. -
Applying Sigmoid: Finally, applying the sigmoid function to
score_andgives theand_scores_tensorwith the same shape and results as before.
This vectorized approach not only simplifies your code but can also lead to performance improvements due to optimized operations in PyTorch's backend.
Note: Make sure that your tensors are correctly shaped and that broadcasting rules apply. It's always a good idea to verify the dimensions using
tensor.shapeat each step during development.Final Code without the for-loop:
def forward(self, inputs, **kwargs): users, items = inputs batch_size = users.shape[0] # Get user embeddings and reshape for rules gu = self.Gu.weight[users] gu = gu.view(batch_size, self.n_rules, self.embed_k) # Get item embeddings gamma_i = self.Gi.weight[items] # Compute conjunctions for each rule without the for-loop score_and = torch.sum(gu * gamma_i.unsqueeze(1), dim=-1) # Apply sigmoid activation and_scores_tensor = torch.sigmoid(score_and)Yes; you can replace the loop with vectorized operations to get the same results without explicitly looping.
-