i have tensor torch.Size([512]) with values -1 or 1 i want to make it [512,2] where…
- User
i have tensor torch.Size([512])
with values -1 or 1
i want to make it [512,2] where first element would be 1 if value is 1 and 0 if value is -1
- Assistant
You can achieve this by creating a new tensor with the desired shape and values using PyTorch. Here's how you can do it:
- First, create a new tensor of shape
[512, 2]with all zeros. - Then, use the original tensor to index and set the appropriate positions to 1.
Here's a step-by-step example:
import torch # Your original tensor original_tensor = torch.randint(0, 2, (512,)) * 2 - 1 # Example tensor with -1 and 1 # Initialize the new tensor with zeros new_tensor = torch.zeros(512, 2) # Set the appropriate positions based on the original tensor values new_tensor[original_tensor == 1, 0] = 1 # Set (row, 0) to 1 where original tensor is 1 new_tensor[original_tensor == -1, 1] = 1 # Set (row, 1) to 1 where original tensor is -1 # Check the result print(new_tensor)In this code:
torch.randint(0, 2, (512,)) * 2 - 1generates an example tensor with values -1 or 1.torch.zeros(512, 2)initializes the new tensor with the shape[512, 2].new_tensor[original_tensor == 1, 0] = 1sets the first column to 1 for rows where the original tensor value is 1.new_tensor[original_tensor == -1, 1] = 1sets the second column to 1 for rows where the original tensor value is -1.
This way, you obtain a
[512, 2]shaped tensor where each row reflects the value in the original tensor appropriately. - First, create a new tensor of shape
- User
lr = 5e-5 optimizer = torch.optim.AdamW(list(my_model.parameters()), lr = lr )
loader = DataLoader(dataset, batch_size=32, shuffle=True, drop_last=True)
criterion = nn.CrossEntropyLoss().to('cuda') for i in range(5): bar = tqdm(loader)
for batch in bar: optimizer.zero_grad() labels = my_model(batch['f1'].to('cuda'), batch['f2'].to('cuda')) label_numbers = batch['v'] new_tensor = torch.zeros(len(labels), 2) new_tensor[label_numbers == 1, 0] = 1 new_tensor[label_numbers == -1, 1] = 1 loss = criterion(labels, new_tensor.to('cuda')) loss.backward() optimizer.step() print(loss.item()) #torch.save(my_model.state_dict(), dataset_dir + f'model/classifier_score_{i}.pth') #torch.save(model.state_dict(), dataset_dir + f'model/vit_score_{i}.pth')make it print loss every 50 steps
- Assistant
To print the loss value every 50 steps, you can use a simple counter that increments with each step in the inner loop. You can then check if the counter is divisible by 50 to determine whether to print the loss. Here’s how you can modify your code to achieve this:
import torch import torch.nn as nn from torch.utils.data import DataLoader from tqdm import tqdm lr = 5e-5 optimizer = torch.optim.AdamW(list(my_model.parameters()), lr=lr) loader = DataLoader(dataset, batch_size=32, shuffle=True, drop_last=True) criterion = nn.CrossEntropyLoss().to('cuda') for epoch in range(5): bar = tqdm(loader) step = 0 # Initialize step counter for batch in bar: optimizer.zero_grad() labels = my_model(batch['f1'].to('cuda'), batch['f2'].to('cuda')) label_numbers = batch['v'] new_tensor = torch.zeros(len(labels), 2) new_tensor[label_numbers == 1, 0] = 1 new_tensor[label_numbers == -1, 1] = 1 loss = criterion(labels, new_tensor.to('cuda')) loss.backward() optimizer.step() # Increment the step counter step += 1 # Print loss every 50 steps if step % 50 == 0: print(f"Epoch {epoch}, Step {step}, Loss: {loss.item()}") # Uncomment these lines if you need to save your model after each epoch # torch.save(my_model.state_dict(), dataset_dir + f'model/classifier_score_{epoch}.pth') # torch.save(model.state_dict(), dataset_dir + f'model/vit_score_{epoch}.pth')Here's the breakdown of the added/modified parts:
- Initialize the
stepcounter:step = 0. - Increment the
stepcounter in each iteration of the inner loop:step += 1. - Conditionally print the loss: Use a conditional statement to check if the step count is divisible by 50 (
if step % 50 == 0). If true, print the loss.
This will print the loss every 50 steps within each epoch.
- Initialize the