Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-2024-05-13· OpenAIAi2 WildChat-4.8M (AI2)

i have tensor torch.Size([512]) with values -1 or 1 i want to make it [512,2] where…

  1. User

    i have tensor torch.Size([512])

    with values -1 or 1

    i want to make it [512,2] where first element would be 1 if value is 1 and 0 if value is -1

  2. Assistant

    You can achieve this by creating a new tensor with the desired shape and values using PyTorch. Here's how you can do it:

    1. First, create a new tensor of shape [512, 2] with all zeros.
    2. Then, use the original tensor to index and set the appropriate positions to 1.

    Here's a step-by-step example:

    import torch
    
    # Your original tensor
    original_tensor = torch.randint(0, 2, (512,)) * 2 - 1  # Example tensor with -1 and 1
    
    # Initialize the new tensor with zeros
    new_tensor = torch.zeros(512, 2)
    
    # Set the appropriate positions based on the original tensor values
    new_tensor[original_tensor == 1, 0] = 1  # Set (row, 0) to 1 where original tensor is 1
    new_tensor[original_tensor == -1, 1] = 1  # Set (row, 1) to 1 where original tensor is -1
    
    # Check the result
    print(new_tensor)
    

    In this code:

    • torch.randint(0, 2, (512,)) * 2 - 1 generates an example tensor with values -1 or 1.
    • torch.zeros(512, 2) initializes the new tensor with the shape [512, 2].
    • new_tensor[original_tensor == 1, 0] = 1 sets the first column to 1 for rows where the original tensor value is 1.
    • new_tensor[original_tensor == -1, 1] = 1 sets the second column to 1 for rows where the original tensor value is -1.

    This way, you obtain a [512, 2] shaped tensor where each row reflects the value in the original tensor appropriately.

  3. User

    lr = 5e-5 optimizer = torch.optim.AdamW(list(my_model.parameters()), lr = lr )

    loader = DataLoader(dataset, batch_size=32, shuffle=True, drop_last=True)

    criterion = nn.CrossEntropyLoss().to('cuda') for i in range(5): bar = tqdm(loader)

    	for batch in bar:
    		optimizer.zero_grad()
    		labels = my_model(batch['f1'].to('cuda'), batch['f2'].to('cuda'))
    		
    		label_numbers = batch['v']
    		new_tensor = torch.zeros(len(labels), 2)
    		new_tensor[label_numbers == 1, 0] = 1 
    		new_tensor[label_numbers == -1, 1] = 1  
    
    		loss = criterion(labels, new_tensor.to('cuda'))
    
    		loss.backward()
    		optimizer.step()
    		print(loss.item())
    
    		#torch.save(my_model.state_dict(), dataset_dir + f'model/classifier_score_{i}.pth')
    		#torch.save(model.state_dict(), dataset_dir + f'model/vit_score_{i}.pth')
    

    make it print loss every 50 steps

  4. Assistant

    To print the loss value every 50 steps, you can use a simple counter that increments with each step in the inner loop. You can then check if the counter is divisible by 50 to determine whether to print the loss. Here’s how you can modify your code to achieve this:

    import torch
    import torch.nn as nn
    from torch.utils.data import DataLoader
    from tqdm import tqdm
    
    lr = 5e-5
    optimizer = torch.optim.AdamW(list(my_model.parameters()), lr=lr)
    
    loader = DataLoader(dataset, batch_size=32, shuffle=True, drop_last=True)
    
    criterion = nn.CrossEntropyLoss().to('cuda')
    
    for epoch in range(5):
        bar = tqdm(loader)
        
        step = 0  # Initialize step counter
        
        for batch in bar:
            optimizer.zero_grad()
            labels = my_model(batch['f1'].to('cuda'), batch['f2'].to('cuda'))
            
            label_numbers = batch['v']
            new_tensor = torch.zeros(len(labels), 2)
            new_tensor[label_numbers == 1, 0] = 1
            new_tensor[label_numbers == -1, 1] = 1
            
            loss = criterion(labels, new_tensor.to('cuda'))
    
            loss.backward()
            optimizer.step()
            
            # Increment the step counter
            step += 1
            
            # Print loss every 50 steps
            if step % 50 == 0:
                print(f"Epoch {epoch}, Step {step}, Loss: {loss.item()}")
    
            # Uncomment these lines if you need to save your model after each epoch
            # torch.save(my_model.state_dict(), dataset_dir + f'model/classifier_score_{epoch}.pth')
            # torch.save(model.state_dict(), dataset_dir + f'model/vit_score_{epoch}.pth')
    

    Here's the breakdown of the added/modified parts:

    1. Initialize the step counter: step = 0.
    2. Increment the step counter in each iteration of the inner loop: step += 1.
    3. Conditionally print the loss: Use a conditional statement to check if the step count is divisible by 50 (if step % 50 == 0). If true, print the loss.

    This will print the loss every 50 steps within each epoch.