the following pytorch model is giving me "TypeError: forward() got an unexpected keyword…
- User
the following pytorch model is giving me "TypeError: forward() got an unexpected keyword argument 'input_mask'":
class transformer_test(nn.Module): def init(self, dim_feedforward=100, num_encoder_layers=5): super(transformer_test, self).init() self.vocab_size = 256 self.emb_size = 512 self.nhead = 4 self.embedding = nn.Embedding(self.vocab_size, self.emb_size) self.transformer = nn.Transformer(d_model=self.emb_size, nhead=self.nhead, num_encoder_layers=num_encoder_layers, dim_feedforward=dim_feedforward) self.decoder = nn.Linear(self.emb_size, self.vocab_size)
def forward(self, input): input = input.cuda() input = self.embedding(input) input_mask = self._generate_square_subsequent_mask(len(input)).to(input.device) output = self.transformer(input, input_mask=input_mask) output = self.decoder(output[-1, :, :]) return output def _generate_square_subsequent_mask(self, sz): mask = (torch.triu(torch.ones(sz, sz)) == 1).transpose(0, 1) mask = mask.float().masked_fill(mask == 0, float('-inf')).masked_fill(mask == 1, float(0.0)) return mask - Assistant
The error message suggests that the
forward()function is being called with an unexpected keyword argumentinput_mask. This could be happening if you are calling theforward()function with incorrect argument names.To solve the issue, make sure that you call the
forward()function with the correct argument names. In this case, theinput_maskargument is expected by thetransformermodule inside theforward()function. Therefore, you need to pass theinput_maskargument as shown below:model = transformer_test() input = torch.rand(10, 20) input_mask = model._generate_square_subsequent_mask(20) output = model(input, input_mask=input_mask)Here,
input_maskis generated using the_generate_square_subsequent_mask()function defined in the model. You can also pass a pre-definedinput_masktensor instead of generating it using the function. - User
This error happend at the line "output = self.transformer(input, input_mask=input_mask)".
- Assistant
Yes, that's correct. The
forward()method is trying to passinput_maskas a keyword argument to theself.transformer()method, but theself.transformer()method is not expecting this keyword argument. So, you need to modify theforward()method to remove this argument.To fix this issue, you can change the
forward()method as follows:class transformer_test(nn.Module): def __init__(self, dim_feedforward=100, num_encoder_layers=5): super(transformer_test, self).__init__() self.vocab_size = 256 self.emb_size = 512 self.nhead = 4 self.embedding = nn.Embedding(self.vocab_size, self.emb_size) self.transformer = nn.Transformer(d_model=self.emb_size, nhead=self.nhead, num_encoder_layers=num_encoder_layers, dim_feedforward=dim_feedforward) self.decoder = nn.Linear(self.emb_size, self.vocab_size) def forward(self, input, input_mask=None): input = input.cuda() input = self.embedding(input) if input_mask is not None: output = self.transformer(input, input_mask=input_mask) else: output = self.transformer(input) output = self.decoder(output[-1, :, :]) return outputIn this modified version of the
forward()method, we have added an optional parameterinput_mask=Noneto the function header. If an input mask is provided, then we use that mask while calling theself.transformer()method. If no input mask is provided, we call theself.transformer()method without any keyword arguments.This should solve the current error that you are facing.