Explain how a transformer processes a two token prompt. Explain how the keys and values…
- User
Explain how a transformer processes a two token prompt. Explain how the keys and values are calculated.
- Assistant
Transformers are neural network models that process input sequences by applying self-attention. Here is a high-level overview of how a transformer processes a two token prompt:
-
Tokenization: The input prompt "Hello world" is split into tokens - ["Hello", "world"]. Each token is assigned an index, so they become ["Hello" -> 0, "world" -> 1].
-
Input Embeddings: Each token is converted into a vector representation by looking up the corresponding embedding. This results in a sequence of input embedding vectors [x0, x1].
-
Positional Encodings: The model adds positional encodings to each input embedding to encode their position in the sequence. This gives [x0 + p0, x1 + p1] where p0 and p1 are positional encodings.
-
Self-Attention: The self-attention layer allows each token to attend to all the other tokens in the sequence. It computes attention scores between each pair of tokens, which are used to derive a new representation per token that aggregates information from the entire sequence.
Specifically, it computes three vectors for each token:
- Query vector (Q): Derived from the token's input embedding + positional encoding
- Key vector (K): Derived similarly
- Value vector (V): Derived similarly
It then computes the dot product of the query with all keys, divides each by sqrt(dim) to get the attention scores, and uses a softmax to normalize the scores to get the attention weights. The output representation for each token is computed as the weighted sum of the values, using the attention weights.
- Feed Forward: Additional layers like MLPs are applied to the representations from the self-attention layer to derive the final representations for each token.
So in this way, the transformer architecture allows the model to build contextual representations of each token that depend on all the other tokens in the input sequence. The keys, values, and query vectors allow relating different tokens to each other through the self-attention mechanism.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.