Skip to content
The Chat Archive
o1-preview-2024-09-12· OpenAI WildChat-4.8M (AI2)

Generate code for a simple ai language model in python

  1. User

    Generate code for a simple ai language model in python

  2. Assistant

    Certainly! Below is a Python code for a simple n-gram language model. This model will learn the probabilities of word sequences from a given text corpus and then use those probabilities to generate new text.

    In this example, we'll use a bigram model (n=2), which looks at pairs of words to predict the next word.

    import random
    import nltk
    from nltk.util import ngrams
    from collections import defaultdict, Counter
    
    # Ensure you have the necessary NLTK data files
    nltk.download('punkt')
    
    class NGramLanguageModel:
        def __init__(self, n):
            self.n = n  # The 'n' in n-gram
            self.ngrams = defaultdict(Counter)
            self.vocab = set()
    
        def train(self, text):
            """
            Train the n-gram language model with the given text.
            :param text: The training text as a string.
            """
            tokens = nltk.word_tokenize(text.lower())
            self.vocab.update(tokens)
            
            # Create n-grams from the tokens
            n_grams = ngrams(tokens, self.n, pad_left=True, pad_right=True, 
                             left_pad_symbol='<s>', right_pad_symbol='</s>')
            
            for gram in n_grams:
                prefix = gram[:-1]
                word = gram[-1]
                self.ngrams[prefix][word] += 1
    
        def generate(self, num_words=50):
            """
            Generate text using the trained language model.
            :param num_words: The number of words to generate.
            :return: A generated text string.
            """
            current = ('<s>',) * (self.n - 1)
            generated_words = []
    
            for _ in range(num_words):
                next_words = self.ngrams[current]
                if not next_words:
                    break  # Cannot proceed if there are no next words
                next_word = random.choices(list(next_words.keys()), 
                                           weights=next_words.values())[0]
                if next_word == '</s>':
                    break
                generated_words.append(next_word)
                current = (*current[1:], next_word)
    
            return ' '.join(generated_words)
    
    # Example usage:
    if __name__ == "__main__":
        # Sample training text
        training_text = """
        Once upon a time, there was a kingdom far away. The king and the queen ruled the land with kindness.
        The people of the kingdom were happy and prosperous. One day, a dragon appeared in the kingdom.
        """
    
        # Create and train the model
        model = NGramLanguageModel(n=2)  # Bigram model
        model.train(training_text)
    
        # Generate text
        generated_text = model.generate(num_words=30)
        print("Generated Text:")
        print(generated_text)
    

    Explanation:

    • Imports:

      • random for random word selection.
      • nltk for tokenizing text and handling n-grams.
      • defaultdict and Counter for counting occurrences of word sequences.
    • NGramLanguageModel Class:

      • The constructor __init__ initializes the n-gram model.
      • The train method processes the input text:
        • Tokenizes the text into words.
        • Updates the vocabulary.
        • Creates n-grams with padding symbols <s> (start) and </s> (end).
        • Counts occurrences of each n-gram prefix and next word.
      • The generate method produces new text:
        • Starts with the start symbol(s).
        • At each step, chooses the next word based on the learned probabilities.
        • Stops if it reaches the end symbol </s> or after generating the specified number of words.
    • Example Usage:

      • A sample text is used to train the model.
      • The model generates new text based on the learned probabilities.

    Note: Before running this code, ensure that you have the NLTK library installed and that you've downloaded the necessary data files with nltk.download('punkt').

    Output Example:

    Generated Text:
    one day , a dragon appeared in the kingdom . the king and the queen ruled the land with kindness .
    

    This simple n-gram model learns word transition probabilities from the training text and can generate new sentences that mimic the style of the input.