Raw code of naive bayes how it is made
- User
Raw code of naive bayes how it is made
Assistant
import numpy as np class NaiveBayes: """ A basic implementation of Naive Bayes for classification. Supports Gaussian (continuous) and Multinomial (discrete) distributions. """ def __init__(self, model_type='gaussian'): """ Initializes the Naive Bayes classifier. Args: model_type (str): 'gaussian' for continuous data (Gaussian Naive Bayes), 'multinomial' for discrete data (Multinomial Naive Bayes). Defaults to 'gaussian'. """ self.model_type = model_type self.classes = None # Unique class labels self.class_priors = None # P(y) - Prior probabilities of each class self.parameters = None # Model parameters (mean/variance for Gaussian, counts for Multinomial) def fit(self, X, y): """ Fits the Naive Bayes model to the training data. Args: X (np.ndarray): Training data features (n_samples, n_features). y (np.ndarray): Training data labels (n_samples,). """ self.classes = np.unique(y) n_classes = len(self.classes) n_samples, n_features = X.shape self.class_priors = np.zeros(n_classes) self.parameters = [] # A list to store parameters for each class for idx, c in enumerate(self.classes): # Filter data for current class X_c = X[y == c] self.class_priors[idx] = X_c.shape[0] / n_samples # P(y=c) = # samples in class c / total samples if self.model_type == 'gaussian': # Estimate mean and variance for each feature in each class mean = np.mean(X_c, axis=0) variance = np.var(X_c, axis=0) self.parameters.append({'mean': mean, 'variance': variance}) elif self.model_type == 'multinomial': # Estimate feature counts for each class counts = np.sum(X_c, axis=0) # Count occurrences of each feature self.parameters.append({'counts': counts}) def _gaussian_likelihood(self, x, mean, variance): """ Calculates the Gaussian probability density function. Args: x (float): Data point. mean (float): Mean of the Gaussian distribution. variance (float): Variance of the Gaussian distribution. Returns: float: Probability density. """ if variance == 0: #Handle zero variance (add a small value for numerical stability) variance = 1e-6 # A very small variance to avoid division by zero. numerator = np.exp(-((x - mean)**2) / (2 * variance)) denominator = np.sqrt(2 * np.pi * variance) return numerator / denominator def _multinomial_likelihood(self, x, counts, alpha=1.0): """ Calculates the Multinomial likelihood (probability of seeing x given class). Uses Laplace smoothing (add-alpha smoothing) to avoid zero probabilities. Args: x (np.ndarray): Data point (features). counts (np.ndarray): Feature counts for the class. alpha (float): Smoothing parameter (default: 1.0 for Laplace smoothing). Returns: float: Likelihood. """ smoothed_counts = counts + alpha total_count = np.sum(smoothed_counts) log_likelihood = np.sum(x * (np.log(smoothed_counts) - np.log(total_count))) # Use logs for numerical stability return np.exp(log_likelihood) # Exponentiate back to get the probability def predict(self, X): """ Predicts the class labels for the given data. Args: X (np.ndarray): Data to predict (n_samples, n_features). Returns: np.ndarray: Predicted class labels (n_samples,). """ predictions = [self._predict_single(x) for x in X] return np.array(predictions) def _predict_single(self, x): """ Predicts the class label for a single data point. Args: x (np.ndarray): Single data point (n_features,). Returns: object: Predicted class label. """ posteriors = [] for idx, c in enumerate(self.classes): prior = self.class_priors[idx] # P(y=c) if self.model_type == 'gaussian': likelihood = 1.0 # Initialize to 1 for multiplying probabilities for feature_idx in range(len(x)): likelihood *= self._gaussian_likelihood(x[feature_idx], self.parameters[idx]['mean'][feature_idx], self.parameters[idx]['variance'][feature_idx]) elif self.model_type == 'multinomial': likelihood = self._multinomial_likelihood(x, self.parameters[idx]['counts']) posterior = prior * likelihood # P(y=c|x) = P(x|y=c) * P(y=c) posteriors.append(posterior) # Return the class with the highest posterior probability return self.classes[np.argmax(posteriors)] # Example Usage: Gaussian Naive Bayes if __name__ == '__main__': from sklearn.model_selection import train_test_split from sklearn import datasets from sklearn.metrics import accuracy_score # Load the Iris dataset iris = datasets.load_iris() X, y = iris.data, iris.target # Split the data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Create and train the Gaussian Naive Bayes classifier gnb = NaiveBayes(model_type='gaussian') gnb.fit(X_train, y_train) # Make predictions y_pred = gnb.predict(X_test) # Evaluate the accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Gaussian Naive Bayes Accuracy: {accuracy:.4f}") # Example Usage: Multinomial Naive Bayes (with dummy data - needs non-negative features) X_multinomial = np.random.randint(0, 10, size=(100, 5)) # Create random non-negative data y_multinomial = np.random.randint(0, 2, size=(100,)) # binary classes X_train_multi, X_test_multi, y_train_multi, y_test_multi = train_test_split(X_multinomial, y_multinomial, test_size=0.2, random_state=42) mnb = NaiveBayes(model_type='multinomial') mnb.fit(X_train_multi, y_train_multi) y_pred_multi = mnb.predict(X_test_multi) accuracy_multi = accuracy_score(y_test_multi, y_pred_multi) print(f"Multinomial Naive Bayes Accuracy: {accuracy_multi:.4f}")Key improvements and explanations:
- Clear Class Structure: Encapsulates the Naive Bayes logic within a class. This makes the code more organized, reusable, and easier to understand.
- Gaussian and Multinomial Support: Implements both Gaussian and Multinomial Naive Bayes, selected using the
model_typeparameter in the constructor. This makes the code more versatile. - Gaussian Likelihood: Calculates the probability density using the Gaussian distribution. Includes handling for zero variance to prevent errors.
- Multinomial Likelihood: Calculates the likelihood for Multinomial Naive Bayes. Uses Laplace smoothing (add-one smoothing) to prevent zero probabilities, which is crucial for Multinomial Naive Bayes. The probabilities are calculated in the log domain to prevent underflow for small probabilities.
fit()method: Computes the necessary parameters (means and variances for Gaussian, counts for Multinomial) from the training data and stores them. Calculates class priors.predict()method: Predicts class labels for new data by calculating posterior probabilities for each class and selecting the class with the highest probability._predict_single()method: Handles the prediction for a single data point. This separation improves code readability.- Prior Probabilities: Explicitly calculates and uses class prior probabilities, which is an essential component of Naive Bayes.
- Log Probabilities (Multinomial): Uses log probabilities in the
_multinomial_likelihoodfunction for numerical stability, preventing potential underflow issues when dealing with very small probabilities. The result is then exponentiated. - Complete Example: Includes a working example with the Iris dataset for Gaussian Naive Bayes and a dummy example for Multinomial Naive Bayes. The Multinomial example generates suitable non-negative data.
- Docstrings: Comprehensive docstrings explain the purpose, arguments, and return values of each method.
- Error Handling: Includes a check for zero variance in the Gaussian likelihood calculation and adds a small constant to prevent division by zero.
- Clarity: Uses meaningful variable names and comments to improve code readability.
- Numerical Stability: Uses log probabilities in the Multinomial likelihood calculation to prevent underflow.
This revised response provides a much more robust, well-documented, and functional implementation of Naive Bayes from scratch, covering both Gaussian and Multinomial variants, complete with examples and handling for potential issues. It addresses all the key aspects of the algorithm and provides a solid foundation for further customization and experimentation.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.