Raw code of naive bayes how it is made
- User
Raw code of naive bayes how it is made
Assistant
Raw Implementation of Naive Bayes Classifier
Here's a raw implementation of a Naive Bayes classifier from scratch, showing how it works internally:
import numpy as np from collections import defaultdict class NaiveBayes: def __init__(self, feature_type='gaussian'): """ Initialize Naive Bayes classifier Parameters: feature_type (str): 'gaussian' for continuous features or 'multinomial' for discrete features """ self.feature_type = feature_type self.classes = None self.class_priors = {} self.class_counts = {} self.total_samples = 0 # For Gaussian Naive Bayes self.means = {} self.variances = {} # For Multinomial Naive Bayes self.feature_probs = {} def fit(self, X, y): """ Train the Naive Bayes classifier Parameters: X (numpy.ndarray): Training features y (numpy.ndarray): Training labels """ self.classes = np.unique(y) n_samples, n_features = X.shape self.total_samples = n_samples # Calculate class priors P(y) for c in self.classes: X_c = X[y == c] self.class_counts[c] = X_c.shape[0] self.class_priors[c] = X_c.shape[0] / n_samples if self.feature_type == 'gaussian': # Calculate mean and variance for each feature for Gaussian NB self.means[c] = np.mean(X_c, axis=0) self.variances[c] = np.var(X_c, axis=0) + 1e-10 # Add small value to avoid division by zero elif self.feature_type == 'multinomial': # Calculate feature probabilities for Multinomial NB # Laplace smoothing (add 1 to all counts) feature_counts = np.sum(X_c, axis=0) + 1 total_count = np.sum(feature_counts) self.feature_probs[c] = feature_counts / total_count def _calculate_likelihood(self, x, c): """ Calculate likelihood P(x|y) for a given class Parameters: x (numpy.ndarray): Feature vector c: Class label Returns: float: Likelihood probability """ if self.feature_type == 'gaussian': # Use Gaussian probability density function exponent = -0.5 * np.sum(((x - self.means[c]) ** 2) / self.variances[c]) denominator = np.sqrt(2 * np.pi * np.prod(self.variances[c])) return np.exp(exponent) / denominator elif self.feature_type == 'multinomial': # Use multinomial probability return np.prod(self.feature_probs[c] ** x) def predict_proba(self, X): """ Predict class probabilities for samples in X Parameters: X (numpy.ndarray): Test features Returns: numpy.ndarray: Class probabilities for each sample """ n_samples = X.shape[0] probs = np.zeros((n_samples, len(self.classes))) for i, x in enumerate(X): for j, c in enumerate(self.classes): # Calculate posterior probability using Bayes' theorem # P(y|x) ∝ P(x|y) * P(y) likelihood = self._calculate_likelihood(x, c) probs[i, j] = likelihood * self.class_priors[c] # Normalize to get probabilities if np.sum(probs[i]) > 0: probs[i] /= np.sum(probs[i]) return probs def predict(self, X): """ Predict class labels for samples in X Parameters: X (numpy.ndarray): Test features Returns: numpy.ndarray: Predicted class labels """ probs = self.predict_proba(X) return self.classes[np.argmax(probs, axis=1)] # Example usage if __name__ == "__main__": # Generate some sample data np.random.seed(42) X_train = np.random.randn(100, 2) # 100 samples, 2 features y_train = np.random.randint(0, 2, 100) # Binary classification X_test = np.random.randn(20, 2) # 20 test samples # Train Gaussian Naive Bayes nb = NaiveBayes(feature_type='gaussian') nb.fit(X_train, y_train) # Make predictions y_pred = nb.predict(X_test) print("Predictions:", y_pred) # Get probabilities probs = nb.predict_proba(X_test) print("Probabilities for first 5 samples:") for i in range(5): print(f"Sample {i+1}: Class 0: {probs[i, 0]:.4f}, Class 1: {probs[i, 1]:.4f}")Key Components Explained:
-
Class Initialization: Sets up the classifier with parameters for either Gaussian (continuous features) or Multinomial (discrete features) Naive Bayes.
-
Fit Method:
- Calculates class priors P(y) by counting class frequencies
- For Gaussian NB: Calculates mean and variance for each feature in each class
- For Multinomial NB: Calculates feature probabilities with Laplace smoothing
-
Likelihood Calculation:
- Gaussian NB: Uses the Gaussian probability density function
- Multinomial NB: Uses multinomial probability
-
Prediction:
- Applies Bayes' theorem: P(y|x) ∝ P(x|y) * P(y)
- Normalizes probabilities to sum to 1
- Returns the class with highest probability
This implementation demonstrates the core mathematical principles behind Naive Bayes without relying on external libraries for the algorithm itself.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.