Write a program that implements machine learning model inference.
- User
Write a program that implements machine learning model inference.
Assistant
The user wants a program that implements machine learning model inference.
1. Understand the Goal: The goal is to demonstrate how to use a pre-trained machine learning model to make predictions (inference). It shouldn't be about training a model from scratch (that would be "model training"), but rather loading an existing one and running data through it.
2. Choose the Tools/Libraries:
- Python: The standard language for ML.
- Libraries:
- Scikit-learn: Good for traditional ML (Random Forest, SVMs), simple to save/load with
pickleorjoblib. - TensorFlow/Keras: Good for Deep Learning, widely used.
- PyTorch: Also good for Deep Learning.
- ONNX: Interoperable format.
- Scikit-learn: Good for traditional ML (Random Forest, SVMs), simple to save/load with
Decision: I should provide a few examples because "inference" varies depending on the type of model. * Scenario A: Scikit-learn (Classical). Easy to understand, good for tabular data. * Scenario B: TensorFlow/Keras (Deep Learning). Good for images or complex patterns.
3. Structure of the Response:
- Introduction: Briefly explain what inference is (Input -> Model -> Output).
- Prerequisites: Mention libraries needed (
pip install ...). - Example 1: Scikit-learn (The "Hello World" of ML inference).
- Need a dummy model to load. Since I can't provide an external file, I will write a script that creates a small model, saves it, and then performs inference on it. This makes the code self-contained and runnable.
- Example 2: Deep Learning (TensorFlow/Keras).
- Similar structure: build a simple NN, save it, load it, predict.
- Example 3: Loading a real pre-trained model (Hugging Face/Transformers).
- This is very popular right now. It shows how to use a massive model like BERT or GPT for a task.
- Best Practices: Mention things like preprocessing, batching, and API serving (Flask/FastAPI).
4. Drafting the Code (Mental or Scratchpad):
-
Scikit-learn Approach:
import joblib from sklearn.ensemble import RandomForestClassifier # 1. Train & Save (just for demo) model = ... joblib.dump(model, 'model.pkl') # 2. Load loaded_model = joblib.load('model.pkl') # 3. Inference prediction = loaded_model.predict([[5.1, 3.5, 1.4, 0.2]]) -
TensorFlow Approach:
import tensorflow as tf # Similar flow: Load -> Predict
5. Refining the "Self-Contained" aspect: To make the answer most helpful, I should write a script that simulates the whole lifecycle but focuses on the inference part. Or, better yet, I'll provide a "Complete Simulation" (Train+Save+Load+Infer) so the user can copy-paste-run immediately without needing external files.
Let's structure the solution into two distinct parts:
- Classical ML (Scikit-Learn): Great for tabular data, regression, simple classification.
- Deep Learning (TensorFlow/Keras): Great for images, text, complex data.
6. Writing the specific code blocks:
-
Part 1: Scikit-learn (Iris dataset example)
- Import
sklearn,joblib,numpy. - Load Iris data.
- Train a simple classifier (Random Forest).
- Save it to
model.joblib. - The Inference Step: Load
model.joblib. Create a new sample (numpy array). Runpredict. Print result.
- Import
-
Part 2: TensorFlow (Simple Neural Net)
- Import
tensorflow. - Define a simple Sequential model.
- Save it (
model.save). - The Inference Step: Load (
tf.keras.models.load_model). Create input tensor. Runmodel.predict.
- Import
7. Adding Context/Explanation:
- Explain why we use
joblibvsh5vsonnx. - Explain the importance of Pre-processing (the model expects data in the same format it was trained on).
8. Review against User Request: The user asked for a program implementing inference. If I just give the inference part, they can't run it because they don't have the model file. Solution: I will generate a script that trains a dummy model first, then performs inference on it, clearly commenting where the "Inference Service" logic would start.
9. Final Polish: Make sure variable names are clear. Add comments. Ensure formatting is clean.
Self-Correction during drafting: I should avoid over-engineering with Flask/FastAPI unless requested, but adding a small note about it at the end is good practice for production context.
Let's stick to Python as the standard.
Plan:
- Scikit-Learn Example: Focuses on loading a
.joblibfile (standard for classical ML). - TensorFlow/Keras Example: Focuses on loading a
.h5or SavedModel format. - Key Concepts: Preprocessing and Postprocessing.
Let's write the code. To provide a working example, I need to make a few assumptions because "inference" depends heavily on the library (Scikit-Learn, TensorFlow, PyTorch) and the type of data (images, text, numbers).
Below is a complete, runnable Python program. To make it easy to test, this program includes a "Training Phase" to create a dummy model file, followed by the "Inference Phase" that loads and uses it.
I will provide two examples:
- Classical ML (Scikit-Learn): Best for tabular data (CSV, numbers).
- Deep Learning (TensorFlow/Keras): Best for images or complex patterns.
Example 1: Classical Machine Learning (Scikit-Learn)
This example uses
joblibto save/load a Random Forest model. This is the standard approach for regression and classification tasks on structured data.Prerequisites:
pip install scikit-learn joblib numpyThe Program:
import os import joblib import numpy as np from sklearn.datasets import load_iris from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split # ========================================== # PART 1: Training & Saving (One-time setup) # ========================================== def train_and_save_model(): print("--- Training dummy model ---") # Load sample data (Iris dataset) data = load_iris() X, y = data.data, data.target # Split data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Train a simple model model = RandomForestClassifier(n_estimators=100, random_state=42) model.fit(X_train, y_train) # Save the model to disk filename = 'iris_model.joblib' joblib.dump(model, filename) accuracy = model.score(X_test, y_test) print(f"Model trained with accuracy: {accuracy:.2f}") print(f"Model saved to '{filename}'\n") # ========================================== # PART 2: Inference (The core request) # ========================================== class ModelInference: def __init__(self, model_path): self.model_path = model_path self.model = None self.load_model() def load_model(self): """Loads the model from disk into memory.""" if not os.path.exists(self.model_path): raise FileNotFoundError(f"Model file not found at {self.model_path}") print(f"--- Loading model from {self.model_path} ---") self.model = joblib.load(self.model_path) def preprocess(self, raw_input): """ Convert raw input into the format the model expects. In a real app, this handles normalization, one-hot encoding, etc. """ # Expecting a list of features: [sepal_length, sepal_width, petal_length, petal_width] return np.array(raw_input).reshape(1, -1) def predict(self, input_data): """Run inference on the input data.""" if self.model is None: raise Exception("Model not loaded.") # 1. Preprocess processed_data = self.preprocess(input_data) # 2. Infer # Note: sklearn uses .predict() for classes, .predict_proba() for probabilities prediction = self.model.predict(processed_data) probabilities = self.model.predict_proba(processed_data) return prediction[0], probabilities[0] # ========================================== # PART 3: Execution # ========================================== if __name__ == "__main__": MODEL_FILE = 'iris_model.joblib' # 1. Ensure model exists (Run training simulation) if not os.path.exists(MODEL_FILE): train_and_save_model() # 2. Initialize Inference Engine inference_engine = ModelInference(MODEL_FILE) # 3. Define new, unseen data # Example: [5.1, 3.5, 1.4, 0.2] (Likely Setosa) new_flower_data = [5.1, 3.5, 1.4, 0.2] # 4. Run Inference prediction_class, confidence = inference_engine.predict(new_flower_data) # 5. Post-processing (Human readable) target_names = load_iris().target_names print(f"Input Data: {new_flower_data}") print(f"Predicted Class: {target_names[prediction_class]}") print(f"Confidence Scores: {confidence}")
Example 2: Deep Learning (TensorFlow/Keras)
If you are doing deep learning (e.g., recognizing images), the syntax differs slightly. Here is the inference logic for a Keras model.
Prerequisites:
pip install tensorflow numpyThe Inference Snippet:
import tensorflow as tf import numpy as np class KerasInference: def __init__(self, model_path): self.model = tf.keras.models.load_model(model_path) def predict(self, input_data): # 1. Preprocess (Example: Normalizing pixel values to 0-1) input_data = np.array(input_data) / 255.0 # 2. Add batch dimension if missing (Model expects [Batch, Height, Width, Channels]) if len(input_data.shape) == 3: input_data = np.expand_dims(input_data, axis=0) # 3. Run Inference predictions = self.model.predict(input_data) return predictions # Usage # inference = KerasInference('my_image_model.h5') # result = inference.predict(image_array)Key Concepts in the Code
- Serialization (
joblib.dump/load): In the real world, training takes hours. You do it once on a powerful server, save the "weights" to a file, and move that file to the inference server. - Preprocessing: Notice the
preprocessstep in the class. This is where most ML projects fail in production. The data must be formatted exactly the same way during inference as it was during training (e.g., same scaling, same order of columns). - Batch Dimension: Libraries like TensorFlow and PyTorch expect inputs in "batches" (lists of items), even if you are only predicting one item. You often have to reshape an array from
[features]to[1, features].
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.