I am using the tensorflow function below to evaluate the accuracy of tflite models. Can…
- User
I am using the tensorflow function below to evaluate the accuracy of tflite models. Can you please add more evaluation metrics (precision, recall, f1-score).
def evaluate_tflite(model_path, dataset): # Load the TFLite model and allocate tensors. interpreter = tf.lite.Interpreter(model_path=model_path) interpreter.allocate_tensors() # Get input and output tensors. input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() # Define a list to store the predictions and labels predictions = [] labels = [] # Loop through the entire evaluation dataset for images, labels_batch in dataset: print('-', end=' ') # Loop through each image in the batch for image in images: # Add a batch dimension image = tf.expand_dims(image, axis=0) # Set the tensor to point to the input data to be inferred interpreter.set_tensor(input_details[0]['index'], image) # Run inference interpreter.invoke() # Get the output tensor output_data = interpreter.get_tensor(output_details[0]['index']) # Append the prediction to the list predictions.append(output_data) # Append the labels to the list labels.extend(labels_batch) # Convert the predictions and labels to numpy arrays predictions = np.array(predictions) #predictions = predictions.astype(int) predictions = predictions.reshape(256, 23) labels = np.array(labels) #labels = labels.astype(int) # Calculate the accuracy using categorical accuracy accuracy = tf.keras.metrics.categorical_accuracy(labels, predictions) # Calculate the average accuracy across all batches average_accuracy = tf.reduce_mean(accuracy) # Print the average accuracy print(f"Average accuracy: {average_accuracy}") Assistant
Here is one way to add additional evaluation metrics like precision, recall, and F1 score to the evaluate_tflite function:
from sklearn.metrics import precision_recall_fscore_support def evaluate_tflite(model_path, dataset): # Existing code to get predictions and labels # Calculate precision, recall, F1 score precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average='micro') # Print metrics print(f"Average accuracy: {average_accuracy}") print(f"Precision: {precision}") print(f"Recall: {recall}") print(f"F1 score: {f1}")The key steps are:
-
Import precision_recall_fscore_support from sklearn.metrics
-
Pass the true labels and predictions to this function to calculate the metrics
-
Specify average='micro' to calculate the metrics globally across all classes
-
Print out the precision, recall, and F1 score along with accuracy
This will add some commonly used classification evaluation metrics to complement accuracy. You can also calculate metrics like confusion matrix, AUC-ROC, etc in a similar way.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.