I builded an model to predict handwritten word but it it does not work well this is my…
- User
I builded an model to predict handwritten word but it it does not work well this is my code can you give some improvement: import numpy as np import pandas as pd import os import cv2 import imutils import random import matplotlib.pyplot as plt from sklearn.model_selection import train_test_split from keras.models import Sequential from sklearn.preprocessing import LabelBinarizer from keras.layers import Flatten, Dense,Dropout from keras.layers import Conv2D, MaxPooling2D
dir = "Final-state-Total" dir2 = "dataset1" df = pd.read_csv(os.path.join(dir2,"english.csv")) df['label'] = df['label'].str.upper() data = [] img_size = 32 non_chars = ["#","$","&","@"] for i in os.listdir(dir): if i in non_chars: continue listeToAdd = df[ df['label'] == i] listeToAdd = listeToAdd['image'].tolist() for j in listeToAdd: img = cv2.imread(os.path.join(dir2,j),0) img = cv2.resize(img,(img_size,img_size)) data.append([img,i]) count = 0 sub_directory = os.path.join(dir,i) for j in os.listdir(sub_directory): count+=1 if count > 4000: break img = cv2.imread(os.path.join(sub_directory,j),0) img = cv2.resize(img,(img_size,img_size)) data.append([img,i])
random.shuffle(data) len(data) # 21174 dataX = [] dataY = [] for feautures,label in data: dataX.append(feautures) dataY.append(label) nb_classes = len(np.unique(dataY)) # 64 classes LB = LabelBinarizer() dataY = LB.fit_transform(dataY) dataX = np.array(dataX)/255.0 dataX = dataX.reshape(-1,img_size,img_size,1) X_train, X_val, y_train, y_val = train_test_split(dataX, dataY, test_size=0.2, random_state=42) # (16939, 32, 32, 1) (4235, 32, 32, 1) (16939, 64) (4235, 64) model = Sequential() model.add(Conv2D(32, (3, 3), padding = "same", activation='relu', input_shape=(32,32,1))) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(64, (3, 3), activation='relu')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(128, (3, 3), activation='relu')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Dropout(0.25))
model.add(Flatten()) model.add(Dense(128, activation='relu')) model.add(Dropout(0.2)) model.add(Dense(64, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) history = model.fit(X_train,y_train, epochs=50, batch_size=32, validation_data = (X_val, y_val), verbose=1) def sort_contours(cnts, method="left-to-right"): reverse = False i = 0 if method == "right-to-left" or method == "bottom-to-top": reverse = True if method == "top-to-bottom" or method == "bottom-to-top": i = 1 boundingBoxes = [cv2.boundingRect(c) for c in cnts] (cnts, boundingBoxes) = zip(*sorted(zip(cnts, boundingBoxes), key=lambda b:b[1][i], reverse=reverse)) # return the list of sorted contours and bounding boxes return (cnts, boundingBoxes) def get_letters(img): letters = [] image = cv2.imread(img) gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) ret,thresh1 = cv2.threshold(gray ,127,255,cv2.THRESH_BINARY_INV) dilated = cv2.dilate(thresh1, None, iterations=2)
cnts = cv2.findContours(dilated.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) cnts = imutils.grab_contours(cnts) cnts = sort_contours(cnts, method="left-to-right")[0] # loop over the contours for c in cnts: if cv2.contourArea(c) > 10: (x, y, w, h) = cv2.boundingRect(c) cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2) roi = gray[y:y + h, x:x + w] thresh = cv2.threshold(roi, 0, 255,cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1] thresh = cv2.resize(thresh, (32, 32), interpolation = cv2.INTER_CUBIC) thresh = thresh.astype("float32") / 255.0 thresh = np.expand_dims(thresh, axis=-1) thresh = thresh.reshape(1,32,32,1) ypred = model.predict(thresh) ypred = LB.inverse_transform(ypred) [x] = ypred letters.append(x) return letters, imagedef get_word(letter): word = "".join(letter) return word letter,image = get_letters("brother.jpg") word = get_word(letter) print(word) plt.imshow(image) # what it supposed to give BROTHER ## what give FFQFFFF
- Assistant
Here are a few suggestions to improve your model's performance:
-
Data augmentation:
Increase the size of your dataset by augmenting the dataset using techniques such as rotation, scaling, flipping, and translation.
-
Increase the model complexity.
Add more layers to your model to increase its capacity to learn from the dataset. You can add more convolutional layers, for example.
-
Change the optimizer.
Try using different optimizers like 'RMSprop' or 'SGD' with momentum, which could sometimes yield better results.
-
Adjust the learning rate.
Experiment with different learning rates to ensure your model achieves the best performance possible.
-
Regularization:
Use techniques like L1 or L2 regularization or increase the dropout rate to reduce overfitting.
Here's an updated version of your model with some improvements:
from keras.layers import BatchNormalization from keras.optimizers import RMSprop model = Sequential() model.add(Conv2D(32, (3, 3), padding = "same", activation='relu', input_shape=(32,32,1))) model.add(BatchNormalization()) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(64, (3, 3), activation='relu')) model.add(BatchNormalization()) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(128, (3, 3), activation='relu')) model.add(BatchNormalization()) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(512, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(128, activation='relu')) model.add(BatchNormalization()) model.add(Dropout(0.2)) model.add(Dense(64, activation='softmax')) optimizer = RMSprop(learning_rate=0.001) model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy']) history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_data=(X_val, y_val), verbose=1)You may need to tweak some of the parameters and layers in the above model to achieve optimal performance for your dataset. In addition, make sure you verify the quality of the dataset and the preprocessing steps to make sure everything is in order. Lastly, if the model still does not perform well, you may need to consider using models pretrained on OCR datasets such as Tesseract or OCRopus.
-
- User
integrate a code to augment data
- Assistant
You can use Keras'
ImageDataGeneratorto easily augment your data. Here is an example of how to integrate data augmentation into your existing code:- Import the necessary module:
from keras.preprocessing.image import ImageDataGenerator- Create an
ImageDataGeneratorinstance with the desired augmentations:
datagen = ImageDataGenerator( rotation_range=10, # Randomly rotate the images (in degrees) width_shift_range=0.1, # Fraction of total width, used for random horizontal shifts height_shift_range=0.1, # Fraction of total height, used for random vertical shifts zoom_range=0.1, # Randomly zoom in and out the images horizontal_flip=True, # Randomly flip inputs horizontally fill_mode='nearest' # Fill empty points with the nearest pixel value )- Fit the generator with your training data:
datagen.fit(X_train)- Replace the
model.fitline with the following:
history = model.fit(datagen.flow(X_train, y_train, batch_size=32), epochs=50, validation_data=(X_val, y_val), verbose=1)This will train your model using the augmented data, which should improve its performance on handwritten words. Adjust the parameters of the
ImageDataGeneratorbased on your specific needs, and experiment with different values to achieve optimal performance.