Skip to content
TheChatArchiveThe Chat Archive
gpt-4-0125-preview· openaiHugging Face Suzhen/CodeChat

class LRA(keras.callbacks.Callback): reset=False count=0 stopcount=0 tepochs=0 def…

  1. User

    class LRA(keras.callbacks.Callback): reset=False count=0 stop_count=0 tepochs=0 def init(self,model, patience,stop_patience, threshold, factor, dwell, model_name, freeze, initial_epoch): super(LRA, self).init() self.model=model self.patience=patience # specifies how many epochs without improvement before learning rate is adjusted self.stop_patience=stop_patience self.threshold=threshold # specifies training accuracy threshold when lr will be adjusted based on validation loss self.factor=factor # factor by which to reduce the learning rate self.dwell=dwell self.lr=float(tf.keras.backend.get_value(model.optimizer.lr)) # get the initiallearning rate and save it in self.lr self.highest_tracc=0.0 # set highest training accuracy to 0 self.lowest_vloss=np.inf # set lowest validation loss to infinity #self.count=0 # initialize counter that counts epochs with no improvement #self.stop_count=0 # initialize counter that counts how manytimes lr has been adjustd with no improvement
    self.initial_epoch=initial_epoch #self.epochs=epochs best_weights=self.model.get_weights() # set a class vaiable so weights can be loaded after training is completed
    msg=' ' if freeze==True: msgs=f' Starting training using base model { model_name} with weights frozen to imagenet weights initializing LRA callback' else: msgs=f' Starting training using base model { model_name} training all layers '
    print_in_color (msgs, (244, 252, 3), (55,65,80))

    def on_epoch_begin(self,epoch, logs=None):
        self.now= time.time()
        
    def on_epoch_end(self, epoch, logs=None):  # method runs on the end of each epoch
        later=time.time()
        duration=later-self.now        
        if epoch== self.initial_epoch or LRA.reset==True:  
            LRA.reset=False           
            msg='{0:^8s}{1:^10s}{2:^9s}{3:^9s}{4:^9s}{5:^9s}{6:^9s}{7:^11s}{8:^8s}'.format('Epoch', 'Loss', 'Accuracy','V_loss','V_acc', 'LR', 'Next LR', 'Monitor', 'Duration')
            print_in_color(msg, (244,252,3), (5,165,80)) 
            
        lr=float(tf.keras.backend.get_value(self.model.optimizer.lr)) # get the current learning rate
        current_lr=lr
        v_loss=logs.get('val_loss')  # get the validation loss for this epoch
        acc=logs.get('accuracy')  # get training accuracy 
        v_acc=logs.get('val_accuracy')
        loss=logs.get('loss')
        #print ( '\n',v_loss, self.lowest_vloss, acc, self.highest_tracc)
        if acc < self.threshold: # if training accuracy is below threshold adjust lr based on training accuracy
            monitor='accuracy'
            if acc>self.highest_tracc: # training accuracy improved in the epoch                
                self.highest_tracc=acc # set new highest training accuracy
                LRA.best_weights=self.model.get_weights() # traing accuracy improved so save the weights
                self.count=0 # set count to 0 since training accuracy improved
                self.stop_count=0 # set stop counter to 0
                if v_loss<self.lowest_vloss:
                    self.lowest_vloss=v_loss
                color= (0,255,0)
                self.lr=lr
            else: 
                # training accuracy did not improve check if this has happened for patience number of epochs
                # if so adjust learning rate
                if self.count>=self.patience -1:
                    color=(245, 170, 66)
                    self.lr= lr* self.factor # adjust the learning by factor
                    tf.keras.backend.set_value(self.model.optimizer.lr, self.lr) # set the learning rate in the optimizer
                    self.count=0 # reset the count to 0
                    self.stop_count=self.stop_count + 1
                    if self.dwell:
                        self.model.set_weights(LRA.best_weights) # return to better point in N space                        
                    else:
                        if v_loss<self.lowest_vloss:
                            self.lowest_vloss=v_loss                                    
                else:
                    self.count=self.count +1 # increment patience counter                    
        else: # training accuracy is above threshold so adjust learning rate based on validation loss
            monitor='val_loss'
            if v_loss< self.lowest_vloss: # check if the validation loss improved 
                self.lowest_vloss=v_loss # replace lowest validation loss with new validation loss                
                LRA.best_weights=self.model.get_weights() # validation loss improved so save the weights
                self.count=0 # reset count since validation loss improved  
                self.stop_count=0  
                color=(0,255,0)
                self.lr=lr
            else: # validation loss did not improve
                if self.count>=self.patience-1:
                    color=(245, 170, 66)
                    self.lr=self.lr * self.factor # adjust the learning rate
                    self.stop_count=self.stop_count + 1 # increment stop counter because lr was adjusted 
                    self.count=0 # reset counter
                    tf.keras.backend.set_value(self.model.optimizer.lr, self.lr) # set the learning rate in the optimizer
                    if self.dwell:
                        self.model.set_weights(LRA.best_weights) # return to better point in N space
                else: 
                    self.count =self.count +1 # increment the patience counter                    
                if acc>self.highest_tracc:
                    self.highest_tracc= acc
                    msg=f'{str(epoch+1):^3s}/{str(LRA.tepochs):4s} {loss:^9.3f}{acc*100:^9.3f}{v_loss:^9.5f}{v_acc*100:^9.3f}{current_lr:^9.5f}{self.lr:^9.5f}{monitor:^11s}{duration:^8.2f}'
                    print_in_color (msg,(244,252,3), (55,65,80))
        if self.stop_count> self.stop_patience - 1: # check if learning rate has been adjusted stop_count times with no improvement
            msg=f' training has been halted at epoch {epoch + 1} after {self.stop_patience} adjustments of learning rate with no improvement'
            print_in_color(msg, (0,255,0), (55,65,80))
            self.model.stop_training = True # stop training
    

    add Codeadd Markdown epochs =10 patience=9 # number of epochs to wait to adjust lr if monitored value does not improve stop_patience =3 # number of epochs to wait before stopping training if monitored value does not improve threshold=.9 # if train accuracy is < threshhold adjust monitor accuracy, else monitor validation loss factor=.5 # factor to reduce lr by dwell=True # experimental, if True and monitored metric does not improve on current epoch set modelweights back to weights of previous epoch freeze=False # if true free weights of the base model callbacks=[LRA(model=model,patience=patience,stop_patience=stop_patience, threshold=threshold, factor=factor,dwell=dwell,model_name= model_name, freeze=freeze, initial_epoch=0 )] LRA.tepochs=epochs # used to determine value of last epoch for printing history=model.fit(x=train_gen, epochs=epochs, callbacks=callbacks, verbose=0, validation_data=valid_gen, validation_steps=None, shuffle=False, initial_epoch=0)

    AttributeError Traceback (most recent call last) Cell In[37], line 8 6 dwell=True # experimental, if True and monitored metric does not improve on current epoch set modelweights back to weights of previous epoch 7 freeze=False # if true free weights of the base model ----> 8 callbacks=[LRA(model=model,patience=patience,stop_patience=stop_patience, threshold=threshold, 9 factor=factor,dwell=dwell,model_name= model_name, freeze=freeze, initial_epoch=0 )] 10 LRA.tepochs=epochs # used to determine value of last epoch for printing 11 history=model.fit(x=train_gen, epochs=epochs, callbacks=callbacks, verbose=0, validation_data=valid_gen, 12 validation_steps=None, shuffle=False, initial_epoch=0)

    Cell In[35], line 8, in LRA.init(self, model, patience, stop_patience, threshold, factor, dwell, model_name, freeze, initial_epoch) 6 def init(self,model, patience,stop_patience, threshold, factor, dwell, model_name, freeze, initial_epoch): 7 super(LRA, self).init() ----> 8 self.model=model 9 self.patience=patience # specifies how many epochs without improvement before learning rate is adjusted 10 self.stop_patience=stop_patience

    AttributeError: can't set attribute 'model'

  2. Assistant

    The error you're encountering, AttributeError: can't set attribute 'model', suggests that there's an issue with setting the model attribute in your LRA class. The specific cause for this error in the TensorFlow Keras context is that some versions of Keras or modifications to the Callback class might protect the model attribute, making it read-only directly after initialization or due to some internal mechanics not shown in the shared code.

    To potentially resolve this issue, consider checking these aspects:

    1. Protected model Attribute: If the TensorFlow version you're using has made model a protected attribute for Callback classes, then trying to directly assign model inside your __init__ method would indeed cause an error. The typical workflow doesn't require manually setting self.model since Keras automatically sets the model attribute of a callback when the callback is used in model.fit.

    2. Assignment Timing: Check if the assignment to self.model is necessary at all. Normally, Keras callbacks have access to the self.model attribute after being passed to model.fit, as Keras handles this assignment internally.

    3. Version Compatibility: Ensure your code is compatible with the version of TensorFlow (and by extension, Keras) that you're using. If your pattern of usage diverges from what the framework expects, that could lead to such errors. It might be helpful to double-check the documentation for your specific TensorFlow version related to custom callbacks.

    4. Alternative Strategy: If you need to use the model in your __init__ method for some reason (like getting the initial learning rate), consider using a different approach. For example, you could pass parameters such as the initial learning rate directly to your callback's constructor instead of extracting them from the model inside the constructor.

    Given the above points, a simple workaround, if you're indeed required to set attributes or perform initial operations with the model, is to delay such operations until the on_train_begin method of your callback, which is the recommended place to perform initial setup actions that require access to the model. At this point, self.model would be set and usable within your callback:

    def on_train_begin(self, logs=None):
        # Perform your setup actions here, using self.model.
        # Example:
        self.lr = float(tf.keras.backend.get_value(self.model.optimizer.lr))
        self.best_weights = self.model.get_weights()
        # Any other setup that requires the model.
    

    This method ensures you're accessing the model only after Keras has properly initialized and assigned it, thus avoiding direct setting and potential conflicts.