Setting up a LearningRateScheduler in Keras
<p>I'm setting up a Learning Rate Scheduler in Keras, using history loss as an updater to self.model.optimizer.lr, but the value on self.model.optimizer.lr does not get inserted in the SGD optimizer and the optimizer is using the dafault learning rate. The code is:</p>

<pre><code>from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.optimizers import SGD
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.preprocessing import StandardScaler

class LossHistory(keras.callbacks.Callback):
    def on_train_begin(self, logs={}):
        self.losses = []
        self.model.optimizer.lr=3
    def on_batch_end(self, batch, logs={}):
        self.losses.append(logs.get('loss'))
        self.model.optimizer.lr=lr-10000*self.losses[-1]

def base_model():
    model=Sequential()
    model.add(Dense(4, input_dim=2, init='uniform'))
    model.add(Dense(1, init='uniform'))
    sgd = SGD(decay=2e-5, momentum=0.9, nesterov=True)


model.compile(loss='mean_squared_error',optimizer=sgd,metrics['mean_absolute_error'])
    return model

history=LossHistory()

estimator = KerasRegressor(build_fn=base_model,nb_epoch=10,batch_size=16,verbose=2,callbacks=[history])

estimator.fit(X_train,y_train,callbacks=[history])

res = estimator.predict(X_test)
</code></pre>

<p>Everything works fine using Keras as a regressor for continuous variables, But I want to reach a smaller derivative by updating the optimizer learning rate.</p>

<p>The learning rate is a variable on the computing device, e.g. a GPU if you are using GPU computation. That means that you have to use <code>K.set_value</code>, with <code>K</code> being <code>keras.backend</code>. For example:</p>

<pre><code>import keras.backend as K
K.set_value(opt.lr, 0.01)
</code></pre>

<p>or in your example</p>

<pre><code>K.set_value(self.model.optimizer.lr, lr-10000*self.losses[-1])
</code></pre>

python optimization keras