python - How to get layer weight while training?

Question

I have a model and I would like to get the weight matrix of a specific layer to use it while defining custom loss function.

Is there any way to get a weight of specific layer, inside the model?

P.S. I am currently working with tensorflow 2, and keras functional API. I tested How do I get the weights of a layer in Keras? approach, but it did not work.

P.P.S. By using the above described approach, I get the following error:

AttributeError                            Traceback (most recent call last)
<ipython-input-26-e0bd481102a7> in <module>
      1 A_DENSE = Dense(1, use_bias = True, name = "A_DENSE")(INPUT)
----> 2 A_DENSE.get_weights()

AttributeError: 'Tensor' object has no attribute 'get_weights'

P.P.P.S. As answered below, combining custom callback and get_weights solves the problem. Good luck for people who were in similar situation with me.

Answer 1

You can write a custom Callback and use it each time an epoch ends. I show it for printing the weights, but you could use it as part of your custom loss.

class CustomCallback(keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        rand_int = tf.random.uniform((), 0, 2, dtype=tf.int32)
        print(rand_int)
        
model.fit(X, y epochs = 10, batch_size = 20, validation_split=0.1, callbacks=[CustomCallback()])

More details here.

---

For example, here is a dummy code to print the weights and biases of layer[1] after each epoch. You can set up the function in a way that you prefer.

from tensorflow.keras import layers, Model, callbacks

class CustomCallback(callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        print(' ')
        print(' ')
        print(model.layers[1].get_weights())
        

X, y = np.random.random((10,5)), np.random.random((10,))

inp = layers.Input((5,))
x = layers.Dense(3)(inp)
out = layers.Dense(1)(x)

model = Model(inp, out)

model.compile(loss='MAE',metrics=['accuracy'])
model.fit(X,y,callbacks=[CustomCallback()], epochs=3)

Epoch 1/3
1/1 [==============================] - ETA: 0s - loss: 0.2346 - accuracy: 0.0000e+00 
 
[array([[ 0.16518219, -0.44628695, -0.07702655],
       [-0.1993848 ,  0.03855793, -0.62964785],
       [ 0.5592851 , -0.28281152, -0.23358124],
       [ 0.05242977,  0.4023881 , -0.19522922],
       [ 0.07936202, -0.40436065,  0.10003945]], dtype=float32), array([ 0.01530731, -0.01565045, -0.01581042], dtype=float32)]
1/1 [==============================] - 0s 2ms/step - loss: 0.2346 - accuracy: 0.0000e+00
Epoch 2/3
1/1 [==============================] - ETA: 0s - loss: 0.2337 - accuracy: 0.0000e+00 
 
[array([[ 0.16814367, -0.4492649 , -0.08000461],
       [-0.19710523,  0.03622784, -0.6319782 ],
       [ 0.55797213, -0.28144714, -0.23221655],
       [ 0.05509637,  0.3996864 , -0.19793113],
       [ 0.07731982, -0.40226308,  0.10213734]], dtype=float32), array([ 0.01846951, -0.01881272, -0.01897269], dtype=float32)]
1/1 [==============================] - 0s 7ms/step - loss: 0.2337 - accuracy: 0.0000e+00
Epoch 3/3
1/1 [==============================] - ETA: 0s - loss: 0.2322 - accuracy: 0.0000e+00 
 
[array([[ 0.16706704, -0.448164  , -0.07889817],
       [-0.19894598,  0.0381193 , -0.63007975],
       [ 0.5558067 , -0.27921563, -0.22997847],
       [ 0.05663134,  0.3981127 , -0.19951159],
       [ 0.07536169, -0.400249  ,  0.10415838]], dtype=float32), array([ 0.01846951, -0.01881272, -0.01897269], dtype=float32)]
1/1 [==============================] - 0s 2ms/step - loss: 0.2322 - accuracy: 0.0000e+00