I am trying to change the threshold value of the activation function Relu
while building my neural network.
So, the initial code was the one written below where the default value of the relu threshold is 0.
model = Sequential([
Dense(n_inputs, input_shape=(n_inputs, ), activation = 'relu'),
Dense(32, activation = 'relu'),
Dense(2, activation='softmax')
])
However, Keras provides a function implementation of the same which can be reffered to here and adding a screenshot as well.
So, I changed my code to the following to pass a custom function only to get the following error.
from keras.activations import relu
model = Sequential([
Dense(n_inputs, input_shape=(n_inputs, ), activation = relu(threshold = 2)),
Dense(32, activation = relu(threshold = 2)),
Dense(2, activation='softmax')
])
Error: TypeError: relu() missing 1 required positional argument: 'x'
I understand the error is that I am not using x in the relu function but there is no way for me to pass something like that. The syntax expects me to write model.add(layers.Activation(activations.relu))
but then I won't be able to change the threshold. This is where I need a workaround or solution.
I then used the Layer implementation of the ReLU function which worked for me as written below but I want to find out if there is a way I can make the activation function implementation work because the layer is not always convenient to add and I want to make more modifications inside the Dense function.
Code which worked for me:-
from keras.layers import ReLU
model = Sequential([
Dense(n_inputs, input_shape=(n_inputs, )),
ReLU(threshold=4),
Dense(32),
ReLU(threshold=4),
Dense(2, activation='softmax')
])
See Question&Answers more detail:
os