python - What does model.eval() do in pytorch?

Question

I am using this code, and saw model.eval() in some cases.

I understand it is supposed to allow me to "evaluate my model", but I don't understand when I should and shouldn't use it, or how to turn if off.

I would like to run the above code to train the network, and also be able to run validation every epoch. I wasn't able to do it still.

Answer 1

model.eval() is a kind of switch for some specific layers/parts of the model that behave differently during training and inference (evaluating) time. For example, Dropouts Layers, BatchNorm Layers etc. You need to turn off them during model evaluation, and .eval() will do it for you. In addition, the common practice for evaluating/validation is using torch.no_grad() in pair with model.eval() to turn off gradients computation:

# evaluate model:
model.eval()

with torch.no_grad():
    ...
    out_data = model(data)
    ...

BUT, don't forget to turn back to training mode after eval step:

# training step
...
model.train()
...