Computer Vision MT25, Learning curves
Flashcards
How would you diagnose the problem with the red curve?
The learning rate is probably too high, you have exploding gradients.
How would you diagnose the problem with the yellow curve?
You have a low learning rate and can’t escape local minima.
How would you diagnose the problem with the blue curve?
The learning rate is probably too high, you have a big step down and then caught on a local minimum.
How would you diagnose the problem with the red dotted curve?
You are likely overfitting the training data.
What is the black gap called here, and how could you reduce it?
::
This is variance: try more data, augmentations, regularisation.
What is the grey gap called here, and how could you reduce it?
::
Try a bigger model or train for longer.
Why have a distinction between the trainval set and the val set?
Trainval is used to tune within the training domain, the validation set is used to assess cross-domain transfer.