The exploding gradient problem is another issue related to the training of deep neural networks, somewhat like the flip side of the vanishing gradient problem.