Parable of the Hill Climber

Once upon a time, there was a hill climber.

max0001-1

The hill climber wanted to climb the tallest hill. However, the land was foggy and they couldn't see far. So, they climbed in whatever nearby direction seemed uphill.

max0002

They climbed & climbed, and finally reached a peak. “At last,” the hill climber said, “I've climbed the tallest hill!”

max0003

What our protagonist has done is known as the "Hill Climbing" algorithm. Given a point on a landscape of possibilities – where higher points represent better states – the algorithm just looks at nearby points (because evaluating all points would be too costly), moves to the "highest" nearby point, and repeats. Eventually, the algorithm will reach a peak.

Just one problem.

max0004

That peak may not be the peak.

Our hero is now stuck in what's called a "local maximum".

max0005

The hill climber looked to the left. Doesn't seem uphill.

max0006

The hill climber looked to the right. Doesn't seem uphill.

max0007

The hill climber felt trapped. “Is this all there is?” they thought. “Is life literally all downhill from here?”

Little did they know, there are better algorithms. Algorithms like “Stochastic Hill Climbing” and “Simulated Annealing” solve the local-maximum problem by adding randomness. This causes the algorithm to take worse steps in the short-run, so it can locate better states in the long-run.

Sometimes, you have to leave what's been working for you, and take a step down into the fog...

max0008

...to climb to a better place.

max0009

(this cheesy story was originally posted on my Patreon)