Lessons Learned from AI Training Game Coast Runners

AI Training Game Coast Runners

Back at a school when you first try to train an algorithm they train you through a game because that is very interesting to students.  There is this very famous game for AI training called Coast Runners.  Basically, you have a speedboat and a watercourse.  You control the speedboat through the watercourse.  You have to run several laps and your result in the game depends on three factors.

One is the amount of time it takes you to complete all 6 laps.

Second is your rankings, so you will need to beat your competition.

The third one has to do with what are called “turbos” along the race course, and every turbo you hit them you get some extra points.

AI Looks for the Optimal Strategy

So no matter who trains the algorithm, the machine algorithm always picks the most optimal strategy.  So instead of finishing the course, the AI will find an area along the water where there are many turbos and would never finish the course itself.  It would just go into a little loop and just keep hitting the turbos.  It is optimizing for the points.

The Lesson Learned

What the AI has found is that your success is the weighted average of three objectives and it can cut the first two objectives.  It never needs to do the actual “work”, but by literally scoring so high on the third objective it can win the game.  So in a way, the AI would cheat.  The reason for this is because it doesn’t understand the human context.  It doesn’t get the grand mission or the vision, or if we apply it to the real world….the good you are trying to bring to the society.  It takes the objective function very literally, whatever you said is the objective function is what it will optimize.  This is why setting the objective function is key when dealing with AI.