Leveled environment

Up to this point, there have been two modes of game play: slippery and non-slippery. Slippery mode is extremely difficult and counter-intuitive. Non-slippery mode is trivial. By default, the only knob you can use to change the difficulty of the Frozen Lake game is to create different boards. But in order to create multiple levels, it’s better to be able to adjust the probability of making a mistake as well.

This is done for you in the frozen-lake package by subclassing the FrozenLakeEnv class and overriding the connection between actions and new states. You can see how this works by looking at frozen_lake/environment.py. The LeveledFrozenLake class has a random() class method that takes a probability of making a mistake (p_mistake) and generates a random board for it. This combination of board and probabiliy of mistake will be considered a level.

  1. Use the random() method to generate a level with p_mistake of 0.2. Try this a few times – the game board will be different each time:

    random_level = LeveledFrozenLake.random(0.2)
    play_manually(random_level)
    

    These levels are harder than non-slippery mode levels, but much easier than the default game that comes in the OpenAI gym (which has a 67% probability of making a mistake).

  2. Try out a few levels with different p_mistake values to confirm that higher levels of p_mistake make the game harder.