Random agent

Play a level with a random agent

Now, you should create a random agent that can play the level automatically. This will serve as a baseline for the performance you can expect from the learned agent – the learned agent should outperform the random agent.

The play_level() function from the frozen-lake package is a helper that allows an agent to play a level of the Frozen Lake game. The function takes two arguments: env and get_action(). The env argument is a Frozen Lake environment. The other argument is a callable – a function that defines the behavior of game play. A get_action() callable will take an environment and return an action.

The play_level() function resets the environment and takes actions until either the game is done or the agent has taken 100 steps, whichever comes first. Then it returns the reward which will be either 0.0 or 1.0 depending on whether the level was won.

To evaluate a random agent, you need a test level and a get_action() callable that returns a random action at each step.

  1. Run the following code in a cell in your SageMaker notebook to define the get_action() callable for a random agent:

    def random_action(env: FrozenLakeEnv) -> int:
        """Choose a random action"""
        return np.random.randint(env.nA)
    
  2. Run the play_level() function with the a standard test level and the random_action() callable.

    play_level(get_test_level(), random_action)
    

    The result of any given call is random and can be 0.0 or 1.0. The get_test_level() function returns a level with the default Frozen Lake board and p_mistake of 0.2. The default board is trickier than most randomly generated boards. This same test level will be used to quickly compare different agents.

  3. To figure out how well the random agent does overall, call play_level() 10,000 times – storing the reward for each attempt. To simulate 10,000 attempts, run the following cell:

    # Run a random agent on the same level 10,000 times
    np.random.seed(1)
    n_attempts = 10000
    test_level = get_test_level()
    rewards = [
        play_level(test_level, random_action)
        for _ in range(n_attempts)
    ]
    
  4. Now calculate win percentage for the random agent:

    sum(rewards) / n_attempts