The aim of this notebook is to help you getting started with the Gymnasium environment. Gymnasium (formerly Gym) is a toolkit for developing and comparing reinforcement learning algorithms. It supports teaching agents everything from walking to playing games like Pong or Pinball.
We are now going to:
- see how to install the gym toolkit
- learn how to use it
- have funThe simplest way to install gymnasium is to use pip. This is easily done, apart from a slight option activation to get all the available environments installed, using the code below:
pip install gymnasium
Requirement already satisfied: gymnasium in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (0.27.0) Requirement already satisfied: jax-jumpy>=0.2.0 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (0.2.0) Requirement already satisfied: shimmy<1.0,>=0.1.0 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (0.2.0) Requirement already satisfied: gymnasium-notices>=0.0.1 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (0.0.1) Requirement already satisfied: typing-extensions>=4.3.0 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (4.3.0) Requirement already satisfied: numpy>=1.21.0 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (1.21.5) Requirement already satisfied: importlib-metadata>=4.8.0 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (4.11.3) Requirement already satisfied: cloudpickle>=1.2.0 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from gymnasium) (2.0.0) Requirement already satisfied: zipp>=0.5 in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (from importlib-metadata>=4.8.0->gymnasium) (3.8.0) Note: you may need to restart the kernel to use updated packages.
You may also have to install the pygame library as well.
pip install pygame
Requirement already satisfied: pygame in /Users/mathieu/opt/anaconda3/lib/python3.9/site-packages (2.1.2) Note: you may need to restart the kernel to use updated packages.
We are now all set to start learning how to interact with the gymnasium toolkit!
import gymnasium as gym
import pygame
The gymnasium library is a collection of test problems, often called environments, that you can use to work out your reinforcement learning algorithms. These environments have a shared interface, allowing you to write general algorithms. Let's have a look at the CartPole environments and play a bunch of games.s
env = gym.make("CartPole-v1", render_mode="human")
observation, info = env.reset(seed=42)
for _ in range(50):
action = env.action_space.sample() # this is where you would insert your policy
observation, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
observation, info = env.reset()
env.close()
pygame.display.quit()
pygame.quit()
1 HIToolbox 0x00007ff823bb0726 _ZN15MenuBarInstance22EnsureAutoShowObserverEv + 102 2 HIToolbox 0x00007ff823bb02b8 _ZN15MenuBarInstance14EnableAutoShowEv + 52 3 HIToolbox 0x00007ff823b54908 SetMenuBarObscured + 408 4 HIToolbox 0x00007ff823b544ca _ZN13HIApplication15HandleActivatedEP14OpaqueEventRefhP15OpaqueWindowPtrh + 164 5 HIToolbox 0x00007ff823b4e996 _ZN13HIApplication13EventObserverEjP14OpaqueEventRefPv + 252 6 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 7 HIToolbox 0x00007ff823b4e3e6 AcquireEventFromQueue + 494 8 HIToolbox 0x00007ff823b3d3ec ReceiveNextEventCommon + 285 9 HIToolbox 0x00007ff823b3d2b3 _BlockUntilNextEventMatchingListInModeWithFilter + 70 10 AppKit 0x00007ff81d344f33 _DPSNextEvent + 909 11 AppKit 0x00007ff81d343db4 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1219 12 libSDL2-2.0.0.dylib 0x0000000108ab28bd Cocoa_PumpEvents + 125 13 libSDL2-2.0.0.dylib 0x00000001089f22c1 SDL_PumpEvents_REAL + 33 14 event.cpython-39-darwin.so 0x000000010711f447 pg_event_pump + 23 15 python3.9 0x0000000105e5b350 cfunction_vectorcall_NOARGS + 96 16 python3.9 0x0000000105f2cb3e call_function + 174 17 python3.9 0x0000000105f25988 _PyEval_EvalFrameDefault + 27608 18 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 19 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 20 python3.9 0x0000000105f2cb3e call_function + 174 21 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 22 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 23 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 24 python3.9 0x0000000105e02cde method_vectorcall + 158 25 python3.9 0x0000000105dff633 PyVectorcall_Call + 163 26 python3.9 0x0000000105f244a7 _PyEval_EvalFrameDefault + 22263 27 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 28 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 29 python3.9 0x0000000105dff633 PyVectorcall_Call + 163 30 python3.9 0x0000000105f244a7 _PyEval_EvalFrameDefault + 22263 31 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 32 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 33 python3.9 0x0000000105e02cde method_vectorcall + 158 34 python3.9 0x0000000105dff633 PyVectorcall_Call + 163 35 python3.9 0x0000000105f244a7 _PyEval_EvalFrameDefault + 22263 36 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 37 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 38 python3.9 0x0000000105e02cde method_vectorcall + 158 39 python3.9 0x0000000105dff633 PyVectorcall_Call + 163 40 python3.9 0x0000000105f244a7 _PyEval_EvalFrameDefault + 22263 41 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 42 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 43 python3.9 0x0000000105e02cde method_vectorcall + 158 44 python3.9 0x0000000105f2cb3e call_function + 174 45 python3.9 0x0000000105f2035c _PyEval_EvalFrameDefault + 5548 46 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 47 python3.9 0x0000000105f17c99 builtin_exec + 377 48 python3.9 0x0000000105e5b565 cfunction_vectorcall_FASTCALL + 85 49 python3.9 0x0000000105f2cb3e call_function + 174 50 python3.9 0x0000000105f24c1d _PyEval_EvalFrameDefault + 24173 51 python3.9 0x0000000105e18b7f gen_send_ex + 175 52 python3.9 0x0000000105f26cf3 _PyEval_EvalFrameDefault + 32579 53 python3.9 0x0000000105e18b7f gen_send_ex + 175 54 python3.9 0x0000000105f26cf3 _PyEval_EvalFrameDefault + 32579 55 python3.9 0x0000000105e18b7f gen_send_ex + 175 56 python3.9 0x0000000105e0d817 method_vectorcall_O + 103 57 python3.9 0x0000000105f2cb3e call_function + 174 58 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 59 python3.9 0x0000000105dffa7b function_code_fastcall + 139 60 python3.9 0x0000000105f2cb3e call_function + 174 61 python3.9 0x0000000105f24c1d _PyEval_EvalFrameDefault + 24173 62 python3.9 0x0000000105dffa7b function_code_fastcall + 139 63 python3.9 0x0000000105f2cb3e call_function + 174 64 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 65 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 66 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 67 python3.9 0x0000000105e02cde method_vectorcall + 158 68 python3.9 0x0000000105dff633 PyVectorcall_Call + 163 69 python3.9 0x0000000105f244a7 _PyEval_EvalFrameDefault + 22263 70 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 71 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 72 python3.9 0x0000000105e02cde method_vectorcall + 158 73 python3.9 0x0000000105f2cb3e call_function + 174 74 python3.9 0x0000000105f2035c _PyEval_EvalFrameDefault + 5548 75 python3.9 0x0000000105e18b7f gen_send_ex + 175 76 python3.9 0x0000000105f26cf3 _PyEval_EvalFrameDefault + 32579 77 python3.9 0x0000000105e18b7f gen_send_ex + 175 78 python3.9 0x0000000105f26cf3 _PyEval_EvalFrameDefault + 32579 79 python3.9 0x0000000105e18b7f gen_send_ex + 175 80 python3.9 0x0000000105f26cf3 _PyEval_EvalFrameDefault + 32579 81 python3.9 0x0000000105e18b7f gen_send_ex + 175 82 python3.9 0x0000000105f26cf3 _PyEval_EvalFrameDefault + 32579 83 python3.9 0x0000000105e18b7f gen_send_ex + 175 84 _asyncio.cpython-39-darwin.so 0x000000010627e97a task_step_impl + 442 85 _asyncio.cpython-39-darwin.so 0x000000010627e72e task_step + 62 86 _asyncio.cpython-39-darwin.so 0x000000010627e5c8 task_wakeup + 104 87 _asyncio.cpython-39-darwin.so 0x000000010627e4bd TaskWakeupMethWrapper_call + 109 88 python3.9 0x0000000105dfed36 _PyObject_MakeTpCall + 134 89 python3.9 0x0000000105f48321 context_run + 81 90 python3.9 0x0000000105e5b49c cfunction_vectorcall_FASTCALL_KEYWORDS + 76 91 python3.9 0x0000000105f244a7 _PyEval_EvalFrameDefault + 22263 92 python3.9 0x0000000105dffa7b function_code_fastcall + 139 93 python3.9 0x0000000105f2cb3e call_function + 174 94 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 95 python3.9 0x0000000105dffa7b function_code_fastcall + 139 96 python3.9 0x0000000105f2cb3e call_function + 174 97 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 98 python3.9 0x0000000105dffa7b function_code_fastcall + 139 99 python3.9 0x0000000105f2cb3e call_function + 174 100 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 101 python3.9 0x0000000105dffa7b function_code_fastcall + 139 102 python3.9 0x0000000105f2cb3e call_function + 174 103 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 104 python3.9 0x0000000105dffa7b function_code_fastcall + 139 105 python3.9 0x0000000105f2cb3e call_function + 174 106 python3.9 0x0000000105f2076a _PyEval_EvalFrameDefault + 6586 107 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 108 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 109 python3.9 0x0000000105e02cde method_vectorcall + 158 110 python3.9 0x0000000105f2cb3e call_function + 174 111 python3.9 0x0000000105f25988 _PyEval_EvalFrameDefault + 27608 112 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 113 python3.9 0x0000000105f17c99 builtin_exec + 377 114 python3.9 0x0000000105e5b565 cfunction_vectorcall_FASTCALL + 85 115 python3.9 0x0000000105f2cb3e call_function + 174 116 python3.9 0x0000000105f24c1d _PyEval_EvalFrameDefault + 24173 117 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 118 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 119 python3.9 0x0000000105f2cb3e call_function + 174 120 python3.9 0x0000000105f24c1d _PyEval_EvalFrameDefault + 24173 121 python3.9 0x0000000105f1d6e9 _PyEval_EvalCode + 489 122 python3.9 0x0000000105dff9bc _PyFunction_Vectorcall + 236 123 python3.9 0x0000000105fb0ec2 pymain_run_module + 258 124 python3.9 0x0000000105fb0781 pymain_run_python + 241 125 python3.9 0x0000000105fb0645 Py_RunMain + 37 126 python3.9 0x0000000105fb1d10 pymain_main + 64 127 python3.9 0x0000000105d96238 main + 56 128 dyld 0x00007ff819dec310 start + 2432
If everythings went fine, you should have seen a window where "someone" played the CartPole game. Let me summarize what the above lines did. First we load the gymnasium library (but you should have already guessed it, no?). Next we create our first gym environment which here happens to be the CartPole game. We then initialize the environment from the env.reset() call while the env.render() call allows us to display the current game state. Next we just do a bunch of random actions using successive env.step calls, more details later. Finally we do not forget to close our environment.
Of course there are many available environments and you can get a peek either by browing the gym website or by invoking:
We are now going to take a closer the look to our gym environment. We will briefly detail:
env1 = gym.make("CartPole-v1", render_mode = "human")
init1 = env1.reset()
env2 = gym.make("FrozenLake-v1", render_mode = "human")
init2 = env2.reset()
print(env1.action_space)## -> Discrete(2) indicates that we have only 2 possible actions '0' and '1'
print(env2.action_space)## -> Discrete(4) we now have 4 possible actions '0', ..., '4'
print(env1.observation_space)## -> Box(4,) indicates that a state is a vector of size 4
print(env2.observation_space)## -> Discrete(16) indicates that we have 16 possible states
print(init1)## this is indeed a vector of size 4
print(init2)## this is indeed an integer
1 HIToolbox 0x00007ff823bb0726 _ZN15MenuBarInstance22EnsureAutoShowObserverEv + 102 2 HIToolbox 0x00007ff823bb02b8 _ZN15MenuBarInstance14EnableAutoShowEv + 52 3 HIToolbox 0x00007ff823b54908 SetMenuBarObscured + 408 4 HIToolbox 0x00007ff823b544ca _ZN13HIApplication15HandleActivatedEP14OpaqueEventRefhP15OpaqueWindowPtrh + 164 5 HIToolbox 0x00007ff823b4e996 _ZN13HIApplication13EventObserverEjP14OpaqueEventRefPv + 252 6 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 7 HIToolbox 0x00007ff823b4e3e6 AcquireEventFromQueue + 494 8 HIToolbox 0x00007ff823b3d3ec ReceiveNextEventCommon + 285 9 HIToolbox 0x00007ff823b3d2b3 _BlockUntilNextEventMatchingListInModeWithFilter + 70 10 AppKit 0x00007ff81d344f33 _DPSNextEvent + 909 11 AppKit 0x00007ff81d343db4 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1219 12 libSDL2-2.0.0.dylib 0x0000000108918910 Cocoa_PumpEvents + 208 13 libSDL2-2.0.0.dylib 0x00000001088582c1 SDL_PumpEvents_REAL + 33 14 event.cpython-39-darwin.so 0x0000000106f85447 pg_event_pump + 23 15 python3.9 0x0000000105cc1350 cfunction_vectorcall_NOARGS + 96 16 python3.9 0x0000000105d92b3e call_function + 174 17 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 18 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 19 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 20 python3.9 0x0000000105d92b3e call_function + 174 21 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 22 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 23 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 24 python3.9 0x0000000105c68e1f method_vectorcall + 479 25 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 26 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 27 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 28 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 29 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 30 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 31 python3.9 0x0000000105c68e1f method_vectorcall + 479 32 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 33 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 34 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 35 python3.9 0x0000000105c68e1f method_vectorcall + 479 36 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 37 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 38 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 39 python3.9 0x0000000105c68cde method_vectorcall + 158 40 python3.9 0x0000000105d92b3e call_function + 174 41 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 42 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 43 python3.9 0x0000000105d7dc99 builtin_exec + 377 44 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 45 python3.9 0x0000000105d92b3e call_function + 174 46 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 47 python3.9 0x0000000105c7eb7f gen_send_ex + 175 48 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 49 python3.9 0x0000000105c7eb7f gen_send_ex + 175 50 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 51 python3.9 0x0000000105c7eb7f gen_send_ex + 175 52 python3.9 0x0000000105c73817 method_vectorcall_O + 103 53 python3.9 0x0000000105d92b3e call_function + 174 54 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 55 python3.9 0x0000000105c65a7b function_code_fastcall + 139 56 python3.9 0x0000000105d92b3e call_function + 174 57 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 58 python3.9 0x0000000105c65a7b function_code_fastcall + 139 59 python3.9 0x0000000105d92b3e call_function + 174 60 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 61 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 62 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 63 python3.9 0x0000000105c68cde method_vectorcall + 158 64 python3.9 0x0000000105c65633 PyVectorcall_Call + 163 65 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 66 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 67 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 68 python3.9 0x0000000105c68cde method_vectorcall + 158 69 python3.9 0x0000000105d92b3e call_function + 174 70 python3.9 0x0000000105d8635c _PyEval_EvalFrameDefault + 5548 71 python3.9 0x0000000105c7eb7f gen_send_ex + 175 72 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 73 python3.9 0x0000000105c7eb7f gen_send_ex + 175 74 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 75 python3.9 0x0000000105c7eb7f gen_send_ex + 175 76 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 77 python3.9 0x0000000105c7eb7f gen_send_ex + 175 78 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 79 python3.9 0x0000000105c7eb7f gen_send_ex + 175 80 _asyncio.cpython-39-darwin.so 0x00000001060e497a task_step_impl + 442 81 _asyncio.cpython-39-darwin.so 0x00000001060e472e task_step + 62 82 _asyncio.cpython-39-darwin.so 0x00000001060e45c8 task_wakeup + 104 83 _asyncio.cpython-39-darwin.so 0x00000001060e44bd TaskWakeupMethWrapper_call + 109 84 python3.9 0x0000000105c64d36 _PyObject_MakeTpCall + 134 85 python3.9 0x0000000105dae321 context_run + 81 86 python3.9 0x0000000105cc149c cfunction_vectorcall_FASTCALL_KEYWORDS + 76 87 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 88 python3.9 0x0000000105c65a7b function_code_fastcall + 139 89 python3.9 0x0000000105d92b3e call_function + 174 90 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 91 python3.9 0x0000000105c65a7b function_code_fastcall + 139 92 python3.9 0x0000000105d92b3e call_function + 174 93 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 94 python3.9 0x0000000105c65a7b function_code_fastcall + 139 95 python3.9 0x0000000105d92b3e call_function + 174 96 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 97 python3.9 0x0000000105c65a7b function_code_fastcall + 139 98 python3.9 0x0000000105d92b3e call_function + 174 99 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 100 python3.9 0x0000000105c65a7b function_code_fastcall + 139 101 python3.9 0x0000000105d92b3e call_function + 174 102 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 103 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 104 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 105 python3.9 0x0000000105c68cde method_vectorcall + 158 106 python3.9 0x0000000105d92b3e call_function + 174 107 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 108 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 109 python3.9 0x0000000105d7dc99 builtin_exec + 377 110 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 111 python3.9 0x0000000105d92b3e call_function + 174 112 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 113 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 114 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 115 python3.9 0x0000000105d92b3e call_function + 174 116 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 117 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 118 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 119 python3.9 0x0000000105e16ec2 pymain_run_module + 258 120 python3.9 0x0000000105e16781 pymain_run_python + 241 121 python3.9 0x0000000105e16645 Py_RunMain + 37 122 python3.9 0x0000000105e17d10 pymain_main + 64 123 python3.9 0x0000000105bfc238 main + 56 124 dyld 0x00007ff819dec310 start + 2432
Discrete(2)
Discrete(4)
Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32)
Discrete(16)
(array([ 0.04129543, -0.01724758, -0.01258496, 0.00614324], dtype=float32), {})
(0, {'prob': 1})
We are now going to see the main method which is step. It takes as an argument the action label, i.e., an integer, and performs this action to the current state of the environment and outputs four values:
Let's try to perform an action to our CartPole problem!
state = env1.reset()
env1.render()
print(env1.step(1))
new_state, reward, done, truncated, info = env1.step(1)
print(new_state) ## --> here the new state
print(reward)## -> we have just been rewarded of 1
print(done)## -> is the game over?
print(truncated)## -> did you reach the maximum number of steps, i.e., you win
print(info)## -> nothing to say
(array([ 0.00705511, 0.15714493, 0.00417313, -0.31796804], dtype=float32), 1.0, False, False, {})
[ 0.01019801 0.3522072 -0.00218624 -0.60933197]
1.0
False
False
{}
1 HIToolbox 0x00007ff823cb752b _ZN15MenuBarInstance21IsAutoShowHideAllowedEv + 259 2 HIToolbox 0x00007ff823bb033e _ZN15MenuBarInstance24UpdateAutoShowVisibilityE5Pointh + 34 3 HIToolbox 0x00007ff823b1f7a4 _ZN15MenuBarInstance16ForEachMenuBarDoEU13block_pointerFvPS_E + 46 4 HIToolbox 0x00007ff823bb093d _ZN15MenuBarInstance20AutoShowHideObserverEjP14OpaqueEventRefPv + 165 5 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 6 HIToolbox 0x00007ff823b48fb8 PostEventToQueueInternal + 700 7 HIToolbox 0x00007ff823b4a871 _ZL29CreateAndPostEventWithCGEventP9__CGEventjhP17__CFMachPortBoost + 404 8 HIToolbox 0x00007ff823b56ee9 _ZL15Convert1CGEventh + 246 9 HIToolbox 0x00007ff823b56d91 _ZL16MainLoopObserverjP14OpaqueEventRefPv + 41 10 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 11 HIToolbox 0x00007ff823b3d726 RunCurrentEventLoopInMode + 228 12 HIToolbox 0x00007ff823b3d396 ReceiveNextEventCommon + 199 13 HIToolbox 0x00007ff823b3d2b3 _BlockUntilNextEventMatchingListInModeWithFilter + 70 14 AppKit 0x00007ff81d344f33 _DPSNextEvent + 909 15 AppKit 0x00007ff81d343db4 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1219 16 libSDL2-2.0.0.dylib 0x0000000108918910 Cocoa_PumpEvents + 208 17 libSDL2-2.0.0.dylib 0x00000001088582c1 SDL_PumpEvents_REAL + 33 18 event.cpython-39-darwin.so 0x0000000106f85447 pg_event_pump + 23 19 python3.9 0x0000000105cc1350 cfunction_vectorcall_NOARGS + 96 20 python3.9 0x0000000105d92b3e call_function + 174 21 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 22 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 23 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 24 python3.9 0x0000000105d92b3e call_function + 174 25 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 26 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 27 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 28 python3.9 0x0000000105c68e1f method_vectorcall + 479 29 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 30 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 31 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 32 python3.9 0x0000000105c68e1f method_vectorcall + 479 33 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 34 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 35 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 36 python3.9 0x0000000105c68e1f method_vectorcall + 479 37 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 38 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 39 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 40 python3.9 0x0000000105c68cde method_vectorcall + 158 41 python3.9 0x0000000105d92b3e call_function + 174 42 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 43 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 44 python3.9 0x0000000105d7dc99 builtin_exec + 377 45 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 46 python3.9 0x0000000105d92b3e call_function + 174 47 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 48 python3.9 0x0000000105c7eb7f gen_send_ex + 175 49 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 50 python3.9 0x0000000105c7eb7f gen_send_ex + 175 51 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 52 python3.9 0x0000000105c7eb7f gen_send_ex + 175 53 python3.9 0x0000000105c73817 method_vectorcall_O + 103 54 python3.9 0x0000000105d92b3e call_function + 174 55 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 56 python3.9 0x0000000105c65a7b function_code_fastcall + 139 57 python3.9 0x0000000105d92b3e call_function + 174 58 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 59 python3.9 0x0000000105c65a7b function_code_fastcall + 139 60 python3.9 0x0000000105d92b3e call_function + 174 61 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 62 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 63 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 64 python3.9 0x0000000105c68cde method_vectorcall + 158 65 python3.9 0x0000000105c65633 PyVectorcall_Call + 163 66 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 67 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 68 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 69 python3.9 0x0000000105c68cde method_vectorcall + 158 70 python3.9 0x0000000105d92b3e call_function + 174 71 python3.9 0x0000000105d8635c _PyEval_EvalFrameDefault + 5548 72 python3.9 0x0000000105c7eb7f gen_send_ex + 175 73 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 74 python3.9 0x0000000105c7eb7f gen_send_ex + 175 75 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 76 python3.9 0x0000000105c7eb7f gen_send_ex + 175 77 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 78 python3.9 0x0000000105c7eb7f gen_send_ex + 175 79 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 80 python3.9 0x0000000105c7eb7f gen_send_ex + 175 81 _asyncio.cpython-39-darwin.so 0x00000001060e497a task_step_impl + 442 82 _asyncio.cpython-39-darwin.so 0x00000001060e472e task_step + 62 83 _asyncio.cpython-39-darwin.so 0x00000001060e45c8 task_wakeup + 104 84 _asyncio.cpython-39-darwin.so 0x00000001060e44bd TaskWakeupMethWrapper_call + 109 85 python3.9 0x0000000105c64d36 _PyObject_MakeTpCall + 134 86 python3.9 0x0000000105dae321 context_run + 81 87 python3.9 0x0000000105cc149c cfunction_vectorcall_FASTCALL_KEYWORDS + 76 88 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 89 python3.9 0x0000000105c65a7b function_code_fastcall + 139 90 python3.9 0x0000000105d92b3e call_function + 174 91 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 92 python3.9 0x0000000105c65a7b function_code_fastcall + 139 93 python3.9 0x0000000105d92b3e call_function + 174 94 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 95 python3.9 0x0000000105c65a7b function_code_fastcall + 139 96 python3.9 0x0000000105d92b3e call_function + 174 97 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 98 python3.9 0x0000000105c65a7b function_code_fastcall + 139 99 python3.9 0x0000000105d92b3e call_function + 174 100 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 101 python3.9 0x0000000105c65a7b function_code_fastcall + 139 102 python3.9 0x0000000105d92b3e call_function + 174 103 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 104 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 105 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 106 python3.9 0x0000000105c68cde method_vectorcall + 158 107 python3.9 0x0000000105d92b3e call_function + 174 108 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 109 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 110 python3.9 0x0000000105d7dc99 builtin_exec + 377 111 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 112 python3.9 0x0000000105d92b3e call_function + 174 113 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 114 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 115 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 116 python3.9 0x0000000105d92b3e call_function + 174 117 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 118 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 119 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 120 python3.9 0x0000000105e16ec2 pymain_run_module + 258 121 python3.9 0x0000000105e16781 pymain_run_python + 241 122 python3.9 0x0000000105e16645 Py_RunMain + 37 123 python3.9 0x0000000105e17d10 pymain_main + 64 124 python3.9 0x0000000105bfc238 main + 56 125 dyld 0x00007ff819dec310 start + 2432 1 HIToolbox 0x00007ff823cb752b _ZN15MenuBarInstance21IsAutoShowHideAllowedEv + 259 2 HIToolbox 0x00007ff823bb033e _ZN15MenuBarInstance24UpdateAutoShowVisibilityE5Pointh + 34 3 HIToolbox 0x00007ff823b1f7a4 _ZN15MenuBarInstance16ForEachMenuBarDoEU13block_pointerFvPS_E + 46 4 HIToolbox 0x00007ff823bb093d _ZN15MenuBarInstance20AutoShowHideObserverEjP14OpaqueEventRefPv + 165 5 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 6 HIToolbox 0x00007ff823b48fb8 PostEventToQueueInternal + 700 7 HIToolbox 0x00007ff823b4a871 _ZL29CreateAndPostEventWithCGEventP9__CGEventjhP17__CFMachPortBoost + 404 8 HIToolbox 0x00007ff823b56ee9 _ZL15Convert1CGEventh + 246 9 HIToolbox 0x00007ff823b56d91 _ZL16MainLoopObserverjP14OpaqueEventRefPv + 41 10 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 11 HIToolbox 0x00007ff823b3d726 RunCurrentEventLoopInMode + 228 12 HIToolbox 0x00007ff823b3d396 ReceiveNextEventCommon + 199 13 HIToolbox 0x00007ff823b3d2b3 _BlockUntilNextEventMatchingListInModeWithFilter + 70 14 AppKit 0x00007ff81d344f33 _DPSNextEvent + 909 15 AppKit 0x00007ff81d343db4 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1219 16 libSDL2-2.0.0.dylib 0x0000000108918910 Cocoa_PumpEvents + 208 17 libSDL2-2.0.0.dylib 0x00000001088582c1 SDL_PumpEvents_REAL + 33 18 event.cpython-39-darwin.so 0x0000000106f85447 pg_event_pump + 23 19 python3.9 0x0000000105cc1350 cfunction_vectorcall_NOARGS + 96 20 python3.9 0x0000000105d92b3e call_function + 174 21 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 22 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 23 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 24 python3.9 0x0000000105d92b3e call_function + 174 25 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 26 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 27 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 28 python3.9 0x0000000105c68e1f method_vectorcall + 479 29 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 30 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 31 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 32 python3.9 0x0000000105c68e1f method_vectorcall + 479 33 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 34 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 35 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 36 python3.9 0x0000000105c68e1f method_vectorcall + 479 37 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 38 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 39 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 40 python3.9 0x0000000105c68cde method_vectorcall + 158 41 python3.9 0x0000000105d92b3e call_function + 174 42 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 43 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 44 python3.9 0x0000000105d7dc99 builtin_exec + 377 45 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 46 python3.9 0x0000000105d92b3e call_function + 174 47 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 48 python3.9 0x0000000105c7eb7f gen_send_ex + 175 49 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 50 python3.9 0x0000000105c7eb7f gen_send_ex + 175 51 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 52 python3.9 0x0000000105c7eb7f gen_send_ex + 175 53 python3.9 0x0000000105c73817 method_vectorcall_O + 103 54 python3.9 0x0000000105d92b3e call_function + 174 55 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 56 python3.9 0x0000000105c65a7b function_code_fastcall + 139 57 python3.9 0x0000000105d92b3e call_function + 174 58 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 59 python3.9 0x0000000105c65a7b function_code_fastcall + 139 60 python3.9 0x0000000105d92b3e call_function + 174 61 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 62 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 63 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 64 python3.9 0x0000000105c68cde method_vectorcall + 158 65 python3.9 0x0000000105c65633 PyVectorcall_Call + 163 66 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 67 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 68 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 69 python3.9 0x0000000105c68cde method_vectorcall + 158 70 python3.9 0x0000000105d92b3e call_function + 174 71 python3.9 0x0000000105d8635c _PyEval_EvalFrameDefault + 5548 72 python3.9 0x0000000105c7eb7f gen_send_ex + 175 73 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 74 python3.9 0x0000000105c7eb7f gen_send_ex + 175 75 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 76 python3.9 0x0000000105c7eb7f gen_send_ex + 175 77 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 78 python3.9 0x0000000105c7eb7f gen_send_ex + 175 79 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 80 python3.9 0x0000000105c7eb7f gen_send_ex + 175 81 _asyncio.cpython-39-darwin.so 0x00000001060e497a task_step_impl + 442 82 _asyncio.cpython-39-darwin.so 0x00000001060e472e task_step + 62 83 _asyncio.cpython-39-darwin.so 0x00000001060e45c8 task_wakeup + 104 84 _asyncio.cpython-39-darwin.so 0x00000001060e44bd TaskWakeupMethWrapper_call + 109 85 python3.9 0x0000000105c64d36 _PyObject_MakeTpCall + 134 86 python3.9 0x0000000105dae321 context_run + 81 87 python3.9 0x0000000105cc149c cfunction_vectorcall_FASTCALL_KEYWORDS + 76 88 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 89 python3.9 0x0000000105c65a7b function_code_fastcall + 139 90 python3.9 0x0000000105d92b3e call_function + 174 91 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 92 python3.9 0x0000000105c65a7b function_code_fastcall + 139 93 python3.9 0x0000000105d92b3e call_function + 174 94 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 95 python3.9 0x0000000105c65a7b function_code_fastcall + 139 96 python3.9 0x0000000105d92b3e call_function + 174 97 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 98 python3.9 0x0000000105c65a7b function_code_fastcall + 139 99 python3.9 0x0000000105d92b3e call_function + 174 100 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 101 python3.9 0x0000000105c65a7b function_code_fastcall + 139 102 python3.9 0x0000000105d92b3e call_function + 174 103 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 104 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 105 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 106 python3.9 0x0000000105c68cde method_vectorcall + 158 107 python3.9 0x0000000105d92b3e call_function + 174 108 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 109 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 110 python3.9 0x0000000105d7dc99 builtin_exec + 377 111 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 112 python3.9 0x0000000105d92b3e call_function + 174 113 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 114 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 115 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 116 python3.9 0x0000000105d92b3e call_function + 174 117 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 118 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 119 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 120 python3.9 0x0000000105e16ec2 pymain_run_module + 258 121 python3.9 0x0000000105e16781 pymain_run_python + 241 122 python3.9 0x0000000105e16645 Py_RunMain + 37 123 python3.9 0x0000000105e17d10 pymain_main + 64 124 python3.9 0x0000000105bfc238 main + 56 125 dyld 0x00007ff819dec310 start + 2432 1 HIToolbox 0x00007ff823b3d0c2 _ZN15MenuBarInstance22RemoveAutoShowObserverEv + 30 2 HIToolbox 0x00007ff823b547e3 SetMenuBarObscured + 115 3 HIToolbox 0x00007ff823b5b29e _ZN13HIApplication11FrontUILostEv + 34 4 HIToolbox 0x00007ff823b54622 _ZN13HIApplication15HandleActivatedEP14OpaqueEventRefhP15OpaqueWindowPtrh + 508 5 HIToolbox 0x00007ff823b4e950 _ZN13HIApplication13EventObserverEjP14OpaqueEventRefPv + 182 6 HIToolbox 0x00007ff823b16bd2 _NotifyEventLoopObservers + 153 7 HIToolbox 0x00007ff823b4e3e6 AcquireEventFromQueue + 494 8 HIToolbox 0x00007ff823b3d3ec ReceiveNextEventCommon + 285 9 HIToolbox 0x00007ff823b3d2b3 _BlockUntilNextEventMatchingListInModeWithFilter + 70 10 AppKit 0x00007ff81d344f33 _DPSNextEvent + 909 11 AppKit 0x00007ff81d343db4 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1219 12 libSDL2-2.0.0.dylib 0x0000000108918910 Cocoa_PumpEvents + 208 13 libSDL2-2.0.0.dylib 0x00000001088582c1 SDL_PumpEvents_REAL + 33 14 event.cpython-39-darwin.so 0x0000000106f85447 pg_event_pump + 23 15 python3.9 0x0000000105cc1350 cfunction_vectorcall_NOARGS + 96 16 python3.9 0x0000000105d92b3e call_function + 174 17 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 18 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 19 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 20 python3.9 0x0000000105d92b3e call_function + 174 21 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 22 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 23 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 24 python3.9 0x0000000105c68e1f method_vectorcall + 479 25 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 26 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 27 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 28 python3.9 0x0000000105c68e1f method_vectorcall + 479 29 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 30 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 31 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 32 python3.9 0x0000000105c68e1f method_vectorcall + 479 33 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 34 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 35 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 36 python3.9 0x0000000105c68cde method_vectorcall + 158 37 python3.9 0x0000000105d92b3e call_function + 174 38 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 39 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 40 python3.9 0x0000000105d7dc99 builtin_exec + 377 41 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 42 python3.9 0x0000000105d92b3e call_function + 174 43 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 44 python3.9 0x0000000105c7eb7f gen_send_ex + 175 45 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 46 python3.9 0x0000000105c7eb7f gen_send_ex + 175 47 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 48 python3.9 0x0000000105c7eb7f gen_send_ex + 175 49 python3.9 0x0000000105c73817 method_vectorcall_O + 103 50 python3.9 0x0000000105d92b3e call_function + 174 51 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 52 python3.9 0x0000000105c65a7b function_code_fastcall + 139 53 python3.9 0x0000000105d92b3e call_function + 174 54 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 55 python3.9 0x0000000105c65a7b function_code_fastcall + 139 56 python3.9 0x0000000105d92b3e call_function + 174 57 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 58 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 59 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 60 python3.9 0x0000000105c68cde method_vectorcall + 158 61 python3.9 0x0000000105c65633 PyVectorcall_Call + 163 62 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 63 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 64 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 65 python3.9 0x0000000105c68cde method_vectorcall + 158 66 python3.9 0x0000000105d92b3e call_function + 174 67 python3.9 0x0000000105d8635c _PyEval_EvalFrameDefault + 5548 68 python3.9 0x0000000105c7eb7f gen_send_ex + 175 69 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 70 python3.9 0x0000000105c7eb7f gen_send_ex + 175 71 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 72 python3.9 0x0000000105c7eb7f gen_send_ex + 175 73 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 74 python3.9 0x0000000105c7eb7f gen_send_ex + 175 75 python3.9 0x0000000105d8ccf3 _PyEval_EvalFrameDefault + 32579 76 python3.9 0x0000000105c7eb7f gen_send_ex + 175 77 _asyncio.cpython-39-darwin.so 0x00000001060e497a task_step_impl + 442 78 _asyncio.cpython-39-darwin.so 0x00000001060e472e task_step + 62 79 _asyncio.cpython-39-darwin.so 0x00000001060e45c8 task_wakeup + 104 80 _asyncio.cpython-39-darwin.so 0x00000001060e44bd TaskWakeupMethWrapper_call + 109 81 python3.9 0x0000000105c64d36 _PyObject_MakeTpCall + 134 82 python3.9 0x0000000105dae321 context_run + 81 83 python3.9 0x0000000105cc149c cfunction_vectorcall_FASTCALL_KEYWORDS + 76 84 python3.9 0x0000000105d8a4a7 _PyEval_EvalFrameDefault + 22263 85 python3.9 0x0000000105c65a7b function_code_fastcall + 139 86 python3.9 0x0000000105d92b3e call_function + 174 87 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 88 python3.9 0x0000000105c65a7b function_code_fastcall + 139 89 python3.9 0x0000000105d92b3e call_function + 174 90 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 91 python3.9 0x0000000105c65a7b function_code_fastcall + 139 92 python3.9 0x0000000105d92b3e call_function + 174 93 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 94 python3.9 0x0000000105c65a7b function_code_fastcall + 139 95 python3.9 0x0000000105d92b3e call_function + 174 96 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 97 python3.9 0x0000000105c65a7b function_code_fastcall + 139 98 python3.9 0x0000000105d92b3e call_function + 174 99 python3.9 0x0000000105d8676a _PyEval_EvalFrameDefault + 6586 100 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 101 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 102 python3.9 0x0000000105c68cde method_vectorcall + 158 103 python3.9 0x0000000105d92b3e call_function + 174 104 python3.9 0x0000000105d8b988 _PyEval_EvalFrameDefault + 27608 105 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 106 python3.9 0x0000000105d7dc99 builtin_exec + 377 107 python3.9 0x0000000105cc1565 cfunction_vectorcall_FASTCALL + 85 108 python3.9 0x0000000105d92b3e call_function + 174 109 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 110 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 111 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 112 python3.9 0x0000000105d92b3e call_function + 174 113 python3.9 0x0000000105d8ac1d _PyEval_EvalFrameDefault + 24173 114 python3.9 0x0000000105d836e9 _PyEval_EvalCode + 489 115 python3.9 0x0000000105c659bc _PyFunction_Vectorcall + 236 116 python3.9 0x0000000105e16ec2 pymain_run_module + 258 117 python3.9 0x0000000105e16781 pymain_run_python + 241 118 python3.9 0x0000000105e16645 Py_RunMain + 37 119 python3.9 0x0000000105e17d10 pymain_main + 64 120 python3.9 0x0000000105bfc238 main + 56 121 dyld 0x00007ff819dec310 start + 2432
Which type of action did we just perform with the step(1) call?
## Give your answer here.
Let's focus on the FrozenLake environment. Please carefully read its description. Try to play a game using random moves.
# %load solutions/FrozenLake.py