Arceus

The repo for the pokemon simulator named Arceus

You can use pip install -e . to install this package

These are the basic classes in Arceus

• PokemonGold
• Recorder
• Interactor
• Bot
• ArceusDataset
• RandCrop
• RandConv
• GaussianNoise
• RayParallelEnv

PokemonGold

This is the main class of the package. The class is a subclass of gym.Env. It use retro as the simulator.

__init__(state="startwalk", scenario="scenario", record_path=False, resolution=(144,160))

Arguments:

• state (str): The starting state of the game.
• scenario (str): The scenario used in the game.
• record_path (str): The path to save the record.
• resolution (Tuple[int, int]): The resolution of the game. This is not changing the observation space, but changing the size of window in render()

Attributes:

• env (retro.Env): The simulator for the game.
• action_space (Discrete(9)): The action space of the env
• observation_space (Box): The observation space of the env
• movie_id (int): The movie id of the env. Used in record to show how many rounds you play. Each reset() called after the first reset() will make it +1.

This class initialize an simulator of the pokemonGold game. Arceus registered gym with id: "PokemonGold-v0", you can call both Arceus.make() or gym.make() with id = "PokemonGold-v0" and parameters to create the environment.

reset()

reset() will reset the game rom to the initial state. It will return the first observation of the state.

Return:

• observation: 3d list shape of (resolution, 3), value range from [0,255].

step(action)

Use the action to control the game. Return the game status after the step. Now the info step will contain ["x_pos", "y_pos", "battle", "HP_current1", "HP_Max1", "type1", "type2", "exp"]

Arguments:

• action (bool list of length 9): Represent to the order of ['B', None, 'SELECT', 'START', 'UP', 'DOWN', 'LEFT', 'RIGHT', 'A']. If multiple place is True, it will only act the preceding one.

Return:

• observation (Box): The observation space after the step.
• reward (int): The reward value after this step. You can define the reward in pokemongold-GbColor/scenario.json
• done (bool): The done value after this step. You can define the done condition in pokemongold-GbColor/scenario.json
• info (dict): The info after this step. You can define the info keys in pokemongold-GbColor/data.json

Info explanation:

• x_pos, y_pos: The x, y value of the character in current map.
• battle: 0 for no battle and 1 for in battle.
• HP_current1, HP_Max1: The current and maximum hp of the first pokemon.
• type1,2: The type of your active pokemon in battle.

render(mode="human", close=False)

Render the observation of the current state. The viewer is based on the gym.envs.classic_control.rendering.SimpleViewer. I made some tiny change to allow it change the screen size. This class (rendering) is removed from gym in gym-0.21.0 (the latest update). So I had to copy the code into my package.

Arguments:

• mode ("human" or "rgb_arary"): The human mode will pop a window shows the image. The "rgb_array" mode will return an array of the current observation.
• close (bool): Close the window of the human mode. (Do this after finish all your work)

seed(seed)

Seed the environment with the seed.

Arguments:

• seed (int): The seed you want to use.

close()

Close the environment. You can only create a new environment after you close the previous one. (This cause a problem when designning the multithread vector env, there's also a issue on Github showing this https://github.com/openai/retro/issues/64)

Example

See examples in Arceus_API_Document.ipynb

Recorder

This is not actually a recorder. It is a tool to read from .bk2 file generated by retro and create a mp4 file for the observation. It use the playback_movie function in retro.utils. You need to install ffmpeg to use the recorder.

__init__(record_path, state, runtime=1)

Arguments:

• record_path (str): The record path you use in creating the env. YOU CANNOT USE RECORDER WITHOUT PASSING RECORD_PATH TO ENV
• state (str): The state you load to the env
• runtime (int): The total number of reset you use to start a new game. This is for find the movie id of the file. Remember in movie_id, movie_id will +1 if you call reset(). The recorder will generate mp4 file for all the .bk2 file under the path. You will get [runtime] of mp4 file.

generate_movie()

Generate mp4 file of all the .bk2 file under the path. The number of mp4 file will be equal to [runtime]. If you call reset() before doing any action, the generate program may get stuck when processing that .bk2 file.

Example

See example in recorder_example.py

Interactor

The interactor for the game. You can pass the env to the interactor to interact with the game. The interactor is based on render() of the env and pynput for dealing with the keyboard interrupt.

__init__(env, fps=60)

Arguments:

• env (PokemonGold): The env you want to interact with.
• fps (int): The fps of the game when using interactor.

run()

Run the interactor. Will pop a window that cannot close by [x].

• Use arrow keys on your keyboard to move
• [z] is for key 'A'
• [x] is for key 'B'
• [ENTER] is for key 'START' (The retro simulator seems have a bug of this. I'm sure the [ENTER] key is detected and passed to step(), but the retro did not respond that key.)
• [TAB] is for key 'SELECT'

Example

See example in interactor_example.py

Bot

A bot that can give action to the env. Now it only have random step and stupid step.

__init__(env)

Arguments:

• env (PokemonGold): The env you want to play

The bot will not change anything in the env. It just read some information from the env.

random_step()

Return:

• action: a purely random action to take

stupid_step(info)

Arguments:

• info : The last info of the step you take.

Return:

• action: Action based on the info. Now it will only judge whether its in battle. If in battle, it will take random step. If not in battle, it will only take four direction step. The bot will continue return the same direction for MIN_STEPS_TO_MOVE frames to make sure have a step.

Example

import Arceus

env = Arceus.make("PokemonGold-v0", record_path=".")
obs = env.reset()
bot = Arceus.Bot(env)

info, rew, done = None, None, None
for i in range(600):
    obs, rew, done, info = env.step(bot.stupid_step(info))
env.close()

ArceusDataset

The dataset is a subclass of torch.utils.data.Dataset. You need first to stall each .bk2 file in a separate folder. Then tell the dataset the path to the folder and the state and number of folders in the path. It will load all the files in that path. There are also two modes of dataset. In single mode, All the entry is stored in one list, not separate for each state. The data dict record the start and end point of each state entry. This allow you to use several indexing method to get item. In full mode, each entry is the whole step list. Will be padded to the max length of the step list.

Details

See Arceus_Dataset_API.ipynb

Wrappers

Design three wrappers for the observation. See Arceus_Wrapper_API.ipynb for detail.

VectorEnv

RayParallelEnv

This is a subclass of gym.vector.VectorEnv. Use ray @remote to do the multi-process part.

__init__(num_envs=1, args=None)

Arguments:

• num_envs (int): The number of parallel environments you want to create.
• args (arr[dict]): If args is None, it will be automatically turned into [{'state':"startwalk", 'scenario': 'scenario', 'record_path': False, 'resolution': (144,160)}] * num_envs. You can define your args for each env in envs separately through args.

reset()

Reset all the envs in the vector.

Return: A list of obs of each env.

step(actions)

Arguments:

• actions (arr[arr[bool]]): The actions you want to step each env. Need to be a array of action you want to take and had to be the same length of num_envs. If you only pass one action space, it will automatically copy num_envs of it and pass them to the envs.

Return: A list of obs of each env.

seed(seeds)

Arguments:

• seeds (arr[arr[bool]]): The seeds you want to seed each env. Need to be a array of seed you want to use and had to be the same length of num_envs. If you only pass one seed, it will automatically copy num_envs of it and pass them to the envs.

Example

See rayparallel_example.py

Redis and Zmq

Not implemented yet, don't have enough time.

Reference

1. Pokemon gold ROM from https://www.emulatorgames.net/roms/gameboy-color/pokemon-gold-version/
2. How I Taught an AI to Play Pokemon Emerald https://youtu.be/9YyQJIuN7n0
3. RAM map for Gold and Silver https://datacrystal.romhacking.net/wiki/Pok%C3%A9mon_Gold_and_Silver:RAM_map
4. gym retro example code https://github.com/openai/retro
5. gym.envs.classic_control.rendering in gym-0.20.0 https://github.com/openai/gym/blob/v0.21.0/gym/envs/classic_control/rendering.py
6. pynput https://pypi.org/project/pynput/
7. randconv code from https://github.com/wildphoton/RandConv/blob/main/lib/networks/rand_conv.py