Skip to content

Open Minecraft AI (OMAI)

This Fontys BeCreative Minor project will use the principles of Machine Learning and Artificial Intelligence to train OMAI to play Minecraft. OMAI will play as a Minecraft agent and learn how to solve mazes. Minecraft is a 3D sandbox video game where players can do whatever they want in a virtual, pixeled world. In this world, players must survive while they craft items, explore in an infinitely large world, and build the biggest structures in a LEGO-like way. Players can build houses, search for rare items inside mines or locate portals to other dimensions and, for example, slay a dragon.

Example of a Minecraft maze.

Project description

In this project, Tesseract is used to convert text from an image to text. This worked well but later during the project we came across Firebase OCR. This OCR gave better converting results. We did not test if Firebase OCR is quicker than Tesseract, or how resource intensive it is.

We developed our own interfacing for the game Minecraft. This was very time-consuming. Instead of doing it our self we could have used project Malmo from Microsoft. The only downside of project Malmo is that it is a modification of Minecraft and cannot run on every Minecraft game. OpenAI Gym is also a system, which can be combined with project Malmo to implement reinforcement learning to the project.

Our project is designed so the Minecraft character can solve 2D mazes without special objects like fences and doors. The character is also limited to only use the keys [W, A, S, D]. The next step would be to solve 3D mazes and include the special obstacles.

To make programming efficient, we used fixed heatmaps to generate a map for the bot to walk through the maze. It would be better if the heatmap gets generated with a given start- and endpoint. This gives the user more freedom for designing parkours.

Q-learning principle.

We used Q-learning for this project: it learns to solve a specific task, in our case parkour/maze. In a follow-up project, deep Q-learning can make the bot smarter by using a neural network implementation.

Project results

The system in our project consists of three components: Webserver, Gaming PC, and Coral Dev Board. Note that the Gaming PC and Coral Dev Board must be in pairs (one player, one game client). If you are interested in the software that was developed in this BeCreative Minor project, feel free to request a copy our report.

The platform we created can be used by other students to improve it and learn about interfacing a game. In the video below we show the capabilities of our system after 5 months of development, where we started from scratch with hardly any knowledge of Machine Learning.

Links

Leave a Reply

Your email address will not be published. Required fields are marked *