Estimated reading time: 2 minutes, 38 seconds

A Look At CraftAssist, Facebook’s Collaborative AI Assistant Bot For Minecraft Featured

A Look At CraftAssist, Facebook’s Collaborative AI Assistant Bot For Minecraft "Minecraft Earth impresses you with stunning 8-bit pixel looks and many AR (Augmented Reality) features."

Minecraft is a popular sandbox multiplayer online building game. It starts with a generated world where terrain and features are created with building blocks. There are also generated characters called “mobs” that appear throughout the world. The two types of gameplay in Minecraft are survival mode and creative mode. In survival mode the player can be harmed and has limited resources. In creative mode the player does not have to worry about being harmed and has access to all resources. In creative mode the player’s imagination is the limit and there have been endless creative projects that have been carried out in the game including recreations of cities and landmarks among others. 

CraftAssist is a framework created in 2019 that features dialogue-enabled interactive assistants. These assistants collaborate with human players in the game. The assistants can build, move objects, spawn mobs and communicate with players by chat. The framework is intended to research how agents interact with players and discover the possibilities between them. The framework is open source to encourage AI researchers to use the toolset. 

Players can interact with other players and the AI assistants, just like in a regular game of Minecraft. CraftAssist includes a baseline assistant, who brings over intelligence of previous interactions. There is also a house dataset that was put together using crowd sourced participants. These participants were tasked with constructing a house within 30 minutes and their actions were recorded. Both of these datasets are part of the core CraftAssist release.

Minecraft is played through a client/server architecture and the bot communicates through the Minecraft protocol. The Minecraft protocol does not require server side mods, which allows the bot to join in with other players and bots. The bot assistant is modular and this allows research to be conducted based on the individual models. These include modules related to perception and memory among others. The bot modules work together to perform tasks and interact with the human players. A human player may request that a bot build part of a house such as the roof. The bot then takes the chat commands and performs high-level actions that carry out the informal chat instructions to build the roof. 

Another example would be a player telling the AI assistant to build a house in the shape of a rectangle near the water. The modules take the given instructions from chat and create a program for the assistant. In this example the object water would be identified as being close and the assistant would be given coordinates to move to and start building. Machine learning works well for difficult and well defined tasks. One of the larger goals with CraftAssist is a framework that takes the often ambiguous human language commands and creates executable actions through the assistant. The interaction between humans and bots is a core focus of CraftAssist. 

More data from past interactions is being added to the assistant for further improvement. If you are a Minecraft player I would recommend giving CraftAssist a try to help build and improve the CraftAssist project!

CraftAssist Documentation:

https://research.fb.com/wp-content/uploads/2019/07/CraftAssist-A-Framework-for-Dialogue-enabled-Interactive-Agents-v3.pdf?

https://github.com/facebookresearch/craftassist

Read 3258 times
Rate this item
(0 votes)
Craig Gehrig

Craig Gehrig is a systems administrator with Rainbow Resource Center, an online retailer of educational materials in Peoria, IL. He is also the founder of Vanova IT, a security research and IT solutions provider. In his spare time, he can be found on the golf course and spending time with his wife Vanessa and their two children- Sasha and Craig.

https://twitter.com/CraigGehrig

Visit other PMG Sites:

We use cookies on our website. Some of them are essential for the operation of the site, while others help us to improve this site and the user experience (tracking cookies). You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.