CraftAssist is a framework created in 2019 that features dialogue-enabled interactive assistants. These assistants collaborate with human players in the game. The assistants can build, move objects, spawn mobs and communicate with players by chat. The framework is intended to research how agents interact with players and discover the possibilities between them. The framework is open source to encourage AI researchers to use the toolset.
Players can interact with other players and the AI assistants, just like in a regular game of Minecraft. CraftAssist includes a baseline assistant, who brings over intelligence of previous interactions. There is also a house dataset that was put together using crowd sourced participants. These participants were tasked with constructing a house within 30 minutes and their actions were recorded. Both of these datasets are part of the core CraftAssist release.
Minecraft is played through a client/server architecture and the bot communicates through the Minecraft protocol. The Minecraft protocol does not require server side mods, which allows the bot to join in with other players and bots. The bot assistant is modular and this allows research to be conducted based on the individual models. These include modules related to perception and memory among others. The bot modules work together to perform tasks and interact with the human players. A human player may request that a bot build part of a house such as the roof. The bot then takes the chat commands and performs high-level actions that carry out the informal chat instructions to build the roof.
Another example would be a player telling the AI assistant to build a house in the shape of a rectangle near the water. The modules take the given instructions from chat and create a program for the assistant. In this example the object water would be identified as being close and the assistant would be given coordinates to move to and start building. Machine learning works well for difficult and well defined tasks. One of the larger goals with CraftAssist is a framework that takes the often ambiguous human language commands and creates executable actions through the assistant. The interaction between humans and bots is a core focus of CraftAssist.
More data from past interactions is being added to the assistant for further improvement. If you are a Minecraft player I would recommend giving CraftAssist a try to help build and improve the CraftAssist project!
CraftAssist Documentation:
