Codex Client Control started from a simple problem: most AI tools can explain Minecraft, talk about Minecraft, and write code for Minecraft, but they still cannot actually operate a running Minecraft client.
That gap is what this mod is meant to close. It opens a token-protected local HTTP and WebSocket interface on 127.0.0.1, so an AI agent on the same machine can read client state and send real in-game actions back to the client it is supposed to control.
This project was built with Codex in mind first. The name is not accidental: it exists to give Codex a practical way to see what the client is doing, inspect the current UI, and perform actual gameplay-side operations instead of stopping at text-only reasoning.
In practice, that means an external tool can read chat, inspect the current screen, check the focused block, look at container contents, move the player, turn the camera, switch hotbar slots, type into GUI fields, click widgets, take screenshots, or run short action sequences.
HTTP was chosen on purpose. It is simple, easy to inspect, easy to debug, and easy for AI tools to call. If an agent can make a normal local request, it can control the client without needing a special plugin host or an embedded scripting environment.
Why It Exists
- To let AI tools operate a real Minecraft client instead of only talking about one
- To give Codex a straightforward control path through a local API
- To make client automation easy to inspect, test, and extend
- To bridge normal scripts, desktop tools, and AI agents with a running game client
What It Can Control
- Player state, chat history, crosshair target, container contents, current screen, and player list
- Movement keys, look control, hotbar selection, chat sending, and command execution
- GUI clicks, scroll input, key input, text entry, and widget targeting
- Screenshots, action sequences, managed actions, and local debug helpers
Notes
- The API is local-only by default and token-protected.
- It controls the client you are already running; it does not log in a second account.
- It was built for Codex first, but anything that can call the local API can use it.
- If you need very heavy continuous automation, the repository also includes an optional
lan-botside project for that style of workload.
