GhostOS

The AI Ghosts wonder in the Shells.

• Documents
• Discord Server

(This document is translated from zh-cn to english by Moonshot)

Example

Using Python code SpheroBoltGPT, an intelligent robot with a SpheroBolt as its body is defined. If you have a SpheroBolt, running ghostos web ghostos.demo.sphero.bolt_gpt can start this robot.

The demo initially implements the following features:

1. Real-time voice conversation.
2. Control of body movements and drawing graphics on an 8x8 LED matrix.
3. Learning skills that include actions and animations through natural language dialogue.
4. Expressing emotions through movements during conversation.

Introduce

GhostOS is an AI Agent framework designed to replace JSON Schema with a Turing-complete code interaction interface (Moss Protocol), becoming the core method for interaction between LLM and Agent system capabilities. For more details: MOSS: Enabling Code-Driven Evolution and Context Management for AI Agents

The expected objects called through code include tools, personality, agent swarm, workflows, thinking, planning, knowledge, and memory. This allows a Meta-Agent to become an intelligent entity capable of continuous learning and growth through code generation and project management.

And such an intelligent agent implemented with a code repository can also be shared and installed in the form of a repository.

GhostOS Still in the early experimental developing, the current version mainly implements out-of-the-box capabilities, including:

• Turn a python file into a web agent
• Agent web UI built by Streamlit Web
• Support llms like OpenAI, Moonshot
• Support OpenAI vision
• Support OpenAI Realtime Beta

Quick Start

GhostOS remains a beta AI project, strongly recommending installation in containers such as Docker rather than running locally.

Install GhostOS package:

pip install ghostos

Initialize workspace (directory app as default), The runtime files of the current version will be stored in the directory.

ghostos init

Configure the model. Default to use OpenAI gpt-4o, requiring the environment variable OPENAI_API_KEY.

export OPENAI_API_KEY="your openai api key"
# Optionals: 
export OPENAI_PROXY="sock5://localhost:[your-port]" # setup openai proxy
export DEEPSEEK_API_KEY="your deepseek api key"
epoxrt MOONSHOT_API_KEY="your moonshot api key"

Or you can use configuration ui by streamlit:

ghostos config

Then test the default agent:

# run an agent with python filename or modulename
ghostos web ghostos.demo.agents.jojo

Or turn a local Python file into an Agent, that can be instructed to call functions or methods within the file through natural language conversations.

ghostos web [my_path_file_path]

some demo agents

ghostos web ghostos.demo.agents.jojo
ghostos web ghostos.demo.test_agents.moonshot         # moonshot-v1-32k model
ghostos web ghostos.demo.test_agents.deepseek_chat    # deepseek chat model
ghostos web ghostos.demo.test_agents.openai_o1_mini   # openai o1 mini model

You can create a local Python file and define your own Agents. For more details

• Chatbot: simplest chatbot
• MossAgent: an agent that can interact with the python module

Install Realtime

GhostOS support OpenAI Realtime, using pyaudio to handle realtime audio i/o. Need to install the dependencies first:

pip install 'ghostos[realtime]'

You may face some difficulties while install pyaudio on your device, I'm sure gpt-4o, google or stackoverflow will offer you solutions.

Use In Python

from ghostos.bootstrap import make_app_container, get_ghostos
from ghostos.ghosts.chatbot import Chatbot

# create your own root ioc container.
# register or replace the dependencies by IoC service providers.
container = make_app_container(...)

# fetch the GhostOS instance.
ghostos = get_ghostos(container)

# Create a shell instance, which managing sessions that keep AI Ghost inside it.
# and initialize the shell level dependency providers.
shell = ghostos.create_shell("your robot shell")
# Shell can handle parallel ghosts running, and communicate them through an EventBus.
# So the Multi-Agent swarm in GhostOS is asynchronous.
shell.background_run()  # Optional

# need an instance implements `ghostos.abcd.Ghost` interface.
my_chatbot: Chatbot = ...

# use Shell to create a synchronous conversation channel with the Ghost.
conversation = shell.sync(my_chatbot)

# use the conversation channel to talk
event, receiver = conversation.talk("hello?")
with receiver:
    for chunk in receiver.recv():
        print(chunk.content)

Developing Features

• Out-of-the-box Agent capability libraries.
• Variable type messaging and Streamlit rendering.
• Asynchronous Multi-Agent.
• Long-term task planning and execution.
• Atomic thinking capabilities.
• Automated execution and management of tree-based projects.
• Configurable components of the framework.
• Experiments with toy-level embodied intelligence.

GhostOS, as a personal project, currently lacks the energy to focus on improving documentation, storage modules, stability, or security issues.

The project's iteration will be centered on validating three directions for a long time: code-driven embodied intelligence, code-based thinking capabilities, and code-based learning. I will also aim to optimize out-of-the-box agent abilities.

So What is GhostOS purpose?

The GhostOS project is developed by the author for exploring AI applications. The basic idea is as follows:

AI Agent technology has two parallel evolutionary paths: one is the perfection of the model's own capabilities, and the other is the evolution of the Agent engineering framework. The productivity level of the Agent framework determines the feasibility of AI models in practical application scenarios.

GhostOS reflects the capabilities of an Agent from code into prompts, providing them to AI models, and the code generated by the models runs directly in the environment. Expecting the large language model do everything through a Turing-complete programming language interface, including computation, tool invocation, body control, personality switching, thinking paradigms, state scheduling, Multi-Agent, memory and recall, and other actions.

This will have stronger interaction capabilities and lower overhead than methods based on JSON schema. The conversation data generated in this process can be used for post-training or reinforcement learning of the model, thereby continuously optimizing the code generation.

The AI Agent itself is also defined by code. Therefore, a Meta-Agent can develop other Agents just like a normal programming task.

Ideally, the Meta-Agent can write code, write its own tools, define memories and chain of thoughts with data structures, and develop other Agents for itself.

Furthermore, most complex tasks with rigorous steps can be described using tree or graph data structures. Constructing a nested graph or tree using methods like JSON is very difficult, while using programming languages is the most efficient.

models can consolidate the results learned from conversations into nodes in the code, and then plan them into trees or graphs, thereby executing sufficiently complex tasks.

In this way, an AI Agent can store the knowledge and capabilities learned from natural language in the form of files and code, thereby evolving itself. This is a path of evolution beyond model iteration.

Based on this idea, GhostOS aims to turn an Agent swarm into a project constructed through code. The Agents continuously precipitate new knowledge and capabilities in the form of code, enriching the project. The Agent project can be copied, shared, or deployed in the form of repositories,

In this new form of productivity, interacting purely through code is the most critical step.

The author's ultimate goal is not GhostOS itself, but to verify and promote the code interaction design and applications. The hope is that one day, agents, paradigms, bodies, and tools for AI Agents can all be designed based on the same programming language protocols, achieving cross-project universality.