Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stable multi-agent & multi-world MAPPO on example scene with road lines #89

Merged
merged 10 commits into from
Apr 25, 2024

Conversation

daphne-cornelisse
Copy link
Contributor

@daphne-cornelisse daphne-cornelisse commented Apr 19, 2024

This PR adds support for the 3 types of collision behavior in the gym environment (base_environment.py).

Tested: 100% success rate in the following settings

  • multi-agent and multi-world in example_scenario.json with road lines (using the "remove" option for collision behavior).
  • multi-agent and multi-world in W = 2 worlds with different different maps. The maps contain different numbers of agents.
  • used selfObservationTensor, agentMapObservationsTensor, and partnerObservationsTensor

Gym environment logic

  • We use "nan" values to indicate invalid agents in environments.
  • Since the number of valid agents varies per scenario (map), scenarios with fewer than kMaxAgentCount controlled agents are padded with nan values.
  • Example: Suppose we have two scenes, one with two and the other with three valid agents. We set kMaxAgentCount to five (in consts.hpp) and max_cont_agents to three:
 env = Env(
        config=config,
        num_worlds=2,
        max_cont_agents=3,
        data_dir="waymo_data",
        device="cuda",
    )
  • When we step the environment, we get tensors of shape (num_worlds, kMaxAgentCount), where we control at most max_cont_agents per environment. Using the current base environment, the done tensor may look like:
done = torch.Tensor(
    [0, 1, nan, nan, 0],
    [0, 0, nan, 0, nan],
)

The above tensor is interpreted as follows:

  • We have one valid agent that is done (as marked by the value 1)
  • We have five valid agents that are not done (marked by 0)
  • We have four invalid agents (as marked by the value nan)

Note

  • Due to garbage values in controlledStateTensor, it is possible that I'm still controlling a few invalid agents. This will be tested later.

Sorry, something went wrong.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@daphne-cornelisse daphne-cornelisse merged commit d168a18 into main Apr 25, 2024
1 check passed
@daphne-cornelisse daphne-cornelisse deleted the dc/end_to_end branch August 28, 2024 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants