Frenetix-RL

This repository includes a PPO Reinforcement Learning accelerated trajectory planning algorithm in the CommonRoad scenario format. The trajectories of the analytical planner are generated according to the sampling-based approach in [1-5] including two different implementations. The Repo provides a python-based and a C++-accelerated Motion Planner Frenetix implementation.

Requirements

The software is developed and tested on recent versions of Linux. We strongly recommend to use Ubuntu 22.04 or higher. For the python installation, we suggest the usage of a Virtual Environment with Python 3.10 or Python 3.9. For the development IDE we suggest PyCharm

🔧 Optional Pre-Installation Instructions (Ubuntu)

Make sure that the following dependencies are installed on your system for the C++ implementation:

• Eigen3
  • On Ubuntu: sudo apt-get install libeigen3-dev
• Boost
  • On Ubuntu: sudo apt-get install libboost-all-dev
• OpenMP
  • On Ubuntu: sudo apt-get install libomp-dev
• python3.10-full
  • On Ubuntu: sudo apt-get install python3.10-full and sudo apt-get install python3.10-dev

1. Clone this repository & create a new virtual environment python3.10 -m venv venv

2. Install the package:

   • Source & Install the package via pip: source venv/bin/activate & pip install . or with poetry poetry install
   • Frenetix should be installed automatically. If not please write [email protected].

3. Optional download of additional scenarios and copy them to the folder scenarios or scenarios_validation:

4. Optional change of configurations in the following files & folders:

   1. configurations --> Analytic planner configs
   2. frenetix-rl/gym_environment/configs.yaml --> RL training environment config
   3. frenetix-rl/hyperparams/ppo2.yml --> PPO hyperparameter settings

5. There is already a best_model to execute if you do not want to train one by yourself. Skip step 6 to skip the training procedure.

6. Start Training with python3 train.py

7. Logs can be found in the logs folder. tensorboard_logs can be found in the logs_tensorboard folder. If you want to visualize them, install tensorboard with pip install tensorboard and execute tensorboard --logdir logs_tensorboard/PPO_1/.

8. Execution of the trained model can be done with python3 execute.py. The plot visualizations of the executed model will be saved in logs again.

📈 Test Data

Additional scenarios can be found here. Load the files and add them to scenarios for training data or scenarios_validation for validation data.

🔧 Modules

Detailed documentation of the functionality behind the single modules can be found below.

1. General RL Algorithm

2. Frenetix Motion Planner

3. Frenetix C++ Trajectory Handler

4. Wale-Net

5. Risk-Assessment

📇 Contact Info

Rainer Trauth, Institute of Automotive Technology, School of Engineering and Design, Technical University of Munich, 85748 Garching, Germany

Johannes Betz, Professorship Autonomous Vehicle Systems, School of Engineering and Design, Technical University of Munich, 85748 Garching, Germany

📃 Citation

If you use this repository for any academic work, please cite our code:

• RL-Paper
• Analytical Planner Paper

@misc{trauth2024reinforcement,
      title={A Reinforcement Learning-Boosted Motion Planning Framework: Comprehensive Generalization Performance in Autonomous Driving}, 
      author={Rainer Trauth and Alexander Hobmeier and Johannes Betz},
      year={2024},
      eprint={2402.01465},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}

@misc{frenetix2024,
      title={Frenetix Motion Planner: High-Performance and Modular Trajectory Planning Algorithm for Complex Autonomous Driving Scenarios}, 
      author={Korbinian Moller and Rainer Trauth and Gerald Wuersching and Johannes Betz},
      year={2024},
      eprint={2402.01443},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}