This unexpected new version has focused on supporting the training and evaluation of reinforcement learning algorithms in NVIDIA Isaac Orbit. Added: a) Isaac Orbit environment loader, b) Wrap an Isaac Orbit environment, c) Gaussian-Deterministic shared model instantiator
Added
- Support for Farama Gymnasium interface
- Wrapper for robosuite environments
- Weights & Biases integration (by @juhannc)
- Set the running mode (training or evaluation) of the agents
- Allow clipping the gradient norm for DDPG, TD3 and SAC agents
- Initialize model biases
- Add RNN (RNN, LSTM, GRU and any other variant) support for A2C, DDPG, PPO, SAC, TD3 and TRPO agents
- Allow disabling training/evaluation progressbar
- Farama Shimmy and robosuite examples
- KUKA LBR iiwa real-world example
Changed
- Forward model inputs as a Python dictionary [*breaking change*]
- Returns a Python dictionary with extra output values in model calls [*breaking change*]
- Adopt the implementation of `terminated` and `truncated` over `done` for all environments
Fixed
- Omniverse Isaac Gym simulation speed for the Franka Emika real-world example
- Call agents' method `record_transition` instead of parent method
to allow storing samples in memories during evaluation
- Move TRPO policy optimization out of the value optimization loop
- Access to the categorical model distribution
- Call reset only once for Gym/Gymnasium vectorized environments