Many real-world problems involve optimizing multiple conflicting objectives simultaneously, often by a series of decisions. Multi-objective reinforcement learning addresses these challenges by training agents to learn policies that balance multiple reward signals from the environment.

In recent years, the community has increasingly paid attention to these problems. However, the necessity for more effective solutions persists. In this context, we propose a methodology for approximating optimal policies using evolutionary algorithms.

Our approach enhances the search process by leveraging uncertainty quantification in the hypervolume computation of candidate solutions. We adapt the classical evolutionary process, changing the evaluation of agents to prioritize those contributing more to the hypervolume and using uncertainty quantification as a guiding metric.

Further, we analyzed four evolutionary algorithms under different uncertainty measures and tested them in diverse multi-objective reinforcement learning environments with up to four objectives. The work revealed that the expected improvement was the most effective uncertainty quantification for guiding the search across all algorithms.