13th IIAI International Congress on Advanced Applied Informatics, pp. 130-137, Dec. 12, 2022
12th International Conference on Smart Computing and Artificial Intelligence (SCAI 2022-Winter) in 13th IIAI International Congress on Advanced Applied Informatics (IIAI AAI 2022 winter)
This study proposes a reinforcement-learning-based method for efficient parameter tuning in multi-agent simulations (MAS). Usually, MAS has a high computational burden because of the agents involved; thus, it is important to tune its parameters efficiently. Our proposed method is centered around actor-critic-based reinforcement learning methods, such as deep deterministic policy gradient (DDPG) and soft actor-critic (SAC). In addition, this study proposes three additional components: an action converter (only for DDPG), a redundant full neural network actor, and a seed fixer. As an experiment, a parameter-tuning task is employed in an artificial financial market simulation. A Bayesian estimation-based method is used as the baseline. The results show that our model outperforms the baseline in terms of tuning performance, indicating that additional components of the proposed method are essential. Moreover, the critics of our models effectively function as surrogate models, that is, as an approximate function of the simulation, which allows the actor to appropriately tune the parameters.
multi-agent simulation; parameter tuning; deep reinforcement learning; artificial financial markets;
10.1109/IIAI-AAI-Winter58034.2022.00035
@inproceedings{Hirano2022-scai-winter,
title={{Efficient Parameter Tuning for Multi-agent Simulation Using Deep Reinforcement Learning}},
author={Masanori HIRANO and Kiyoshi IZUMI},
booktitle={13th IIAI International Congress on Advanced Applied Informatics},
pages={130-137},
doi={10.1109/IIAI-AAI-Winter58034.2022.00035},
year={2022}
}