左烔菲,陈逢林,芮康旭

• 1:安庆师范大学数理学院

摘要(Abstract)：

多旅行商问题（multiple traveling salesman problem, MTSP）是旅行商问题的延伸。随着城市数量的增加，传统精确算法的求解速度缓慢，而启发式算法不具有泛化性，因此研究快速且有效的求解算法具有重要意义。文章提出了一种采用虚拟点的单智能体深度强化学习来求解MTSP的模型，其将加入一定数量虚拟点的城市节点坐标输入到编码器中，并通过解码器配合注意力机制来进行解码以输出一条完整路径。再采用Actor-Critic框架计算差值并进行训练，达到收敛后获得一种快速求解的模型，该模型将每一个虚拟点的路径作为每一个代理商人的路径。将传统方法和多智能体深度强化学习求解MTSP结果对比可知，采用虚拟点的深度强化学习所取得的计算效果均有一定程度的提高。

关键词(KeyWords)： 多旅行商问题;深度强化学习;虚拟点;智能体

Abstract:

Keywords:

基金项目(Foundation): 安徽省教育厅自然科学研究重点项目（KJ2019A0580）

作者(Author): 左烔菲,陈逢林,芮康旭

DOI: 10.13757/j.cnki.cn34-1328/n.2025.03.009

参考文献(References)：

• [1]李凯文,张涛,王锐,等.基于深度强化学习的组合优化研究进展[J].自动化学报, 2021, 47(11):2521-2537.
• [2]WANG Q, TANG C L. Deep reinforcement learning for transportation network combinatorial optimization:a survey[J]. Knowledge-Based Systems, 2021, 233:107526.
• [3]王扬,陈智斌,吴兆蕊,等.强化学习求解组合最优化问题的研究综述[J].计算机科学与探索, 2022, 16(2):261-279.
• [4]SVESTKA J A, HUCKFELDT V E. Computational experience with an m-salesman traveling salesman algorithm[J]. Management Science,1973, 19(7):790-799.
• [5]BELLMORE M, HONG S. Transformation of multisalesman problem to the standard traveling salesman problem[J]. Journal of the ACM,1974, 21(3):500-504.
• [6]LENSTRA J K, KAN A H G R. Some simple applications of the travelling salesman problem[J]. Journal of the Operational Research Society,1975, 26(4):717-733.
• [7]MAHMOUDINAZLOU S, KWON C. A hybrid genetic algorithm for the min-max multiple traveling salesman problem[J]. Computers&Operations Research, 2024, 162:106455.
• [8]KIRALY A, ABONYI J. A novel approach to solve multiple traveling salesmen problem by genetic algorithm[J]. Computational Intelligence in Engineering, 2010, 313:141-151.
• [9]ZHENG J Z, HONG Y W, XU W C, et al. An effective iterated two-stage heuristic algorithm for the multiple traveling salesmen problem[J].Computers&Operations Research, 2022, 143:150772.
• [10]GARN W. Balanced dynamic multiple travelling salesmen:algorithms and continuous approximations[J]. Computers&Operations Research,2021, 136:105509.
• [11]LAPORTE G, NOBERT Y. A cutting planes algorithm for the m-salesmen problem[J]. Journal of the Operational Research Society, 1980,31:1017-1023.
• [12]KAEMPFER Y, WOLF L. Learning the multiple traveling salesmen problem with permutation invariant pooling networks[J]. arXiv Preprint arXiv. 1803.09621, 2018.
• [13]LI J W, MA Y L, GAO R Z, et al. Deep reinforcement learning for solving the heterogeneous capacitated vehicle routing problem[J]. IEEE Transactions on Cybernetics, 2021, 52(12):13572-13585.
• [14]SON J, KIM M, CHOI S, et al. Equity-transformer:solving NP-hard min-max routing problems as sequential generation with equity context[J].arXiv Preprint arXiv:2306.02689, 2023.
• [15]VINYALS O, FORTUNATO M, JAITLY N. Pointer networks[J]. arXiv Preprint arXiv:1506.03134, 2017.
• [16]BELLO I, PHAM H, LE Q V, et al. Neural combinatorial optimization with reinforcement learning[J]. arXiv Preprint arXiv:1611.09940, 2016.
• [17]HU Y J, YAO Y, LEE W S. A reinforcement learning approach for optimizing multiple traveling salesman problems over graphs[J]. KnowledgeBased Systems, 2020, 204:106244.
• [18]PARK J, BAKHTIYAR S, PARK J. Schedulenet:learn to solve multiagent scheduling problems with reinforcement learning[J]. arXiv Preprint arXiv:2106.03051, 2021.
• [19]CAO Y H, SUN Z H, SARTORETTI G. Dan:decentralized attention-based neural network for the min-max multiple traveling salesman problem[J]. arXiv Preprint arXiv:2109.04205, 2021.
• [20]ZHANG R K, ZHANG C, CAO Z G, et al. Learning to solve multiple-tsp with time window and rejections via deep reinforcement learning[J].IEEE Transactions on Intelligent Transportation Systems, 2022, 24(1):1325-1336.
• [21]LE H M, YUE Y, CARR P, et al. Coordinated multi-agent imitation learning[J]. arXiv Preprint arXiv:1703.03121, 2018.
• [22]IQBAL S, SHA F. Actor-attention-critic for multiagent reinforcement learning[J]. arXiv Preprint arXiv:1810.02912, 2019.
• [23]VOUDOURIS C, TSANG E P, ALSHEDDY A. Handbook of metaheuristics:guided local search[M]. 2nd ed. Boston:Springer, 2010.