《多智能体增强式学习及其在多机器人协调中的应用研究》由会员分享，可在线阅读，更多相关《多智能体增强式学习及其在多机器人协调中的应用研究（129页珍藏版）》请在金锄头文库上搜索。
1、上海交通大学博士学位论文多智能体增强式学习及其在多机器人协调中的应用研究姓名：杨玉君申请学位级别：博士专业：控制理论与控制工程指导教师：程君实20030501多智能体增强式学习及其在多机器人协调中 的应用研究 摘 要 目前的工业控制系统正向大型复杂动态和开放的方向转变传统的多机器人系统在许多关键问题上遇到了困难分布式人工智能DAIDistributed Artificial Intelligence及其分支多智能体系统MASMulti-Agent System理论为解决这些问题提供了一种契机将 MAS 充分应用于多机器人系统的结果便产生了一门新兴的机器人技术领域多智能体机器人系统( MARSM
2、ulti-Agent Robot System) MARS 技术正在蓬勃发展之中 在工业管道传输中有许多细小管道使用过程中将受到腐蚀或机械损伤而积垢存污若不定期维护和清理可能会使管道破裂或堵塞而酿成事故例如核电站的热交换器长年累月形成的积垢会腐蚀管壁是核泄漏的潜在隐患本博士论文得到了国家自然科学基金面向管道检测的多微机器人移动控制协调及集成技术No. 6 9 8 8 9 0 5 0 项目的资助研究管道间的多微小型移动机器人动作协调和故障检测由于管道之间的距离大约在 1520mm所以这微小型移动机器人本体的体积设计也在毫米级上 由于体积的微型化 单个微机器人的工作能力很弱需要多个微机器人协调合作
3、来共同完成复杂的任务所以在提高单个机器人性能的基础上要求对多个微小型移动机器人进行协调控制这构成了本博士论文研究的动因 在多智能体系统( MAS) 中 最集中和关键的问题表现在多智能体的学习及相应的协调合作机制上我们根据国际上多智能体系统的研究情况, 并针对面向管道检测多微小型移动机器人实际特点提出了我们的研究课题本博士论文的主要内容及创新点如下 1) 采用 Multi-agent SystemMAS理论来研究多机器人协调问题 2) 将博弈论思想引入到多智能体的增强式学习中提出一种特殊联合奖赏函数结构形式 3) 提出一种基于传导径迹的多智能体 Q 学习智能体不需要知道其他智能体的 Q 函数结构
4、只需维持其他智能体动作行为的传导径迹 4) 采用 MLPs多层感知器来泛化多智能体的状态空间结合 Q 学习在线地更新 MLPs的连接权值解决了 MLPs 的过拟合缺陷同时还可以节约存储空间 5) 采用国家自然科学基金课题组研制的毫米级微小型移动机器人 SJTU-MR和 SJTU-MR搭建了模拟面向管道检测的多微小型移动机器人环境系统建立了多微小型移动机器人系统实验平台 由于当前国内微器件的加工水平我们研制的微小型移动机器人还只能用于特定的工作环境中 我们建立了一个面向管道检测的毫米级多微小型移动机器人实验平台 虽然该平台还不完善但它为面向管道检测的多微小型移动机器人协调研究提供了一个良好的开端
5、随着我国微器件加工工艺的不断提高和科研人员的努力相信一定可以研制出实用的面向管道检测的多微小型移动机器人系统 关键词多智能体系统多智能体机器人系统多智能体增强式学习博弈微小型移动机器人多层感知器MLPs泛化 Study on Multi-agent Learning and Its Application to Multiple Robots with Coordination Abstract The industry control system is a complex, dynamic and open system, and the traditional industry syst
6、em and multiple robots system face difficulties on some key problems. Distributed Artificial Intelligence (DAI) and its embranchment, Multi-Agent System (MAS) theory supports a chance to settle these problems. The results using the MAS theory to the multiple robots system, give birth to the new robo
7、t technique fieldMulti-Agent Robot System (MARS), which is developed full of vitality. In the industry pipeline transmission, where the pipelines usually are exiguous, the pipelines have much begrime for being cankered or scathed. The pipeline should be cleaned up, otherwise which would make for acc
8、ident, once the pipelines burst or wall up. The heat exchanger in the nuclear power plant, for example, forms the smudginess for long time and cankers the tube well, which is the hidden trouble of nucleus leakage. This Ph.D. thesis is supported by the National Nature Science Foundation of China (No.
9、 69889050) “The technology of multiple robots moving, control, cooperation, integration facing pipeline inspect”, which studies on the multiple robots actions cooperation and fault detection. For the distance between the pipelines usually is about 1520mm, the micro-mobile robot noumenal size should
10、on the mm class. The ability of the micro-mobile robot is weak for its oversize, so multiple micro-mobile robots should accomplish complex with cooperation. The study on the cooperation control mechanism for the multiple micro-mobile robots is the important cause of this thesis on the basis of impro
11、ving the ability of single micro-mobile robot. The system architecture and cooperation mechanism are the key problem in the Multi-Agent System. Based on condition of the international multiple robots system, we bring forward our study task according to the practical situation of the multiple micro-m
12、obile robots facing pipeline fault detection. The primary contents of this thesis are as follows: 1) Study the cooperation problem of multiple micro-mobile robots adopting Multi-agent SystemMAStheory. 2) Study the Multi-agent Q-learning with the Game Theory; propose an especial structure of payoff f
13、unction. 3) Propose a Multi-agent Q-learning based on eligibility traces. The agent would not know the structure of other agents Q functions, and only maintain the eligibility traces of the other agents past actions. 4) Adopt MLPsMulti-Layer Perceptronsto generalize the Multi-agent state space, inte
14、grate MLPs with Q-learning, which on- line updates the connectionist weights of the MLPs. This algorithm settles the over-fitted shortcoming of MLPs, at the same time economize the memory. 5) Adopt two micro-mobile robots, SJTU-MRa n d SJTU-MR on the mm class, which is designed by the Group of the N
15、ational Nature Science Foundation, and build a experimentation system for the multiple micro-mobile robots facing pipeline fault detection. For the micro apparatus machining level, our micro-mobile robots only work in the specifically environment. We build a experimentation system for the multiple m
16、icro-mobile robots facing pipeline fault detection. Although the system is not perfect, it supports the favorable outset for the cooperation mechanism of multiple micro-mobile robots. We can develop applied multiple micro-mobile robots system with the development of the micro apparatus. Key WordsMulti-agent SystemMulti-agent Robot SystemMulti-agent Reinforcement LearningGame TheoryMicro-mobile Robot 上海交通大学学位论文原创性声明本人郑重声明所 呈 交 的 学 位 论 文是 本 人 在 导 师 的 指 导 下独 立 进 行 研究 工 作 所 取 得 的 成 果除 文 中
