《基于自动交换光网络的mpi的设计与实现》由会员分享，可在线阅读，更多相关《基于自动交换光网络的mpi的设计与实现（70页珍藏版）》请在金锄头文库上搜索。
1、上海交通大学 硕士学位论文 基于自动交换光网络的MPI的设计与实现 姓名：毕旋 申请学位级别：硕士 专业：计算机技术与应用 指导教师：李明禄 20080101 I 基于自动交换光网络的 MPI 的设计与实现 摘 要 集群（Cluster）节点间互联通信技术对集群系统进行并行计算的效 率和扩展性有着极其重要的影响。光纤作为区域存储网络（Storage Area Network，SAN）在集群系统中有着迫切的应用需求和良好的应用前景： 1）集群网络对带宽的需求在不断增加。从 100M 发展到 1Gb，再到目 前的 20Gb，未来仍有持续增加的趋势；2）目前主流的集群网络，如 InfiniBand，
2、Myrinet 以及 GE（千兆以太网）等在提供带宽（20Gb）服 务时运行成本较高，且部分技术不够成熟。相比之下，光纤目前最高已 可达 40Gb，且具有技术成熟、运行成本较低的优点。 将光纤应用到集群系统中的难点在于与集群软件的兼容性和性能 优化。 1） 光纤网络是基于电路交换 （circuit switching） 而非包交换 （packet switching） 。一旦连接建立，该连接上的带宽资源就一直保留为该连接 的用户服务，直至通信双方断开连接资源才被释放。因此直接将光纤网 络应用到集群系统中去，会造成当前流行的一些针对基于包交换的内部 互联网络的集群管理软件的兼容性问题；2）光纤网
3、络只能建立点到点 的连接，不能直接建立一对多或多对一的连接。虽然可以使用 TCP/IP 来间接实现组播的功能，但仍然需要在使用组播时考虑性能优化的问 题。 本文基于 MPI（Message Passing Interface）对光纤网络在集群中的 应用进行研究。MPI 是目前最流行的并行编程消息传递规范，它为消息 传递和相关操作提供了库例程。很多集群上的并行应用都是基于 MPI 开发的，这其中就包括了用来评测 TOP500 (Top 500 Super Computers) 的 HPL（High Performance LINPAC）基准程序。MPICH2 是目前比较流 行的，对 MPI 标准
4、的一个免费开源实现。基于 MPICH2 的源代码，我 们可以方便地进行二次开发，并进行相关的实验工作。 本文利用 ASON（Automatically Switched Optical Network）协议， 主要研究如何将光纤网络应用于集群系统的并行环境中。首先，我们设 II 计并实现了 BUNIA（Beyond UNI Agent） ，对 ASON 的 UNI-C（UNI 客 户侧信令代理）协议进行封装，用于向客户端提供可以远程调用的动态 拆建光链路的接口。其次，结合对 BUNIA 的接口调用，我们通过改进 MPICH2 在消息传递中的点对点通信和组通信操作，设计并实现了 ASON 自动交
5、换光网络上的 MPI 消息传递库 AN_MPI。最后，我们通过 实验验证了将光纤网络应用到集群系统的并行环境下可以带来更高的 通信性能。 本文的创新点主要包括两个方面：1）ASON 自动交换光网络的 UNI-C 协议提供了建立和拆除光链路的接口，但是该接口不能被客户端 远程调用，而我们对系统的设计期望是松耦合的。因此，我们设计并实 现了 BUNIA，它对 UNI-C 协议进行封装，并以 Socket 形式提供给客户 可远程调用的拆建光链路的接口；2）设计并实现 AN_MPI，使得在点 对点通信和集合通信操作中可以通过调用 BUNIA 的接口来动态的拆建 光链路。 关键词： 光纤网络，ASON，
6、MPICH2，动态拆建光链路 III DESIGN AND IMPLEMETATION OF MPI OVER ASON ABSTRACT The inter-connection among nodes of cluster plays an important role on the efficiency and scalability of the parallel computing. As the implementation of SAN (Storage Area Network) for clusters, optical network has a high performa
7、nce advantage and possibility to be applied in cluster system: 1) The bandwidth requirement of cluster network has grown from 100Mb to 1Gb, until currently 20Gb, and this trend is even continuing; 2) Today, most cluster networks, such as InfiniBand, Myrinet, and GE (Giga Ethernet) not only require h
8、igh cost under 20Gb bandwidth but also need mature technology. On the contrary, optical network can provide 40GB bandwidth using mature technology while maintain low cost. The difficulty of applying optical network into cluster system lies in software compatibility and performance optimization. Firs
9、tly, optical network is based on circuit switching, which determines its difference to packet switching ones. Resources of certain connection will be reserved for users from the time when the connection is established. Network resource is exclusive and will not be released until the disconnection of
10、 link. Therefore, applying optical network into cluster system directly will lead to problems of compatibility, especially for some management software of packet switching network. Secondly, optical network can only setup point to point but one-to-multiple or multiple-to-one connections; even if usi
11、ng TCP/IP can implement multi-cast indirectly, performance optimization should be taken into consideration In this paper, our research on optical network in cluster mainly bases on MPI (Message Passing Interface). MPI is the most popular message passing specification currently. It provides standard
12、library for parallel message IV delivery and operation. There are so many parallel applications which developed on MPI, such as the HPL (High Performance LINPACK) which is used to test the performance of TOP 500 (Top 500 Super Computers). MPICH2 is a popular implementation of MPI, based on its sourc
13、e code, the author can implement the second development and related experiment works. This paper mainly focuses on research of applying ASON (Automatically Switched Optical Network) into parallel environment of cluster system. At first, we design and implement the BUNIA (Beyond UNI Agent), which enc
14、apsulates UNI-C (UNI Client Side Signal Agent Protocol) and provides remote invocation interface of dynamically establishing and destroying network connection to clients. Secondly, on the basis of MPICH2 source code as well as BUNIA interface invocation, we re-factor the communication and operation
15、of P2P message delivery to implement ASON parallel message library as AN_MPI. At last, we verify that optical network can be applied into cluster environment for higher efficiency by performance experiments. The innovation of this paper lies in two aspects: 1) The UNI-C of ASON provides interfaces o
16、f establishing and disconnecting light path links. However such APIs can not be invoked by client using remote call directly. Since it is expected that the system should be a loose coupling one, we design and implement the BUNIA which encapsulates the UNI-C protocol of ASON, providing remote invocation APIs for clients by means of socket connection.; 2) we design and implement the AN_MPI, which includes re-factoring of MPI-1 on peer-to-peer communication and collective communication operations
