《云计算技术与应用大连理工PPT优秀课件》由会员分享，可在线阅读，更多相关《云计算技术与应用大连理工PPT优秀课件（40页珍藏版）》请在金锄头文库上搜索。
1、云计算技术及应用大连理工大学计算机科学与技术学院2010年春季2021/5/261基本情况申彦明B810助教：齐恒B812Office hour: Fri 3:30-4:30 PMCourse website:http:/ CenterBigTableAppEngine2021/5/263GradingHW：40%Final Project: 60%Final project proposalProject reports12 teams, 4-5 students2021/5/264Syllabus (Subject to change)Week 2Mar 8: Lecture 1: Int
2、roduction Mar 10: Lecture 2: Map/Reduce Theory and Implementation, HadoopWeek 3Mar 15: Lecture 3 & 4: Guest Speaker (8:00 AM-11:35AM研教楼102)Mar 17: Lecture 5: Distributed File System and the Google File SystemWeek 4Mar 22: Lecture 6 & 7: Guest Speaker(8:00 AM-11:35AM研教楼102)Mar 24: Lecture 8: Distribu
3、ted Graph Algorithms and PageRankWeek 5Mar 29: Lecture 9: Introduction to Some ProjectsMar 31: Lecture 10: Data Centers2021/5/265Syllabus (Subject to change)Week 6Apr 5: Lecture 11: Some Google TechnologiesApr 7: Lecture 12: VirtualizationWeek 7Lecture 13 & 14: Project PresentationWeek 8: No class W
4、eek 9:Lecture 15 &16: Project Presentation2021/5/266Gartner ReportTop 10 Strategic Technology Areasfor 2009 VirtualizationCloud ComputingServers: Beyond BladesWeb-Oriented ArchitecturesEnterprise MashupsSpecialized SystemsSocial Software and Social NetworkingUnified CommunicationsBusiness Intelligen
5、ceGreen Information TechnologyTop 10 Strategic Technology Areas for 2010Cloud Computing Advanced AnalyticsClient Computing IT for GreenReshaping the Data CenterSocial ComputingSecurity Activity Monitoring Flash MemoryVirtualization for AvailabilityMobile Applications2021/5/267From Desktop/HPC/Grids
6、to Internet Clouds in 30 YearsHPC moving from centralized supercomputers to geographically distributed desktops, clusters, and grids to clouds over last 30 yearsR/D efforts on HPC, clusters, Grids, P2P, and virtual machines has laid the foundation of cloud computing that has been greatly advocated s
7、ince 2007Location of computing infrastructure in areas with lower costs in hardware, software, datasets, space, and power requirements moving from desktop computing to datacenter-based clouds2021/5/268What is Cloud Computing?1. Web-scale problems2. Large data centers3. Different models of computing4
8、. Highly-interactive Web applications2021/5/2691. “Web-Scale” ProblemsCharacteristics:Definitely data-intensiveMay also be processing intensiveExamples:Crawling, indexing, searching, mining the WebData warehousesSensor networks“Post-genomics” life sciences researchOther scientific data (physics, ast
9、ronomy, etc.)Web 2.0 applications 2021/5/2610How much data?Google processes 20 PB a day (2008)“all words ever spoken by human beings” 5 EBCERNs LHC will generate 10-15 PB a year640K ought to be enough for anybody.2021/5/2611What to do with more data?Answering factoid questionsPattern matching on the
10、 WebWorks amazingly wellLearning relationsStart with seed instancesSearch for patterns on the WebUsing patterns to find more instances2021/5/2612How do I make money?Petabytes of valuable customer dataSitting idle in existing data warehousesOverflowing out of existing data warehousesSimply being thro
11、wn awaySource of data:OLTPUser behavior logsCall-center logsWeb crawls, public datasets Structured data (today) vs. unstructured data (tomorrow)How can an organization derive value from all this data?2021/5/26132. Large Data CentersWeb-scale problems? Throw more machines at it!Centralization of reso
12、urces in large data centersNecessary ingredients: fiber, juice, and landWhat do Oregon, Iceland, and abandoned mines have in common?Important Issues:EfficiencyRedundancyUtilizationSecurityManagement overhead2021/5/26143. Different Computing ModelsUtility computingWhy buy machines when you can rent c
13、ycles?Examples: Amazons EC2Platform as a Service (PaaS)Give me nice API and take care of the implementationExample: Google App EngineSoftware as a Service (SaaS)Just run it for me!Example: Gmail“Why do it yourself if you can pay someone to do it for you?”2021/5/26154. Web ApplicationsWhat is the nat
14、ure of future software applications?From the desktop to the browserSaaS = Web-based applicationsExamples: Google Maps, FacebookHow do we deliver highly-interactive Web-based applications? AJAX (asynchronous JavaScript and XML)A hack on top of a mistake built on sand, all held together by duct tape a
15、nd chewing gum?2021/5/2616Some Cloud DefinitionsIan Foster et al defined cloud computing as a large-scale distributed computing paradigm, that is driven by economics of scale, in which a pool of abstracted virtualized, dynamically-scalable, managed computing power, storage, platforms, and services a
16、re delivered on demand to external customers over the internet(云计算是一种商业计算模型。它将计算任务分布在大量计算机构成的资源池上，使各种应用系统能够根据需要获取计算力、存储空间和各种软件服务。)IBM experts consider clouds that can:Host a variety of different workloads, including batch-style backend interactive, user-facing applicationsAllow workloads to be deplo
17、yed and scaled-out quickly through the rapid provisioning of virtual machines or physical machinesSupport redundant, self-recovering, highly scalable programming models that allow workloads to recover from HW/SW failuresMonitor resource use in real time to rebalance allocations on demand 2021/5/2617
18、Internet Cloud Goals Sharing of peak-load capacity among a large pool of users, improving overall resource utilizationSeparation of infrastructure maintenance duties from domain-specific application developmentMajor cloud applications include upgraded web services, distributed data storage, raw supe
19、rcomputing, and access to specialized Grid, P2P, data-mining, and content networking services2021/5/2618Three Aspects in Hardware that are New in Cloud ComputingThe illusion of infinite computing resources available on demand, thereby eliminating the need for cloud users to plan far ahead for provis
20、ioningThe elimination of an up-front commitment by cloud users, thereby allowing companies to start small and increase hardware resources when neededThe ability to pay computing resources on a short-term basis as needed (e.g., processors by the hour and storage by the day) and release them after don
21、e and thereby rewarding resource conservation2021/5/2619Some Innovative Cloud Services and Application OpportunitiesSmart and pervasive cloud applications for individuals, homes, communities, companies, and governments, etc.Coordinated Calendar, Itinerary, job management, events, and consumer record
22、 management (CRM) servicesCoordinated word processing, on-line presentations, web-based desktops, sharing on-line documents, datasets, photos, video, and databases, etcDeploy conventional cluster, grid, P2P, social networking applications in cloud environments, more cost-effectivelyEarthbound Applic
23、ations that Demand Elasticity and Parallelism rather data movement Costs2021/5/2620Operations in Cloud ComputingUsers interact with the cloud to request serviceProvisioning tool carves out the systems from the cloud configuration or reconfiguration, or deprovision The servers can be either real or v
24、irtual machinesSupporting resources include distributed storage system, datacenters, security devices, etc.2021/5/2621Cloud Computing InstancesGoogleAmazonMicrosoft AzureIBM Blue Cloud2021/5/2622Google Cloud InfrastructureSchedulerChubbyGFS masterNodeNodeNodeUserApplicationSchedulerslaveGFSchunkserv
25、erLinuxNodeMapReduceJobBigTableServerGoogle Cloud Infrastructure2021/5/2623S3EBSEC2EBSEC2EBSEC2EBSEC2SimpleDBSQSUserDeveloperAmazon Elastic Computing CloudSQS: Simple Queue ServiceEC2: Running Instance of Virtual MachinesEBS: Elastic Block Service, Providing the Block Interface, Storing Virtual Mach
26、ine ImagesS3: Simple Storage Service, SOAP, Object InterfaceSimpleDB: Simplified Database2021/5/2624 Azure Services PlatformMicrosoft Azure Platform2021/5/2625DeveloperMonitoringApplicationServerProvisioningManagerUserOpen Source Linux with XenTivoli Monitoring AgentIBM Blue Cloud2021/5/2626Cost Con
27、siderations : Power, Cooling, Physical Plant, and Operational CostsCosttechnology costscost of securityetc. Benefitsavailabilityopportunityconsolidationetc.2021/5/2627Cost Breakdown+ Storage ($/MByte/year)+ Computing ($/CPU Cycles)+ Networking ($/bit)2021/5/2628Research Challenges Service availabili
28、tyS3 outage: authentication service overload leading to unavailabilityAppEngine partial outageprogramming errorGmail: site unavailable Solutions:The management of a Cloud Computing service by a single company results in a single point of failure (SPF).In the Internet, a large ISP uses multiple netwo
29、rk providers so that failure by a single company will not take them off the air. Similarly, we need multiple Cloud Computing providers to support each other to eliminate SPF.2021/5/2629Research ChallengesData SecurityCurrent cloud offerings are essentially public rather than private networks, exposi
30、ng the system to more attacks such as DDoS attacks.Solutions:There are many well understood technologies such as encrypted storage, virtual local area networks, and network middle boxes.2021/5/2630Research ChallengesData Transfer BottlenecksApplications continue to become more data-intensive. If App
31、lications continue to become more data-intensive. If we assume applications may be “pulled apart” across we assume applications may be “pulled apart” across the boundaries of clouds, this may complicate data the boundaries of clouds, this may complicate data placement and transport.placement and tra
32、nsport.Both WAN bandwidth and intra-cloud networking technology Both WAN bandwidth and intra-cloud networking technology are performance bottleneck.are performance bottleneck.Industrial solutions:Industrial solutions:It is estimated that 2/3 of the cost of WAN bandwidth is It is estimated that 2/3 o
33、f the cost of WAN bandwidth is consumed by high-end routers, whereas only 1/3 charged consumed by high-end routers, whereas only 1/3 charged by fiber industry. by fiber industry. We can lower the cost by using simpler routers built We can lower the cost by using simpler routers built from commodity
34、components with centralized control, but from commodity components with centralized control, but research is heading towards using high-end distributed research is heading towards using high-end distributed routers .routers .2021/5/2631Research ChallengesSoftware LicensingCurrent software licenses c
35、ommonly restrict the computers on which the software can run. Users pay for the software and then pay an annual maintenance fee.Many cloud computing providers originally relied on open source software in part because the licensing model for commercial software is not a good match to Utility Computin
36、g.Some ideas:We can encourage sales forces of software companies to sell products into Cloud Computing. Or they can implement pay-per-use model to the software to adapt to a cloud environment.2021/5/2632Research ChallengesScalable storageDifferences between common storage and cloud storageThe system
37、 is built from many inexpensive commodity components that often fail The system stores a modest number of large filesThe workloads primarily consist both large streaming reads and small random reads. The workloads many large, sequential writes that append data to files and once written, files are se
38、ldom modified again.The cloud storage (file) system needs to share many of the same goals as previous distributed file systems such as performance, scalability, reliability, and availability. In addition, its design needs to be driven by key observations of the specific workloads and technological e
39、nvironment, both current and anticipated, that reflect a marked departure from some earlier file system design assumptions.GFSFiles are divided into fixed-size chunks, Chunk size is one of the key design parameters. GFS chooses 64 MB, which is much larger than typical file system block sizes.The mas
40、ter stores three major types of metadata: the file and chunk namespaces, the mapping from files to chunks, and the locations of each chunks replicas.GFS supports the usual operations to create, delete, open, close, read, and write files.2021/5/2633Research ChallengesTransparent Programming ModelProg
41、rams written for cloud implementation need to be automatically parallelized and executed on a large cluster of commodity machines. The run-time system should take care of the details of partitioning the input data, scheduling the programs execution across a set of machines, handling machine failures
42、, and managing the required inter-machine communication. The programming model should allow programmers without many experiences with parallel and distributed systems to easily utilize the resources of a large distributed system.MapReduceScalable Data Processing on Large ClustersA web programming mo
43、del implemented for fast processing and generating large datasets Applied mainly in web-scale search and cloud computing applications Users specify a map function to generate a set of intermediate key/value pairs Users use a reduce function to merge all intermediate values with the same intermediate
44、 key. 2021/5/2634Research Challenges2021/5/2635Steve Ballmers View on the Future of CloudCloud creates opportunities and Cloud creates opportunities and responsibilitiesresponsibilitiesCloud learns and helps you learn, decide and Cloud learns and helps you learn, decide and take actiontake action Cl
45、oud enhances social and professional Cloud enhances social and professional interactionsinteractionsThe cloud wants smarter devicesThe cloud wants smarter devicesCloud drives server advances that, in turn, Cloud drives server advances that, in turn, drive the clouddrive the cloud 2021/5/2636Cloud Co
46、mputing SkepticismCLOUD COMPUTING2021/5/2637Cloud computing is simply a buzzword used to repackage grid computing and utility computing, both of which have existed for decades.“Cloud computing is simply a buzzword used to repackage grid computing and utility computing, both of which have existed for
47、 decades.”Definition of Cloud Computing2021/5/2638Larry Ellison“The interesting thing about cloud computing is that weve redefined cloud computing to include everything that we already do. The computer industry is the only industry that is more fashion-driven than womens fashion. Maybe Im an idiot, but I have no idea what anyone is talking about. What is it? Its complete gibberish. Its insane. When is this idiocy going to stop?”2021/5/2639部分资料从网络收集整理而来，供大家参考，感谢您的关注！
