【正文】 Cloud puting platform, infrastructure and theory 云計(jì)算平臺(tái)，架構(gòu)和理論 Jiaheng Lu Key Lab of Data Engineering and Knowledge Engineering Renmin University of China 主要內(nèi)容 3 ? 云計(jì)算概述 ? Google 云計(jì)算技術(shù)： GFS， Bigtable 和Mapreduce ?開(kāi)源平臺(tái) Hadoop介紹 ?云計(jì)算理論 ?事務(wù)處理理論 ? DataLog理論 新編教材 《 分布式系統(tǒng)與云計(jì)算概述 》 ? 系統(tǒng)講述分布式系統(tǒng)與云計(jì)算的教材 ? 2022年 9月 清華大學(xué)出版社出版 ? 歡迎使用，并提寶貴意見(jiàn)！ 課堂作業(yè)和編程習(xí)題 ? 上課時(shí)，請(qǐng)認(rèn)真回答三道關(guān)于云計(jì)算的問(wèn)答題 ? 課后請(qǐng)到 提交Mapreduce的程序。 ? 測(cè)試結(jié)果作為頒發(fā)畢業(yè)證書(shū)和向微軟研究院推薦學(xué)生的重要依據(jù)之一 Cloud puting Why we use cloud puting? Why we use cloud puting? Case 1: Write a file Save Computer down, file is lost Files are always stored in cloud, never lost Why we use cloud puting? Case 2: Use IE download, install, use Use download, install, use Use C++ download, install, use …… Get the serve from the cloud What is cloud and cloud puting? Cloud Demand resources or services over Inter scale and reliability of a data center. What is cloud and cloud puting? Cloud puting is a style of puting in which dynamically scalable and often virtualized resources are provided as a serve over the Inter. Users need not have knowledge of, expertise in, or control over the technology infrastructure in the cloud that supports them. Characteristics of cloud puting ? Virtual. software, databases, Web servers, operating systems, storage and working as virtual servers. ? On demand. add and subtract processors, memory, work bandwidth, storage. IaaS Infrastructure as a Service PaaS Platform as a Service SaaS Software as a Service Types of cloud service Software delivery model ? No hardware or software to manage ? Service delivered through a browser ? Customers use the service on demand ? Instant Scalability SaaS Examples ? Your current CRM package is not managing the load or you simply don’t want to host it inhouse. Use a SaaS provider such as ? Your is hosted on an exchange server in your office and it is very slow. Outsource this using Hosted Exchange. SaaS Platform delivery model ? Platforms are built upon Infrastructure, which is expensive ? Estimating demand is not a science! ? Platform management is not fun! PaaS Examples ? You need to host a large file (5Mb) on your website and make it available for 35,000 users for only two months duration. Use Cloud Front from Amazon. ? You want to start storage services on your work for a large number of files and you do not have the storage capacity…use Amazon S3. PaaS Computer infrastructure delivery model ? A platform virtualization environment ? Computing resources, such as storing and processing capacity. ? Virtualization taken a step further IaaS Examples ? You want to run a batch job but you don’t have the infrastructure necessary to run it in a timely manner. Use Amazon EC2. ? You want to host a website, Use Google App Engine. IaaS Cloud puting and other puting techniques The 21st Century Vision Of Computing Sun Microsystems cofounder Bill Joy The 21st Century Vision Of Computing Definitions Cloud Grid Cluster utility Definitions Cloud Grid Cluster utility Utility puting is the packaging of puting resources, such as putation and storage, as a metered service similar to a traditional public utility Definitions Cloud Grid Cluster utility A puter cluster is a group of linked puters, working together closely so that in many respects they form a single puter. Definitions Cloud Grid Cluster utility Grid puting is the application of several puters to a single problem at the same time — usually to a scientific or technical problem that requires a great number of puter processing cycles or access to large amounts of data Definitions Cloud Grid Cluster utility Cloud puting is a style of puting in which dynamically scalable and often virtualized resources are provided as a service over the Inter. Grid Computing amp。 Cloud Computing ? share a lot monality intention, architecture and technology ? Difference programming model, business model, pute model, applications, and Virtualization. Grid Computing amp。 Cloud Computing ? the problems are mostly the same ? manage large facilities。 ? define methods by which consumers discover, request and use resources provided by the central facilities。 ? implement the often highly parallel putations that execute on those resources. Grid Computing amp。 Cloud Computing ? Virtualization ? Grid ? do not rely on virtualization as much as Clouds do, each individual anization maintain full control of their resources ? Cloud ? an indispensable ingredient for almost every Cloud 2022/5/24 34 歡迎提問(wèn)和討論！ 主要內(nèi)容 35 ? 云計(jì)算概述 ? Google 云計(jì)算技術(shù)： GFS， Bigtable 和Mapreduce ?開(kāi)源平臺(tái) Hadoop介紹 ?云計(jì)算理論 ?事務(wù)處理理論 ? DataLog理論 The Google File System The Google File System (GFS) A scalable distributed file system for large distributed data intensive applications Multiple GFS clusters are currently deployed. The largest ones have: 1000+ storage nodes 300+ TeraBytes of disk storage heavily accessed by hundreds of clients on distinct machines Introduction Shares many same goals as previous distributed file systems performance, scalability, reliability, etc GFS design has been driven by four key observation of Google application workloads and technological environment Intro: Observations 1 ?1. Component failures are the norm constant monitoring, error detection, fault tolerance and automatic recovery are integral to the system