【正文】 website, but only for a few days. Use Flexiscale. IaaS Cloud puting and other puting techniques The 21st Century Vision Of Computing Leonard Kleinrock , one of the chief scientists of the original Advanced Research Projects Agency Network (ARPANET) project which seeded the Inter, said: ― As of now, puter works are still in their infancy, but as they grow up and bee sophisticated, we will probably see the spread of ?puter utilities’ which, like present electric and telephone utilities, will service individual homes and offices across the country.‖ The 21st Century Vision Of Computing Sun Microsystems cofounder Bill Joy He also indicated ―It would take time until these markets to mature to generate this kind of value. Predicting now which panies will capture the value is impossible. Many of them have not even been created yet.‖ The 21st Century Vision Of Computing Definitions Cloud Grid Cluster utility Definitions Cloud Grid Cluster utility Utility puting is the packaging of puting resources, such as putation and storage, as a metered service similar to a traditional public utility Definitions Cloud Grid Cluster utility A puter cluster is a group of linked puters, working together closely so that in many respects they form a single puter. Definitions Cloud Grid Cluster utility Grid puting is the application of several puters to a single problem at the same time — usually to a scientific or technical problem that requires a great number of puter processing cycles or access to large amounts of data Definitions Cloud Grid Cluster utility Cloud puting is a style of puting in which dynamically scalable and often virtualized resources are provided as a service over the Inter. Grid Computing amp。 Cloud Computing ? share a lot monality intention, architecture and technology ? Difference programming model, business model, pute model, applications, and Virtualization. Grid Computing amp。 ? define methods by which consumers discover, request and use resources provided by the central facilities。 Cloud Computing ? Virtualization ? Grid ? do not rely on virtualization as much as Clouds do, each individual anization maintain full control of their resources ? Cloud ? an indispensable ingredient for almost every Cloud 2022/4/12 36 Any question and any ments ? 主要內(nèi)容 37 ? 云計算概述 ? Google 云計算技術(shù)： GFS， Bigtable 和Mapreduce ? Yahoo云計算技術(shù)和 Hadoop ?云數(shù)據(jù)管理的挑戰(zhàn) Google Cloud puting techniques The Google File System The Google File System (GFS) A scalable distributed file system for large distributed data intensive applications Multiple GFS clusters are currently deployed. The largest ones have: 1000+ storage nodes 300+ TeraBytes of disk storage heavily accessed by hundreds of clients on distinct machines Introduction Shares many same goals as previous distributed file systems performance, scalability, reliability, etc GFS design has been driven by four key observation of Google application workloads and technological environment Intro: Observations 1 ?1. Component failures are the norm constant monitoring, error detection, fault tolerance and automatic recovery are integral to the system ?2. Huge files (by traditional standards) Multi GB files are mon I/O operations and blocks sizes must be revisited Intro: Observations 2 ?3. Most files are mutated by appending new data This is the focus of performance optimization and atomicity guarantees ?4. Codesigning the applications and APIs benefits overall system by increasing flexibility The Design Cluster consists of a single master and multiple chunkservers and is accessed by multiple clients The Master Maintains all file system metadata. names space, access control info, file to chunk mappings, chunk (including replicas) location, etc. Periodically municates with chunkservers in HeartBeat messages to give instructions and check state The Master Helps make sophisticated chunk placement and replication decision, using global knowledge For reading and writing, client contacts Master to get chunk locations, then deals directly with chunkservers Master is not a bottleneck for reads/writes Chunkservers Files are broken into chunks. Each chunk has a immutable globally unique 64bit chunkhandle. handle is assigned by the master at chunk creation Chunk size is 64 MB Each chunk is replicated on 3 (default) servers Clients Linked to apps using the file system API. Communicates with master and chunkservers for reading and writing Master interactions only for metadata Chunkserver interactions for data Only caches metadata information Data is too large to cache. Chunk Locations Master does not keep a persistent record of locations of chunks and replicas. Polls chunkservers at startup, and when new chunkservers join/leave for this. Stays up to date by controlling placement of new chunks and through HeartBeat messages (when monitoring chunkservers) Operation Log Record of all critical metadata changes Stored on Master and replicated on other machines Defines order of concurrent operations Also used to recover the file system state System Interactions: Leases and Mutation Order Leases maintain a mutation order across all chunk replicas Master grants a lease to a replica, called the primary The primary choses the serial mutation order, and all replicas follow this order Minimizes management overhead for the Master Atomic Record Append Client specifies the data to write。 reduce(String output_key, Iterator intermediate_values): // output_key: a word // output_values: a list of counts int result = 0。 Emit(AsString(result