【正文】 demand ? Instant Scalability SaaS Examples ? Your current CRM package is not managing the load or you simply don’t want to host it inhouse. Use a SaaS provider such as ? Your is hosted on an exchange server in your office and it is very slow. Outsource this using Hosted Exchange. SaaS Platform delivery model ? Platforms are built upon Infrastructure, which is expensive ? Estimating demand is not a science! ? Platform management is not fun! PaaS Examples ? You need to host a large file (5Mb) on your website and make it available for 35,000 users for only two months duration. Use Cloud Front from Amazon. ? You want to start storage services on your work for a large number of files and you do not have the storage capacity…use Amazon S3. PaaS Computer infrastructure delivery model ? A platform virtualization environment ? Computing resources, such as storing and processing capacity. ? Virtualization taken a step further IaaS Examples ? You want to run a batch job but you don’t have the infrastructure necessary to run it in a timely manner. Use Amazon EC2. ? You want to host a website, Use Google App Engine. IaaS Cloud puting and other puting techniques The 21st Century Vision Of Computing Sun Microsystems cofounder Bill Joy The 21st Century Vision Of Computing Definitions Cloud Grid Cluster utility Definitions Cloud Grid Cluster utility Utility puting is the packaging of puting resources, such as putation and storage, as a metered service similar to a traditional public utility Definitions Cloud Grid Cluster utility A puter cluster is a group of linked puters, working together closely so that in many respects they form a single puter. Definitions Cloud Grid Cluster utility Grid puting is the application of several puters to a single problem at the same time — usually to a scientific or technical problem that requires a great number of puter processing cycles or access to large amounts of data Definitions Cloud Grid Cluster utility Cloud puting is a style of puting in which dynamically scalable and often virtualized resources are provided as a service over the Inter. Grid Computing amp。 Cloud Computing ? the problems are mostly the same ? manage large facilities。 ? implement the often highly parallel putations that execute on those resources. Grid Computing amp。 GFS chooses and returns the offset it writes to and appends the data to each replica at least once Heavily used by Google’s Distributed applications. No need for a distributed lock manager GFS choses the offset, not the client Atomic Record Append: How? ? Follows similar control flow as mutations ? Primary tells secondary replicas to append at the same offset as the primary ? If a replica append fails at any replica, it is retried by the client. So replicas of the same chunk may contain different data, including duplicates, whole or in part, of the same record Atomic Record Append: How? ? GFS does not guarantee that all replicas are bitwise identical. Only guarantees that data is written at least once in an atomic unit. Data must be written at the same offset for all chunk replicas for success to be reported. Detecting Stale Replicas ? Master has a chunk version number to distinguish up to date and stale replicas ? Increase version when granting a lease ? If a replica is not available, its version is not increased ? master detects stale replicas when a chunkservers report chunks and versions ? Remove stale replicas during garbage collection Garbage collection When a client deletes a file, master logs it like other changes and changes filename to a hidden file. Master removes files hidden for longer than 3 days when scanning file system name space metadata is also erased During HeartBeat messages, the chunkservers send the master a subset of its chunks, and the master tells it which files have no metadata. Chunkserver removes these files on its own Fault Tolerance: High Availability ? Fast recovery Master and chunkservers can restart in seconds ? Chunk Replication ? Master Replication “shadow” masters provide readonly access when primary master is down mutations not done until recorded on all master replicas Fault Tolerance: Data Integrity Chunkservers use checksums to detect corrupt data Since replicas are not bitwise identical, chunkservers maintain their own checksums For reads, chunkserver verifies checksum before sending chunk Update checksums during writes Introduction to MapReduce MapReduce: Insight ?‖Consider the problem of counting the number of occurrences of each word in a large collection of documents‖ ? How would you do it in parallel ? MapReduce Programming Model ?Inspired from map and reduce operations monly used in functional programming languages like Lisp. ?Users implement interface of two primary methods: ?1. Map: (key1, val1) → (key2, val2) ?2. Reduce: (key2, [val2]) → [val3] Map operation ? Map, a pure function, written by the user, takes an input key/value pair and produces a set of intermediate key/value pairs. ?. (doc—id, doccontent) ? Draw an analogy to SQL, map can be visualized as groupby clause of an aggregate query. Reduce operation ? On pletion of map phase, all the intermediate values for a given output key are bined together into a list and given to a reducer. ? Can be visualized as aggregate function (., average) that is puted over all the rows with the same groupby attribute. Pseudocode map(String input_key, String input