【正文】 nternet.大數(shù)據(jù)要求特殊的技術(shù)在可容忍的時間內(nèi)來處理大量的數(shù)據(jù)。s effective capacity to exchange information through telemunication networks was 281 petabytes in 1986, 471 petabytes in 1993, exabytes in 2000, 65 exabytes in 2007[15] and it is predicted that the amount of traffic flowing over the internet will reach 667 exabytes annually by 2013.[5]發(fā)達的經(jīng)濟越來越多地使用數(shù)據(jù)密集型技術(shù)。 The volume of business data worldwide, across all panies, doubles every years, according to estimates.[31]據(jù)估計。 Walmart handles more than 1 million customer transactions every hour, which is imported into databases estimated to contain more than petabytes of data – the equivalent of 167 times the information contained in all the books in the US Library of Congress.[5]Walmart每小時處理100萬個顧客事務(wù)，這些事務(wù)被輸入進數(shù)據(jù)庫，——是美國國會圖書館所有圖書信息量的167倍216。該計劃由跨越6個政府部門的84個大數(shù)據(jù)程序項目所組成。他們比較了每個國家GDP的未來去向指數(shù)，發(fā)現(xiàn)具有較高的GDP國家的用戶使用Google搜索的關(guān)于未來的信息的強烈傾向。216。216。 When the Sloan Digital Sky Survey (SDSS) began collecting astronomical data in 2000, it amassed more in its first few weeks than all data collected in the history of astronomy. Continuing at a rate of about 200 GB per night, SDSS has amassed more than 140 terabytes of information. When the Large Synoptic Survey Telescope, successor to SDSS, es online in 2016 it is anticipated to acquire that amount of data every five days.[5]在Sloan Digital Sky Survey (SDSS)于2000年開始采集天文數(shù)據(jù)時，在最初的幾周內(nèi)它積累了比天文史上收集的所有數(shù)據(jù)還要多的數(shù)據(jù)。Gartner和目前業(yè)界大多數(shù)（人）延續(xù)使用這種“3V“模型來描述大數(shù)據(jù)。大數(shù)據(jù)的尺寸是一個不斷變化的目標(biāo)，截至到2012年在一個單一數(shù)據(jù)集中的數(shù)據(jù)范圍從十?dāng)?shù)TB到數(shù)個PB。Big data is difficult to work with using relational databases and desktop statistics and visualization packages, requiring instead massively parallel software running on tens, hundreds, or even thousands of servers.[17] What is considered big data varies depending on the capabilities of the organization managing the set, and on the capabilities of the applications that are traditionally used to process and analyze the data set in its domain. For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size bees a significant consideration.[18]使用關(guān)系型數(shù)據(jù)庫和桌面統(tǒng)計和可視化軟件包對大數(shù)據(jù)進行處理是困難的，它需要“將大規(guī)模并行軟件運行在數(shù)十臺、數(shù)百臺或甚至數(shù)千臺服務(wù)器（來處理）”。在許多領(lǐng)域科學(xué)家們經(jīng)常遇到大數(shù)據(jù)集的限制，這些領(lǐng)域包括氣象學(xué)、基因?qū)W、connectomics、復(fù)雜的物理仿真、以及生物和環(huán)境研究。更大的數(shù)據(jù)集的趨勢是由于從相關(guān)數(shù)據(jù)的單一大數(shù)據(jù)集推導(dǎo)而來的額外信息，與分離的較小的具有相同數(shù)據(jù)總量的數(shù)據(jù)集相比，能夠發(fā)現(xiàn)相關(guān)性來“識別商業(yè)趨勢（spot business trends）、確定研究的質(zhì)量、預(yù)防疾病、法律引用鏈接、打擊犯罪以及實時確定道路交通狀態(tài)”。其挑戰(zhàn)包括采集、管理、存儲、搜索、共享、分析和可視化。[15] as of 2012, every day quintillion (1018) bytes of data were created.[16]截至2012年，數(shù)據(jù)集大小尺寸的限制是exabyte數(shù)量級的數(shù)據(jù)，這種規(guī)模是指以可行的處理方式在合理的時間內(nèi)進行數(shù)據(jù)處理。從80年代起，全球存儲信息人均信息存儲能力在技術(shù)上大致每40個月就翻一番；as of 2012, every day quintillion (1018) bytes of data were created.[16]截至到2012年， quintillion（*10^18）字節(jié)。DefinitionBig data usually includes data sets with sizes beyond the ability of monlyused software tools to capture, curate, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set. With this difficulty, a new platform of big data tools has arisen to handle sensemaking over large quantities of data, as in the Apache Hadoop Big Data Platform.大數(shù)據(jù)通常包括在尺寸上超出常用軟件工具對數(shù)據(jù)在一定的可容忍時間間隔內(nèi)進行采集、管理和處理的能力的數(shù)據(jù)集。In a 2001 research report[19] and related lectures, META Group (now Gartner) analyst Doug Laney defined data growth challenges and opportunities as being threedimensi