hadoop安裝過(guò)程word版(編輯修改稿)
2025-09-14 16:27 本頁(yè)面
【文章內(nèi)容簡(jiǎn)介】 0427 12:17:50,901 [myid:] INFO [main:Environment@98] Client environment:=/home/app/20120427 12:17:50,903 [myid:] INFO [main:ZooKeeper@433] Initiating client connection, connectString=:2181 sessionTimeout=30000 watcher=$MyWatcher@4ef5c3a620120427 12:17:50,937 [myid:] INFO [mainSendThread():ClientCnxn$SendThread@933] Opening socket connection to server /:2181Wele to ZooKeeper!20120427 12:17:50,965 [myid:] INFO [mainSendThread(localhost:2181):ZooKeeperSaslClient@125] Client will not SASLauthenticate because the default JAAS configuration section 39。Client39。 could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.JLine support is enabled20120427 12:17:51,006 [myid:] INFO [mainSendThread(localhost:2181):ClientCnxn$SendThread@846] Socket connection established to localhost/:2181, initiating session[zk: :2181(CONNECTING) 0] 20120427 12:17:51,094 [myid:] INFO [mainSendThread(localhost:2181):ClientCnxn$SendThread@1175] Session establishment plete on server localhost/:2181, sessionid = 0x136f1fa35960000, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:nullHive安裝（數(shù)據(jù)庫(kù)）Cd /media/shared/sudo tar xzvf ./ C /home/app/vim ~/.bashrcexport HIVE_HOME={{pwd}}修改環(huán)境變量export PATH=${JAVA_HOME}bin:${HIVE_HOME}bin:$PATHvim export HADOOP_HOME=/home/app/cp 設(shè)置主要環(huán)境變量（手動(dòng)）export HADOOP_HOME=/home/app/(你自己的hadoop安裝路徑) 在hdfs建立目錄用來(lái)保存hive數(shù)據(jù)$ $HADOOP_HOME/bin/hadoop fs mkdir /tmp $ $HADOOP_HOME/bin/hadoop fs mkdir /user/hive/warehouse $ $HADOOP_HOME/bin/hadoop fs chmod g+w /tmp $ $HADOOP_HOME/bin/hadoop fs chmod g+w /user/hive/warehouse 運(yùn)行運(yùn)行 bin/hive 即可hive SET =localhost:50030。 hive SET v。hive show tables。hive create table log_stat(ip STRING,time STRING,_request STRING,uri STRING, STRING,status int,code STRING)。(此表創(chuàng)建時(shí)沒(méi)有指定數(shù)據(jù)分隔方式和行分隔方式,下面將刪除它重新建)OKTime taken: secondshive drop table log_stat。OKTime taken: secondshive create table log_stat(ip STRING,time STRING,_request STRING,uri STRING, STRING,status int,code STRING) row format delimited fields terminated by 39。\t39。 lines terminated by 39。\n39。 stored as textfile。(指定數(shù)據(jù)分隔方式’\t’,行分隔’\n’,以文本存儲(chǔ))hive load data local inpath 39。/home/app/39。 overwrite into table log_stat。(如果數(shù)據(jù)在hdfs上,則不需要Local關(guān)鍵字)hive dfs ls /user/hive/warehouse。Found 1 itemsdrwxrxrx root supergroup 0 20120621 10:19 /user/hive/warehouse/log_stathive dfs ls /user/hive/warehouse/log_stat。Found 1 itemsrwrr 1 root supergroup 1593 20120621 10:19 /user/hive/warehouse/log_stat/partm00000hive發(fā)現(xiàn)hive中select count(*) 會(huì)花費(fèi)很多時(shí)間而且執(zhí)行mapreducehive insert overwrite local directory 39。/media/shared/reg_339。 select a.* from log_stat a。(將表數(shù)據(jù)導(dǎo)入本地文件)hive load data inpath 39。/user/root/load_result/partm0000039。 overwrite into table tt。（hdfs文件導(dǎo)入表）load data inpath 39。hdfs://localhost:9000/user/root/outlog/click_mini/2012062839。 into table click_mini。hive set =true。hiveset 。外部表創(chuàng)建桶不會(huì)拆分目錄hive select * from ad_3rd tablesample(bucket 3 out of 3 on rand())。select count(*) from ad_3rd tablesample(bucket 2 out of 3 on access_date)。這句執(zhí)行有問(wèn)題select count(*) from ad_3rd tablesample(bucket 1 out of 3 on access_date)。這句可以安裝mysqlaptget install root@zhuweiVirtualBox:/etc/mysql service mysql statusroot@zhuweiVirtualBox:
法律信息相關(guān)推薦
鄂ICP備17016276號(hào)-1