【正文】 一種按照一定的規(guī)則，自動(dòng)的抓取因特網(wǎng)信息的程序或者腳本。 MultiThreading 成都學(xué)院學(xué)士學(xué)位論文（設(shè)計(jì)） I 目 錄 緒 論 ............................................................................................................................................1 1 概述 ..........................................................................................................................................2 研究目的和意義 .................................................................................................................2 網(wǎng)絡(luò)爬蟲的歷史 .................................................................................................................2 研究現(xiàn)狀和發(fā)展趨勢 ..........................................................................................................3 2 網(wǎng)絡(luò)爬蟲相關(guān)技術(shù) ....................................................................................................................5 URL ....................................................................................................................................5 HTTP 協(xié)議 ..........................................................................................................................5 HTTP 協(xié)議簡介 .............................................................................................................5 HTTP 消息的結(jié)構(gòu) .........................................................................................................6 請求方法 ......................................................................................................................7 HTTP 狀態(tài)代碼 .............................................................................................................8 Java 多線程 .........................................................................................................................8 定義 .............................................................................................................................8 優(yōu)點(diǎn) .............................................................................................................................8 不足 .............................................................................................................................8 Java 多線程編程 ............................................................................................................9 Java 網(wǎng)絡(luò)編程 ................................................................................................................... 11 TCP/IP ........................................................................................................................ 11 網(wǎng)絡(luò)編程步驟 ............................................................................................................ 11 Java 網(wǎng)絡(luò)編程技術(shù) ...................................................................................................... 13 正則表達(dá)式 ...................................................................................................................... 14 正則表達(dá)式特點(diǎn) ......................................................................................................... 14 符號功能 ..................................................................................................................... 15 網(wǎng)頁搜索策略 ................................................................................................................... 15 廣度優(yōu)先搜索 ............................................................................................................. 15 深度優(yōu)先搜索 ............................................................................................................. 15 最佳優(yōu)先搜索 ............................................................................................................. 16 3 需求分析和模型設(shè)計(jì) .............................................................................................................. 17 網(wǎng)絡(luò)爬蟲的定義 ............................................................................................................... 17 網(wǎng)絡(luò)爬蟲的分類 ................................................................................................................ 17 成都學(xué)院學(xué)士學(xué)位論文（設(shè)計(jì)） II 系統(tǒng)需求分析 ................................................................................................................... 17 下載網(wǎng)頁 .................................................................................................................... 18 分解網(wǎng)頁 .................................................................................................................... 18 遍歷網(wǎng)絡(luò) .................................................................................................................... 18 存儲(chǔ)網(wǎng)頁 .................................................................................................................... 18 模型設(shè)計(jì) .......................................................................................................................... 18 數(shù)據(jù)庫結(jié)構(gòu) ...................................................................................................................... 19 技術(shù)選型 .......................................................................................................................... 20 Java 簡介 .................................................................................................................... 20 eclipse 基本介紹 .......................................................................................................... 20 4 網(wǎng)絡(luò)爬蟲模型的總體設(shè)計(jì) ....................................................................................................... 21 網(wǎng)絡(luò)爬蟲總體設(shè)計(jì) ........................................................................................................... 21 URL 處理 .......................................................................................................................... 21 URL 抽取 .................................................................................................................... 21 URL 解析 .............