【正文】 I 貴 州 民 族 大 學(xué) 本 科 畢 業(yè) 生 論 文 基于 LUCENE的搜索引擎的研究與實(shí)現(xiàn) 張三 摘要 : 在 Inter蓬勃發(fā)展的當(dāng)下， Inter 上面的信息 是廣 如大海。人們在享受 Inter所帶來的便利的同時(shí)，也面臨著一個(gè)如何在此浩瀚的內(nèi)容中精確，快捷地找到自己所需要的信息，由此Inter搜索引擎應(yīng)運(yùn)而生，而對搜索引擎的研究也理所當(dāng)然的成為了 Inter技術(shù)中最熱門的領(lǐng)域。 本文對搜索引擎的原理 組成 數(shù)據(jù)結(jié)構(gòu) 工作流程等方面做了深入細(xì)致地研究與分析。搜索引擎的未來發(fā)展方向是個(gè) 性化 智能化的。對于不同類型的用戶群搜索相同的內(nèi)容將得到不同的更適合用戶的搜索結(jié)果，這就是搜索引擎的個(gè)性化：而智能化則指搜索引擎具有自我學(xué)習(xí)的功能，能自動(dòng)地適應(yīng)用戶的查詢需求，并能對用戶進(jìn)行智能分類從而為搜索引擎的個(gè)性化提供依據(jù)。本文詳細(xì)地介紹了 LUCENE的特點(diǎn) 系統(tǒng)結(jié)構(gòu)及索引機(jī)制。 LUCENE是 Apache軟件基金會(huì)下的一個(gè)開源項(xiàng)目，完全支持 JAVA技術(shù)，適用于需要全文搜索能力的應(yīng)用程序，并具有很好的跨平臺能力。 在理論研究的基礎(chǔ)上，本文最后利用 JAVA技術(shù)實(shí)現(xiàn)了一個(gè)新聞搜索引擎系統(tǒng)。該搜索引擎系統(tǒng) 的網(wǎng)絡(luò)蜘蛛部分采用了非遞歸爬行方式和 JAVA的多線程機(jī)制，實(shí)現(xiàn)對網(wǎng)頁的抓取。而索引和搜索部分借助 LUCENE全文搜索引擎庫中的 JAVA類庫進(jìn)行實(shí)現(xiàn)。最后采用 JSP(Java server pages)技術(shù)設(shè)計(jì) 一個(gè)簡單的新聞搜索引擎客戶端 關(guān)鍵字 :搜索引擎、 LUCENE ； 、 網(wǎng)絡(luò)蜘蛛 、 JAVA 、 JSP、 HTML II Research and Implementation of Web Search Engine Based on LUCENE CHENXIANGGAO Abstract： The contents on the Web are increasing exponentially as the rapid development of the Inter. A problem how to obtain the useful information from vast contents quickly and accurately is facing us while people are enjoying the convenience of the Inter .the solver of this problem is Web search engine. Nowadays, search engine is one of the hottest topics in the field of Inter technology. The future is contentoriented .People browse the Inter through the search results of search engines. Firstly this paper does an indepth research and analysis on the theory, framework, and data structure of a Web search engine .Meanwhile this paper discusses the future developing trend of search engine which includes individuation and intelligence . Different types of users search the same content will get different search results that are more suitable for users and this is individuation of search engine 。 and intelligence of search engine refers to that search engine has selflearning function , can automatically adapt to user is query needs and classified users intelligently so as to provide foundation for its intelligence . then this paper describes the characteristics , system structure and indexing mechanism of LUCENE . LUCENE is an open source project of the Apache Software Foundation, which is implemented by Java pletely, is fit for the application required fulltext search capability, and has a good crossplatform capability On the basis of the theory, use the Java technology, this paper implements a news search engine system. In this search engine system , the part of the work spider using a nonbased queue manager to take responsibility for operating the joining , distribution , handling and other operation of URL ink in the process of web crawling using thread pool manager multiple crawling threads , concurrently crawl web pages . The implementation of index and search is with the Java class in the fulltext search engine library of LUCENE. And then, use JSP (Java server Pages) technology to design a simple news search engine clients. Keywords: Search Engine、 LUCENE、 Web Spider、 Java 、 HTML III 目錄 第一章 緒論 ................................................................................................................. 1 研究背景 .............................................................................................................. 1 研究現(xiàn)狀 .............................................................................................................. 1 研究的目及意義 .................................................................................................. 2 本文的組織結(jié)構(gòu) .................................................................................................. 3 第二章 相關(guān)技術(shù)研究 ................................................................................................. 5 搜索引擎 .............................................................................................................. 5 搜索引擎的結(jié)構(gòu) ........................................................................................... 5 搜索引擎的工作原理 ................................................................................... 5 網(wǎng)絡(luò)機(jī)器人 .......................................................................................................... 6 如何解析 HTML ........................................................................................... 6 Spider 程序結(jié)構(gòu) ............................................................................................ 7 如何構(gòu)造 Spider 程序 ................................................................................... 8 網(wǎng)絡(luò)機(jī)器人的代碼分析 ............................................................................. 10 概述 .................................................................................................... 12 簡介 ............................................................................................. 12 的特點(diǎn)和優(yōu)勢 ............................................................................. 13 全文索引與數(shù)據(jù)庫索引的比較 ................................................. 15 第 三 章 基于 LUCENE 的 搜索引擎設(shè)計(jì) ................................................................. 16 索引的建立 ....................................................................................................... 16 索引工具 IndexWriter ............................................................................... 16 LUCENE 索引的建立過程 .........