实时索引管理类IndexManager
最后更新于：2022-04-01 19:48:39
转载请注明出处：[http://blog.csdn.net/xiaojimanman/article/details/44015983](http://blog.csdn.net/xiaojimanman/article/details/44015983)
[http://www.llwjy.com/blogdetail/5757ce8c007754704b563dd6a47ca1ca.html](http://www.llwjy.com/blogdetail/5757ce8c007754704b563dd6a47ca1ca.html)
个人的博客小站也搭建成功，网址：[www.llwjy.com](http://www.llwjy.com) ，欢迎大家来吐槽~
在前一篇博客中，对实时索引的实现原理做了一些简单的介绍，这里就介绍下，如何利用Lucene来实现索引的管理（Lucene中已经实现了大部分的功能，我们只需要对其再次封装即可）。
**逐个击破**
在Lucene4.3.1中，实现实时索引时，需要将IndexWrite的相关操作委托给TrackingIndexWriter来处理，具体代码实现如下：
ps:关于如何创建索引这里就不再介绍了，可以参照之前的博客或者该博客后面的完整代码。
~~~
this.trackingIndexWriter = new TrackingIndexWriter(this.indexWriter);
~~~
同时初始化索引管理对象，代码如下：
~~~
this.nrtManager = new NRTManager(this.trackingIndexWriter, new SearcherFactory());
~~~
到这里还需要开启两个守护线程：内存索引重读线程和内存数据commit线程。内存索引重读线程执行的频率也就是实时索引的时差，由于内存中的数据不会太多，所以这个延时一般也就是在十几毫秒左右；内存数据commit线程是将内存中的数据写到磁盘上，不至于数据丢失，如果研究过Lucene源码的童鞋也许会发现，即使你不执行commit操作，到内存中的数据达到一定的程度，也会将一部分数据写到磁盘上，只不过重启服务这部分数据就丢失了同时还会造成一系列的问题，[http://bbs.csdn.net/topics/390677902](http://bbs.csdn.net/topics/390677902) 这个地址下就是commit线程死掉之后造成的一系列问题，感兴趣的童鞋可以了解下。
内存重读线程我们只需要配置下参数启动即可，代码如下：
~~~
this.nrtManagerReopenThread = new NRTManagerReopenThread(this.nrtManager, indexReopenMaxStaleSec, indexReopenMinStaleSec);
this.nrtManagerReopenThread.setName("NRTManager Reopen Thread");
this.nrtManagerReopenThread.setPriority(Math.min(Thread.currentThread().getPriority()+2, Thread.MAX_PRIORITY));
this.nrtManagerReopenThread.setDaemon(true);
this.nrtManagerReopenThread.start();
~~~
内存数据commit线程需要自己写代码实现，然后启动该线程即可，代码如下：
~~~
private class IndexCommitThread extends Thread{
private boolean flag;
public IndexCommitThread(String name){
super(name);
}
@SuppressWarnings("deprecation")
public void run(){
flag = true;
while(flag) {
try {
indexWriter.commit();
if (bprint) {
System.out.println(new Date().toLocaleString() + "\t" + IndexManagerName + "\tcommit");
}
TimeUnit.SECONDS.sleep(indexCommitSeconds);
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
}
}
~~~
~~~
this.indexCommitThread = new IndexCommitThread(IndexManagerName + "Index Commit Thread");
this.indexCommitThread.setDaemon(true);
this.indexCommitThread.start();
~~~
那又如何像普通的索引那样使用IndexSearcher呢？当然NrtManager类也提供了相关的方法，可以获取最新可用的IndexSearcher，代码如下：
~~~
public IndexSearcher getIndexSearcher(){
try {
return this.nrtManager.acquire();
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
~~~
当然在使用之后别忘记释放，代码如下：
~~~
public void release(IndexSearcher searcher){
try {
nrtManager.release(searcher);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
~~~
**另类单例模式**
在之前的博客中，我也多次提到，加载索引是一个相当消耗资源的事情，所以我们不可能每一次索引操作都加载一次索引，所以我们就必须使用单例模式来实现IndexManager类。这里的单例模式又和我们常见的单例模式有所区别，普通的单例模式该类只有一个对象，这里的单例模式是该类有多个对象，下面就简单的介绍下此处另类的单例模式。
通过前一篇博客最后，你也许会注意到，系统中关于索引的配置信息是存在HashSet对象中，这也就是说这里IndexManager类会实例化多少次取决于HashSet对象，也就是你配置文件让他实例化多少次就会实例化多少次。既然这样，怎么还能叫单例模式呢？这里的单例是索引的单例，也就是说一个索引只有一个IndexManager对象，不会存在两个IndexManager对象去操作同一个索引的情况。具体代码实现如下：
~~~
/**
* Initialization on Demand Holder式初始化IndexManager
*/
private static class LazyLoadIndexManager {
private static final HashMap indexManager = new HashMap();
static {
for (ConfigBean configBean : IndexConfig.getConfigBean()) {
indexManager.put(configBean.getIndexName(), new IndexManager(configBean));
}
}
}
/**
*@Description: IndexManager私有构造方法
*@Author: lulei
*@Version: 1.1.0
*/
private IndexManager(ConfigBean configBean){
//...
}
public static IndexManager getIndexManager(String indexName){
return LazyLoadIndexManager.indexManager.get(indexName);
}
~~~
这样我们就可以通过索引名获取到该索引的IndexManager对象。
**庐山真面目**
说了这么多，下面就把IndexManager的源码附在最后，感兴趣的童鞋可以试试（里面还有一些其他的方法，相信不用介绍也都可以看的懂）
~~~
/**
*@Description: 索引管理类
*/
package com.lulei.lucene.index.manager;
import java.io.File;
import java.io.IOException;
import java.util.Date;
import java.util.HashMap;
import java.util.concurrent.TimeUnit;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.IndexWriterConfig.OpenMode;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.NRTManager;
import org.apache.lucene.search.NRTManager.TrackingIndexWriter;
import org.apache.lucene.search.NRTManagerReopenThread;
import org.apache.lucene.search.SearcherFactory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.NIOFSDirectory;
import org.apache.lucene.util.Version;
import com.lulei.lucene.index.model.ConfigBean;
import com.lulei.lucene.index.model.IndexConfig;
public class IndexManager {
private IndexWriter indexWriter;
//更新索引文件的IndexWriter
private TrackingIndexWriter trackingIndexWriter;
//索引文件采用的分词器
private Analyzer analyzer;
//索引管理对象
private NRTManager nrtManager;
//索引重读线程
private NRTManagerReopenThread nrtManagerReopenThread;
//索引写入磁盘线程
private IndexCommitThread indexCommitThread;
//索引地址
private String indexPath;
//索引重读最大、最小时间间隔
private double indexReopenMaxStaleSec;
private double indexReopenMinStaleSec;
//索引commit时间
private int indexCommitSeconds;
//索引名
private String IndexManagerName;
//commit时是否输出相关信息
private boolean bprint = true;
/**
* Initialization on Demand Holder式初始化IndexManager
*/
private static class LazyLoadIndexManager {
private static final HashMap indexManager = new HashMap();
static {
for (ConfigBean configBean : IndexConfig.getConfigBean()) {
indexManager.put(configBean.getIndexName(), new IndexManager(configBean));
}
}
}
/**
*@Description: IndexManager私有构造方法
*@Author: lulei
*@Version: 1.1.0
*/
private IndexManager(ConfigBean configBean){
//设置相关属性
analyzer = configBean.getAnalyzer();
indexPath = configBean.getIndexPath();
IndexManagerName = configBean.getIndexName();
indexReopenMaxStaleSec = configBean.getIndexReopenMaxStaleSec();
indexReopenMinStaleSec = configBean.getIndexReopenMinStaleSec();
indexCommitSeconds = configBean.getIndexCommitSeconds();
bprint = configBean.isBprint();
String indexFile = indexPath + IndexManagerName + "/";
//创建或打开索引
IndexWriterConfig indexWriterConfig = new IndexWriterConfig(Version.LUCENE_43, analyzer);
indexWriterConfig.setOpenMode(OpenMode.CREATE_OR_APPEND);
Directory directory = null;
try {
directory = NIOFSDirectory.open(new File(indexFile));
if (IndexWriter.isLocked(directory)){
IndexWriter.unlock(directory);
}
this.indexWriter = new IndexWriter(directory, indexWriterConfig);
this.trackingIndexWriter = new TrackingIndexWriter(this.indexWriter);
this.nrtManager = new NRTManager(this.trackingIndexWriter, new SearcherFactory());
} catch(IOException e){
e.printStackTrace();
}
//开启守护进程
this.setThread();
}
/**
* @Author: lulei
* @Description: 创建索引管理线程
*/
private void setThread(){
this.nrtManagerReopenThread = new NRTManagerReopenThread(this.nrtManager, indexReopenMaxStaleSec, indexReopenMinStaleSec);
this.nrtManagerReopenThread.setName("NRTManager Reopen Thread");
this.nrtManagerReopenThread.setPriority(Math.min(Thread.currentThread().getPriority()+2, Thread.MAX_PRIORITY));
this.nrtManagerReopenThread.setDaemon(true);
this.nrtManagerReopenThread.start();
this.indexCommitThread = new IndexCommitThread(IndexManagerName + "Index Commit Thread");
this.indexCommitThread.setDaemon(true);
this.indexCommitThread.start();
}
/**
* @return
* @Author:lulei
* @Description: 重启索引commit线程
*/
public String setCommitThread() {
try {
if (this.indexCommitThread.isAlive()){
return "is alive";
}
this.indexCommitThread = new IndexCommitThread(IndexManagerName + "Index Commit Thread");
this.indexCommitThread.setDaemon(true);
this.indexCommitThread.start();
} catch (Exception e) {
e.printStackTrace();
return "failed";
}
return "reload";
}
/**
*@Description: 索引commit线程
*@Author: lulei
*@Version: 1.1.0
*/
private class IndexCommitThread extends Thread{
private boolean flag;
public IndexCommitThread(String name){
super(name);
}
@SuppressWarnings("deprecation")
public void run(){
flag = true;
while(flag) {
try {
indexWriter.commit();
if (bprint) {
System.out.println(new Date().toLocaleString() + "\t" + IndexManagerName + "\tcommit");
}
TimeUnit.SECONDS.sleep(indexCommitSeconds);
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
}
}
/**
* @return IndexManager
* @Author: lulei
* @Description: 获取索引管理类
*/
public static IndexManager getIndexManager(String indexName){
return LazyLoadIndexManager.indexManager.get(indexName);
}
/**
* @@Description:释放IndexSearcher资源
* @param searcher
*/
public void release(IndexSearcher searcher){
try {
nrtManager.release(searcher);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
/**
* @return IndexSearcher
* @Author: lulei
* @Description: 返回IndexSearcher对象，使用完之后，调用release方法进行释放
*/
public IndexSearcher getIndexSearcher(){
try {
return this.nrtManager.acquire();
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
public NRTManager getNRTManager(){
return this.nrtManager;
}
public IndexWriter getIndexWriter(){
return this.indexWriter;
}
public TrackingIndexWriter getTrackingIndexWriter(){
return this.trackingIndexWriter;
}
public Analyzer getAnalyzer(){
return analyzer;
}
/**
* @return
* @Author: lulei
* @Description: 获取索引中的记录条数
*/
public int getIndexNum(){
return indexWriter.numDocs();
}
}
~~~
ps:最近发现其他网站可能会对博客转载，上面并没有源链接，如想查看更多关于 [基于lucene的案例开发](http://blog.csdn.net/xiaojimanman/article/category/2841877) 请[点击这里](http://blog.csdn.net/xiaojimanman/article/category/2841877)。或访问网址http://blog.csdn.net/xiaojimanman/article/category/2841877 或 [www.llwjy.com](http://www.llwjy.com)
';