Solr5之solrconfig.xml配置详解

来源:互联网 发布:阳煤集团知乎 编辑:程序博客网 时间:2024/06/08 06:26

solrconfig.xml 配置文件中包含了很多 solr 自身配置相关的参数 ,solrconfig.xml 配置文件示例可以从 solr

的解压目录下找到,如图:


用文本编辑软件打开 solrconfig.xml 配置,你将会看到以下配置内容:
<?xml version="1.0" encoding="UTF-8" ?><!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.  See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.  You may obtain a copy of the License at     http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.--><!--      For more details about configurations options that may appear in     this file, see http://wiki.apache.org/solr/SolrConfigXml. --><config>  <!-- In all configuration below, a prefix of "solr." for class names       is an alias that causes solr to search appropriate packages,       including org.apache.solr.(search|update|request|core|analysis)       You may also specify a fully qualified Java classname if you       have your own custom plugins.    -->  <!-- Controls what version of Lucene various components of Solr       adhere to.  Generally, you want to use the latest version to       get all bug fixes and improvements. It is highly recommended       that you fully re-index after changing this setting as it can       affect both how text is indexed and queried.  -->  <luceneMatchVersion>5.1.0</luceneMatchVersion>  <!-- Data Directory       Used to specify an alternate directory to hold all index data       other than the default ./data under the Solr home.  If       replication is in use, this should match the replication       configuration.    -->  <!--  <dataDir>${solr.data.dir:}</dataDir>  -->  <dataDir>C:\solr_home\core1\data</dataDir>  <!-- The DirectoryFactory to use for indexes.       solr.StandardDirectoryFactory is filesystem       based and tries to pick the best implementation for the current       JVM and platform.  solr.NRTCachingDirectoryFactory, the default,       wraps solr.StandardDirectoryFactory and caches small files in memory       for better NRT performance.       One can force a particular implementation via solr.MMapDirectoryFactory,       solr.NIOFSDirectoryFactory, or solr.SimpleFSDirectoryFactory.       solr.RAMDirectoryFactory is memory based, not       persistent, and doesn't work with replication.    -->  <directoryFactory name="DirectoryFactory"       class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}">  </directoryFactory>   <!-- The CodecFactory for defining the format of the inverted index.       The default implementation is SchemaCodecFactory, which is the official Lucene       index format, but hooks into the schema to provide per-field customization of       the postings lists and per-document values in the fieldType element       (postingsFormat/docValuesFormat). Note that most of the alternative implementations       are experimental, so if you choose to customize the index format, it's a good       idea to convert back to the official format e.g. via IndexWriter.addIndexes(IndexReader)       before upgrading to a newer version to avoid unnecessary reindexing.  -->  <codecFactory class="solr.SchemaCodecFactory"/>  <schemaFactory class="ClassicIndexSchemaFactory"/>  <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~       Index Config - These settings control low-level behavior of indexing       Most example settings here show the default value, but are commented       out, to more easily see where customizations have been made.       Note: This replaces <indexDefaults> and <mainIndex> from older versions       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->  <indexConfig>    <!-- LockFactory   This option specifies which Lucene LockFactory implementation  to use.  single = SingleInstanceLockFactory - suggested for a    read-only index or when there is no possibility of    another process trying to modify the index.  native = NativeFSLockFactory - uses OS native file locking.    Do not use when multiple solr webapps in the same    JVM are attempting to share a single index.  simple = SimpleFSLockFactory  - uses a plain file for locking  Defaults: 'native' is default for Solr3.6 and later, otherwise     'simple' is the default  More details on the nuances of each LockFactory...  http://wiki.apache.org/lucene-java/AvailableLockFactories    -->    <lockType>${solr.lock.type:native}</lockType>    <!-- Lucene Infostream  To aid in advanced debugging, Lucene provides an "InfoStream"  of detailed information when indexing.  Setting the value to true will instruct the underlying Lucene  IndexWriter to write its info stream to solr's log. By default,  this is enabled here, and controlled through log4j.properties.      -->     <infoStream>true</infoStream>  </indexConfig>  <!-- JMX       This example enables JMX if and only if an existing MBeanServer       is found, use this if you want to configure JMX through JVM       parameters. Remove this to disable exposing Solr configuration       and statistics to JMX.       For more details see http://wiki.apache.org/solr/SolrJmx    -->  <jmx />  <!-- If you want to connect to a particular server, specify the       agentId     -->  <!-- <jmx agentId="myAgent" /> -->  <!-- If you want to start a new MBeanServer, specify the serviceUrl -->  <!-- <jmx serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>    -->  <!-- The default high-performance update handler -->  <updateHandler class="solr.DirectUpdateHandler2">    <!-- Enables a transaction log, used for real-time get, durability, and  and solr cloud replica recovery.  The log can grow as big as  uncommitted changes to the index, so use of a hard autoCommit  is recommended (see below).  "dir" - the target directory for transaction logs, defaults to the         solr data directory.  -->     <updateLog>      <str name="dir">${solr.ulog.dir:}</str>    </updateLog>    <!-- AutoCommit  Perform a hard commit automatically under certain conditions.  Instead of enabling autoCommit, consider using "commitWithin"  when adding documents.   http://wiki.apache.org/solr/UpdateXmlMessages  maxDocs - Maximum number of documents to add since the last     commit before automatically triggering a new commit.  maxTime - Maximum amount of time in ms that is allowed to pass     since a document was added before automatically     triggering a new commit.   openSearcher - if false, the commit causes recent index changes    to be flushed to stable storage, but does not cause a new    searcher to be opened to make those changes visible.  If the updateLog is enabled, then it's highly recommended to  have some sort of hard autoCommit to limit the log size.      -->     <autoCommit>        <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>        <openSearcher>false</openSearcher>      </autoCommit>    <!-- softAutoCommit is like autoCommit except it causes a  'soft' commit which only ensures that changes are visible  but does not ensure that data is synced to disk.  This is  faster and more near-realtime friendly than a hard commit.      -->     <autoSoftCommit>        <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>      </autoSoftCommit>  </updateHandler>  <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~       Query section - these settings control query time things like caches       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->  <query>    <!-- Max Boolean Clauses  Maximum number of clauses in each BooleanQuery,  an exception  is thrown if exceeded.  ** WARNING **  This option actually modifies a global Lucene property that  will affect all SolrCores.  If multiple solrconfig.xml files  disagree on this property, the value at any given moment will  be based on the last SolrCore to be initialized.      -->    <maxBooleanClauses>1024</maxBooleanClauses>    <!-- Solr Internal Query Caches  There are two implementations of cache available for Solr,  LRUCache, based on a synchronized LinkedHashMap, and  FastLRUCache, based on a ConcurrentHashMap.    FastLRUCache has faster gets and slower puts in single  threaded operation and thus is generally faster than LRUCache  when the hit ratio of the cache is high (> 75%), and may be  faster under other scenarios on multi-cpu systems.    -->    <!-- Filter Cache  Cache used by SolrIndexSearcher for filters (DocSets),  unordered sets of *all* documents that match a query.  When a  new searcher is opened, its caches may be prepopulated or  "autowarmed" using data from caches in the old searcher.  autowarmCount is the number of items to prepopulate.  For  LRUCache, the autowarmed items will be the most recently  accessed items.  Parameters:    class - the SolrCache implementation LRUCache or        (LRUCache or FastLRUCache)    size - the maximum number of entries in the cache    initialSize - the initial capacity (number of entries) of        the cache.  (see java.util.HashMap)    autowarmCount - the number of entries to prepopulate from        and old cache.        -->    <filterCache class="solr.FastLRUCache"          size="512"          initialSize="512"          autowarmCount="0"/>    <!-- Query Result Cache  Caches results of searches - ordered lists of document ids  (DocList) based on a query, a sort, and the range of documents requested.        -->    <queryResultCache class="solr.LRUCache"       size="512"       initialSize="512"       autowarmCount="0"/>    <!-- Document Cache  Caches Lucene Document objects (the stored fields for each  document).  Since Lucene internal document ids are transient,  this cache will not be autowarmed.        -->    <documentCache class="solr.LRUCache"     size="512"     initialSize="512"     autowarmCount="0"/>    <!-- custom cache currently used by block join -->     <cache name="perSegFilter"      class="solr.search.LRUCache"      size="10"      initialSize="0"      autowarmCount="10"      regenerator="solr.NoOpRegenerator" />    <!-- Lazy Field Loading  If true, stored fields that are not requested will be loaded  lazily.  This can result in a significant speed improvement  if the usual case is to not load all stored fields,  especially if the skipped fields are large compressed text  fields.    -->    <enableLazyFieldLoading>true</enableLazyFieldLoading>   <!-- Result Window Size        An optimization for use with the queryResultCache.  When a search        is requested, a superset of the requested number of document ids        are collected.  For example, if a search for a particular query        requests matching documents 10 through 19, and queryWindowSize is 50,        then documents 0 through 49 will be collected and cached.  Any further        requests in that range can be satisfied via the cache.       -->   <queryResultWindowSize>20</queryResultWindowSize>   <!-- Maximum number of documents to cache for any entry in the        queryResultCache.      -->   <queryResultMaxDocsCached>200</queryResultMaxDocsCached>    <!-- Use Cold Searcher  If a search request comes in and there is no current  registered searcher, then immediately register the still  warming searcher and use it.  If "false" then all requests  will block until the first searcher is done warming.      -->    <useColdSearcher>false</useColdSearcher>    <!-- Max Warming Searchers  Maximum number of searchers that may be warming in the  background concurrently.  An error is returned if this limit  is exceeded.  Recommend values of 1-2 for read-only slaves, higher for  masters w/o cache warming.      -->    <maxWarmingSearchers>2</maxWarmingSearchers>  </query>  <!-- Request Dispatcher       This section contains instructions for how the SolrDispatchFilter       should behave when processing requests for this SolrCore.       handleSelect is a legacy option that affects the behavior of requests       such as /select?qt=XXX       handleSelect="true" will cause the SolrDispatchFilter to process       the request and dispatch the query to a handler specified by the        "qt" param, assuming "/select" isn't already registered.       handleSelect="false" will cause the SolrDispatchFilter to       ignore "/select" requests, resulting in a 404 unless a handler       is explicitly registered with the name "/select"       handleSelect="true" is not recommended for new users, but is the default       for backwards compatibility    -->  <requestDispatcher handleSelect="false" >    <!-- Request Parsing  These settings indicate how Solr Requests may be parsed, and  what restrictions may be placed on the ContentStreams from  those requests  enableRemoteStreaming - enables use of the stream.file  and stream.url parameters for specifying remote streams.  multipartUploadLimitInKB - specifies the max size (in KiB) of  Multipart File Uploads that Solr will allow in a Request.  formdataUploadLimitInKB - specifies the max size (in KiB) of  form data (application/x-www-form-urlencoded) sent via  POST. You can use POST to pass request parameters not  fitting into the URL.  addHttpRequestToContext - if set to true, it will instruct  the requestParsers to include the original HttpServletRequest  object in the context map of the SolrQueryRequest under the   key "httpRequest". It will not be used by any of the existing  Solr components, but may be useful when developing custom   plugins.  *** WARNING ***  The settings below authorize Solr to fetch remote files, You  should make sure your system has some authentication before  using enableRemoteStreaming="true"      -->     <requestParsers enableRemoteStreaming="true"       multipartUploadLimitInKB="2048000"      formdataUploadLimitInKB="2048"      addHttpRequestToContext="false"/>    <!-- HTTP Caching  Set HTTP caching related parameters (for proxy caches and clients).  The options below instruct Solr not to output any HTTP Caching  related headers      -->    <httpCaching never304="true" />  </requestDispatcher>  <!-- Request Handlers        http://wiki.apache.org/solr/SolrRequestHandler       Incoming queries will be dispatched to a specific handler by name       based on the path specified in the request.       Legacy behavior: If the request path uses "/select" but no Request       Handler has that name, and if handleSelect="true" has been specified in       the requestDispatcher, then the Request Handler is dispatched based on       the qt parameter.  Handlers without a leading '/' are accessed this way       like so: http://host/app/[core/]select?qt=name  If no qt is       given, then the requestHandler that declares default="true" will be       used or the one named "standard".       If a Request Handler is declared with startup="lazy", then it will       not be initialized until the first request that uses it.    -->  <!-- SearchHandler       http://wiki.apache.org/solr/SearchHandler       For processing Search Queries, the primary Request Handler       provided with Solr is "SearchHandler" It delegates to a sequent       of SearchComponents (see below) and supports distributed       queries across multiple shards    -->  <!--  <requestHandler name="/dataimport" class="solr.DataImportHandler">    <lst name="defaults">      <str name="config">solr-data-config.xml</str>    </lst>  </requestHandler>  -->  <requestHandler name="/dataimport" class="solr.DataImportHandler">    <lst name="defaults">      <str name="config">data-config.xml</str>    </lst>  </requestHandler>  <requestHandler name="/select" class="solr.SearchHandler">    <!-- default values for query parameters can be specified, these  will be overridden by parameters in the request      -->     <lst name="defaults">       <str name="echoParams">explicit</str>       <int name="rows">10</int>     </lst>    </requestHandler>  <!-- A request handler that returns indented JSON by default -->  <requestHandler name="/query" class="solr.SearchHandler">     <lst name="defaults">       <str name="echoParams">explicit</str>       <str name="wt">json</str>       <str name="indent">true</str>       <str name="df">text</str>     </lst>  </requestHandler>  <!--    The export request handler is used to export full sorted result sets.    Do not change these defaults.  -->  <requestHandler name="/export" class="solr.SearchHandler">    <lst name="invariants">      <str name="rq">{!xport}</str>      <str name="wt">xsort</str>      <str name="distrib">false</str>    </lst>    <arr name="components">      <str>query</str>    </arr>  </requestHandler>  <initParams path="/update/**,/query,/select,/tvrh,/elevate,/spell">    <lst name="defaults">      <str name="df">text</str>    </lst>  </initParams>  <!-- Field Analysis Request Handler       RequestHandler that provides much the same functionality as       analysis.jsp. Provides the ability to specify multiple field       types and field names in the same request and outputs       index-time and query-time analysis for each of them.       Request parameters are:       analysis.fieldname - field name whose analyzers are to be used       analysis.fieldtype - field type whose analyzers are to be used       analysis.fieldvalue - text for index-time analysis       q (or analysis.q) - text for query time analysis       analysis.showmatch (true|false) - When set to true and when    query analysis is performed, the produced tokens of the    field value analysis will be marked as "matched" for every    token that is produces by the query analysis   -->  <requestHandler name="/analysis/field"     startup="lazy"    class="solr.FieldAnalysisRequestHandler" />  <!-- Document Analysis Handler       http://wiki.apache.org/solr/AnalysisRequestHandler       An analysis handler that provides a breakdown of the analysis       process of provided documents. This handler expects a (single)       content stream with the following format:       <docs>  <doc>    <field name="id">1</field>    <field name="name">The Name</field>    <field name="text">The Text Value</field>  </doc>  <doc>...</doc>  <doc>...</doc>  ...       </docs>    Note: Each document must contain a field which serves as the    unique key. This key is used in the returned response to associate    an analysis breakdown to the analyzed document.    Like the FieldAnalysisRequestHandler, this handler also supports    query analysis by sending either an "analysis.query" or "q"    request parameter that holds the query text to be analyzed. It    also supports the "analysis.showmatch" parameter which when set to    true, all field tokens that match the query tokens will be marked    as a "match".   -->  <requestHandler name="/analysis/document"     class="solr.DocumentAnalysisRequestHandler"     startup="lazy" />  <!-- Echo the request contents back to the client -->  <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >    <lst name="defaults">     <str name="echoParams">explicit</str>      <str name="echoHandler">true</str>    </lst>  </requestHandler>  <!-- Search Components       Search components are registered to SolrCore and used by        instances of SearchHandler (which can access them by name)       By default, the following components are available:       <searchComponent name="query"     class="solr.QueryComponent" />       <searchComponent name="facet"     class="solr.FacetComponent" />       <searchComponent name="mlt"       class="solr.MoreLikeThisComponent" />       <searchComponent name="highlight" class="solr.HighlightComponent" />       <searchComponent name="stats"     class="solr.StatsComponent" />       <searchComponent name="debug"     class="solr.DebugComponent" />     -->  <!-- Terms Component       http://wiki.apache.org/solr/TermsComponent       A component to return terms and document frequency of those       terms    -->  <searchComponent name="terms" class="solr.TermsComponent"/>  <!-- A request handler for demonstrating the terms component -->  <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">     <lst name="defaults">      <bool name="terms">true</bool>      <bool name="distrib">false</bool>    </lst>         <arr name="components">      <str>terms</str>    </arr>  </requestHandler>  <!-- Legacy config for the admin interface -->  <admin>    <defaultQuery>*:*</defaultQuery>  </admin></config>

下面我将对其中关键地方加以解释说明:

lib

<lib>   标签指令可以用来告诉 Solr 如何去加载 solr plugins(Solr 插件 ) 依赖的 jar 包,在 solrconfig.xml 配置文件的注释中有配置示例,例如:

<lib dir="./lib" regex=”lucene-\w+\.jar”/>

这里的 dir 表示一个 jar 包目录路径,该目录路径是相对于你当前 core 根目录的; regex 表示一个正则表达式,用来过滤文件名的,符合正则表达式的 jar 文件将会被加载

dataDir parameter

用来指定一个 solr 的索引数据目录, solr 创建的索引会存放在 data\index 目录下,默认 dataDir 是相对于当前 core 目录 ( 如果 solr_home 下存在 core 的话 ) ,如果 solr_home 下不存在 core 的话,那 dataDir 默认就是相对于 solr_home 啦,不过一般 dataDir 都在 core.properties 下配置。

<dataDir>/var/data/solr</dataDir>

codecFactory

用来设置Lucene倒排索引的编码工厂类,默认实现是官方提供的SchemaCodecFactory类。

indexConfig Section

在 solrconfig.xml 的 <indexConfig> 标签中间有很多关于此配置项的说明:

<!-- maxFieldLength was removed in 4.0. To get similar behavior, include a

LimitTokenCountFilterFactory in your fieldType definition. E.g.

<filter class="solr.LimitTokenCountFilterFactory" maxTokenCount="10000"/>

提供我们 maxFieldLength 配置项已经从 4.0 版本开始就已经被移除了,可以使用配置一个 filter 达到相似的效果, maxTokenCount 即在对某个域分词的时候,最多只提取前 10000 个 Token ,后续的域值将被抛弃。 maxFieldLength 若表示 1000 ,则意味着只会对域值的 0~1000 范围内的字符串进行分词索引。

<writeLockTimeout>1000</writeLockTimeout>

writeLockTimeout 表示 IndexWriter 实例在获取写锁的时候最大等待超时时间,超过指定的超时时间仍未获取到写锁,则 IndexWriter 写索引操作将会抛出异常

<maxIndexingThreads>8</maxIndexingThreads>

表示创建索引的最大线程数,默认是开辟 8 个线程来创建索引

<useCompoundFile>false</useCompoundFile>

是否开启复合文件模式,启用了复合文件模式即意味着创建的索引文件数量会减少,这样占用的文件描述符也会减少,但这会带来性能的损耗,在 Lucene 中,它默认是开启,而在 Solr 中,自从 3.6 版本开始,默认就是禁用的

<ramBufferSizeMB>100</ramBufferSizeMB>

表示创建索引时内存缓存大小,单位是 MB, 默认最大是 100M,

<maxBufferedDocs>1000</maxBufferedDocs>

表示在 document 写入到硬盘之前,缓存的 document 最大个数,超过这个最大值会触发索引的 flush 操作。

<mergePolicy class="org.apache.lucene.index.TieredMergePolicy">    <int name="maxMergeAtOnce">10</int><int name="segmentsPerTier">10</int></mergePolicy>

用来配置 Lucene 索引段合并策略的,里面有两个参数:

maxMergeAtOne: 一次最多合并段个数

segmentPerTier:  每个层级的段个数,同时也是内存 buffer 递减的等比数列的公比,看源码:

// Compute max allowed segs in the index    long levelSize = minSegmentBytes;    long bytesLeft = totIndexBytes;    double allowedSegCount = 0;    while(true) {      final double segCountLevel = bytesLeft / (double) levelSize;      if (segCountLevel < segsPerTier) {        allowedSegCount += Math.ceil(segCountLevel);        break;      }      allowedSegCount += segsPerTier;      bytesLeft -= segsPerTier * levelSize;      levelSize *= maxMergeAtOnce;    }int allowedSegCountInt = (int) allowedSegCount;

<mergeFactor>10</mergeFactor>

要理解 mergeFactor 因子的含义,还是先看看 lucene in action 中给出的解释:

IndexWriter’s mergeFactor lets you control how many Documents to store in memorybefore writing them to the disk, as well as how often to merge multiple indexsegments together. (Index segments are covered in appendix B.) With the defaultvalue of 10, Lucene stores 10 Documents in memory before writing them to a singlesegment on the disk. The mergeFactor value of 10 also means that once thenumber of segments on the disk has reached the power of 10, Lucene mergesthese segments into a single segment.For instance, if you set mergeFactor to 10, a new segment is created on the diskfor every 10 Documents added to the index. When the tenth segment of size 10 isadded, all 10 are merged into a single segment of size 100. When 10 such segmentsof size 100 have been added, they’re merged into a single segment containing1,000 Documents, and so on. Therefore, at any time, there are no more than 9segments in the index, and the size of each merged segment is the power of 10.There is a small exception to this rule that has to do with maxMergeDocs,another IndexWriter instance variable: While merging segments, Lucene ensuresthat no segment with more than maxMergeDocs Documents is created. For instance,suppose you set maxMergeDocs to 1,000. When you add the ten-thousandth Document,instead of merging multiple segments into a single segment of size 10,000,Lucene creates the tenth segment of size 1,000 and keeps adding new segmentsof size 1,000 for every 1,000 Documents added.

IndexWriter 的 mergeFactory 允许你来控制索引在写入磁盘之前内存中能缓存的 document 数量,以及合并

多个段文件的频率。默认这个值为 10. 当往内存中存储了 10 个 document, 此时 Lucene 还没有把单个段文件

写入磁盘, mergeFactor 值等于 10 也意味着当硬盘上的段文件数量达到 10 , lucene 将会把这 10 个段文件合

并到一个段文件中。例如:如果你把 mergeFactor 设置为 10 ,当你往索引中添加了 10 个 document, 一个段

文件将会在硬盘上被创建,当第 10 个段文件被添加时,这 10 个段文件就会被合并到 1 个段文件,此时这个

段文件中有 100 个 document, 当 10 个这样的包含了 100 个 document 的段文件被添加时,他们又会被合并到一

个新的段文件中,而此时这个段文件包含 1000 个 document, 以此类推。所以,在任何时候,在索引中不

存在超过 9 个段文件。每个被合并的段文件包含的 document 个数都是 10 ,但这样有点小问题,我们还必须

设置一个maxMergeDocs变量,当合并段文件的时候,lucene必须确保没有哪个段文件超过maxMergeDocs

变量规定的最大document数量。设置maxMergeDocs的目的是为了防止单个段文件中包含的document数量

过大,假定你把 maxMergeDocs 设置为 1000 ,当你创建第 10 个包含 1000 个 document 段文件的时候,这时并

不会触发段文件合并(如果没有设置maxMergeDocs为100的话,按理来说,这10个包含了1000个document

的段文件将会被合并到一个包含了 10000 个 document 的段文件当中,但 maxMergeDocs 限制了单个段文件中

最多包含 1000 个 document, 所以此时并不会触发段合并操作 ) 。影响段合并还有一些其他参数,比如:

mergeFactor :当大小几乎相当的段的数量达到此值的时候,开始合并。

minMergeSize :所有大小小于此值的段,都被认为是大小几乎相当,一同参与合并。

maxMergeSize :当一个段的大小大于此值的时候,就不再参与合并。

maxMergeDocs :当一个段包含的文档数大于此值的时候,就不再参与合并。

段合并分两个步骤:

1.首先筛选出哪些段需要合并,这一步由MergePolicy合并策略类来决定

2. 然后就是真正的段合并过程了,这一步是交给 MergeScheduler 来完成的, MergeScheduler 类主要做两件事:

A.对存储域,项向量,标准化因子即norms等信息进行合并

B.对倒排索引信息进行合并

尼玛扯远了,接着继续我们的 solrconfig.xml 中影响索引创建的一些参数配置;

<mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"/>

mergeScheduler 刚才提到过了,这是用来配置段合并操作的处理类。默认实现类是 Lucene 中自带的 ConcurrentMergeScheduler 。

<lockType>${solr.lock.type:native}</lockType>

这个是用来指定 Lucene 中 LockFactory 实现的,可配置项如下:

single = SingleInstanceLockFactory - suggested for a  read-only index or when there is no possibility of  another process trying to modify the index.native = NativeFSLockFactory - uses OS native file locking.  Do not use when multiple solr webapps in the same  JVM are attempting to share a single index.simple = SimpleFSLockFactory  - uses a plain file for lockingDefaults: 'native' is default for Solr3.6 and later, otherwise   'simple' is the default

single :表示只读锁,没有另外一个处理线程会去修改索引数据

native :即 Lucene 中的 NativeFSLockFactory 实现,使用的是基于操作系统的本地文件锁

simple :即 Lucene 中的 SimpleFSLockFactory 实现,通过在硬盘上创建 write.lock 锁文件实现

Defaults :从 solr3.6 版本开始,这个默认值是 native, 否则,默认值就是 simple, 意思就是说,你如果配置为 Defaults ,到底使用哪种锁实现,取决于你当前使用的 Solr 版本。

<unlockOnStartup>false</unlockOnStartup>

如果这个设置为 true, 那么在 solr 启动后, IndexWriter 和 commit 提交操作拥有的锁将会被释放,这会打破 Lucene 的锁机制,请谨慎使用。如果你的 lockType 设置为 single, 那么这个配置 true or false 都不会产生任何影响。

<deletionPolicy class="solr.SolrDeletionPolicy">

用来配置索引删除策略的,默认使用的是 Solr 的 SolrDeletionPolicy 实现。如果你需要自定义删除策略,那么你需要实现 Lucene 的 org.apache.lucene.index.IndexDeletionPolicy 接口。

<jmx />

这个配置是用来在 Solr 中启用 JMX ,有关这方面的详细信息,请移步到 Solr 官方 Wiki ,访问地址如下:

http://wiki.apache.org/solr/SolrJmx

<updateHandler class="solr.DirectUpdateHandler2">

指定索引更新操作处理类, DirectUpdateHandler2 是一个高性能的索引更新处理类,它支持软提交

<updateLog>

<str name="dir">${solr.ulog.dir:}</str>

</updateLog>

<updateLog> 用来指定上面的 updateHandler 的处理事务日志存放路径的,默认值是 solr 的 data 目录即 solr 的 dataDir 配置的目录。

<query> 标签是有关索引查询相关的配置项

<maxBooleanClauses>1024</maxBooleanClauses>

表示 BooleanQuery 最大能链接多少个子 Query, 当不同的 core 下的 solrconfig.xml 中此配置项的参数值配置的不一样时,以最后一个初始化的 core 的配置为准。

<filterCache class="solr.FastLRUCache"

size="512"

initialSize="512"

autowarmCount="0"/>

用来配置 filter 过滤器的缓存相关的参数

<queryResultCache class="solr.LRUCache"

size="512"

initialSize="512"

autowarmCount="0"/>

用来配置对 Query 返回的查询结果集即 TopDocs 的缓存

<documentCache class="solr.LRUCache"

size="512"

initialSize="512"

autowarmCount="0"/>

用来配置对 Document 中存储域的缓存,因为每次从硬盘上加载存储域的值都是很昂贵的操作,这里说的存储域指的是那些 Store.YES 的 Field ,所以你懂的。

<fieldValueCache class="solr.FastLRUCache"

size="512"

autowarmCount="128"

showItems="32" />

这个配置是用来缓存 Document id 的,用来快速访问你的 Document id 的。这个配置项默认就是开启的,无需显式配置。

<cache name="myUserCache"

class="solr.LRUCache"

size="4096"

initialSize="1024"

autowarmCount="1024"

regenerator="com.mycompany.MyRegenerator"

/>

这个配置是用来配置你的自定义缓存的,你自己的 Regenerator 需要实现 Solr 的 CacheRegenerator 接口。

<enableLazyFieldLoading>true</enableLazyFieldLoading>

表示启用存储域的延迟加载,前提是你的存储域在 Query 的时候没有显式指定需要 return 这个域。

<useFilterForSortedQuery>true</useFilterForSortedQuery>

表示当你的 Query 没有使用 score 进行排序时,是否使用 filter 来替代 Query.

<listener event="newSearcher" class="solr.QuerySenderListener">      <arr name="queries">        <!--           <lst><str name="q">solr</str><str name="sort">price asc</str></lst>           <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst>          -->      </arr></listener>

QuerySenderListener 用来监听查询发送过程,即你可以在 Query 请求发送之前追加一些请求参数,如上面给的示例中,可以追加 qery 关键字以及 sort 排序规则。

<requestDispatcher handleSelect="false" >

设置为 false 即表示 Solr 服务器端不接收 /select 请求,即如果你请求 http://localhost:8080/solr/coreName/select?qt=xxxx 时,将会返回一个 404 ,

这个 select 请求是为了兼容先前的旧版本,已经不推荐使用。

<httpCaching never304="true" />

表示 solr 服务器段永远不返回 304 ,那 http 响应状态码 304 表示什么呢?表示服务器端告诉客户端,你请求的资源尚未被修改过,我返回给你的是上次缓存的内容。 Never304 即告诉服务器,不管我访问的资源有没有更新过,都给我重新返回不走 Http 缓存。这属于 Http 协议相关知识,不清楚的请去 Google HTTP 协议详细了解去。

<requestHandler name="/query" class="solr.SearchHandler">    <lst name="defaults">      <str name="echoParams">explicit</str>      <str name="wt">json</str>      <str name="indent">true</str>    </lst>  </requestHandler>

这个 requestHandler 配置的是请求 URL  /query 跟请求处理类 SearcherHandler 之间的一个映射关系,即你访问 http://localhost:8080/solr/coreName/query?q=xxx 时,会交给 SearcherHandler 类来处理这个 http 请求,你可以配置一些参数来干预 SearcherHandler 处理细节,比如 echoParams 表示是否打印 HTTP 请求参数, wt 即 writer type, 即返回的数据的 MIME 类型,如 json,xml 等等, indent 表示返回的 json 或者 XML 数据是否需要缩进,否则返回的数据没有缩进也没有换行,不利于阅读。

其他的一些 requestHandler 说明就略过了,其实都大同小异,就是一个请求 URL 跟请求处理类的一个映射 , 就好比 SpringMVC 中请求 URL 和 Controller 类的一个映射。

<searchComponent name="spellcheck" class="solr.SpellCheckComponent">

用来配置查询组件比如 SpellCheckComponent 拼写检查,有关拼写检查的详细配置说明留到以后说到 SpellCheck 时再说吧。

<searchComponent name="terms" class="solr.TermsComponent"/>

用来返回所有的 Term 以及每个 document 中 Term 的出现频率

<searchComponent class="solr.HighlightComponent" name="highlight">

用来配置关键字高亮的, Solr 高亮配置的详细说明这里暂时先略过,这篇我们只是先暂时大致了解下每个配置项的含义即可,具体如何使用留到后续再深入研究。

有关 searchComponent 查询组件的其他配置我就不一一说明了,太多了。你们自己看里面的英文注释吧,如果你实在看不懂再来问我。

<queryResponseWriter name="json" class="solr.JSONResponseWriter">    <!-- For the purposes of the tutorial, JSON responses are written as     plain text so that they are easy to read in *any* browser.     If you expect a MIME type of "application/json" just remove this override.    -->    <str name="content-type">text/plain; charset=UTF-8</str></queryResponseWriter>

这个是用来配置 Solr 响应数据转换类, JSONResponseWriter 就是把 HTTP 响应数据转成 JSON 格式, content-type 即 response 响应头信息中的 content-type, 即告诉客户端返回的数据的 MIME 类型为 text/plain ,且 charset 字符集编码为 UTF-8.

内置的响应数据转换器还有 velocity , xslt 等,如果你想自定义一个基于 FreeMarker 的转换器,那你需要实现 Solr 的 QueryResponseWriter 接口,模仿其他实现类,你懂的,然后在 solrconfig.xml 中添加类似的 <queryResponseWriter 配置即可

最后需要说明下的是 solrconfig.xml 中有大量类似 <arr> <list> <str> <int>

这样的自定义标签,下面做个统一的说明:


这张图摘自于 Solr in Action 这本书,由于是英文的,所以我稍微解释下:


arr: 即 array 的缩写,表示一个数组, name 即表示这个数组参数的变量名

lst 即 list 的缩写,但注意它里面存放的是 key-value 键值对

bool 表示一个 boolean 类型的变量 ,name 表示 boolean 变量名,

同理还有 int,long,float,str 等等

Str 即 string 的缩写,唯一要注意的是 arr 下的 str 子元素是没有 name 属性的,而 list 下的 str 元素是有 name 属性的

最后总结下:

solrconfig.xml 中的配置项主要分以下几大块:

1. 依赖的 lucene 版本配置,这决定了你创建的 Lucene 索引结构,因为 Lucene 各版本之间的索引结构并不是完全兼容的,这个需要引起你的注意。

2. 索引创建相关的配置,如索引目录, IndexWriterConfig 类中的相关配置 ( 它决定了你的索引创建性能 )

3.solrconfig.xml 中依赖的外部 jar 包加载路径配置

4.JMX 相关配置

5. 缓存相关配置,缓存包括过滤器缓存,查询结果集缓存, Document 缓存,以及自定义缓存等等

6.updateHandler 配置即索引更新操作相关配置

7.RequestHandler 相关配置,即接收客户端 HTTP 请求的处理类配置

8. 查询组件配置如 HightLight , SpellChecker 等等

9.ResponseWriter 配置即响应数据转换器相关配置,决定了响应数据是以什么样格式返回给客户端的。

10. 自定义 ValueSourceParser 配置,用来干预 Document 的权重、评分,排序

原创粉丝点击