[
https://issues.apache.org/jira/browse/LUCENE-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155894#comment-13155894
]
Luca Cavanna edited comment on LUCENE-3586 at 11/23/11 2:23 PM:
----------------------------------------------------------------
Hi Uwe,
I'm working with lucene 3.4 and jvm 1.6.0_23 64 bit. I have ulimit unlimited
and /proc/sys/vm/max_map_count 65530. The index size is 76GB, I have problems
with just one segment, the first and definitely the biggest one, on which the
fdt is 22GB, tvf is 13GB, prx is 4.2GB, frq is 2.8GB, and so on (others are
under 1GB). I tried to run the checkindex with -Xmx512m and -Xms512m. Here is
my output:
Segments file=segments_kj numSegments=31 version=3.4 format=FORMAT_3_1 [Lucene
3.1+]
1 of 31: name=_1iz docCount=3186709
compound=false
hasProx=true
numFiles=12
size (MB)=41,902.362
diagnostics = {optimize=true, mergeFactor=23,
os.version=2.6.27.59-1POi-x86_64, os=Linux, lucene.version=3.4.0 1167142 - mike
- 2011-09-09 09:02:09, source=merge, os.arch=amd64, java.version=1.6.0_23,
java.vendor=Sun Microsystems Inc.}
has deletions [delFileName=_1iz_r.del]
test: open reader.........FAILED
Thanks, and please let me know if you have any questions.
was (Author: lucacavanna):
Hi Uwe,
I'm working with lucene 3.4 and jvm 1.6.0_23 64 bit. I have ulimit unlimited
and /proc/sys/vm/max_map_count 65530. The index size is 76GB, I have problems
with just one segment, the first and definitely the bigger one, on which the
fdt is 22GB, tvf is 13GB, prx is 4.2GB, frq is 2.8GB, and so on (others are
under 1GB). I tried to run the checkindex with -Xmx512m and -Xms512m. Here is
my output:
Segments file=segments_kj numSegments=31 version=3.4 format=FORMAT_3_1 [Lucene
3.1+]
1 of 31: name=_1iz docCount=3186709
compound=false
hasProx=true
numFiles=12
size (MB)=41,902.362
diagnostics = {optimize=true, mergeFactor=23,
os.version=2.6.27.59-1POi-x86_64, os=Linux, lucene.version=3.4.0 1167142 - mike
- 2011-09-09 09:02:09, source=merge, os.arch=amd64, java.version=1.6.0_23,
java.vendor=Sun Microsystems Inc.}
has deletions [delFileName=_1iz_r.del]
test: open reader.........FAILED
Thanks, and please let me know if you have any questions.
> Choose a specific Directory implementation running the CheckIndex main
> ----------------------------------------------------------------------
>
> Key: LUCENE-3586
> URL: https://issues.apache.org/jira/browse/LUCENE-3586
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Luca Cavanna
> Assignee: Luca Cavanna
> Priority: Minor
>
> It should be possible to choose a specific Directory implementation to use
> during the CheckIndex process when we run it from its main.
> What about an additional main parameter?
> In fact, I'm experiencing some problems with MMapDirectory working with a big
> segment, and after some failed attempts playing with maxChunkSize, I decided
> to switch to another FSDirectory implementation but I needed to do that on my
> own main.
> Should we also consider to use a FileSwitchDirectory?
> I'm willing to contribute, could you please let me know your thoughts about
> it?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
