I will look a little deeper into the information you supplied and
comment, but will suggest this on my initial cursory review:
1 - You have 32GB of memory. Using the 64bit VM, try using a 16GB or 24GB heap;
2 - Turn-on huge pages:
-XX:+UseLargePages
-XX:LargePageSizeInBytes=256m
3 - Turn on aggressive opts:
-XX:+AggressiveOpts
4 - Try using bound threads:
-XX:+UseBoundThreads
5 - Try using thread-local object allocation:
-XX:+UseTLAB
6 - Try Biased Locking:
-XX:+UseBiasedLocking
I consider #1-3 most likely to help you. I would first try them
separately then in combination. #4-6 I am not sure given your hardware
(of which I am very envious!), but would try nonetheless.
I am more used to working with quad dual core ia64 machines with 8GB
memory running Linux. And large indexes, with tests up to 1B documents
and ~700GB in size, in a single Lucene index.
Please let me know how these things help, if at all.
thanks,
Glen
2009/2/19 Varun Dhussa <va...@mapmyindia.com>:
> Hi,
>
> The details are as follows:
> Solaris version: Solaris 10 U5 and U6
> For the Java Setup, I have tried with:
> Sun JDK 1.5 (32 & 64)
> Sun JDK 1.6 (32 & 64)
> Heap Space: 2G from 32 bit and 4G for 64 bit (Set the same values for both
> XMS and XMX)
>
> Disk: Tried with ZFS (U6) and UFS (U5)
> I reduced the Inode density on the partition since page faults were
> happening initially. Also, the block size was 8K.
> The following reports are on Solaris 10 U5 with UFS (8K block size).
>
>
> The vmstat report is below:
>
> # vmstat 3
> kthr memory page disk faults cpu
> r b w swap free re mf pi po fr de sr s0 s2 -- -- in sy cs us sy
> id
> 0 0 0 43408368 29603112 13 107 4 7 6 0 0 2 -0 0 0 458 589 321 0 0
> 99
> 0 0 0 22517072 30241872 830 7709 0 0 0 0 0 0 0 0 0 2537 30699 3899 36 3
> 61
> 0 0 0 24666416 30483248 860 7942 0 0 0 0 0 0 0 0 0 2235 31886 3362 28 3
> 69
> 0 0 0 24184008 30450728 791 7813 0 0 0 0 0 0 0 0 0 2335 31426 3740 33 3
> 65
> 0 0 0 23472024 30442168 813 7822 0 0 0 0 0 0 0 0 0 2494 31312 3923 35 3
> 62
> 0 0 0 22751768 30297240 787 7654 0 0 0 0 0 0 0 0 0 2498 31820 3892 35 3
> 62
> 0 0 0 25385568 30527600 835 7963 0 0 0 0 0 0 0 0 0 2360 32144 3691 29 3
> 68
> 0 0 0 22741056 30281376 787 7643 0 0 0 0 0 0 0 0 0 2249 30008 3484 37 3
> 60
> 0 0 0 22753800 30373072 847 7923 0 0 0 0 0 0 0 0 0 2501 31375 3854 33 3
> 64
> 0 0 0 23725600 30431312 841 8049 0 0 0 0 0 0 0 0 0 2362 32514 3752 32 3
> 65
> 0 0 0 22743008 30256368 826 7719 0 0 0 0 0 0 0 0 0 2427 32101 3773 34 3
> 63
> 0 0 0 23950336 30405312 818 7925 0 0 0 0 0 0 0 0 0 2337 32095 3606 31 3
> 66
> 0 0 0 22752872 30319312 778 7486 0 0 0 0 0 0 0 0 0 2435 30277 3769 37 3
> 60
> 0 0 0 24657848 30401408 836 7797 0 0 0 0 0 0 0 0 0 2469 31879 3770 32 3
> 65
> 0 0 0 23460408 30331056 776 7560 0 0 0 0 0 0 0 0 0 2461 31344 3883 37 3
> 60
> 0 0 0 22748352 30302248 792 7534 0 0 0 0 0 0 0 0 0 2424 30683 3733 36 3
> 61
> 0 0 0 23953848 30389280 837 7868 0 0 0 0 0 0 0 0 0 2280 30710 3511 34 3
> 63
> 0 0 0 22736592 30249672 810 7716 0 0 0 0 0 0 0 0 0 2345 30980 3593 34 3
> 63
> kthr memory page disk faults cpu
> r b w swap free re mf pi po fr de sr s0 s2 -- -- in sy cs us sy
> id
> 0 0 0 23951544 30400016 834 7992 0 0 0 0 0 0 0 0 0 2431 32647 3776 31 3
> 66
> 0 0 0 21797992 30245536 812 7635 0 0 0 0 0 0 0 0 0 2402 30213 3693 39 3
> 59
> 0 0 0 22745264 30307728 781 7539 0 0 0 0 0 0 0 0 0 2437 30744 3860 38 3
> 59
> 0 0 0 22748536 30286456 851 7987 0 0 0 0 0 0 0 0 0 2433 32842 3678 29 3
> 68
> 0 0 0 22747680 30297544 791 7514 0 0 0 0 0 0 0 0 0 2369 30625 3655 38 3
> 59
> 0 0 0 23104848 30309488 793 7524 0 0 0 0 0 0 0 0 0 2566 31190 3942 36 3
> 61
> 0 0 0 23460368 30340688 810 7784 0 0 0 0 0 0 0 0 0 2302 31430 3688 33 3
> 64
> 0 0 0 22031344 30215504 792 7397 0 0 0 0 0 0 0 0 0 2477 30733 3850 39 3
> 58
> 0 0 0 22744168 30277312 834 7829 0 0 0 0 0 0 0 0 0 2370 29926 3592 35 3
> 63
> 0 0 0 24177728 30432600 806 7789 0 0 0 0 0 0 0 0 0 2379 31870 3731 32 3
> 65
> 0 0 0 23467896 30382728 802 7810 0 0 0 0 0 0 0 0 0 2527 31866 3980 33 3
> 64
> 0 0 0 22757912 30331080 823 7847 0 0 0 0 0 0 0 0 0 2496 31580 3871 33 3
> 64
> 0 0 0 22043312 30215904 819 7558 0 0 0 0 0 0 0 0 0 2253 30252 3455 36 3
> 61
> 0 0 0 22744952 30275008 793 7657 0 0 0 0 0 0 0 0 0 2499 31840 3937 36 3
> 61
> 0 0 0 23715200 30419008 843 7906 0 0 0 0 0 0 0 0 0 2404 31741 3707 33 3
> 64
> 0 0 0 21307184 30148472 788 7515 0 0 0 0 0 0 0 0 0 2430 31340 3819 39 3
> 58
> 0 0 0 23459544 30332160 817 7687 0 0 0 0 0 0 0 0 0 2353 31011 3526 34 3
> 63
> 0 0 0 23467304 30376392 819 7846 0 0 0 0 0 0 0 0 0 2522 31918 3933 32 3
> 65
> 0 0 0 23473464 30421992 830 7854 0 0 0 0 0 0 0 0 0 2407 31714 3827 33 3
> 65
> kthr memory page disk faults cpu
> r b w swap free re mf pi po fr de sr s0 s2 -- -- in sy cs us sy
> id
> 0 0 0 22508648 30240256 799 7447 0 0 0 0 0 0 0 0 0 2524 30483 3911 39 3
> 58
> 0 0 0 22737496 30276864 837 7731 0 0 0 0 0 0 0 0 0 2310 30289 3473 36 3
> 61
> 0 0 0 22515488 30312728 792 7646 0 0 0 0 0 0 0 0 0 2472 31610 3867 35 3
> 62
> 0 0 0 22756248 30251584 823 7669 0 0 0 0 0 0 0 0 0 2637 31413 4119 36 3
> 60
> 0 0 0 23940168 30379336 833 7973 0 0 0 0 0 0 0 0 0 2240 32325 3472 31 3
> 66
> 0 0 0 24184736 30455928 846 7838 0 0 0 0 0 0 0 0 0 2288 31508 3538 32 3
> 65
> 0 0 0 22751808 30309984 776 7614 0 0 0 0 0 0 0 0 0 2480 31486 3906 36 3
> 61
> 0 0 0 22989624 30309536 813 7626 0 0 0 0 0 0 0 0 0 2537 30347 3988 38 3
> 59
> 0 0 0 22739224 30276936 814 7692 0 0 0 0 0 0 0 0 0 2261 31528 3409 34 3
> 63
> 0 0 0 23236200 30350216 853 7968 0 0 0 0 0 0 0 0 0 2676 32636 4167 32 3
> 65
> 0 0 0 24194016 30506960 836 8005 0 0 0 0 0 0 0 0 0 2222 31503 3382 30 3
> 67
> 0 0 0 23480504 30424536 834 7848 0 0 0 0 0 0 0 0 0 2390 32039 3736 32 3
> 65
> 0 0 0 22739048 30233176 812 7742 0 0 0 0 0 0 0 0 0 2604 31693 4029 35 3
> 62
> 0 0 0 23461664 30358528 805 7700 0 0 0 0 0 0 0 0 0 2463 30936 3922 36 3
> 61
> 0 0 0 22031616 30257808 796 7533 0 0 0 0 0 0 0 0 0 2226 29829 3377 37 3
> 60
> 0 0 0 23458168 30327688 813 7773 0 0 0 0 0 0 0 0 0 2388 31519 3763 33 3
> 64
> 0 0 0 23465184 30371904 834 7796 0 0 0 0 0 0 0 0 0 2510 32101 3933 34 3
> 63
> 0 0 0 22513648 30312240 800 7596 0 0 0 0 0 0 0 0 0 2416 30538 3734 38 3
> 59
> 0 0 0 22272824 30228912 802 7640 0 0 0 0 0 0 0 0 0 2292 31395 3469 34 3
> 63
> kthr memory page disk faults cpu
> r b w swap free re mf pi po fr de sr s0 s2 -- -- in sy cs us sy
> id
> 0 0 0 22744032 30253480 830 7758 0 0 0 0 0 0 0 0 0 2568 31896 3935 35 3
> 62
> 0 0 0 23287696 30368432 808 7736 0 0 0 0 0 0 0 0 0 2439 30929 3844 35 3
> 62
> 0 0 0 22038504 30253112 794 7480 0 0 0 0 0 0 0 0 0 2213 29774 3433 37 3
> 60
> 0 0 0 24660144 30480240 846 8103 0 0 0 0 0 0 0 0 0 2548 32825 3944 30 3
> 66
> 0 0 0 24183648 30445120 823 7895 0 0 0 0 0 0 0 0 0 2441 32699 3822 31 3
> 66
> 0 0 0 23469136 30391744 830 7916 0 0 0 0 0 0 0 0 0 2441 31731 3813 32 3
> 65
> 0 0 0 22516312 30289688 796 7592 0 0 0 0 0 0 0 0 0 2318 30310 3616 36 3
> 61
> 0 0 0 23943448 30363248 821 7821 0 0 0 0 0 0 0 0 0 2355 31789 3641 31 3
> 66
> 0 0 0 22742768 30284896 795 7602 0 0 0 0 0 0 0 0 0 2387 30608 3675 37 3
> 60
> 0 0 0 23475456 30416136 852 7991 0 0 0 0 0 0 0 0 0 2489 32066 3871 31 3
> 66
> 0 0 0 22756848 30233968 798 7588 0 0 0 0 0 0 0 0 0 2464 31106 3836 37 3
> 60
>
> Cooltest report said virtually nothing.
>
> System Configuration
>
> Host name mmiAS02
> System name SUNW,SPARC-Enterprise-T5120
> Effective UID 0
> Cooltst version 3.0.1
> OS Solaris
> OS release 5.10
> OS version Generic_127127-11
> Distro Solaris
> BIOS/PROM OBP 4.28.0 2008/01/22 21:10
> Memory 32640 MB
> Chip UltraSPARC-T2
> MHz 1165
> Architecture SPARC
> # of Virtual CPUs 64
> P0: 1165 MHz UltraSPARC-T2
> P1: 1165 MHz UltraSPARC-T2
> P2: 1165 MHz UltraSPARC-T2
> P3: 1165 MHz UltraSPARC-T2
> P4: 1165 MHz UltraSPARC-T2
> P5: 1165 MHz UltraSPARC-T2
> P6: 1165 MHz UltraSPARC-T2
> P7: 1165 MHz UltraSPARC-T2
> P8: 1165 MHz UltraSPARC-T2
> P9: 1165 MHz UltraSPARC-T2
> P10: 1165 MHz UltraSPARC-T2
> P11: 1165 MHz UltraSPARC-T2
> P12: 1165 MHz UltraSPARC-T2
> P13: 1165 MHz UltraSPARC-T2
> P14: 1165 MHz UltraSPARC-T2
> P15: 1165 MHz UltraSPARC-T2
> P16: 1165 MHz UltraSPARC-T2
> P17: 1165 MHz UltraSPARC-T2
> P18: 1165 MHz UltraSPARC-T2
> P19: 1165 MHz UltraSPARC-T2
> P20: 1165 MHz UltraSPARC-T2
> P21: 1165 MHz UltraSPARC-T2
> P22: 1165 MHz UltraSPARC-T2
> P23: 1165 MHz UltraSPARC-T2
> P24: 1165 MHz UltraSPARC-T2
> P25: 1165 MHz UltraSPARC-T2
> P26: 1165 MHz UltraSPARC-T2
> P27: 1165 MHz UltraSPARC-T2
> P28: 1165 MHz UltraSPARC-T2
> P29: 1165 MHz UltraSPARC-T2
> P30: 1165 MHz UltraSPARC-T2
> P31: 1165 MHz UltraSPARC-T2
> P32: 1165 MHz UltraSPARC-T2
> P33: 1165 MHz UltraSPARC-T2
> P34: 1165 MHz UltraSPARC-T2
> P35: 1165 MHz UltraSPARC-T2
> P36: 1165 MHz UltraSPARC-T2
> P37: 1165 MHz UltraSPARC-T2
> P38: 1165 MHz UltraSPARC-T2
> P39: 1165 MHz UltraSPARC-T2
> P40: 1165 MHz UltraSPARC-T2
> P41: 1165 MHz UltraSPARC-T2
> P42: 1165 MHz UltraSPARC-T2
> P43: 1165 MHz UltraSPARC-T2
> P44: 1165 MHz UltraSPARC-T2
> P45: 1165 MHz UltraSPARC-T2
> P46: 1165 MHz UltraSPARC-T2
> P47: 1165 MHz UltraSPARC-T2
> P48: 1165 MHz UltraSPARC-T2
> P49: 1165 MHz UltraSPARC-T2
> P50: 1165 MHz UltraSPARC-T2
> P51: 1165 MHz UltraSPARC-T2
> P52: 1165 MHz UltraSPARC-T2
> P53: 1165 MHz UltraSPARC-T2
> P54: 1165 MHz UltraSPARC-T2
> P55: 1165 MHz UltraSPARC-T2
> P56: 1165 MHz UltraSPARC-T2
> P57: 1165 MHz UltraSPARC-T2
> P58: 1165 MHz UltraSPARC-T2
> P59: 1165 MHz UltraSPARC-T2
> P60: 1165 MHz UltraSPARC-T2
> P61: 1165 MHz UltraSPARC-T2
> P62: 1165 MHz UltraSPARC-T2
> P63: 1165 MHz UltraSPARC-T2
> OS release detail:
> Solaris 10 5/08 s10s_u5wos_10 SPARC Copyright 2008 Sun Microsystems, Inc.
> All Rights Reserved. Use is subject to license terms. Assembled 24 March
> 2008
>
> Workload Measurements
>
> Observed system for 10 min
> in intervals of 10 sec
> Cycles 44768051692942
> Instructions 3980806371547
> CPI 11.25 **
> FP instructions 5821938521
> Emulated FP instructions 0
> FP Percentage 0.1%
> The following applies to the measurement interval with the
> busiest single thread or process:
> Peak thread utilization at 2009-02-13 14:36:16
> Corresponding file name 1234515976
> CPU utilization 39.4%
> Command java
> PID/LWPID 16396/1
> Thread utilization 0.6%
> More detail on processes and threads is in data/process.out
>
> **Cycles per Instruction (CPI) is not comparable between UltraSPARC
> T1 and T2 processors and conventional processors. Conventional
> processors execute an idle loop when there is no work to do, so
> CPI may be artificially low, especially when the system is
> somewhat idle. The UltraSPARC T1 and T2 "park" idle threads,
> consuming no energy, when there is no work to do, so CPI may
> be artificially high, especially when the system is somewhat idle.
>
> Advice
>
> Floating Point GREEN
> Observed floating point content was not excessive for
> an UltraSPARC T1 processor. Floating point content is not
> a limitation for UltraSPARC T2.
>
> Parallelism GREEN
> The observed workload has sufficient threads of execution to
> efficiently utilize the multiple cores and threads of an
> UltraSPARC T1 or UltraSPARC T2 processor.
>
>
>
> Varun Dhussa
> Product Architect
> CE InfoSystems (P) Ltd
> http://www.mapmyindia.com
>
>
>
> Glen Newton wrote:
>>
>> Could you give some configuration details:
>> - Solaris version
>> - Java VM version, heap size, and any other flags
>> - disk setup
>>
>> You should also consider using huge pages (see
>>
>> http://zzzoot.blogspot.com/2009/02/java-mysql-increased-performance-with.html)
>>
>> I will also be posting performance gains using huge pages for Java
>> Lucene Linux large scale indexing in the next week or so...
>>
>> -glen
>>
>>
>>
>> 2009/2/18 Varun Dhussa <va...@mapmyindia.com>:
>>
>>>
>>> Hi,
>>>
>>> I have had a bad experience when migrating my application from Intel Xeon
>>> based servers to Sun UltraSparc T2 T5120 servers. Lucene fuzzy search
>>> just
>>> does not perform. A search which took approximately 500 ms takes more
>>> than 6
>>> seconds to execute.
>>>
>>> The index has about 100,000,000 records. So, I tried to split it into 10
>>> indices and used the ParallelSearcher on it, but still got similar
>>> results.
>>>
>>> I am guessing that this is because the distance implementation used by
>>> Lucene requires higher clock speed and can't be parallelized much.
>>>
>>> Please advice
>>>
>>> --
>>> Varun Dhussa
>>> Product Architect
>>> CE InfoSystems (P) Ltd
>>> http://www.mapmyindia.com
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>>
>>>
>>>
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
--
-
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
