Hi Rus, so OpenJDK is 17 times slower than Oracle JDK... About the delete we know it's a slow operation now, you could remove indexes + truncate to speed up things and when you delete an entire cluster. You could also drop and re-create it. This should be much faster.
What are the Oracle rules for the JVM? This is from the official web site ( https://blogs.oracle.com/henrik/entry/oracle_releases_jdk_for_linux): *Is the ARM JDK free (gratis) or does it require a commercial license?* Like all general-purpose JDK and JRE binaries, the ARM JDK is free for development and production use on general-purpose hardware, and can be redistributed for free with applications targeting a general-purpose computer. See the end-user license<http://www.oracle.com/technetwork/java/javase/terms/license/index.html> for the exact license grants & restrictions. To take a couple of examples, an ARM server deployed in your datacenter running Tomcat or Glassfish is general-purpose, as is a Raspberry Pi board when you use it like a PC. An industrial controller or a kiosk appliance is not general purpose, and both would require a commercial license. So what's the problem on using Oracle JVM? Lvc@ On 22 May 2014 07:18, Sfinx <[email protected]> wrote: > Hi Luca, > > Yes, the graph-example-2.xml import times are already present here at > this thread, in short : > > - Orks JDK8 - [java] Imported in *158111ms*. Vertexes: 809 > - Orks JDK7 - [java] Imported in *154536ms*. Vertexes: 809 > - OpenJDK - [java] Imported in *2636541ms*. Vertexes: 809 > > The only operation that is evenly slow at all three JDK's is 'delete from > ' - it takes nearly *30 seconds per 1500 rows* > > Rus > > понедельник, 19 мая 2014 г., 14:09:11 UTC+3 пользователь Lvc@ написал: >> >> Hi Rus, >> Have you tried the same test with Oracle JDK8? Just to understand if the >> bottleneck is on OrientDB or Open JDK? >> >> Thanks, >> Lvc@ >> >> >> >> On 18 May 2014 10:38, Sfinx <[email protected]> wrote: >> >>> No problem. I think that sometime you will decide to tune the OrientDB >>> perfomance, it is can be done with the following : >>> >>> - use the OpenJDK only >>> - limit the Linux kernel used memory size say to 2G (use mem=2G kernel >>> parameter) >>> - limit the CPU cores to 1 or 2 (maxcpus=2 kernel parameter) >>> - use the slowest HDD ;) >>> >>> So you can try the Raspberry again ;) >>> >>> Cheers ! >>> >>> Rus >>> >>> воскресенье, 18 мая 2014 г., 11:09:56 UTC+3 пользователь Andrey Lomakin >>> написал: >>>> >>>> Sorry for later delay we have enormous amount of tasks to do. >>>> >>>> >>>> On Sun, May 18, 2014 at 11:09 AM, Andrey Lomakin >>>> <[email protected]>wrote: >>>> >>>>> Hi, >>>>> The point here that you always in swap, you see you have 2gb of ram >>>>> but disk cache by default has 4gb. >>>>> Could you kindle set the parameter storage.diskCache.bufferSize to >>>>> 1024 and try your test ? >>>>> >>>>> >>>>> On Sun, May 18, 2014 at 8:12 AM, Sfinx <[email protected]> wrote: >>>>> >>>>>> Hi Luca, >>>>>> >>>>>> Never mind - switched to MongoDB. Seems like running OrientDB on such >>>>>> low hardware was initially bad idea ;) >>>>>> >>>>>> [proto]:root:/Arhiv/mongo # echo "{nThreads:1,fileSizeMB:1,r:true}" >>>>>> | /opt/mongo/bin/mongoperf >>>>>> mongoperf >>>>>> use -h for help >>>>>> parsed options: >>>>>> { nThreads: 1, fileSizeMB: 1, r: true } >>>>>> creating test file size:1MB ... >>>>>> testing... >>>>>> optoins:{ nThreads: 1, fileSizeMB: 1, r: true } >>>>>> wthr 1 >>>>>> new thread, total running : 1 >>>>>> read:1 write:0 >>>>>> 5343 ops/sec 20 MB/sec >>>>>> 5384 ops/sec 21 MB/sec >>>>>> 5359 ops/sec 20 MB/sec >>>>>> 5397 ops/sec 21 MB/sec >>>>>> 5389 ops/sec 21 MB/sec >>>>>> 5341 ops/sec 20 MB/sec >>>>>> ^C >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> среда, 14 мая 2014 г., 11:03:07 UTC+3 пользователь Sfinx написал: >>>>>> >>>>>>> Hi Luca, >>>>>>> >>>>>>> Please see the attached VisualVM and YourKit snapshots made during >>>>>>> importing src/test/resources/graph-example-2.xml to the snapshot >>>>>>> branch under OpenJDK. Another bottleneck noticed when deleting records >>>>>>> from >>>>>>> the table using simple "delete * from" SQL statement. Can profile this >>>>>>> too >>>>>>> if interested. >>>>>>> >>>>>>> Rus >>>>>>> >>>>>>> понедельник, 12 мая 2014 г., 19:46:13 UTC+3 пользователь Lvc@написал: >>>>>>>> >>>>>>>> Hi Rus, >>>>>>>> could you profile the server on such HW/SW cfg (JVisualVM or better >>>>>>>> YourKit)? When we tested OrientDB on Raspberry we found many >>>>>>>> bottlenecks >>>>>>>> detectable only on such architecture. >>>>>>>> >>>>>>>> Lvc@ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Lvc@ >>>>>>>> >>>>>>>> >>>>>>>> On 12 May 2014 18:30, Sfinx <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> The specifying -Dmemory.useUnsafe=false prevented the hangs >>>>>>>>> (thanks for hint !), the second option do not influence at all. Though >>>>>>>>> insert speed is still too low (counting that database dir resides as >>>>>>>>> fast >>>>>>>>> SSD and the object is the dummy object from the post.txt): >>>>>>>>> >>>>>>>>> ........ >>>>>>>>> [proto]:root:/opt/orientdb/benchmarks # ab -n1000 -A >>>>>>>>> root:root_pass -k -c2 -p post.txt http://127.0.0.1:2480/document >>>>>>>>> /demo/9:1 >>>>>>>>> This is ApacheBench, Version 2.3 <$Revision: 1528965 $> >>>>>>>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >>>>>>>>> http://www.zeustech.net/ >>>>>>>>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>>>>>>>> >>>>>>>>> Benchmarking 127.0.0.1 (be patient) >>>>>>>>> Completed 100 requests >>>>>>>>> Completed 200 requests >>>>>>>>> Completed 300 requests >>>>>>>>> Completed 400 requests >>>>>>>>> Completed 500 requests >>>>>>>>> Completed 600 requests >>>>>>>>> Completed 700 requests >>>>>>>>> Completed 800 requests >>>>>>>>> Completed 900 requests >>>>>>>>> Completed 1000 requests >>>>>>>>> Finished 1000 requests >>>>>>>>> >>>>>>>>> >>>>>>>>> Server Software: OrientDB >>>>>>>>> Server Hostname: 127.0.0.1 >>>>>>>>> Server Port: 2480 >>>>>>>>> >>>>>>>>> Document Path: /document/demo/9:1 >>>>>>>>> Document Length: 123 bytes >>>>>>>>> >>>>>>>>> Concurrency Level: 2 >>>>>>>>> Time taken for tests: 17.209 seconds >>>>>>>>> Complete requests: 1000 >>>>>>>>> Failed requests: 0 >>>>>>>>> Keep-Alive requests: 1000 >>>>>>>>> Total transferred: 486503 bytes >>>>>>>>> Total body sent: 320000 >>>>>>>>> HTML transferred: 123000 bytes >>>>>>>>> Requests per second: * 58.11 *[#/sec] (mean) >>>>>>>>> Time per request: 34.419 [ms] (mean) >>>>>>>>> Time per request: 17.209 [ms] (mean, across all concurrent >>>>>>>>> requests) >>>>>>>>> Transfer rate: 27.61 [Kbytes/sec] received >>>>>>>>> 18.16 kb/s sent >>>>>>>>> 45.77 kb/s total >>>>>>>>> >>>>>>>>> Connection Times (ms) >>>>>>>>> min mean[+/-sd] median max >>>>>>>>> Connect: 0 0 0.3 0 10 >>>>>>>>> Processing: 11 34 26.9 31 487 >>>>>>>>> Waiting: 11 34 26.9 31 487 >>>>>>>>> Total: 11 34 27.1 31 490 >>>>>>>>> >>>>>>>>> Percentage of the requests served within a certain time (ms) >>>>>>>>> 50% 31 >>>>>>>>> 66% 36 >>>>>>>>> 75% 40 >>>>>>>>> 80% 43 >>>>>>>>> 90% 54 >>>>>>>>> 95% 63 >>>>>>>>> 98% 80 >>>>>>>>> 99% 93 >>>>>>>>> 100% 490 (longest request) >>>>>>>>> [proto]:root:/opt/orientdb/benchmarks # >>>>>>>>> ........... >>>>>>>>> >>>>>>>>> So deleting from the table took too much : >>>>>>>>> >>>>>>>>> ..... >>>>>>>>> orientdb {demo}> select count(*) from Devices; >>>>>>>>> >>>>>>>>> ----+------+----- >>>>>>>>> # |@RID |count >>>>>>>>> ----+------+----- >>>>>>>>> 0 |#-1:-1|8207 >>>>>>>>> ----+------+----- >>>>>>>>> >>>>>>>>> 1 item(s) found. Query executed in 0.017 sec(s). >>>>>>>>> orientdb {demo}> delete from Devices >>>>>>>>> >>>>>>>>> Delete record(s)* '8261' in 30.190001 sec(s).* >>>>>>>>> >>>>>>>>> orientdb {demo}> >>>>>>>>> ........ >>>>>>>>> >>>>>>>>> >>>>>>>>> Rus >>>>>>>>> >>>>>>>>> понедельник, 12 мая 2014 г., 19:16:41 UTC+3 пользователь Lvc@написал: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> Can you run the server by trying a combination of these options >>>>>>>>>> (edit bin/server.sh, last line)? >>>>>>>>>> >>>>>>>>>> -Djna.disable.system.library=true >>>>>>>>>> -Dmemory.useUnsafe=false >>>>>>>>>> >>>>>>>>>> Lvc@ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Lvc@ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 12 May 2014 18:06, Sfinx <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Ok, for the Oracle java version ["1.7.0_55" Java(TM) SE Runtime >>>>>>>>>>> Environment (build 1.7.0_55-b13) Java HotSpot(TM) Client VM >>>>>>>>>>> (build 24.55-b03, mixed mode)] I have the following results : >>>>>>>>>>> >>>>>>>>>>> - graph-example-2.xml insert time is *154536ms *- nearly the >>>>>>>>>>> same as at 1.8.0 >>>>>>>>>>> - the fault error still present : >>>>>>>>>>> ....... >>>>>>>>>>> [proto]:root:/opt/orientdb/benchmarks # ab -n1000 -A >>>>>>>>>>> root:root_pass -k -c10 -p post.txt http://127.0.0.1:2480/ >>>>>>>>>>> document/demo/9:1 >>>>>>>>>>> This is ApacheBench, Version 2.3 <$Revision: 1528965 $> >>>>>>>>>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >>>>>>>>>>> http://www.zeustech.net/ >>>>>>>>>>> Licensed to The Apache Software Foundation, >>>>>>>>>>> http://www.apache.org/ >>>>>>>>>>> >>>>>>>>>>> Benchmarking 127.0.0.1 (be patient) >>>>>>>>>>> Completed 100 requests >>>>>>>>>>> apr_pollset_poll: The timeout specified has expired (70007) >>>>>>>>>>> Total of 124 requests completed >>>>>>>>>>> [proto]:root:/opt/orientdb/benchmarks # >>>>>>>>>>> ........ >>>>>>>>>>> 2014-05-12 15:56:42:308 INFO OrientDB Server v1.7-SNAPSHOT is >>>>>>>>>>> active. [OServer] >>>>>>>>>>> 2014-05-12 15:56:57:053 INFO - Rebuilding index >>>>>>>>>>> demo.dictionary... [OIndexRebuildOutputListener] >>>>>>>>>>> 2014-05-12 15:56:57:066 INFO --> OK, indexed 0 items in 14 ms >>>>>>>>>>> [OIndexRebuildOutputListener] >>>>>>>>>>> 2014-05-12 15:57:05:450 INFO Created database 'demo' of type >>>>>>>>>>> 'plocal' [ONetworkProtocolBinary] >>>>>>>>>>> 2014-05-12 15:58:40:951 SEVE Internal server error: >>>>>>>>>>> java.lang.InternalError: a fault occurred in a recent unsafe >>>>>>>>>>> memory access operation in compiled Java code >>>>>>>>>>> [ONetworkProtocolHttpDb] >>>>>>>>>>> ........ >>>>>>>>>>> >>>>>>>>>>> I see that fault error happens sporadically even when using low >>>>>>>>>>> "ab -n" numbers, but using high (>= 1000) will hang the server >>>>>>>>>>> immediatly. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Rus >>>>>>>>>>> >>>>>>>>>>> понедельник, 12 мая 2014 г., 18:41:27 UTC+3 пользователь >>>>>>>>>>> Lvc@написал: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> That errors seems caused by JVM as internal... To understand if >>>>>>>>>>>> it's Java8 or new snapshot, can you run new 1.7-snapshot against >>>>>>>>>>>> JDK7? >>>>>>>>>>>> >>>>>>>>>>>> Lvc@ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Lvc@ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 12 May 2014 17:33, Sfinx <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> the fault message never appears if I use "ab -n 100". using >>>>>>>>>>>>> "ab -n 200" and higher immediatly triggers server hang. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> ....... >>>>>>>>>>>>>> ab -n1000 -A root:root_pass -k -c2 -p post.txt >>>>>>>>>>>>>> http://127.0.0.1:2480/document/demo/9:1 >>>>>>>>>>>>>> This is ApacheBench, Version 2.3 <$Revision: 1528965 $> >>>>>>>>>>>>>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >>>>>>>>>>>>>> http://www.zeustech.net/ >>>>>>>>>>>>>> Licensed to The Apache Software Foundation, >>>>>>>>>>>>>> http://www.apache.org/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Benchmarking 127.0.0.1 (be patient) >>>>>>>>>>>>>> Completed 100 requests >>>>>>>>>>>>>> apr_pollset_poll: The timeout specified has expired (70007) >>>>>>>>>>>>>> Total of 184 requests completed >>>>>>>>>>>>>> ...... >>>>>>>>>>>>>> 2014-05-12 14:56:14:208 SEVE Internal server error: >>>>>>>>>>>>>> java.lang.InternalError: a fault occurred in a recent unsafe >>>>>>>>>>>>>> memory access operation in compiled Java code >>>>>>>>>>>>>> [ONetworkProtocolHttpDb] >>>>>>>>>>>>>> ...... >>>>>>>>>>>>>> >>>>>>>>>>>>>> The OrientDB server have been hunged, so restart needed. >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> --- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "OrientDB" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>> >>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> --- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "OrientDB" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> >>>>>>>>> --- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "OrientDB" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>> >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OrientDB" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Andrey Lomakin. >>>>> >>>>> Orient Technologies >>>>> the Company behind OrientDB >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrey Lomakin. >>>> >>>> Orient Technologies >>>> the Company behind OrientDB >>>> >>>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
