Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-10-01 Thread Lambertus
Chris Miller wrote: > If you're willing to experiment a little, one thing that might be worth > trying > is to switch to different GC algorithms to see if that helps stability: > > -XX:+UseConcMarkSweepGC > -XX:+UseParNewGC > -XX:+UseParallelGC > -XX:+UseParallelOldGC > -XX:+UseSerialGC > -XX:+U

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-30 Thread Chris Miller
L> It seems logic that different compilers generates different byte L> code, hence different behavior... Yes that definitely happens; for example I know the Eclipse compiler produces slightly different bytecode in some situations than javac. I'm just surprised that OpenJDK is producing different

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-29 Thread Lambertus
On Tue, 29 Sep 2009 15:53:58 + (UTC), Chris Miller wrote: > AS> make sure you use the same SDK for compiling mkgmap and executing. > AS> mixing makes it even worse. > > Really? Ouch... that shouldn't really be the case, Java bytecode is Java > bytecode and should be happy enough running on

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-29 Thread Apollinaris Schoell
On Tue, Sep 29, 2009 at 8:53 AM, Chris Miller wrote: > AS> make sure you use the same SDK for compiling mkgmap and executing. > AS> mixing makes it even worse. > > Really? Ouch... that shouldn't really be the case, Java bytecode is Java > bytecode and should be happy enough running on any VM regar

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-29 Thread Chris Miller
AS> make sure you use the same SDK for compiling mkgmap and executing. AS> mixing makes it even worse. Really? Ouch... that shouldn't really be the case, Java bytecode is Java bytecode and should be happy enough running on any VM regardless of where it was compiled (class version number limitati

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-29 Thread Steve Ratcliffe
On 29/09/09 07:42, Lambertus wrote: > I've removed openJDK and installed Sun Java RE (afaik 1.6.0-16) and the > problems got even worse: instead of only race situations the VM *also* > crashes randomly. This is tested with r1240 building only one map in > parallel. Oh dear! In this case if there

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-29 Thread Apollinaris Schoell
make sure you use the same SDK for compiling mkgmap and executing. mixing makes it even worse. On 28 Sep 2009, at 23:42 , Lambertus wrote: > Steve Ratcliffe wrote: >> >> I see from them that you are using the openjdk with Ubuntu, >> so I tried openjdk on my Fedora machine and didn't get a proble

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
Steve Ratcliffe wrote: > > I see from them that you are using the openjdk with Ubuntu, > so I tried openjdk on my Fedora machine and didn't get a problem but > then it is 32bit and has a higher build number so probably has > different (perhaps fewer ;) bugs. > > If you can I'd try the Sun jdk ins

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Apollinaris Schoell
> > Apollinaris: when you see a crash, does one of your VMs crash > completely > with an hs_err_pid.log file? Or is something else happening? > yes > Given what you've both said it unfortunately looks fairly likely the > problem > is JDK related rather than mkgmap. > definitely, mkgmap i

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Chris Miller
L> The box is a laptop with a single Intel P8600 CPU (dual core Penryn) L> and when the process hangs it's just 1 core that is at 100% (I use L> only one core so I can render America and Eurasia in parallel). Hmm ok, I thought it was a dual-CPU box based on the crash dump you posted. I guess I mi

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
Thanks for sharing this. Good to know that I don't have to buy a quadcore yet :-) Apollinaris Schoell wrote: > have seen frequent java crashes when I run more than one mkmap in > parallel. Even when the machine has 4 cores and plenty of memory. > it happens for openJDK and sun java. > when I ru

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Apollinaris Schoell
have seen frequent java crashes when I run more than one mkmap in parallel. Even when the machine has 4 cores and plenty of memory. it happens for openJDK and sun java. when I run ~400 tiles for north america it runs without crash, 2 parallel jobs are ok most of the time but 3 jobs is too much

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
The box is a laptop with a single Intel P8600 CPU (dual core Penryn) and when the process hangs it's just 1 core that is at 100% (I use only one core so I can render America and Eurasia in parallel). I've tried the --max-jobs=1 parameter a few times and got it to hang, stackdump (which seems i

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Chris Miller
Ah ok that makes more sense. A crash dump usually indicates the VM has a bug or has died 'violently' (as is the case here). A stack dump on the other hand is produced by the VM and shows you what Java code is currently executing. It looks like you are running on a dual CPU box (two dual-core Xeo

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
Addition: I killed the hanging process with kill -3 and got a stackdump this time. It's attached. Lambertus wrote: Well, you are probably right in that the Java VM itself in hanging: The process hung again and jstack reports: "Unable to open socket file: target process not responding or HotSpo

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
Well, you are probably right in that the Java VM itself in hanging: The process hung again and jstack reports: "Unable to open socket file: target process not responding or HotSpot VM not loaded". Forcing it to attach gave an error about HotSpot not being able to find a "Symbol". Anyway, I gues

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Steve Ratcliffe
On 28/09/09 15:09, Lambertus wrote: > The error files were produced using kill -4. Kill -1 till -3 had > no effect on the process. Just switching to using Sun Java is probably That would explain it then :) But -3 should have caused java to write out the stack traces onto standard output. If you

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
Thanks Steve, Chris, The error files were produced using kill -4 . Kill -1 till -3 had no effect on the process. Just switching to using Sun Java is probably the easy way out, but I reckon it's interesting to find out what happened :-), so next time I'll try jstack before killing the process.

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Steve Ratcliffe
On 28/09/09 13:53, Lambertus wrote: > It happened again, this time straight from the commandline while trying > to render the three failed tiles at once by providing r2228 the *.osm.gz > parameter... Stack dump is attached. Those hs_err_* files were not what I was expecting at all. You normally g

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Chris Miller
Hi Lambertus, What you have attached is a crash dump from the JVM, not a stack dump of the (running) Java application. How are you getting that file? If it's being generated as the result of you killing the process, then it's not useful. You need to instead try and get a stack dump output by fo

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
It happened again, this time straight from the commandline while trying to render the three failed tiles at once by providing r2228 the *.osm.gz parameter... Stack dump is attached. r1067 finishes the same test normally (with errors). Then I tried r2228 again a few times and it finishes normal

Re: [mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Steve Ratcliffe
Hi Lambertus On 28/09/09 08:36, Lambertus wrote: > While trying to get a new world map update out the door I noticed that > Mkgmap sometimes hangs on an OSM data file produced with Splitter. When > this happens the CPU is at 100% constantly for hours (e.g. all night) > until I kill the process. Th

[mkgmap-dev] Mkgmap appears to hang on some split files

2009-09-28 Thread Lambertus
While trying to get a new world map update out the door I noticed that Mkgmap sometimes hangs on an OSM data file produced with Splitter. When this happens the CPU is at 100% constantly for hours (e.g. all night) until I kill the process. The problem occurs in r1222 and also r1228. The followin