Hi Larry, I am so sorry, it was a little bit busy in the office today and than I had to leave it. Now it is late evening and I am at home. Tomorrow morning the first thing I will do when I am back in the office is to sent You the zipped shape file.
Thank You and all others for Your support!! Kindly regards Arnd -------- Original-Nachricht -------- > Datum: Thu, 4 Oct 2007 13:51:41 -0500 > Von: "Larry Becker" <[EMAIL PROTECTED]> > An: "JUMP Users Discussion" <[email protected]> > Betreff: Re: [jump-users] Better Performence for point data > OK, for my PC (P4 3.2Ghz with OJ set to use 256MB) it takes 5 seconds to > load your 351k points u1 layer, and 17 seconds to draw. Pretty bad draw > performance compared with other geometry types, but not really relevant to > the discussion since waiting for the redraw is not necessary. > > 1. I managed to copy and paste about 100k of these to a new layer in about > 10 seconds. Replicate gives similar results. > > 2. Selecting about 1/3 of the points takes about 5 seconds to do, and > about > 8 more to render the handles, although as I mentioned before, you can > proceed as soon as the the status bar indicates the number selected. > > 3. Selecting all points takes about 20 seconds, and another 30 to render. > > Preliminary results for 350000 points: slow but definitely workable. > > I loaded 3 copies of the u1 layer giving a total of over 1 million points. > Memory usage is at 200 MB committed. I suspect it would be quite a bit > more > if Stefan's data had any attributes. > > 4. Redraw is now about 45 seconds, but just ignoring it works for me. > > 5. Selecting about 1/3 failed after 90 seconds and resulted in an Out of > Memory Error. > > After allocating 512MB of memory to OJ, I restarted. 390000 points > selected > after 20 seconds. The selection handles finished rendering 40 seconds > later. 388MB in use. > > 6. Right clicking on the selection takes about 5 seconds to bring up the > menu. > > 7. Replicate took about 20 seconds to replicate to a new layer. > > These are my current findings. I'll update the post when I test with > Arnd's > data. So far I don't see anything too bad. About what you might expect > really. The rendering code is not optimized for points, and the selection > code is dealing with its worst case since the bounding box for points is a > point. > > Larry > On 10/4/07, Stefan Steiniger <[EMAIL PROTECTED]> wrote: > > > > oha.. > > that is interesting. Accidentaly i stored the points as multipoints. > > Now i updated my jump @office and stored the points as point (and wrote > > the file again to the ftp). The file size is now already a bit smaller, > > and also the commited memory... > > > > stefan > > > > Stefan Steiniger wrote: > > > > > Good if Arnd provides the data, but here the link to my data in case > > > it fails: > > > ftp://ftp.geo.uzh.ch/pub/sstein/data/u1_points.zip > > > > > > an option to get one million points is to copy the 350k points and > > > then to move them (if that works ;). > > > Btw. the lascers can data have no attributes (may be therefore smaller > > > than Arnd's; the covered area is relatively small < 200m) > > > (shapefile size is 30mb and zip xyz-ascii was 15mb) > > > > > > stefan > > > > > > Arnd Kielhorn wrote: > > > > > >> Hello Larry, > > >> > > >> first of all, thank You very much! > > >> > > >> I work with delaunay triangulation and building a 3D-Model with our > > >> plugin. > > >> > > >> If we both could make a direct connection (via FTP or a free tool > > >> called teamviewer) I could sent You one file with laserscanning data > > >> with 1 million points (XYZ). It is about 30 MB in the PIROL-CSV > > >> format or about 100 MB as shape file. > > >> > > >> Kindly regards > > >> Arnd > > >> > > >> Larry Becker schrieb: > > >> > > >>> I'm going to start investigating the point performance problem as > > >>> soon as I get some test data. I have been trying to locate a large > > >>> point dataset on the web, but no luck. Perhaps the best approach > > >>> might be to add a new item to the Generate menu, "Random Points". > > >>> > > >>> @Arnd: One thing that people often forget about JUMP is that > > >>> rendering is a background process. You don't have to wait for it > > >>> to finish to start the next operation. > > >>> What kind of terrain maps are you generating? Are you generating > > >>> contour lines from a grid of points? You mentioned deleting > > >>> selected points being impossibly slow. I don't see why this should > > >>> be so. If I can duplicate the problem, it should be an easy fix. > > >>> > > >>> regards, > > >>> Larry > > >>> > > >>> On 10/2/07, *Larry Becker* <[EMAIL PROTECTED] > > >>> <mailto:[EMAIL PROTECTED]>> wrote: > > >>> > > >>> HI Arnd, > > >>> > > >>> Would it be helpful to work around the problem by saving the > > >>> points as a shapefile and using a utility like shp2tile to break > > >>> the point data up into multiple files? > > >>> > > >>> see: http://imaptools.com/download-software.html > > >>> > > >>> regards, > > >>> Larry > > >>> > > >>> > > >>> On 10/2/07, *Arnd Kielhorn* < [EMAIL PROTECTED] > > >>> <mailto:[EMAIL PROTECTED]>> wrote: > > >>> > > >>> Hello Paul, > > >>> > > >>> thank You for the tip with the z-value, but how I can use it > > >>> as a > > >>> z-value on the point itself? > > >>> > > >>> The file fomrat is txt, csv (from PIROL CSV-Format) like > > >>> following example > > >>> $Rechts Hoch heigh > > >>> $grad grad m > > >>> $double double double > > >>> 3434000.00 5800000.00 93.78 > > >>> 3434001.00 5800000.00 93.84 > > >>> ... ... ... > > >>> > > >>> or the ESRI ascii grid format asc (reading plugin from > SIGLE). > > >>> The most files I got have 1 million points and such csv > files > > >>> have about > > >>> 30 MB. > > >>> In my start file for OJ I got 768 MB, because I have 1 GB > RAM. > > >>> > > >>> Kindly regards > > >>> Arnd > > >>> > > >>> > > >>> Paul Austin schrieb: > > >>> > Arnd, > > >>> > > > >>> > What file format are you using? > > >>> > > > >>> > As JUMP loads the whole file into memory I would recommend > > >>> setting the > > >>> > heap size on the JAVA VM to be -Xmx512M if you are dealing > > >>> with large > > >>> > datasets. > > >>> > > > >>> > You mentioned that the height is an attribute on the > > feature, > > >>> if you use > > >>> > the height as a z-value on the point itself there will be > > >>> some memory > > >>> > savings. > > >>> > > > >>> > Paul > > >>> > > > >>> > Arnd Kielhorn wrote: > > >>> > > > >>> >> Hello Larry, > > >>> >> > > >>> >> as in the further discussion just named the main problema > > >>> are (for > > >>> >> example I just have to work with point layers with only a > > >>> heigh > > >>> >> attribute but with 1 million points; but also e.g. > 300,000 > > >>> points are > > >>> >> enough): > > >>> >> - (re)drawing of the points > > >>> >> - strongly restricted or impossible operation (e.g. > > deleting > > >>> some > > >>> >> points/vertices after marking them; e.g. copying hundreds > > or > > >>> a few > > >>> >> thousand in a new layer to have layer with lesser points) > > >>> >> Very often OJ hangs on and I have to restart it. > > >>> >> I use such point layers for creating a digital terrain > > maps. > > >>> >> > > >>> >> Kindly regards > > >>> >> Arnd > > >>> >> > > >>> >> Larry Becker schrieb: > > >>> >> > > >>> >>> Hi Arnd, > > >>> >>> > > >>> >>> I haven't worked with large point datasets much. What > > >>> it is > > >>> >>> exactly that is slow? Is it redraw time? Or perhaps > the > > >>> particular > > >>> >>> operation that you are doing? > > >>> >>> > > >>> >>> I know that point layers do not benefit from the > recent > > >>> decimation > > >>> >>> optimization, so they take longer to draw than > equivalent > > >>> polygons > > >>> >>> and polyline layers. I think Michaël tried a point > > >>> decimation > > >>> >>> optimization, but wasn't satisfied with the results. > > >>> >>> > > >>> >>> regards, > > >>> >>> Larry Becker > > >>> >>> > > >>> >>> On 10/1/07, *Stefan Steiniger* < [EMAIL PROTECTED] > > >>> <mailto:[EMAIL PROTECTED]> > > >>> >>> <mailto: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> > > >>> wrote: > > >>> >>> > > >>> >>> mhm > > >>> >>> from my point of view it is both OJ and JRE. It > would > > >>> be probably > > >>> >>> better > > >>> >>> when the points are stored and processes in a > > database. > > >>> >>> > > >>> >>> stefan > > >>> >>> > > >>> >>> Arnd Kielhorn schrieb: > > >>> >>> > Hello, > > >>> >>> > > > >>> >>> > yes I know, first of all OJ is a GIS for vector > > data. > > >>> But the > > >>> >>> > functionality is also very good for point > datasets. > > >>> But working > > >>> >>> with > > >>> >>> > great point datasets (more 500,000 points) bring > > >>> problems with the > > >>> >>> > memory and the garbage collector works not > > satisfied. > > >>> Every > > >>> >>> command take > > >>> >>> > a lot of time or could not carried out. (CPU P4 > 2.4 > > >>> GHz, 1 GB RAM) > > >>> >>> > Is it only a thing of the JRE or also from OJ? > > >>> >>> > So, it is useful to know if there is a chance to > > >>> increase the > > >>> >>> > performence of OJ in this case and what could be > the > > >>> possible > > >>> >>> milestones > > >>> >>> > to reach this aim? > > >>> >>> > I am very eager to read everyones opion. > > >>> >>> > > > >>> >>> > Kindly regards > > >>> >>> > Arnd > > >>> >>> > _______________________________________________ > > >>> >>> > jump-users mailing list > > >>> >>> > [email protected] > > >>> <mailto:[email protected]> > > >>> >>> <mailto: [email protected] > > >>> <mailto:[email protected]>> > > >>> >>> > > > >>> http://lists.refractions.net/mailman/listinfo/jump-users > > >>> <http://lists.refractions.net/mailman/listinfo/jump-users> > > >>> >>> > > > >>> > > >>> <http://amusingprogrammer.blogspot.com/> > > >>> > > ------------------------------------------------------------------------ > > >>> > > >>> > > >>> _______________________________________________ > > >>> jump-users mailing list > > >>> [email protected] > > >>> http://lists.refractions.net/mailman/listinfo/jump-users > > >>> > > >> > > >> > > >> > > >> _______________________________________________ > > >> jump-users mailing list > > >> [email protected] > > >> http://lists.refractions.net/mailman/listinfo/jump-users > > >> > > >> > > > > > > _______________________________________________ > > > jump-users mailing list > > > [email protected] > > > http://lists.refractions.net/mailman/listinfo/jump-users > > > > > > > > > > _______________________________________________ > > jump-users mailing list > > [email protected] > > http://lists.refractions.net/mailman/listinfo/jump-users > > > > > > -- > http://amusingprogrammer.blogspot.com/ -- GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS. Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail _______________________________________________ jump-users mailing list [email protected] http://lists.refractions.net/mailman/listinfo/jump-users
