Thanks for the catch, Ian.That logic error crept in while we were
experimenting with the documentation. We have fixed that logic error,
but the overall problem still exists in that we are getting a null
shape object which is causing the NullPointerException. This problem
seems to be independent of the validity of the layer. For example, we
have tried loading layers with invalid line objects (specifically,
lines consisting of only two points where both points were identical),
but this does not crash GeoTools.
However, data which checks out as being valid by GeoTools still causes
the NullPointerException intermittently.
As you can see in the method below (which we modified since Ian's
comment), we are checking to see if geom is null when we create a
layer. Although we are not finding any null geom's, geotools is still
crashing intermittently with the NullPointerException. Checking for
null or invalid geoms does seem to slow the system down enough to
limit the number of crashes from the likely race condition. However,
we still have not routed out the problem.
Are there any ideas concerning what could be causing this
NullPointerException? I will send a separate email in a few moments
which details how we were able to reproduce the same
NullPointerException with minor modifications to the Quickstart class
in the tutorials.
Thank you in advance for all of your help.
Marty
private boolean validateGeometryLayer(Layer layer) {
System.out.println("Validating Layer: " + layer.getTitle());
boolean isLayerValid = true;
boolean isnull = false;
boolean isinvalid = false;
FeatureIterator<?> iter;
try {
iter = layer.getFeatureSource().getFeatures().features();
int geom_count = 1;
while (iter.hasNext()) {
SimpleFeature feature = (SimpleFeature) iter.next();
Geometry geom = (Geometry) feature.getDefaultGeometry();
if (geom == null) {
isLayerValid = false;
System.out.println("--- Layer: " +
layer.getTitle() + " has NULL geometry: ");
}
else if (!geom.isValid()) {
isLayerValid = false;
System.out.println("--- Layer: " +
layer.getTitle() + " has INVALID geometry: ");
System.out.println(geom.toText());
}
}
iter.close();
} catch (IOException ex) {
Logger.getLogger(REPLANMapLayerTable.class.getName()).log(Level.SEVERE,
null, ex);
return false;
}
return isLayerValid;
}
On Sat, Feb 27, 2016 at 11:10 AM, Ian Turton <[email protected]> wrote:
> I'm on my phone so I might be reading this wrong, but the break in your loop
> looks like you are only checking the first geometry and then quitting. So
> all the other geometries could be invalid and you would never know.
>
> Ian
>
> On 27 Feb 2016 15:32, "Marty O'Neill II" <[email protected]> wrote:
>>
>> Jody,
>>
>> Thank you for your reply. Based on your suggestions, we have revisited
>> the tutorials and have changed the way we are creating our layers.
>> This did not fix the problem, so we have continued with your
>> suggestion of tracing the call that failed. We have described what we
>> have found below. It looks like we are getting a null shape object for
>> some reason which is causing the problems. We don’t know why we are
>> getting this null shape object since it tests as being !null and
>> valid. However, we can’t figure out why we are (only some of the time)
>> getting this null object variable. Can you please let us know if you
>> have any further suggestions?
>>
>> Creating and adding the layer to the map content intermittently causes
>> an error, but it works 90% of the time. The geometry on the database
>> is not invalid. The layer can be created from the exact same geometry
>> in other instances without problems.
>>
>> We create the layer as such:
>>
>> Layer layer = createLayer(String tableName, Style style, String name);
>> // layer class type = import org.geotools.map.Layer
>>
>> if (layer != null) {
>>
>> map.layers().add(layer); // map class type = import
>> org.geotools.map.MapContent
>>
>> }
>>
>>
>>
>> Where the "createLayer()" method is:
>>
>> public Layer createLayer(String tableName, Style style, String name) {
>>
>> Layer layer = null;
>>
>> try {
>>
>> Connection connection = PLANNER.getConnection();
>>
>>
>> if (pgDataStore == null) {
>>
>> Map params = PLANNER.getPostGIS_DB_Params(); // db
>> params: role, user, pass, port, dbname...
>>
>> pgDataStore = DataStoreFinder.getDataStore(params);
>>
>> }
>>
>>
>>
>> FeatureSource fs = pgDataStore.getFeatureSource(tableName);
>>
>>
>> layer = new FeatureLayer(fs, style, name);
>>
>>
>> if (validateGeometryLayer(layer) == false) {
>>
>> return null;
>>
>> }
>>
>> }
>>
>> return layer;
>>
>> }
>>
>>
>>
>> And our "validateGeometryLayer()" method is:
>>
>> public boolean validateGeometryLayer(Layer layer) {
>>
>> boolean isLayerValid = false;
>>
>> try {
>>
>> iter = layer.getFeatureSource().getFeatures().features();
>>
>> while (iter.hasNext()) {
>>
>> SimpleFeature feature = (SimpleFeature) iter.next();
>>
>> Geometry geom = (Geometry) feature.getDefaultGeometry();
>>
>> geom.isValid();
>>
>> if (geom != null && geom.isValid()) {
>>
>> isLayerValid = true;
>>
>> iter.close();
>>
>> break;
>>
>> }
>>
>> }
>>
>> if (!isLayerValid) {
>>
>> iter.close();
>>
>> }
>>
>> } catch (IOException ex)
>>
>> return false;
>>
>> }
>>
>> return isLayerValid;
>>
>> }
>>
>>
>>
>> Only at random times (e.g., usually with a slower network and larger
>> geometries), we are given this error:
>>
>> java.lang.NullPointerException
>>
>> at sun.java2d.pipe.LoopPipe.draw(LoopPipe.java:191)
>>
>> at
>> sun.java2d.pipe.PixelToParallelogramConverter.draw(PixelToParallelogramConverter.java:148)
>>
>> at sun.java2d.SunGraphics2D.draw(SunGraphics2D.java:2497)
>>
>> at
>> org.geotools.renderer.lite.StyledShapePainter.paint(StyledShapePainter.java:316)
>>
>> at
>> org.geotools.renderer.lite.StreamingRenderer$PaintShapeRequest.execute(StreamingRenderer.java:3264)
>>
>> at
>> org.geotools.renderer.lite.StreamingRenderer$PainterThread.run(StreamingRenderer.java:3525)
>>
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>> When tracing this error back, it states that in the paint() method of
>> StyledShapePainter class that the problem is in this line:
>>
>> graphics.draw(shape); // Line 316 in file
>> org.geotools.renderer.lite.StyledShapePainter -- Geotools Version 12.0
>>
>>
>> Where the "shape" object has the class type LiteShape2 passed from the
>> method params.
>>
>> If we further trace the error back, then we see it passed in the
>> constructor of the PaintShapeRequest class as shown below: // Lines
>> 3230-3269 in file org.geotools.renderer.lite.StreamingRenderer --
>> Geotools Version 12.0
>>
>> /**
>>
>> * A request to paint a shape with a specific Style2D
>>
>> * @author aaime
>>
>> *
>>
>> */
>>
>> class PaintShapeRequest extends RenderingRequest {
>>
>> Graphics2D graphic;
>>
>>
>>
>> LiteShape2 shape;
>>
>>
>> Style2D style;
>>
>>
>> double scale;
>>
>>
>> boolean labelObstacle = false;
>>
>>
>> public PaintShapeRequest(Graphics2D graphic, LiteShape2 shape,
>> Style2D style, double scale) {
>>
>> this.graphic = graphic;
>>
>> this.shape = shape;
>>
>> this.style = style;
>>
>> this.scale = scale;
>>
>> }
>>
>>
>> public void setLabelObstacle(boolean labelObstacle) {
>>
>> this.labelObstacle = labelObstacle;
>>
>> }
>>
>>
>> @Override
>>
>> void execute() {
>>
>> if(graphic instanceof DelayedBackbufferGraphic) {
>>
>> ((DelayedBackbufferGraphic) graphic).init();
>>
>> }
>>
>>
>>
>> try {
>>
>> painter.paint(graphic, shape, style, scale, labelObstacle);
>>
>> } catch(Throwable t) {
>>
>> fireErrorEvent(t);
>>
>> }
>>
>> }
>>
>> }
>>
>>
>> And this is called from the PainterThread class in its run() method.
>> // Lines 3502-3540 in file
>> org.geotools.renderer.lite.StreamingRenderer -- Geotools Version 12.0.
>>
>> So it would appear that even though we are using/creating the layers
>> correctly, and even confirming that the geometry is not null and is
>> valid before adding it to the MapContent, we are still getting the
>> null pointer exception. Again, this happens on layers that we have
>> previously created in the exact same way and process with the exact
>> same geometry and worked perfectly without any issue.
>>
>> Further, to validate the geometry, we used the code from this tutorial:
>> -
>> http://docs.geotools.org/stable/userguide/tutorial/geometry/geometrycrs.html
>>
>> And we even tried to add to use the "Hints" class to see if that would
>> help any:
>> Hints.putSystemDefault(RenderingHints.KEY_RENDERING,
>> RenderingHints.VALUE_RENDER_SPEED);
>>
>> In experimenting with the system, we also found that removing all
>> layers from the MapContent causes a similar (but different)
>> NullPointerException.
>>
>> Thank you in advance for all of your help. We truly appreciate any
>> advice you can give us.
>>
>> Sincerely,
>>
>> Marty
>>
>> On Mon, Feb 22, 2016 at 5:53 PM, Jody Garnett <[email protected]>
>> wrote:
>> > Hey Marty,
>> >
>> > Glad you are contacting us, kind of wish you did so earlier (incase you
>> > are
>> > looking at a race condition or something.). I did change the map/layer
>> > data
>> > structure a few years back.
>> >
>> > But still it looks like you are using things wrong... You are creating
>> > one
>> > connection from your controller, and then a connection pool (maybe 10
>> > connections) for each layer.
>> >
>> >
>> > 1. Create a single datastore - and keep it for use for each layer you
>> > want
>> > to publish.
>> > 2. Use feature source to figure out the count (if you want to skip empty
>> > layers)
>> > 3. Create a Map/Layer for the map you are trying to draw, reuse it
>> > multiple
>> > times if you can (say for different bounding boxes)
>> > 4. be sure to dispose layers that are not in use
>> > 5. be sure to dispose stores that are no longer in use (this returns the
>> > connections in their connection pool).
>> >
>> > If PLANNER.getController(). getConnection() has a connection pool (why
>> > wouldn't it) - consider setting up the DataStore using that connection
>> > pool
>> > to avoid burning through your connections.
>> >
>> > A lot of this is covered in the tutorials, please read through the
>> > tutorials
>> > and let me and the user-list know if you have question.
>> >
>> >
>> > As for your exception - here is LoopPipe.java
>> >
>> > 191 sg2d.loops.drawPathLoop.DrawPath(sg2d,
>> > sg2d.getSurfaceData(),
>> > 192 transX, transY,
>> > p2df);
>> >
>> > So your options for a NPE are:
>> > - sg2d
>> > - sg2d.loops
>> >
>> > sg2d is used a bit further up the method, so sg2d.loops must be null.
>> > Work
>> > back up the call chain trying to see how that could be null.
>> >
>> > Or if you want to cut to the chase assume your geometry might be invalid
>> > or
>> > otherwise harmed (since we are drawing a path and the path is determined
>> > by
>> > the shape of the geometry). So try checking the geometry in the table
>> > that
>> > is causing you trouble...
>> >
>> > --
>> > Jody Garnett
>> >
>> > On 19 February 2016 at 12:58, Marty O'Neill II <[email protected]>
>> > wrote:
>> >>
>> >> We have been using GeoTools to render maps in a software project we
>> >> have been working on for over five years. However, there is one
>> >> problem that has plagued us since day 1, and I'm wondering if anyone
>> >> on the list has any suggestions.
>> >>
>> >> We are currently using GeoTools 12.0, but the problems we are
>> >> experiencing have been around throughout the various versions for over
>> >> five years. However, as our project continues, the scale of the
>> >> regions we are examining is increasing, and the problem is becoming
>> >> more frequent.
>> >>
>> >> The spatial data we are rendering using GeoTools is stored on a
>> >> PostGIS-enabled Postgres DB. The software client we have been writing
>> >> connects directly to the DB to read the layers.
>> >>
>> >> The size of the data we are using isn't tremendous, but it isn't
>> >> exactly small, either. We often ask GeoTools to render maps drawn from
>> >> layers rendered from 100s of MB of spatial data from the DB. For
>> >> example, we may want to draw the entire state of Texas at the Block
>> >> Group Level along with OSM roads for the entire state and a few other
>> >> layers.
>> >>
>> >> We get a Null Pointer Exception message intermittently when we add
>> >> layers or when we toggle layers on and off using the user interface.
>> >> One way to trigger the problem is by loading a large dataset and
>> >> quickly toggling a layer on and off.
>> >>
>> >> The Exception seems to happen more often when the size of the layers
>> >> we are viewing in GeoTools are larger and/or when the network latency
>> >> is higher.
>> >>
>> >> When we zoom into an area of the map which is outside of the extent of
>> >> all of the layers (so that nothing is being rendered in the display)
>> >> before we add new layers, the Exception seems to be less likely.
>> >>
>> >> If we add multiple layers quickly, one after another, the problem
>> >> seems to be more likely.
>> >>
>> >> When we toggle off most or all of the layers before adding new layers,
>> >> the Exception seems to be less likely.
>> >>
>> >> When the latency between the DB and the client is lower, the Exception
>> >> seems to be less likely. For instance, if we are accessing the DB
>> >> across the internet, it seems to be more likely than if we are
>> >> accessing it across the LAN. Further, it seems to be more likely
>> >> across the LAN than if we are accessing a DB on localhost.
>> >>
>> >> Following occurrence of this exception, our program still runs, but
>> >> the GeoTools object stops rendering maps. Sometimes the map area
>> >> becomes all white. Other times, it takes on a fill of one of a color
>> >> from one of the line layers. If it takes on this fill, as soon
>> >> as a layer is toggled or the extent is changed, the map area becomes
>> >> all white, and nothing is rendered.
>> >>
>> >> I have included the Exception Message and the code we are using the
>> >> create the layers below.
>> >>
>> >>
>> >> Please let me know if you have any ideas about how to solve this
>> >> problem and what other information you may need.
>> >>
>> >>
>> >> Thank you,
>> >>
>> >> Marty
>> >>
>> >>
>> >> Exception Message:
>> >>
>> >> 2016-02-12T16:26:07.055-0600 SEVERE null
>> >> java.lang.NullPointerException
>> >> at sun.java2d.pipe.LoopPipe.draw(LoopPipe.java:191)
>> >> at
>> >>
>> >> sun.java2d.pipe.PixelToParallelogramConverter.draw(PixelToParallelogramConverter.java:148)
>> >> at sun.java2d.SunGraphics2D.draw(SunGraphics2D.java:2438)
>> >> at
>> >>
>> >> org.geotools.renderer.lite.StyledShapePainter.paint(StyledShapePainter.java:316)
>> >> at
>> >>
>> >> org.geotools.renderer.lite.StreamingRenderer$PaintShapeRequest.execute(StreamingRenderer.java:3264)
>> >> at
>> >>
>> >> org.geotools.renderer.lite.StreamingRenderer$PainterThread.run(StreamingRenderer.java:3525)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> >> at
>> >>
>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >>
>> >>
>> >>
>> >> This is how we are creating the layers:
>> >>
>> >> public Layer createLayer(String tableName, Style style, String name) {
>> >> Layer layer = null;
>> >>
>> >> try {
>> >> //the vacuum analyze removes the st_estimated_extent
>> >> error. good riddance.
>> >> Connection myconn =
>> >> PLANNER.getController().getConnection();
>> >> String query = "VACUUM ANALYZE " + UserState.userId + "."
>> >> + tableName + ";";
>> >> Statement stmt = myconn.createStatement();
>> >> stmt.executeUpdate(query);
>> >> Statement my_new_statement = myconn.createStatement();
>> >> String check_rows = "SELECT COUNT(*) as my_int FROM " +
>> >> UserState.userId + "." + tableName + ";";
>> >> ResultSet my_results =
>> >> my_new_statement.executeQuery(check_rows);
>> >> long my_int = 0;
>> >> while (my_results.next()) {
>> >> my_int = my_results.getInt("my_int");
>> >> }
>> >>
>> >> if (my_int == 0) {
>> >> //if 0 was returned that means the table is empty.
>> >> Don't create an empty layer.
>> >> return null;
>> >> }
>> >>
>> >> Map params = PLANNER.getController().getPostGIS();
>> >> params.remove("schema");
>> >> params.remove("role");
>> >> params.put("schema", UserState.userId);
>> >> params.put("role", UserState.userId);
>> >>
>> >> DataStore pgDatastore
>> >> = DataStoreFinder.getDataStore(params);
>> >>
>> >> FeatureSource fs = pgDatastore.getFeatureSource(tableName);
>> >> layer = new FeatureLayer(fs, style, name);
>> >> if (layer == null) {
>> >> System.out.println("Could not find stuff in database
>> >> to create layer.");
>> >> }
>> >> map.layers().add(layer);
>> >>
>> >> } catch (IOException ex) {
>> >> Logger.getLogger(ScenarioPanel.class.getName()).
>> >> log(Level.SEVERE, null, ex);
>> >> } catch (Exception e) {
>> >> System.out.println("This is probably: ERROR:
>> >> LWGEOM_estimated_extent: couldn't locate table within current schema
>> >> \n");
>> >> }
>> >>
>> >> return layer;
>> >> }
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> >> Monitor end-to-end web transactions and take corrective actions now
>> >> Troubleshoot faster and improve end-user experience. Signup Now!
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> >> _______________________________________________
>> >> GeoTools-GT2-Users mailing list
>> >> [email protected]
>> >> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>> >
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> _______________________________________________
>> GeoTools-GT2-Users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
