[Geotools-devel] ConcurrentModificationException on MemoryDataStore Layer

[Frank Gasdorf]

Hi folks,

Sorry for cross-posting, I'm unsure which forum is the right choise to
address this

I'm wondering about the current behavior adding features programatically to
a layer. The layer has a MemoryGeoResourceImpl internally which uses
MemoryDataStore (geotools 11.2 & uDig 2.0.0-SNAPSHOT).

I expected to get features stored to DataStore while uDig is (still)
rendering features already added. But the during rendering
ConcurrentModificatinExtepion's are thrown while writing new features or
modifying exiting.


!ENTRY org.locationtech.udig.project 2 0 2015-09-07 16:06:32.517
!MESSAGE class java.util.ConcurrentModificationException occured during
rendering: null
!STACK 0
org.locationtech.udig.project.render.RenderException: class
java.util.ConcurrentModificationException occured during rendering: null
at
org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:489)
at
org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:320)
at
org.locationtech.udig.project.internal.render.impl.RenderJob.startRendering(RenderJob.java:117)
at
org.locationtech.udig.project.internal.render.impl.RenderJob.run(RenderJob.java:222)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.util.ConcurrentModificationException
at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source)
at java.util.LinkedHashMap$ValueIterator.next(Unknown Source)
at
org.geotools.data.memory.MemoryDataStore.getBounds(MemoryDataStore.java:609)
at
org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:318)
at
org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:286)
at
org.locationtech.udig.catalog.memory.internal.MemoryGeoResourceImpl$ScratchResourceInfo.getBounds(MemoryGeoResourceImpl.java:167)
at
org.locationtech.udig.project.internal.impl.GeoResourceInfoInterceptor$Wrapper.getBounds(GeoResourceInfoInterceptor.java:63)
at
org.locationtech.udig.project.internal.impl.LayerImpl.obtainBoundsFromResources(LayerImpl.java:2173)
at
org.locationtech.udig.project.internal.impl.LayerImpl.getBounds(LayerImpl.java:2144)
at
org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.validateBounds(BasicFeatureRenderer.java:567)
at
org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:363)
... 4 more

>From a "user" perspective its quite hard to synchchronize access to the
datastore while there are different extensions accessing the layer
(getBounds, Iterator for Renderer, etc.) and its resources from uDig. I
remember a discussion a few years ago about the same problem but cannot
find the thread anymore.

Q: What's the best aproach to write features to MemoryDataStore while other
having read-access in between? Would it be a good approach using a
ConcurrentHashMap internally or using a different DataStore (e.g. h2)?

Thanks in advance

-- Frank
--
___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Geotools-devel] Reminder: GeoTools / GeoServer Meeting at 19:30 UTC on Tuesday

[Ben Caradoc-Davies]

GeoTools / GeoServer committee meeting on Skype at 19:30 UTC on Tuesday:
http://www.timeanddate.com/worldclock/fixedtime.html?msg=GeoTools+%2F+GeoServer+Meeting&year=2015&month=9&day=8&hour=19&min=30&sec=0&ah=1

-- 
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

--
___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Re: [Geotools-devel] [udig-dev] ConcurrentModificationException on MemoryDataStore Layer

[Jody Garnett]

In general we try and have a Transaction to keep seperate threads
independent, but in this case we are both rendering from (and writing) to
the same transaction.

I would not put any time into debugging this where you are at, The GeoTools
14.x series has been reviewed and approved by Eclipse IP process. This
branch has migrated MemoryDataStore to ContendDataStore - so we are up
against a new thing to test against.

As part of the migration Torben was able to poke a few holes in the Diff
data structure used to keep transactions independent. I would expect that
would be the correct location to resolve any concurrent modification
problems.
--
Jody


--
Jody Garnett

On 7 September 2015 at 12:23, Frank Gasdorf 
wrote:

> Hi folks,
>
> Sorry for cross-posting, I'm unsure which forum is the right choise to
> address this
>
> I'm wondering about the current behavior adding features programatically
> to a layer. The layer has a MemoryGeoResourceImpl internally which uses
> MemoryDataStore (geotools 11.2 & uDig 2.0.0-SNAPSHOT).
>
> I expected to get features stored to DataStore while uDig is (still)
> rendering features already added. But the during rendering
> ConcurrentModificatinExtepion's are thrown while writing new features or
> modifying exiting.
>
>
> !ENTRY org.locationtech.udig.project 2 0 2015-09-07 16:06:32.517
> !MESSAGE class java.util.ConcurrentModificationException occured during
> rendering: null
> !STACK 0
> org.locationtech.udig.project.render.RenderException: class
> java.util.ConcurrentModificationException occured during rendering: null
> at
> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:489)
> at
> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:320)
> at
> org.locationtech.udig.project.internal.render.impl.RenderJob.startRendering(RenderJob.java:117)
> at
> org.locationtech.udig.project.internal.render.impl.RenderJob.run(RenderJob.java:222)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
> Caused by: java.util.ConcurrentModificationException
> at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source)
> at java.util.LinkedHashMap$ValueIterator.next(Unknown Source)
> at
> org.geotools.data.memory.MemoryDataStore.getBounds(MemoryDataStore.java:609)
> at
> org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:318)
> at
> org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:286)
> at
> org.locationtech.udig.catalog.memory.internal.MemoryGeoResourceImpl$ScratchResourceInfo.getBounds(MemoryGeoResourceImpl.java:167)
> at
> org.locationtech.udig.project.internal.impl.GeoResourceInfoInterceptor$Wrapper.getBounds(GeoResourceInfoInterceptor.java:63)
> at
> org.locationtech.udig.project.internal.impl.LayerImpl.obtainBoundsFromResources(LayerImpl.java:2173)
> at
> org.locationtech.udig.project.internal.impl.LayerImpl.getBounds(LayerImpl.java:2144)
> at
> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.validateBounds(BasicFeatureRenderer.java:567)
> at
> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:363)
> ... 4 more
>
> From a "user" perspective its quite hard to synchchronize access to the
> datastore while there are different extensions accessing the layer
> (getBounds, Iterator for Renderer, etc.) and its resources from uDig. I
> remember a discussion a few years ago about the same problem but cannot
> find the thread anymore.
>
> Q: What's the best aproach to write features to MemoryDataStore while
> other having read-access in between? Would it be a good approach using a
> ConcurrentHashMap internally or using a different DataStore (e.g. h2)?
>
> Thanks in advance
>
> -- Frank
>
> ___
> udig-dev mailing list
> udig-...@locationtech.org
> To change your delivery options, retrieve your password, or unsubscribe
> from this list, visit
> https://locationtech.org/mailman/listinfo/udig-dev
>
--
___
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Re: [Geotools-devel] [udig-dev] ConcurrentModificationException on MemoryDataStore Layer

[Frank Gasdorf]

Good to hear gt 14.x has been approved. Sounds like a major upgrade (from
11.2 right now) for uDig with some implications for Implementations. At the
moment I'm upgrating a 1.2 Application to uDig 2.0 which is not straight
forward too.

Tomorrow I'm goint to try 14.x MemoryDataStore in a Test-Setup to verify
whether it would work. My first impression after I reviewed
https://github.com/geotools/geotools/blob/aa0d0346e4bb4a5a209c22d2b12ded7cbd20/modules/library/data/src/main/java/org/geotools/data/memory/MemoryDataStore.java#L314
is:
* readers and writers are synchronized on internal map of featureTypes and
Map of FeatureIds and Features (memory)
* please correct me, but if using accessing Features for FeatureType is not
synchronized which would lead into Concurrent Modification Exceptions
(memory.get(FeatureType).iterator())

However, I try to prepare a test scenario and let you know

- Frank


2015-09-07 21:36 GMT+02:00 Jody Garnett :

> In general we try and have a Transaction to keep seperate threads
> independent, but in this case we are both rendering from (and writing) to
> the same transaction.
>
> I would not put any time into debugging this where you are at, The
> GeoTools 14.x series has been reviewed and approved by Eclipse IP process.
> This branch has migrated MemoryDataStore to ContendDataStore - so we are up
> against a new thing to test against.
>
> As part of the migration Torben was able to poke a few holes in the Diff
> data structure used to keep transactions independent. I would expect that
> would be the correct location to resolve any concurrent modification
> problems.
> --
> Jody
>
>
> --
> Jody Garnett
>
> On 7 September 2015 at 12:23, Frank Gasdorf 
> wrote:
>
>> Hi folks,
>>
>> Sorry for cross-posting, I'm unsure which forum is the right choise to
>> address this
>>
>> I'm wondering about the current behavior adding features programatically
>> to a layer. The layer has a MemoryGeoResourceImpl internally which uses
>> MemoryDataStore (geotools 11.2 & uDig 2.0.0-SNAPSHOT).
>>
>> I expected to get features stored to DataStore while uDig is (still)
>> rendering features already added. But the during rendering
>> ConcurrentModificatinExtepion's are thrown while writing new features or
>> modifying exiting.
>>
>>
>> !ENTRY org.locationtech.udig.project 2 0 2015-09-07 16:06:32.517
>> !MESSAGE class java.util.ConcurrentModificationException occured during
>> rendering: null
>> !STACK 0
>> org.locationtech.udig.project.render.RenderException: class
>> java.util.ConcurrentModificationException occured during rendering: null
>> at
>> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:489)
>> at
>> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:320)
>> at
>> org.locationtech.udig.project.internal.render.impl.RenderJob.startRendering(RenderJob.java:117)
>> at
>> org.locationtech.udig.project.internal.render.impl.RenderJob.run(RenderJob.java:222)
>> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
>> Caused by: java.util.ConcurrentModificationException
>> at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown
>> Source)
>> at java.util.LinkedHashMap$ValueIterator.next(Unknown Source)
>> at
>> org.geotools.data.memory.MemoryDataStore.getBounds(MemoryDataStore.java:609)
>> at
>> org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:318)
>> at
>> org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:286)
>> at
>> org.locationtech.udig.catalog.memory.internal.MemoryGeoResourceImpl$ScratchResourceInfo.getBounds(MemoryGeoResourceImpl.java:167)
>> at
>> org.locationtech.udig.project.internal.impl.GeoResourceInfoInterceptor$Wrapper.getBounds(GeoResourceInfoInterceptor.java:63)
>> at
>> org.locationtech.udig.project.internal.impl.LayerImpl.obtainBoundsFromResources(LayerImpl.java:2173)
>> at
>> org.locationtech.udig.project.internal.impl.LayerImpl.getBounds(LayerImpl.java:2144)
>> at
>> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.validateBounds(BasicFeatureRenderer.java:567)
>> at
>> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:363)
>> ... 4 more
>>
>> From a "user" perspective its quite hard to synchchronize access to the
>> datastore while there are different extensions accessing the layer
>> (getBounds, Iterator for Renderer, etc.) and its resources from uDig. I
>> remember a discussion a few years ago about the same problem but cannot
>> find the thread anymore.
>>
>> Q: What's the best aproach to write features to MemoryDataStore while
>> other having read-access in between? Would it be a good approach using a
>> ConcurrentHashMap internally or using a different DataStore (e.g. h2)?
>>
>> Thanks in advance
>>
>> -- Frank
>>
>> _

Re: [Geotools-devel] [udig-dev] ConcurrentModificationException on MemoryDataStore Layer

[Jody Garnett]

Not that major an upgrade, but yeah finally putting AbstractDataStore
classes to bed. I am still waiting for some downstream projects to check in
(uDig and GeoMesa) before deleting these classes.

Trodden may be able to answer specific questions about the ContentDataStore
implementation of MemoryDataStore (but North America is on holiday today).

I did not expect to find that features( typeName ) method. It goes a bit
against the design of ContentDataStore - but I expect it was done in order
to have a more direct port from the AbstractDataStore original.

What would be better? I would of done a MemoryContentEntry (already
arranged in a Map by type name) each one of which containing its
FeatureCollection. This same approach (subclass ContentEntry and
ContentState) is what is done by JDBCDataStore and the ProperyDataStore
tutorial.

As it is I think you have found your concurrent modification error, and we
have a couple days left before the beta closes to fix it. Even if we
keep/modify that Map we would need to document how it should be locked with
respect to ContentState
--
Jody Garnett

On 7 September 2015 at 13:33, Frank Gasdorf 
wrote:

> Good to hear gt 14.x has been approved. Sounds like a major upgrade (from
> 11.2 right now) for uDig with some implications for Implementations. At the
> moment I'm upgrating a 1.2 Application to uDig 2.0 which is not straight
> forward too.
>
> Tomorrow I'm goint to try 14.x MemoryDataStore in a Test-Setup to verify
> whether it would work. My first impression after I reviewed
> https://github.com/geotools/geotools/blob/aa0d0346e4bb4a5a209c22d2b12ded7cbd20/modules/library/data/src/main/java/org/geotools/data/memory/MemoryDataStore.java#L314
> is:
> * readers and writers are synchronized on internal map of featureTypes and
> Map of FeatureIds and Features (memory)
> * please correct me, but if using accessing Features for FeatureType is
> not synchronized which would lead into Concurrent Modification Exceptions
> (memory.get(FeatureType).iterator())
>
> However, I try to prepare a test scenario and let you know
>
> - Frank
>
>
> 2015-09-07 21:36 GMT+02:00 Jody Garnett :
>
>> In general we try and have a Transaction to keep seperate threads
>> independent, but in this case we are both rendering from (and writing) to
>> the same transaction.
>>
>> I would not put any time into debugging this where you are at, The
>> GeoTools 14.x series has been reviewed and approved by Eclipse IP process.
>> This branch has migrated MemoryDataStore to ContendDataStore - so we are up
>> against a new thing to test against.
>>
>> As part of the migration Torben was able to poke a few holes in the Diff
>> data structure used to keep transactions independent. I would expect that
>> would be the correct location to resolve any concurrent modification
>> problems.
>> --
>> Jody
>>
>>
>> --
>> Jody Garnett
>>
>> On 7 September 2015 at 12:23, Frank Gasdorf 
>> wrote:
>>
>>> Hi folks,
>>>
>>> Sorry for cross-posting, I'm unsure which forum is the right choise to
>>> address this
>>>
>>> I'm wondering about the current behavior adding features programatically
>>> to a layer. The layer has a MemoryGeoResourceImpl internally which uses
>>> MemoryDataStore (geotools 11.2 & uDig 2.0.0-SNAPSHOT).
>>>
>>> I expected to get features stored to DataStore while uDig is (still)
>>> rendering features already added. But the during rendering
>>> ConcurrentModificatinExtepion's are thrown while writing new features or
>>> modifying exiting.
>>>
>>>
>>> !ENTRY org.locationtech.udig.project 2 0 2015-09-07 16:06:32.517
>>> !MESSAGE class java.util.ConcurrentModificationException occured during
>>> rendering: null
>>> !STACK 0
>>> org.locationtech.udig.project.render.RenderException: class
>>> java.util.ConcurrentModificationException occured during rendering: null
>>> at
>>> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:489)
>>> at
>>> org.locationtech.udig.render.internal.feature.basic.BasicFeatureRenderer.render(BasicFeatureRenderer.java:320)
>>> at
>>> org.locationtech.udig.project.internal.render.impl.RenderJob.startRendering(RenderJob.java:117)
>>> at
>>> org.locationtech.udig.project.internal.render.impl.RenderJob.run(RenderJob.java:222)
>>> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
>>> Caused by: java.util.ConcurrentModificationException
>>> at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown
>>> Source)
>>> at java.util.LinkedHashMap$ValueIterator.next(Unknown Source)
>>> at
>>> org.geotools.data.memory.MemoryDataStore.getBounds(MemoryDataStore.java:609)
>>> at
>>> org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:318)
>>> at
>>> org.geotools.data.AbstractFeatureSource.getBounds(AbstractFeatureSource.java:286)
>>> at
>>> org.locationtech.udig.catalog.memory.internal.MemoryGeoResourceImpl$ScratchResourceInfo.getBounds(MemoryGeoResourceImpl.java