Re: [configuration] How to get threadsafe subset() config in a threadsafe way?
Excuse me, mistake on the stacktrace commented "NullPointerException on read of a configuration property" - that was a property setting, not getting; if we find a non-existent property, we add it with a default value. - Tim On Wed, Mar 14, 2012 at 2:48 PM, Tim Bessie wrote: > Surely - here's a few - these are all using commons-config 1.7. Don't > know if 1.8 would fix these; since they seemed linked to concurrency, > I doubt it, but perhaps you know? > > --- > > I attempted to avoid ConcurrentModificationExceptions by creating a > new HashMap from an existing SubsetConfiguration which I'd converted > to a ConfigurationMap. Unfortunately, the HashMap creation iterates > through the underlying Configuration objects, and generates this > error: > > [00:00:37.609]java.util.ConcurrentModificationException > [00:00:37.609] at > org.apache.commons.collections.map.AbstractLinkedMap$LinkIterator.nextEntry(AbstractLinkedMap.java:559) > [00:00:37.609] at > org.apache.commons.collections.map.AbstractLinkedMap$KeySetIterator.next(AbstractLinkedMap.java:459) > [00:00:37.609] at > org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) > [00:00:37.609] at > org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:93) > [00:00:37.609] at > org.apache.commons.configuration.CompositeConfiguration.getKeys(CompositeConfiguration.java:236) > [00:00:37.609] at > org.apache.commons.configuration.SubsetConfiguration.getKeys(SubsetConfiguration.java:196) > [00:00:37.609] at > org.apache.commons.configuration.ConfigurationMap$ConfigurationSet$ConfigurationSetIterator.(ConfigurationMap.java:158) > [00:00:37.609] at > org.apache.commons.configuration.ConfigurationMap$ConfigurationSet$ConfigurationSetIterator.(ConfigurationMap.java:151) > [00:00:37.609] at > org.apache.commons.configuration.ConfigurationMap$ConfigurationSet.iterator(ConfigurationMap.java:202) > [00:00:37.609] at java.util.HashMap.putAllForCreate(HashMap.java:536) > [00:00:37.609] at java.util.HashMap.(HashMap.java:269) > > --- > > NullPointerException on read of a configuration property. Digging > into it, it *looks* like there's a null key value in a Map, but we > don't put any null key values into our configuration. Could be > another error caused this to happen, or some concurrent modification > elsewhere: > > java.lang.NullPointerException > at > org.apache.commons.collections.map.AbstractLinkedMap.removeEntry(AbstractLinkedMap.java:292) > at > org.apache.commons.collections.map.AbstractHashedMap.removeMapping(AbstractHashedMap.java:542) > at > org.apache.commons.collections.map.AbstractHashedMap.remove(AbstractHashedMap.java:324) > at > org.apache.commons.configuration.BaseConfiguration.clearPropertyDirect(BaseConfiguration.java:133) > at > org.apache.commons.configuration.AbstractConfiguration.clearProperty(AbstractConfiguration.java:503) > at > org.apache.commons.configuration.CompositeConfiguration.clearPropertyDirect(CompositeConfiguration.java:269) > at > org.apache.commons.configuration.AbstractConfiguration.clearProperty(AbstractConfiguration.java:503) > at > org.apache.commons.configuration.AbstractConfiguration.setProperty(AbstractConfiguration.java:483) > at > com.dm.avatar.configuration.Configuration.getProperty(Configuration.java:880) > at > com.dm.avatar.configuration.Configuration.getProperty(Configuration.java:835) > at > com.dm.avatar.configuration.Configuration.getProperty(Configuration.java:830) > at com.dm.web.util.CommonService.getHostString(CommonService.java:795) > > --- > > Another NullPointerException case when wrapping a SubsetConfiguration > in a ConfigurationMap with a new HashMap: > > [08:05:18.787]java.lang.NullPointerException > [08:05:18.787] at > org.apache.commons.configuration.AbstractConfiguration$3.evaluate(AbstractConfiguration.java:577) > [08:05:18.787] at > org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:184) > [08:05:18.787] at > org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:93) > [08:05:18.787] at > org.apache.commons.configuration.CompositeConfiguration.getKeys(CompositeConfiguration.java:236) > [08:05:18.787] at > org.apache.commons.configuration.SubsetConfiguration.getKeys(SubsetConfiguration.java:196) > [08:05:18.787] at > org.apache.commons.configuration.ConfigurationMap$ConfigurationSet.size(ConfigurationMap.java:189) > [08:05:18.787] at java.util.AbstractMap.size(AbstractMap.java:67) > [08:05:18.787] at java.util.HashMap.(HashMap.java:267) > [08:05:18.787] at > com.dm.avatar.configuration.Configuration.getPropertiesAsMap(Configuration.java:607) > > > > > On Wed, Mar 14, 2012 at 1:24 PM, Oliver He
Re: [configuration] How to get threadsafe subset() config in a threadsafe way?
Surely - here's a few - these are all using commons-config 1.7. Don't know if 1.8 would fix these; since they seemed linked to concurrency, I doubt it, but perhaps you know? --- I attempted to avoid ConcurrentModificationExceptions by creating a new HashMap from an existing SubsetConfiguration which I'd converted to a ConfigurationMap. Unfortunately, the HashMap creation iterates through the underlying Configuration objects, and generates this error: [00:00:37.609]java.util.ConcurrentModificationException [00:00:37.609] at org.apache.commons.collections.map.AbstractLinkedMap$LinkIterator.nextEntry(AbstractLinkedMap.java:559) [00:00:37.609] at org.apache.commons.collections.map.AbstractLinkedMap$KeySetIterator.next(AbstractLinkedMap.java:459) [00:00:37.609] at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) [00:00:37.609] at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:93) [00:00:37.609] at org.apache.commons.configuration.CompositeConfiguration.getKeys(CompositeConfiguration.java:236) [00:00:37.609] at org.apache.commons.configuration.SubsetConfiguration.getKeys(SubsetConfiguration.java:196) [00:00:37.609] at org.apache.commons.configuration.ConfigurationMap$ConfigurationSet$ConfigurationSetIterator.(ConfigurationMap.java:158) [00:00:37.609] at org.apache.commons.configuration.ConfigurationMap$ConfigurationSet$ConfigurationSetIterator.(ConfigurationMap.java:151) [00:00:37.609] at org.apache.commons.configuration.ConfigurationMap$ConfigurationSet.iterator(ConfigurationMap.java:202) [00:00:37.609] at java.util.HashMap.putAllForCreate(HashMap.java:536) [00:00:37.609] at java.util.HashMap.(HashMap.java:269) --- NullPointerException on read of a configuration property. Digging into it, it *looks* like there's a null key value in a Map, but we don't put any null key values into our configuration. Could be another error caused this to happen, or some concurrent modification elsewhere: java.lang.NullPointerException at org.apache.commons.collections.map.AbstractLinkedMap.removeEntry(AbstractLinkedMap.java:292) at org.apache.commons.collections.map.AbstractHashedMap.removeMapping(AbstractHashedMap.java:542) at org.apache.commons.collections.map.AbstractHashedMap.remove(AbstractHashedMap.java:324) at org.apache.commons.configuration.BaseConfiguration.clearPropertyDirect(BaseConfiguration.java:133) at org.apache.commons.configuration.AbstractConfiguration.clearProperty(AbstractConfiguration.java:503) at org.apache.commons.configuration.CompositeConfiguration.clearPropertyDirect(CompositeConfiguration.java:269) at org.apache.commons.configuration.AbstractConfiguration.clearProperty(AbstractConfiguration.java:503) at org.apache.commons.configuration.AbstractConfiguration.setProperty(AbstractConfiguration.java:483) at com.dm.avatar.configuration.Configuration.getProperty(Configuration.java:880) at com.dm.avatar.configuration.Configuration.getProperty(Configuration.java:835) at com.dm.avatar.configuration.Configuration.getProperty(Configuration.java:830) at com.dm.web.util.CommonService.getHostString(CommonService.java:795) --- Another NullPointerException case when wrapping a SubsetConfiguration in a ConfigurationMap with a new HashMap: [08:05:18.787]java.lang.NullPointerException [08:05:18.787] at org.apache.commons.configuration.AbstractConfiguration$3.evaluate(AbstractConfiguration.java:577) [08:05:18.787] at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:184) [08:05:18.787] at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:93) [08:05:18.787] at org.apache.commons.configuration.CompositeConfiguration.getKeys(CompositeConfiguration.java:236) [08:05:18.787] at org.apache.commons.configuration.SubsetConfiguration.getKeys(SubsetConfiguration.java:196) [08:05:18.787] at org.apache.commons.configuration.ConfigurationMap$ConfigurationSet.size(ConfigurationMap.java:189) [08:05:18.787] at java.util.AbstractMap.size(AbstractMap.java:67) [08:05:18.787] at java.util.HashMap.(HashMap.java:267) [08:05:18.787] at com.dm.avatar.configuration.Configuration.getPropertiesAsMap(Configuration.java:607) On Wed, Mar 14, 2012 at 1:24 PM, Oliver Heger wrote: > > Am 13.03.2012 19:05, schrieb Tim Bessie: > >> That's a possibility which I'll look into. >> >> In the meantime, after initializing our main Configuration wrapper object >> with all of our static configuration data, I copy slices of it to static >> maps that I know will never change, and provide convenience accessors to >> those maps' data. Since the calls that access the data placed in these new >> maps were the only one
Re: [configuration] How to get threadsafe subset() config in a threadsafe way?
Am 13.03.2012 19:05, schrieb Tim Bessie: That's a possibility which I'll look into. In the meantime, after initializing our main Configuration wrapper object with all of our static configuration data, I copy slices of it to static maps that I know will never change, and provide convenience accessors to those maps' data. Since the calls that access the data placed in these new maps were the only ones asking for slices of config data, this eliminates our problem. I was just hoping for a more elegant solution. But at least this way there's no locks necessary. I'll take a look at your suggestions, however - thanks much! A nice feature in the future could be something like ConfigurationUtils.getSynchronizedInstance(Configuration), for example. I'm sure it's been discussed before. :-) - Tim You are right, support for concurrent access to Configuration objects can certainly be improved. Can you provide a stack trace of such a ConcurrentModificationException you receive occasionally? Thanks Oliver On Tue, Mar 13, 2012 at 1:43 AM, Luc Maisonobewrote: Le 13/03/2012 07:34, Tim Bessie a écrit : Hi all... Hi Tim, So we're keeping some config information CompositeConfiguration object, and we need to get subsets of this configuration data. When I call .subset(...), and then do some checks on the subset (isEmpty(), etc.), I sometimes get ConcurrentModificationExceptions. I'm not sure what's modifying the underlying configuration, although we do have occasional setting of configuration values throughout the running of our app. What would be *ideal* would be to: 1. NOT have to synchronize every access to the configuration object, since we have a high-volume application 2. To allow read and write operations to happen to the configuration object without worrying about ConcurrentModificationExceptions 3. To be able to call .subset(...) on the configuration object and 1) not risk a ConcurrentModificationException during this operation, and 2) get a COPY of the subset back, so that further operations on the subset don't risk ConcurrentModificationExceptions Does anyone know of a way to do this? Or is the ONLY way to guarantee lack of CMEs to synchronize EVERY access (reads, writes, subsets, iterations, etc.)? Perhaps you could try java.util.concurrent.locks.ReadWriteLock ? You would have to put the protection by yourself so it may require lots of code wrapping. One of the good thins is that it allows concurrent read (but when a write occurs, only one thread can write and reads are blocked). Luc If this is the case, how have others dealt with situations like ours, where you do mostly just reads on a Configuration object, but very occasional writes, and need to take subsets, and need to avoid exceptions while doing this? With a high volume app, synchronizing every access would slow things waay down, thus my question. - Tim - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [configuration] How to get threadsafe subset() config in a threadsafe way?
That's a possibility which I'll look into. In the meantime, after initializing our main Configuration wrapper object with all of our static configuration data, I copy slices of it to static maps that I know will never change, and provide convenience accessors to those maps' data. Since the calls that access the data placed in these new maps were the only ones asking for slices of config data, this eliminates our problem. I was just hoping for a more elegant solution. But at least this way there's no locks necessary. I'll take a look at your suggestions, however - thanks much! A nice feature in the future could be something like ConfigurationUtils.getSynchronizedInstance(Configuration), for example. I'm sure it's been discussed before. :-) - Tim On Tue, Mar 13, 2012 at 1:43 AM, Luc Maisonobe wrote: > Le 13/03/2012 07:34, Tim Bessie a écrit : > > Hi all... > > Hi Tim, > > > > > So we're keeping some config information CompositeConfiguration object, > and > > we need to get subsets of this configuration data. > > > > When I call .subset(...), and then do some checks on the subset > (isEmpty(), > > etc.), I sometimes get ConcurrentModificationExceptions. I'm not sure > > what's modifying the underlying configuration, although we do have > > occasional setting of configuration values throughout the running of our > > app. > > > > What would be *ideal* would be to: > > > > 1. NOT have to synchronize every access to the configuration object, > since > > we have a high-volume application > > 2. To allow read and write operations to happen to the configuration > object > > without worrying about ConcurrentModificationExceptions > > 3. To be able to call .subset(...) on the configuration object and 1) not > > risk a ConcurrentModificationException during this operation, and 2) get > a > > COPY of the subset back, so that further operations on the subset don't > > risk ConcurrentModificationExceptions > > > > Does anyone know of a way to do this? Or is the ONLY way to guarantee > lack > > of CMEs to synchronize EVERY access (reads, writes, subsets, iterations, > > etc.)? > > Perhaps you could try java.util.concurrent.locks.ReadWriteLock ? You > would have to put the protection by yourself so it may require lots of > code wrapping. One of the good thins is that it allows concurrent read > (but when a write occurs, only one thread can write and reads are blocked). > > Luc > > > > > If this is the case, how have others dealt with situations like ours, > where > > you do mostly just reads on a Configuration object, but very occasional > > writes, and need to take subsets, and need to avoid exceptions while > doing > > this? With a high volume app, synchronizing every access would slow > things > > waay down, thus my question. > > > > - Tim > > > > > - > To unsubscribe, e-mail: user-unsubscr...@commons.apache.org > For additional commands, e-mail: user-h...@commons.apache.org > > -- Tim Bessie Meez, Inc. tbes...@meez.com
Re: [configuration] How to get threadsafe subset() config in a threadsafe way?
Le 13/03/2012 07:34, Tim Bessie a écrit : > Hi all... Hi Tim, > > So we're keeping some config information CompositeConfiguration object, and > we need to get subsets of this configuration data. > > When I call .subset(...), and then do some checks on the subset (isEmpty(), > etc.), I sometimes get ConcurrentModificationExceptions. I'm not sure > what's modifying the underlying configuration, although we do have > occasional setting of configuration values throughout the running of our > app. > > What would be *ideal* would be to: > > 1. NOT have to synchronize every access to the configuration object, since > we have a high-volume application > 2. To allow read and write operations to happen to the configuration object > without worrying about ConcurrentModificationExceptions > 3. To be able to call .subset(...) on the configuration object and 1) not > risk a ConcurrentModificationException during this operation, and 2) get a > COPY of the subset back, so that further operations on the subset don't > risk ConcurrentModificationExceptions > > Does anyone know of a way to do this? Or is the ONLY way to guarantee lack > of CMEs to synchronize EVERY access (reads, writes, subsets, iterations, > etc.)? Perhaps you could try java.util.concurrent.locks.ReadWriteLock ? You would have to put the protection by yourself so it may require lots of code wrapping. One of the good thins is that it allows concurrent read (but when a write occurs, only one thread can write and reads are blocked). Luc > > If this is the case, how have others dealt with situations like ours, where > you do mostly just reads on a Configuration object, but very occasional > writes, and need to take subsets, and need to avoid exceptions while doing > this? With a high volume app, synchronizing every access would slow things > waay down, thus my question. > > - Tim > - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
[configuration] How to get threadsafe subset() config in a threadsafe way?
Hi all... So we're keeping some config information CompositeConfiguration object, and we need to get subsets of this configuration data. When I call .subset(...), and then do some checks on the subset (isEmpty(), etc.), I sometimes get ConcurrentModificationExceptions. I'm not sure what's modifying the underlying configuration, although we do have occasional setting of configuration values throughout the running of our app. What would be *ideal* would be to: 1. NOT have to synchronize every access to the configuration object, since we have a high-volume application 2. To allow read and write operations to happen to the configuration object without worrying about ConcurrentModificationExceptions 3. To be able to call .subset(...) on the configuration object and 1) not risk a ConcurrentModificationException during this operation, and 2) get a COPY of the subset back, so that further operations on the subset don't risk ConcurrentModificationExceptions Does anyone know of a way to do this? Or is the ONLY way to guarantee lack of CMEs to synchronize EVERY access (reads, writes, subsets, iterations, etc.)? If this is the case, how have others dealt with situations like ours, where you do mostly just reads on a Configuration object, but very occasional writes, and need to take subsets, and need to avoid exceptions while doing this? With a high volume app, synchronizing every access would slow things waay down, thus my question. - Tim -- Tim Bessie Meez, Inc. tbes...@meez.com