[jira] [Commented] (AVRO-2051) Thread contention accessing JsonProperties props
[ https://issues.apache.org/jira/browse/AVRO-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159144#comment-16159144 ] Daniel Kulp commented on AVRO-2051: --- Redid the pull request to base off a branch so I can keep my fork more in sync. Anyway, I made the hashCode and propsEquals package protected which should keep those from being exposed.I needed to add these methods as the subclass (Protocol) needs to know those, but I wanted to keep the "props" field private to make sure any subclass or other does not do any direct access to the props field. This forces subclasses (and others) to use the addProp method where we can make sure properties don't "change". (Properties can only be added, not modified) I don't think putting the ConcurrentHashMap into utils makes sense. This is a VERY particular use case and only certain methods are implemented for it. By keeping it anonymous and limiting access (via a private field), we can make sure the proper methods are called. For example, for a proper public class, keySet() would need to be implemented to return the keys in the proper order. I didn't implement it as, in this case, it's not ever called. Likewise, clear/remove/etc... are not implemented as they are not needed in this case. (the comment in the code mentions this) > Thread contention accessing JsonProperties props > > > Key: AVRO-2051 > URL: https://issues.apache.org/jira/browse/AVRO-2051 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Daniel Kulp > > See > https://lists.apache.org/thread.html/dd34ab8439137a81a9de29ad4161f37b17638394cea0806765689976@%3Cuser.avro.apache.org%3E > Basically, the getJsonProp method, being synchronized, is causing thread > contention issues when trying to share schemas between threads.My > proposal (pull request forthcoming) is to treat "props" as an immutable map > and do a copy+add+swap for the addProp method. This will make the addProp > call slower (particularly for large maps of props), but would make the reads > significantly faster as no locking will be needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AVRO-2051) Thread contention accessing JsonProperties props
[ https://issues.apache.org/jira/browse/AVRO-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159136#comment-16159136 ] ASF GitHub Bot commented on AVRO-2051: -- GitHub user dkulp opened a pull request: https://github.com/apache/avro/pull/245 [AVRO-2051] Remove synchronization for JsonProperties.getJsonProp This change does two basic things: Makes "props" a private field and requires the subclasses to access it via the additional methods. This allows some changing of the underlying implementation a bit easier. Change props to an ConcurrentHashMap with an extra ConcurrentLinkedQueue to maintain order. With the (1) change, this is fairly simple. This keeps the O(log N) puts and lookups and a simple entrySet iterator for quick iterations. Synchronized blocks are removed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dkulp/avro AVRO-2051 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/245.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #245 commit ef757c5aefd344a32e100736d419b4717d4dfd9f Author: Daniel Kulp Date: 2017-07-17T19:08:10Z [AVRO-2051] Remove synchronization for JsonProperties.getJsonProp > Thread contention accessing JsonProperties props > > > Key: AVRO-2051 > URL: https://issues.apache.org/jira/browse/AVRO-2051 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Daniel Kulp > > See > https://lists.apache.org/thread.html/dd34ab8439137a81a9de29ad4161f37b17638394cea0806765689976@%3Cuser.avro.apache.org%3E > Basically, the getJsonProp method, being synchronized, is causing thread > contention issues when trying to share schemas between threads.My > proposal (pull request forthcoming) is to treat "props" as an immutable map > and do a copy+add+swap for the addProp method. This will make the addProp > call slower (particularly for large maps of props), but would make the reads > significantly faster as no locking will be needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] avro pull request #245: [AVRO-2051] Remove synchronization for JsonPropertie...
GitHub user dkulp opened a pull request: https://github.com/apache/avro/pull/245 [AVRO-2051] Remove synchronization for JsonProperties.getJsonProp This change does two basic things: Makes "props" a private field and requires the subclasses to access it via the additional methods. This allows some changing of the underlying implementation a bit easier. Change props to an ConcurrentHashMap with an extra ConcurrentLinkedQueue to maintain order. With the (1) change, this is fairly simple. This keeps the O(log N) puts and lookups and a simple entrySet iterator for quick iterations. Synchronized blocks are removed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dkulp/avro AVRO-2051 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/245.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #245 commit ef757c5aefd344a32e100736d419b4717d4dfd9f Author: Daniel Kulp Date: 2017-07-17T19:08:10Z [AVRO-2051] Remove synchronization for JsonProperties.getJsonProp ---
[jira] [Commented] (AVRO-2051) Thread contention accessing JsonProperties props
[ https://issues.apache.org/jira/browse/AVRO-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159134#comment-16159134 ] ASF GitHub Bot commented on AVRO-2051: -- Github user dkulp closed the pull request at: https://github.com/apache/avro/pull/236 > Thread contention accessing JsonProperties props > > > Key: AVRO-2051 > URL: https://issues.apache.org/jira/browse/AVRO-2051 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Daniel Kulp > > See > https://lists.apache.org/thread.html/dd34ab8439137a81a9de29ad4161f37b17638394cea0806765689976@%3Cuser.avro.apache.org%3E > Basically, the getJsonProp method, being synchronized, is causing thread > contention issues when trying to share schemas between threads.My > proposal (pull request forthcoming) is to treat "props" as an immutable map > and do a copy+add+swap for the addProp method. This will make the addProp > call slower (particularly for large maps of props), but would make the reads > significantly faster as no locking will be needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] avro pull request #236: [AVRO-2051] Remove synchronization for JsonPropertie...
Github user dkulp closed the pull request at: https://github.com/apache/avro/pull/236 ---
[jira] [Commented] (AVRO-1213) Revisit dependencies on Jetty, servlet-api, and Netty
[ https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159099#comment-16159099 ] ASF GitHub Bot commented on AVRO-1213: -- GitHub user dkulp opened a pull request: https://github.com/apache/avro/pull/244 [AVRO-1213] Update to latest release of Jetty This updates the Jetty dependency to the latest Jetty release. Note: this does not address the Jetty vs. Netty thing or updates to the latest Netty. I hope to tackle that soon, but the netty update is huge/hard with a much larger impact. The API signatures do change slightly, but that is obviously required due to the org.mortbay -> org.eclipse change. However, getting onto the supported version of Jetty (with the latest security updates and fixes) is important. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dkulp/avro JETTY Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/244.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #244 commit b9ea20ba5700a7df60411a7bd62406f5c4beab42 Author: Daniel Kulp Date: 2017-09-08T17:18:33Z [AVRO-1213] Update to latest release of Jetty > Revisit dependencies on Jetty, servlet-api, and Netty > - > > Key: AVRO-1213 > URL: https://issues.apache.org/jira/browse/AVRO-1213 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.2 >Reporter: Sharmarke Aden >Priority: Blocker > Fix For: 1.9.0 > > > The compile scoped dependency on jetty servlet-api in the IPC pom file can be > problematic if using Avro in a webapp environment. Would it be possible to > make this dependency either optional or provided? Or maybe Avro modularize > into sub-modules in such a way that desired features can be assembled > piecemeal? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] avro pull request #244: [AVRO-1213] Update to latest release of Jetty
GitHub user dkulp opened a pull request: https://github.com/apache/avro/pull/244 [AVRO-1213] Update to latest release of Jetty This updates the Jetty dependency to the latest Jetty release. Note: this does not address the Jetty vs. Netty thing or updates to the latest Netty. I hope to tackle that soon, but the netty update is huge/hard with a much larger impact. The API signatures do change slightly, but that is obviously required due to the org.mortbay -> org.eclipse change. However, getting onto the supported version of Jetty (with the latest security updates and fixes) is important. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dkulp/avro JETTY Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/244.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #244 commit b9ea20ba5700a7df60411a7bd62406f5c4beab42 Author: Daniel Kulp Date: 2017-09-08T17:18:33Z [AVRO-1213] Update to latest release of Jetty ---
[jira] [Updated] (AVRO-2072) ResolvingGrammarGenerator doesn't implement schema resolution correctly for unions
[ https://issues.apache.org/jira/browse/AVRO-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated AVRO-2072: Affects Version/s: 1.7.7 1.8.1 > ResolvingGrammarGenerator doesn't implement schema resolution correctly for > unions > -- > > Key: AVRO-2072 > URL: https://issues.apache.org/jira/browse/AVRO-2072 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7, 1.8.1 >Reporter: Nandor Kollar >Assignee: Nandor Kollar > Attachments: 0001-AVRO-1931-Additional-test-cases.patch, > AVRO-2072_2.patch, AVRO-2072.patch > > > According to > [specification|https://avro.apache.org/docs/current/spec.html#Schema+Resolution], > int and long is promotable to float, but when using SchemaValidator, a union > with a single int or long branch is not readable by an union with a float > branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2072) ResolvingGrammarGenerator doesn't implement schema resolution correctly for unions
[ https://issues.apache.org/jira/browse/AVRO-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated AVRO-2072: Component/s: java > ResolvingGrammarGenerator doesn't implement schema resolution correctly for > unions > -- > > Key: AVRO-2072 > URL: https://issues.apache.org/jira/browse/AVRO-2072 > Project: Avro > Issue Type: Bug > Components: java >Reporter: Nandor Kollar >Assignee: Nandor Kollar > Attachments: 0001-AVRO-1931-Additional-test-cases.patch, > AVRO-2072_2.patch, AVRO-2072.patch > > > According to > [specification|https://avro.apache.org/docs/current/spec.html#Schema+Resolution], > int and long is promotable to float, but when using SchemaValidator, a union > with a single int or long branch is not readable by an union with a float > branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)