[jira] [Created] (SLING-5287) OOME with DiscoveryServiceImplTest.testThirtyInstances on jenkins
Stefan Egli created SLING-5287: -- Summary: OOME with DiscoveryServiceImplTest.testThirtyInstances on jenkins Key: SLING-5287 URL: https://issues.apache.org/jira/browse/SLING-5287 Project: Sling Issue Type: Test Components: Extensions Affects Versions: Discovery Base 1.0.2, Discovery Impl 1.2.0 Reporter: Stefan Egli Assignee: Stefan Egli Fix For: Discovery Impl 1.2.2, Discovery Base 1.1.0 As noted on the [mailinglist|http://markmail.org/message/ksdc4jdmyd2fk5nn] there are OOME happening on jenkins - mostly during DiscoveryServiceImplTest.testThirtyInstances. Either we increase the memory or change this test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLING-5287) OOME with DiscoveryServiceImplTest.testThirtyInstances on jenkins
[ https://issues.apache.org/jira/browse/SLING-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved SLING-5287. Resolution: Fixed reduced the 'thirty' test to be a 'twenty-four' in rev 1713588 - perhaps that's enough. > OOME with DiscoveryServiceImplTest.testThirtyInstances on jenkins > - > > Key: SLING-5287 > URL: https://issues.apache.org/jira/browse/SLING-5287 > Project: Sling > Issue Type: Test > Components: Extensions >Affects Versions: Discovery Impl 1.2.0, Discovery Base 1.0.2 >Reporter: Stefan Egli >Assignee: Stefan Egli > Fix For: Discovery Impl 1.2.2, Discovery Base 1.1.0 > > > As noted on the [mailinglist|http://markmail.org/message/ksdc4jdmyd2fk5nn] > there are OOME happening on jenkins - mostly during > DiscoveryServiceImplTest.testThirtyInstances. > Either we increase the memory or change this test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-5251) discovery.impl should use SyncTokenService (configurable)
[ https://issues.apache.org/jira/browse/SLING-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998253#comment-14998253 ] Stefan Egli commented on SLING-5251: rev 1713589 : bq. add config parameter useSyncTokenService that enables usage of a SyncTokenService if available. This increases synchronization QoS when for topology changes > discovery.impl should use SyncTokenService (configurable) > - > > Key: SLING-5251 > URL: https://issues.apache.org/jira/browse/SLING-5251 > Project: Sling > Issue Type: Improvement > Components: Extensions >Affects Versions: Discovery Impl 1.2.0 >Reporter: Stefan Egli >Assignee: Stefan Egli > Fix For: Discovery Impl 1.2.2 > > > The SyncTokenService introduced with discovery.commons allows discovery > implementors to add additional QoS to a TOPOLOGY_CHANGED event by waiting for > each instance to store a 'sync token' (eg cluster *view* id, not cluster id) > into the repository. The algorithm in detail: > * upon detecting a topology change happening, each instance sends out > TOPOLOGY_CHANGING to its listeners > * *after* that is done, it stores a syncToken to a well-known place into the > repository > * thus when a particular instance sees all syncTokens by the peers, it knows > that all listeners have now gotten the TOPOLOGY_CHANGING event - and that > provides a strong synchronization guarantee. > This mechanism is already in-use by default in discovery.oak (which > additionally also ensures no instance has any backlog with the repository - > which it can do as it requires oak and oak provides this info in the > discovery-lite descriptor). > Now for discovery.impl such a 'strong cluster synchronization' guarantee is > not that relevant as it might look. Since discovery.impl already uses the > repository (and thus its delays) for voting and creating the establishedView. > But at the moment there is no guarantee that the voting happens *after* the > TOPOLOGY_CHANGING was processed by all listeners. It's just rather unlikely > that sending the TOPOLOGY_CHANGING (which is all local) is slower than the > voting (which is going through repository delays). > Nevertheless, using the SyncTokenService would provide mentioned strong > cluster synchronization guarantee and _discovery.impl could just as well make > use of it_. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-5264) DiscoveryServiceImplTest.testThirtyInstances failed on jenkins
[
https://issues.apache.org/jira/browse/SLING-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998284#comment-14998284
]
Stefan Egli commented on SLING-5264:
Seen OOME with this test - handling that separately in SLING-5287
> DiscoveryServiceImplTest.testThirtyInstances failed on jenkins
> --
>
> Key: SLING-5264
> URL: https://issues.apache.org/jira/browse/SLING-5264
> Project: Sling
> Issue Type: Bug
> Components: Extensions
>Affects Versions: Discovery Impl 1.2.0, Discovery Base 1.0.2
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Minor
> Fix For: Discovery Impl 1.2.2, Discovery Base 1.1.0
>
> Attachments: testThirtyInstances.txt.gz
>
>
> DiscoveryServiceImplTest.testThirtyInstances failed on jenkins:
> https://builds.apache.org/job/sling-trunk-1.7/2659/
> {code}
> DiscoveryServiceImplTest>AbstractDiscoveryServiceTest.testThirtyInstances:304->AbstractDiscoveryServiceTest.startRetryLoop:311->AbstractDiscoveryServiceTest.startRetryLoop:315
> RetryLoop failed, condition is false after 100 seconds: null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SLING-5231) Remove getAdministrativeResourceResolver() usage from Discovery components
[
https://issues.apache.org/jira/browse/SLING-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated SLING-5231:
---
Fix Version/s: (was: Discovery Oak 1.1.0)
(was: Discovery Base 1.1.0)
(was: Discovery Commons 1.0.4)
(was: Discovery Impl 1.2.2)
Discovery Impl 1.2.4
Discovery Oak 1.1.2
Discovery Base 1.1.2
Discovery Commons 1.0.6
moving one version out as this requires some synching within the entire teams
> Remove getAdministrativeResourceResolver() usage from Discovery components
> --
>
> Key: SLING-5231
> URL: https://issues.apache.org/jira/browse/SLING-5231
> Project: Sling
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: Discovery Impl 1.2.0, Discovery Oak 1.0.2, Discovery
> Base 1.0.2, Discovery Commons 1.0.2
>Reporter: Antonio Sanso
>Assignee: Stefan Egli
> Fix For: Discovery Commons 1.0.6, Discovery Base 1.1.2, Discovery
> Oak 1.1.2, Discovery Impl 1.2.4
>
>
> * {{org.apache.sling.discovery.base}}: 6
> * {{org.apache.sling.discovery.commons}}: 3
> * {{org.apache.sling.discovery.oak}}: 4
> total of 13 occurrences
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SLING-5269) Sling Pipes IP clearance
[ https://issues.apache.org/jira/browse/SLING-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998390#comment-14998390 ] Nicolas Peltier commented on SLING-5269: [~bdelacretaz] [~olli] received aknowledgment that the software grant is filed, do you need a fwd of the mail? > Sling Pipes IP clearance > > > Key: SLING-5269 > URL: https://issues.apache.org/jira/browse/SLING-5269 > Project: Sling > Issue Type: Task > Components: Extensions >Reporter: Oliver Lietz >Assignee: Oliver Lietz > > This issue tracks the IP clearance for the donation of Sling Pipes > (SLING-5134). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-5269) Sling Pipes IP clearance
[ https://issues.apache.org/jira/browse/SLING-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998393#comment-14998393 ] Bertrand Delacretaz commented on SLING-5269: I got the notification, thanks, I'll have a look later this week > Sling Pipes IP clearance > > > Key: SLING-5269 > URL: https://issues.apache.org/jira/browse/SLING-5269 > Project: Sling > Issue Type: Task > Components: Extensions >Reporter: Oliver Lietz >Assignee: Oliver Lietz > > This issue tracks the IP clearance for the donation of Sling Pipes > (SLING-5134). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLING-5264) DiscoveryServiceImplTest.testThirtyInstances failed on jenkins
[
https://issues.apache.org/jira/browse/SLING-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli resolved SLING-5264.
Resolution: Fixed
Marking fixed as it might well have been a case of running low on memory
(SLING-5287)
> DiscoveryServiceImplTest.testThirtyInstances failed on jenkins
> --
>
> Key: SLING-5264
> URL: https://issues.apache.org/jira/browse/SLING-5264
> Project: Sling
> Issue Type: Bug
> Components: Extensions
>Affects Versions: Discovery Impl 1.2.0, Discovery Base 1.0.2
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Minor
> Fix For: Discovery Impl 1.2.2, Discovery Base 1.1.0
>
> Attachments: testThirtyInstances.txt.gz
>
>
> DiscoveryServiceImplTest.testThirtyInstances failed on jenkins:
> https://builds.apache.org/job/sling-trunk-1.7/2659/
> {code}
> DiscoveryServiceImplTest>AbstractDiscoveryServiceTest.testThirtyInstances:304->AbstractDiscoveryServiceTest.startRetryLoop:311->AbstractDiscoveryServiceTest.startRetryLoop:315
> RetryLoop failed, condition is false after 100 seconds: null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SLING-5288) Restrict which classes can be deserialized
Bertrand Delacretaz created SLING-5288: -- Summary: Restrict which classes can be deserialized Key: SLING-5288 URL: https://issues.apache.org/jira/browse/SLING-5288 Project: Sling Issue Type: Bug Components: General Reporter: Bertrand Delacretaz To avoid a recently reported Java deserialization vulnerability [1], we should restrict which classes are accepted when deserializing binaries. I have created a prototype SafeObjectInputStream at [2], which refuses to operate on classes that are outside a whitelist. We probably also need a wrapper for ObjectInputStreams provided by the environment, that looks a bit harder to create, for now we can already discuss this prototype to see if we want to pursue the idea. [1] https://blogs.apache.org/foundation/entry/apache_commons_statement_to_widespread [2] https://svn.apache.org/repos/asf/sling/whiteboard/bdelacretaz/safe-object-input-stream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
SafeObjectInputStream prototype
Hi, I have created a prototype at SLING-5288 to guard against recently reported Java deserialization risks. Feedback is welcome, and if someone feels like enhancing that with an ObjectInputStream wrapper that would be useful. -Bertrand
Re: SafeObjectInputStream prototype
Thanks a lot Bertrand!! This look promising. I have seen you used a white list approach (that is the best way by far). I was wondering if we can have a combination of white/black list approach though regards antonio On Nov 10, 2015, at 3:09 PM, Bertrand Delacretaz wrote: > Hi, > > I have created a prototype at SLING-5288 to guard against recently > reported Java deserialization risks. > > Feedback is welcome, and if someone feels like enhancing that with an > ObjectInputStream wrapper that would be useful. > > -Bertrand
[jira] [Created] (SLING-5289) react faster when established view changes
Stefan Egli created SLING-5289: -- Summary: react faster when established view changes Key: SLING-5289 URL: https://issues.apache.org/jira/browse/SLING-5289 Project: Sling Issue Type: Improvement Components: Extensions Affects Versions: Discovery Impl 1.2.0 Reporter: Stefan Egli Assignee: Stefan Egli Fix For: Discovery Impl 1.2.2 When a view gets promoted into /establishedView the DiscoveryServiceImpl should immediately analyze the change for possible need to send a new event to its listeners. This used to be done by TopologyChangeHandler but got lost during refactoring into discovery.commons/base -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLING-5289) react faster when established view changes
[ https://issues.apache.org/jira/browse/SLING-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved SLING-5289. Resolution: Fixed done in rev 1713665 > react faster when established view changes > -- > > Key: SLING-5289 > URL: https://issues.apache.org/jira/browse/SLING-5289 > Project: Sling > Issue Type: Improvement > Components: Extensions >Affects Versions: Discovery Impl 1.2.0 >Reporter: Stefan Egli >Assignee: Stefan Egli > Fix For: Discovery Impl 1.2.2 > > > When a view gets promoted into /establishedView the DiscoveryServiceImpl > should immediately analyze the change for possible need to send a new event > to its listeners. This used to be done by TopologyChangeHandler but got lost > during refactoring into discovery.commons/base -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: SafeObjectInputStream prototype
On Tue, Nov 10, 2015 at 4:06 PM, Antonio Sanso wrote: > ...I was wondering if we can have a combination of white/black list approach > though... SerialKiller [1] which was also recently created uses regular expression patterns with both black and white lists (and gets them from an XML config which I don't like). That might be a bit slow for the common cases where you know exactly which class you want, at least in Sling I think most or all use cases are like that. So maybe we can provide the current mode with whitelist of fixed class names, with the option of a set of white + blacklists based on regexp class name patterns. -Bertrand [1] https://github.com/ikkisoft/SerialKiller/blob/master/src/org/nibblesec/tools/SerialKiller.java
[jira] [Resolved] (SLING-5251) discovery.impl should use SyncTokenService (configurable)
[ https://issues.apache.org/jira/browse/SLING-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved SLING-5251. Resolution: Fixed rev 1713675: bq. adding debug infos to the topology webconsole re the SyncTokenService History thus considering this done. > discovery.impl should use SyncTokenService (configurable) > - > > Key: SLING-5251 > URL: https://issues.apache.org/jira/browse/SLING-5251 > Project: Sling > Issue Type: Improvement > Components: Extensions >Affects Versions: Discovery Impl 1.2.0 >Reporter: Stefan Egli >Assignee: Stefan Egli > Fix For: Discovery Impl 1.2.2 > > > The SyncTokenService introduced with discovery.commons allows discovery > implementors to add additional QoS to a TOPOLOGY_CHANGED event by waiting for > each instance to store a 'sync token' (eg cluster *view* id, not cluster id) > into the repository. The algorithm in detail: > * upon detecting a topology change happening, each instance sends out > TOPOLOGY_CHANGING to its listeners > * *after* that is done, it stores a syncToken to a well-known place into the > repository > * thus when a particular instance sees all syncTokens by the peers, it knows > that all listeners have now gotten the TOPOLOGY_CHANGING event - and that > provides a strong synchronization guarantee. > This mechanism is already in-use by default in discovery.oak (which > additionally also ensures no instance has any backlog with the repository - > which it can do as it requires oak and oak provides this info in the > discovery-lite descriptor). > Now for discovery.impl such a 'strong cluster synchronization' guarantee is > not that relevant as it might look. Since discovery.impl already uses the > repository (and thus its delays) for voting and creating the establishedView. > But at the moment there is no guarantee that the voting happens *after* the > TOPOLOGY_CHANGING was processed by all listeners. It's just rather unlikely > that sending the TOPOLOGY_CHANGING (which is all local) is slower than the > voting (which is going through repository delays). > Nevertheless, using the SyncTokenService would provide mentioned strong > cluster synchronization guarantee and _discovery.impl could just as well make > use of it_. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SLING-4380) Use sling mocks for the discovery impl tests
[ https://issues.apache.org/jira/browse/SLING-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated SLING-4380: --- Assignee: (was: Stefan Egli) Fix Version/s: Discovery Impl 1.2.4 > Use sling mocks for the discovery impl tests > > > Key: SLING-4380 > URL: https://issues.apache.org/jira/browse/SLING-4380 > Project: Sling > Issue Type: Improvement > Components: Extensions, Testing >Affects Versions: Discovery Impl 1.0.12 >Reporter: Robert Munteanu > Fix For: Discovery Impl 1.2.4 > > > The org.apache.sling.discovery.impl has its own utility mocks in > https://svn.apache.org/repos/asf/sling/trunk/bundles/extensions/discovery/impl/src/test/java/org/apache/sling/discovery/impl/setup/ > . Most of these can be replaced by the new Sling mocks. > The benefits would be: > - less maintenance for the discovery impl project > - more exposure/coverage for the Sling mocks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-5288) Restrict which classes can be deserialized
[ https://issues.apache.org/jira/browse/SLING-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998939#comment-14998939 ] Alexander Klimetschek commented on SLING-5288: -- For performance reasons I would use a hash set of strings (fully qualified class names) to do the check against and pass that into the object stream already, as you don't want to build the set every time. (I think in an osgi environment you have to use the names and checking for Class equality might not work as these could change over time.) I would also have the class validator be an interface that just does "allows(Class class)", then provide one impl that does the above simple whitelist check, and pass that to the safe object stream. This way applications can provide their own whitelist logic. > Restrict which classes can be deserialized > -- > > Key: SLING-5288 > URL: https://issues.apache.org/jira/browse/SLING-5288 > Project: Sling > Issue Type: Bug > Components: General >Reporter: Bertrand Delacretaz > > To avoid a recently reported Java deserialization vulnerability [1], we > should restrict which classes are accepted when deserializing binaries. > I have created a prototype SafeObjectInputStream at [2], which refuses to > operate on classes that are outside a whitelist. > We probably also need a wrapper for ObjectInputStreams provided by the > environment, that looks a bit harder to create, for now we can already > discuss this prototype to see if we want to pursue the idea. > [1] > https://blogs.apache.org/foundation/entry/apache_commons_statement_to_widespread > [2] > https://svn.apache.org/repos/asf/sling/whiteboard/bdelacretaz/safe-object-input-stream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] sling pull request: SLING-5263: implement some junit test cases fo...
GitHub user tiennv90 opened a pull request: https://github.com/apache/sling/pull/111 SLING-5263: implement some junit test cases for LogSupport.java Hi This is the pull request for SLING-5263, please have a look. Regards, Tien Nguyen You can merge this pull request into a Git repository by running: $ git pull https://github.com/tiennv90/sling SLING-5263 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/sling/pull/111.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #111 commit 3efb8aa5c7ceb74d7a8d968cdb8ff48e22e34ea4 Author: Tien Nguyen Date: 2015-11-11T07:20:55Z SLING-5263: implement some junit test cases for LogSupport.java --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (SLING-5263) Request to implement junit test cases for LogSupport.java
[ https://issues.apache.org/jira/browse/SLING-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1553#comment-1553 ] ASF GitHub Bot commented on SLING-5263: --- GitHub user tiennv90 opened a pull request: https://github.com/apache/sling/pull/111 SLING-5263: implement some junit test cases for LogSupport.java Hi This is the pull request for SLING-5263, please have a look. Regards, Tien Nguyen You can merge this pull request into a Git repository by running: $ git pull https://github.com/tiennv90/sling SLING-5263 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/sling/pull/111.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #111 commit 3efb8aa5c7ceb74d7a8d968cdb8ff48e22e34ea4 Author: Tien Nguyen Date: 2015-11-11T07:20:55Z SLING-5263: implement some junit test cases for LogSupport.java > Request to implement junit test cases for LogSupport.java > - > > Key: SLING-5263 > URL: https://issues.apache.org/jira/browse/SLING-5263 > Project: Sling > Issue Type: Improvement > Components: Commons >Reporter: Tien Nguyen Viet >Priority: Trivial > > org.apache.sling.commons.logservice.internal.LogSupport.java is not 100% > covered by junit test cases, can i request to let me implement the test cases > ? > CC: [~cziegeler] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SLING-5263) Request to implement junit test cases for LogSupport.java
[ https://issues.apache.org/jira/browse/SLING-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1560#comment-1560 ] Tien Nguyen Viet commented on SLING-5263: - Hi [~rombert] Please see the pull request for this jira: https://github.com/apache/sling/pull/111 > Request to implement junit test cases for LogSupport.java > - > > Key: SLING-5263 > URL: https://issues.apache.org/jira/browse/SLING-5263 > Project: Sling > Issue Type: Improvement > Components: Commons >Reporter: Tien Nguyen Viet >Priority: Trivial > > org.apache.sling.commons.logservice.internal.LogSupport.java is not 100% > covered by junit test cases, can i request to let me implement the test cases > ? > CC: [~cziegeler] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SLING-5263) Request to implement junit test cases for LogSupport.java
[ https://issues.apache.org/jira/browse/SLING-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tien Nguyen Viet resolved SLING-5263. - Resolution: Fixed > Request to implement junit test cases for LogSupport.java > - > > Key: SLING-5263 > URL: https://issues.apache.org/jira/browse/SLING-5263 > Project: Sling > Issue Type: Improvement > Components: Commons >Reporter: Tien Nguyen Viet >Priority: Trivial > > org.apache.sling.commons.logservice.internal.LogSupport.java is not 100% > covered by junit test cases, can i request to let me implement the test cases > ? > CC: [~cziegeler] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (SLING-5263) Request to implement junit test cases for LogSupport.java
[ https://issues.apache.org/jira/browse/SLING-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konrad Windszus reopened SLING-5263: [~tiennv90] Please don't close the issue until the patch was applied to the Sling subversion repository. > Request to implement junit test cases for LogSupport.java > - > > Key: SLING-5263 > URL: https://issues.apache.org/jira/browse/SLING-5263 > Project: Sling > Issue Type: Improvement > Components: Commons >Reporter: Tien Nguyen Viet >Priority: Trivial > > org.apache.sling.commons.logservice.internal.LogSupport.java is not 100% > covered by junit test cases, can i request to let me implement the test cases > ? > CC: [~cziegeler] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
