[jira] [Created] (OAK-7229) Build Jackrabbit Oak #1218 failed
Hudson created OAK-7229: --- Summary: Build Jackrabbit Oak #1218 failed Key: OAK-7229 URL: https://issues.apache.org/jira/browse/OAK-7229 Project: Jackrabbit Oak Issue Type: Bug Components: continuous integration Reporter: Hudson No description is provided The build Jackrabbit Oak #1218 has failed. First failed run: [Jackrabbit Oak #1218|https://builds.apache.org/job/Jackrabbit%20Oak/1218/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1218/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348147#comment-16348147 ] Amit Jain commented on OAK-7223: On trunk with http://svn.apache.org/viewvc?rev=1822850=rev > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.9.0, 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-7223: --- Fix Version/s: 1.9.0 > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.9.0, 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7227) MountPermissionProvider getNumEntries prone to overflow
[ https://issues.apache.org/jira/browse/OAK-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Deparvu resolved OAK-7227. --- Resolution: Fixed Fix Version/s: 1.8.2 applied patch provided on OAK-7228 with http://svn.apache.org/viewvc?rev=1822821=rev > MountPermissionProvider getNumEntries prone to overflow > --- > > Key: OAK-7227 > URL: https://issues.apache.org/jira/browse/OAK-7227 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, security >Reporter: Alex Deparvu >Assignee: Alex Deparvu >Priority: Trivial > Fix For: 1.9.0, 1.8.2 > > > It's about this method MountPermissionProvider#getNumEntries [0] > thanks [~anchela] for spotting! > [0] > https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/authorization/permission/MountPermissionProvider.java#L105 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7227) MountPermissionProvider getNumEntries prone to overflow
[ https://issues.apache.org/jira/browse/OAK-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Deparvu updated OAK-7227: -- Issue Type: Bug (was: Improvement) > MountPermissionProvider getNumEntries prone to overflow > --- > > Key: OAK-7227 > URL: https://issues.apache.org/jira/browse/OAK-7227 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, security >Reporter: Alex Deparvu >Assignee: Alex Deparvu >Priority: Trivial > Fix For: 1.9.0 > > > It's about this method MountPermissionProvider#getNumEntries [0] > thanks [~anchela] for spotting! > [0] > https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/authorization/permission/MountPermissionProvider.java#L105 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7227) MountPermissionProvider getNumEntries prone to overflow
[ https://issues.apache.org/jira/browse/OAK-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347034#comment-16347034 ] angela commented on OAK-7227: - see OAK-7228 for a proposed (but untested) patch :) > MountPermissionProvider getNumEntries prone to overflow > --- > > Key: OAK-7227 > URL: https://issues.apache.org/jira/browse/OAK-7227 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, security >Reporter: Alex Deparvu >Assignee: Alex Deparvu >Priority: Trivial > Fix For: 1.9.0 > > > It's about this method MountPermissionProvider#getNumEntries [0] > thanks [~anchela] for spotting! > [0] > https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/authorization/permission/MountPermissionProvider.java#L105 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7228) Potential long overflow in MountPermissionProvider.getNumEntries
[
https://issues.apache.org/jira/browse/OAK-7228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
angela resolved OAK-7228.
-
Resolution: Duplicate
:-)
> Potential long overflow in MountPermissionProvider.getNumEntries
> -
>
> Key: OAK-7228
> URL: https://issues.apache.org/jira/browse/OAK-7228
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: angela
>Priority: Major
>
> [~stillalex], just came across {{MountPermissionProvider.getNumEntries}},
> which looks as follows:
> {code}
> @Override
> public long getNumEntries(String principalName, long max) {
> long num = 0;
> for (PermissionStoreImpl store : stores) {
> num += store.getNumEntries(principalName, max);
> if (num >= max) {
> break;
> }
> }
> return num;
> }
> {code}
> If I am not mistaken this may lead to long overflow similar to the one we
> spotted it in {{PermissionEntryProviderImpl.init}}.
> Proposed (but untested fix) could look as follows:
> {code}
> @Override
> public long getNumEntries(String principalName, long max) {
> long num = 0;
> for (PermissionStoreImpl store : stores) {
> num = LongUtils.safeAdd(num,
> store.getNumEntries(principalName, max))
> if (num >= max) {
> break;
> }
> }
> return num;
> }
> {code}
> wdyt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7228) Potential long overflow in MountPermissionProvider.getNumEntries
[
https://issues.apache.org/jira/browse/OAK-7228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
angela updated OAK-7228:
Description:
[~stillalex], just came across {{MountPermissionProvider.getNumEntries}}, which
looks as follows:
{code}
@Override
public long getNumEntries(String principalName, long max) {
long num = 0;
for (PermissionStoreImpl store : stores) {
num += store.getNumEntries(principalName, max);
if (num >= max) {
break;
}
}
return num;
}
{code}
If I am not mistaken this may lead to long overflow similar to the one we
spotted it in {{PermissionEntryProviderImpl.init}}.
Proposed (but untested fix) could look as follows:
{code}
@Override
public long getNumEntries(String principalName, long max) {
long num = 0;
for (PermissionStoreImpl store : stores) {
num = LongUtils.safeAdd(num, store.getNumEntries(principalName,
max))
if (num >= max) {
break;
}
}
return num;
}
{code}
wdyt?
was:
[~stillalex], just came across {{MountPermissionProvider.getNumEntries}}, which
looks as follows:
{code}
@Override
public long getNumEntries(String principalName, long max) {
long num = 0;
for (PermissionStoreImpl store : stores) {
num += store.getNumEntries(principalName, max);
if (num >= max) {
break;
}
}
return num;
}
{code}
If I am not mistaken this may lead to long overflow similar to the one we
spotted it in {{PermissionEntryProviderImpl.init}}.
Proposed (but untested fix) could look as follows:
{code}
@Override
public long getNumEntries(String principalName, long max) {
long num = 0;
for (PermissionStoreImpl store : stores) {
num = LongUtils.safeAdd(num, store.getNumEntries(principalName,
max))
if (num >= max) {
break;
}
}
return num;
}
{code}
> Potential long overflow in MountPermissionProvider.getNumEntries
> -
>
> Key: OAK-7228
> URL: https://issues.apache.org/jira/browse/OAK-7228
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, security
>Reporter: angela
>Priority: Major
>
> [~stillalex], just came across {{MountPermissionProvider.getNumEntries}},
> which looks as follows:
> {code}
> @Override
> public long getNumEntries(String principalName, long max) {
> long num = 0;
> for (PermissionStoreImpl store : stores) {
> num += store.getNumEntries(principalName, max);
> if (num >= max) {
> break;
> }
> }
> return num;
> }
> {code}
> If I am not mistaken this may lead to long overflow similar to the one we
> spotted it in {{PermissionEntryProviderImpl.init}}.
> Proposed (but untested fix) could look as follows:
> {code}
> @Override
> public long getNumEntries(String principalName, long max) {
> long num = 0;
> for (PermissionStoreImpl store : stores) {
> num = LongUtils.safeAdd(num,
> store.getNumEntries(principalName, max))
> if (num >= max) {
> break;
> }
> }
> return num;
> }
> {code}
> wdyt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7228) Potential long overflow in MountPermissionProvider.getNumEntries
angela created OAK-7228:
---
Summary: Potential long overflow in
MountPermissionProvider.getNumEntries
Key: OAK-7228
URL: https://issues.apache.org/jira/browse/OAK-7228
Project: Jackrabbit Oak
Issue Type: Bug
Components: core, security
Reporter: angela
[~stillalex], just came across {{MountPermissionProvider.getNumEntries}}, which
looks as follows:
{code}
@Override
public long getNumEntries(String principalName, long max) {
long num = 0;
for (PermissionStoreImpl store : stores) {
num += store.getNumEntries(principalName, max);
if (num >= max) {
break;
}
}
return num;
}
{code}
If I am not mistaken this may lead to long overflow similar to the one we
spotted it in {{PermissionEntryProviderImpl.init}}.
Proposed (but untested fix) could look as follows:
{code}
@Override
public long getNumEntries(String principalName, long max) {
long num = 0;
for (PermissionStoreImpl store : stores) {
num = LongUtils.safeAdd(num, store.getNumEntries(principalName,
max))
if (num >= max) {
break;
}
}
return num;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7227) MountPermissionProvider getNumEntries prone to overflow
Alex Deparvu created OAK-7227: - Summary: MountPermissionProvider getNumEntries prone to overflow Key: OAK-7227 URL: https://issues.apache.org/jira/browse/OAK-7227 Project: Jackrabbit Oak Issue Type: Improvement Components: core, security Reporter: Alex Deparvu Assignee: Alex Deparvu Fix For: 1.9.0 It's about this method MountPermissionProvider#getNumEntries [0] thanks [~anchela] for spotting! [0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/authorization/permission/MountPermissionProvider.java#L105 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347006#comment-16347006
]
Michael Dürig commented on OAK-5506:
Agree wrt. performance. In other places we already switched to byte buffers.
Wrt. compatibility, this sounds reasonable. But I think we need to come up with
a test case writing the old way and reading the new way. This should be doable
if there is a way to disable the new behaviour I guess.
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Assignee: Francesco Mari
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-name-conversion.diff, OAK-5506-segment.diff,
> OAK-5506-segment2.diff, OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346957#comment-16346957
]
Julian Reschke commented on OAK-5506:
-
With respect to compatibility with segment-tar storage written by previous
versions: my understanding is that any string value (be it an item name or a
string property) persisted to segment-tar will have gone through
{{String.getBytes(UTF8)}}. In the case of a string that doesn't roundtrip
through UTF-8, this means that those characters will have been replaced by a
replacement character (here: "?"). So the persisted state *does* represent
valid UTF-8, and when read back, will just use that replacement character.
The proposed patch only affects writing of strings that are invalid, thus
previously written garbage shouldn't be an issue.
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Assignee: Francesco Mari
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-name-conversion.diff, OAK-5506-segment.diff,
> OAK-5506-segment2.diff, OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OAK-7162) Race condition on revisions head between compaction and scheduler could result in skipped commit
[
https://issues.apache.org/jira/browse/OAK-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345030#comment-16345030
]
Andrei Dulceanu edited comment on OAK-7162 at 1/31/18 3:01 PM:
---
Fixed in trunk at r1822640.
was (Author: dulceanu):
Fixed at r1822640.
> Race condition on revisions head between compaction and scheduler could
> result in skipped commit
>
>
> Key: OAK-7162
> URL: https://issues.apache.org/jira/browse/OAK-7162
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: 1.8.0
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Blocker
> Labels: scalability
> Fix For: 1.9.0, 1.10, 1.8.2
>
> Attachments: OAK-7162-02.patch, OAK-7162-03.patch, OAK-7162.patch
>
>
> There is a race condition on {{TarRevisions#head}} between a running
> compaction trying to set the new head [0] and the scheduler doing the same
> after executing a specific commit [1]. If the compaction thread is first,
> then the head assignment in the scheduler will fail and not be re-attempted.
> IMO, the simple if statement should be changed to a while loop in which the
> head is refreshed and the commit is re-applied against the new head, before
> attempting again to set a new head in {{TarRevisions}}. This is somehow
> similar to what we previously had [2], but without the unneeded
> optimistic/pessimistic strategies involving tokens.
> [0]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/FileStore.java#L764
> [1]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/scheduler/LockBasedScheduler.java#L253
> [2]
> https://github.com/apache/jackrabbit-oak/blob/1.6/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentNodeStore.java#L686
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346962#comment-16346962
]
Julian Reschke commented on OAK-5506:
-
With respect to performance: this code change opens the door to serialize to
{{ByteBuffer}}s instead of {{byte[]}}, potentially avoiding array copies. Not
sure whether segment-tar could take advantage of that...
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Assignee: Francesco Mari
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-name-conversion.diff, OAK-5506-segment.diff,
> OAK-5506-segment2.diff, OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346949#comment-16346949 ] Julian Reschke commented on OAK-7188: - a) I don't see how, if the test is supposed to run with Oak versions using both old and new Guava - am I missing something? b) If you can change the test not to need this piece of Guava, that would of course be preferable. > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Priority: Major > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346932#comment-16346932 ] Michael Dürig commented on OAK-7188: Can't we have a stable test dependency to Guava? If not, I'd prefer to rewrite that functionality by other means instead of relying on reflection. > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Priority: Major > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7198) Index rule with REGEX_ALL_PROPS includes relative node
[
https://issues.apache.org/jira/browse/OAK-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-7198.
---
Resolution: Fixed
Fix Version/s: 1.9.0
Thanks for the review. Applied the patch to trunk:
http://svn.apache.org/r1822808
> Index rule with REGEX_ALL_PROPS includes relative node
> --
>
> Key: OAK-7198
> URL: https://issues.apache.org/jira/browse/OAK-7198
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Labels: candidate_oak_1_8
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-7198.patch
>
>
> A lucene index with an index rule that includes properties with
> {{LuceneIndexConstants.REGEX_ALL_PROPS}} also includes some child node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7198) Index rule with REGEX_ALL_PROPS includes relative node
[
https://issues.apache.org/jira/browse/OAK-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-7198:
--
Labels: candidate_oak_1_8 (was: )
> Index rule with REGEX_ALL_PROPS includes relative node
> --
>
> Key: OAK-7198
> URL: https://issues.apache.org/jira/browse/OAK-7198
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Labels: candidate_oak_1_8
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-7198.patch
>
>
> A lucene index with an index rule that includes properties with
> {{LuceneIndexConstants.REGEX_ALL_PROPS}} also includes some child node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-7188: Attachment: OAK-7188.diff > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Priority: Major > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346920#comment-16346920 ] Julian Reschke commented on OAK-7188: - [^OAK-7188.diff] uses the proper method using reflection and should be compatible with Guava 15..20 (or more) > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Priority: Major > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-7198) Index rule with REGEX_ALL_PROPS includes relative node
[
https://issues.apache.org/jira/browse/OAK-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger reassigned OAK-7198:
-
Assignee: Marcel Reutegger
> Index rule with REGEX_ALL_PROPS includes relative node
> --
>
> Key: OAK-7198
> URL: https://issues.apache.org/jira/browse/OAK-7198
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-7198.patch
>
>
> A lucene index with an index rule that includes properties with
> {{LuceneIndexConstants.REGEX_ALL_PROPS}} also includes some child node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7213) Avoid call for child node when bundle contains all children
[
https://issues.apache.org/jira/browse/OAK-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346878#comment-16346878
]
Marcel Reutegger commented on OAK-7213:
---
Thanks for the review. I added an additional assertion to the test to check for
the {{_children}} flag.
In trunk: http://svn.apache.org/r1822802
> Avoid call for child node when bundle contains all children
> ---
>
> Key: OAK-7213
> URL: https://issues.apache.org/jira/browse/OAK-7213
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Labels: bundling
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-7213.patch
>
>
> When nodes are bundled in a document, the DocumentNodeStore keeps track of
> whether all children are included in a document. The presence of the hidden
> {{:doc-has-child-non-bundled}} property indicates there are non bundled child
> nodes. For the case when a document contains all children in the bundle, the
> DocumentNodeStore still does a find call on the DocumentStore when asked for
> an unknown child node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346875#comment-16346875
]
Julian Reschke commented on OAK-5506:
-
...was just looking at the last columns. Will rerun later with more iterations.
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Assignee: Francesco Mari
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-name-conversion.diff, OAK-5506-segment.diff,
> OAK-5506-segment2.diff, OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7194) Threads are going in blocked state - OAK segment
[
https://issues.apache.org/jira/browse/OAK-7194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7194:
---
Issue Type: Improvement (was: Bug)
> Threads are going in blocked state - OAK segment
>
>
> Key: OAK-7194
> URL: https://issues.apache.org/jira/browse/OAK-7194
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Affects Versions: 1.6.2
>Reporter: Kshitiz Garg
>Priority: Critical
>
> We are using AEM and it internally is using OAK 1.6.2. Our application is
> going to a choked state many a times. Our thread dump analysis is always
> pointing to this stack trace with many blocked threads. Is it a known issue?
> Or is there a setting to avoid it? We are using tar files based repository
> underneath and are using default settings:
>
> {noformat}
> - nativeId:0x1208 - state:BLOCKED
> stackTrace:
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:121)
> - waiting to lock <0x000414d8c250> (a
> org.apache.jackrabbit.oak.segment.SegmentId)
> at org.apache.jackrabbit.oak.segment.Record.getSegment(Record.java:70)
> at org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:160)
> at org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:192)
> at org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:192)
> at org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:192)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.getChildNode(SegmentNodeState.java:412)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.ImmutableTree.createChild(ImmutableTree.java:125)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.ImmutableTree.getChild(ImmutableTree.java:176)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.ImmutableTree.getChild(ImmutableTree.java:81)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionUtil.getPrincipalRoot(PermissionUtil.java:83)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionStoreImpl.getPrincipalRoot(PermissionStoreImpl.java:132)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionStoreImpl.getNumEntries(PermissionStoreImpl.java:103)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionEntryCache.getNumEntries(PermissionEntryCache.java:102)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionEntryProviderImpl.init(PermissionEntryProviderImpl.java:79)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionEntryProviderImpl.(PermissionEntryProviderImpl.java:72)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.CompiledPermissionImpl.(CompiledPermissionImpl.java:112)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.CompiledPermissionImpl.create(CompiledPermissionImpl.java:126)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionProviderImpl.getCompiledPermissions(PermissionProviderImpl.java:162)
> at
> org.apache.jackrabbit.oak.security.authorization.permission.PermissionProviderImpl.getTreePermission(PermissionProviderImpl.java:151)
> at
> org.apache.jackrabbit.oak.security.authorization.composite.CompositeTreePermission.create(CompositeTreePermission.java:67)
> at
> org.apache.jackrabbit.oak.security.authorization.composite.CompositePermissionProvider.getTreePermission(CompositePermissionProvider.java:147)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.getTreePermission(SecureNodeBuilder.java:357)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.getTreePermission(SecureNodeBuilder.java:360)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.getTreePermission(SecureNodeBuilder.java:360)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.access$100(SecureNodeBuilder.java:49)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder$ReadablePropertyPredicate.apply(SecureNodeBuilder.java:377)
> at
> org.apache.jackrabbit.oak.core.SecureNodeBuilder.getProperty(SecureNodeBuilder.java:184)
> at
> org.apache.jackrabbit.oak.plugins.tree.impl.AbstractTree.getProperty(AbstractTree.java:249)
> at
> org.apache.jackrabbit.oak.core.MutableTree.getProperty(MutableTree.java:128)
> at
> org.apache.jackrabbit.oak.util.TreeUtil.getStringInternal(TreeUtil.java:108)
> at org.apache.jackrabbit.oak.util.TreeUtil.getString(TreeUtil.java:101)
> at
> org.apache.jackrabbit.oak.namepath.GlobalNameMapper.getNamespacesProperty(GlobalNameMapper.java:191)
> at
> org.apache.jackrabbit.oak.namepath.GlobalNameMapper.getOakURIOrNull(GlobalNameMapper.java:187)
> - eliminated <0x00070c692f50> (a
> org.apache.jackrabbit.oak.jcr.session.SessionNamespaces)
> at
>
[jira] [Resolved] (OAK-7226) Warn messages ignoring exception parameter
[
https://issues.apache.org/jira/browse/OAK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-7226.
---
Resolution: Invalid
Fix Version/s: (was: 1.10)
I see, thanks for the pointer. Resolving as invalid then.
> Warn messages ignoring exception parameter
> --
>
> Key: OAK-7226
> URL: https://issues.apache.org/jira/browse/OAK-7226
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Priority: Minor
>
> There are a few usages of Logger.warn() with a pattern similar to this
> example:
> {noformat}
> log.warn("Error occurred while fetching DataRecord for identifier {}", input,
> exeption);
> {noformat}
> The intention probably is that the third parameter is treated as an exception
> and e.g. logged with the stack trace. However, this method signature
> interprets the exception as a second argument for the message format. This
> means the exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346838#comment-16346838
]
Oliver Lietz commented on OAK-7203:
---
[~stillalex], it's ugly main class (those methods are not required, only used
for testing) vs ugly test class and I prefer ugly test classes in general (you
can make it nicer by adding exception to method signatures of overridden
methods).
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (OAK-7226) Warn messages ignoring exception parameter
[
https://issues.apache.org/jira/browse/OAK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346825#comment-16346825
]
Chetan Mehrotra edited comment on OAK-7226 at 1/31/18 1:33 PM:
---
Logback also supports it [1]. Also this is stated in
[Logger|https://www.slf4j.org/api/org/slf4j/Logger.html] docs
bq. Note that logging statements can be parameterized in presence of an
exception/throwable.
[1]
https://github.com/qos-ch/logback/blob/master/logback-classic/src/main/java/ch/qos/logback/classic/spi/LoggingEvent.java#L114
was (Author: chetanm):
Logback also supports it [1]
[1]
https://github.com/qos-ch/logback/blob/master/logback-classic/src/main/java/ch/qos/logback/classic/spi/LoggingEvent.java#L114
> Warn messages ignoring exception parameter
> --
>
> Key: OAK-7226
> URL: https://issues.apache.org/jira/browse/OAK-7226
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Priority: Minor
> Fix For: 1.10
>
>
> There are a few usages of Logger.warn() with a pattern similar to this
> example:
> {noformat}
> log.warn("Error occurred while fetching DataRecord for identifier {}", input,
> exeption);
> {noformat}
> The intention probably is that the third parameter is treated as an exception
> and e.g. logged with the stack trace. However, this method signature
> interprets the exception as a second argument for the message format. This
> means the exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346826#comment-16346826
]
Oliver Lietz commented on OAK-7203:
---
+1 for moving {{MountInfoProviderService}} to {{oak-core}}
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7226) Warn messages ignoring exception parameter
[
https://issues.apache.org/jira/browse/OAK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346825#comment-16346825
]
Chetan Mehrotra commented on OAK-7226:
--
Logback also supports it [1]
[1]
https://github.com/qos-ch/logback/blob/master/logback-classic/src/main/java/ch/qos/logback/classic/spi/LoggingEvent.java#L114
> Warn messages ignoring exception parameter
> --
>
> Key: OAK-7226
> URL: https://issues.apache.org/jira/browse/OAK-7226
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Priority: Minor
> Fix For: 1.10
>
>
> There are a few usages of Logger.warn() with a pattern similar to this
> example:
> {noformat}
> log.warn("Error occurred while fetching DataRecord for identifier {}", input,
> exeption);
> {noformat}
> The intention probably is that the third parameter is treated as an exception
> and e.g. logged with the stack trace. However, this method signature
> interprets the exception as a second argument for the message format. This
> means the exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7226) Warn messages ignoring exception parameter
[
https://issues.apache.org/jira/browse/OAK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346806#comment-16346806
]
Marcel Reutegger commented on OAK-7226:
---
Hmm, now I'm confused. How can slf4j do that? Isn't this up to the actual
logging framework how to implement the interface defined by slf4j? E.g. we use
logback in Oak, which doesn't seem to have this 'feature'.
> Warn messages ignoring exception parameter
> --
>
> Key: OAK-7226
> URL: https://issues.apache.org/jira/browse/OAK-7226
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Priority: Minor
> Fix For: 1.10
>
>
> There are a few usages of Logger.warn() with a pattern similar to this
> example:
> {noformat}
> log.warn("Error occurred while fetching DataRecord for identifier {}", input,
> exeption);
> {noformat}
> The intention probably is that the third parameter is treated as an exception
> and e.g. logged with the stack trace. However, this method signature
> interprets the exception as a second argument for the message format. This
> means the exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346786#comment-16346786
]
Marcel Reutegger commented on OAK-5506:
---
The result doesn't look that useful. The measured time is zero up to the 50
percentile.
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Assignee: Francesco Mari
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-name-conversion.diff, OAK-5506-segment.diff,
> OAK-5506-segment2.diff, OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7226) Warn messages ignoring exception parameter
[
https://issues.apache.org/jira/browse/OAK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346776#comment-16346776
]
Chetan Mehrotra commented on OAK-7226:
--
[~mreutegg] This is a "feature" of slf4j
https://www.slf4j.org/faq.html#paramException
> Warn messages ignoring exception parameter
> --
>
> Key: OAK-7226
> URL: https://issues.apache.org/jira/browse/OAK-7226
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Priority: Minor
> Fix For: 1.10
>
>
> There are a few usages of Logger.warn() with a pattern similar to this
> example:
> {noformat}
> log.warn("Error occurred while fetching DataRecord for identifier {}", input,
> exeption);
> {noformat}
> The intention probably is that the third parameter is treated as an exception
> and e.g. logged with the stack trace. However, this method signature
> interprets the exception as a second argument for the message format. This
> means the exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (OAK-7221) Build Jackrabbit Oak #1212 failed
[ https://issues.apache.org/jira/browse/OAK-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger closed OAK-7221. - > Build Jackrabbit Oak #1212 failed > - > > Key: OAK-7221 > URL: https://issues.apache.org/jira/browse/OAK-7221 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1212 has failed. > First failed run: [Jackrabbit Oak > #1212|https://builds.apache.org/job/Jackrabbit%20Oak/1212/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1212/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7221) Build Jackrabbit Oak #1212 failed
[ https://issues.apache.org/jira/browse/OAK-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-7221. --- Resolution: Duplicate The log doesn't contain any failed tests, but judging from the build time, the job was likely reaching the 90 minutes limit again. > Build Jackrabbit Oak #1212 failed > - > > Key: OAK-7221 > URL: https://issues.apache.org/jira/browse/OAK-7221 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1212 has failed. > First failed run: [Jackrabbit Oak > #1212|https://builds.apache.org/job/Jackrabbit%20Oak/1212/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1212/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346754#comment-16346754 ] Marcel Reutegger commented on OAK-7223: --- See OAK-7226 for the warn messages. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7226) Warn messages ignoring exception parameter
[
https://issues.apache.org/jira/browse/OAK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-7226:
--
Affects Version/s: 1.8.0
> Warn messages ignoring exception parameter
> --
>
> Key: OAK-7226
> URL: https://issues.apache.org/jira/browse/OAK-7226
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob-plugins
>Affects Versions: 1.8.0
>Reporter: Marcel Reutegger
>Priority: Minor
> Fix For: 1.10
>
>
> There are a few usages of Logger.warn() with a pattern similar to this
> example:
> {noformat}
> log.warn("Error occurred while fetching DataRecord for identifier {}", input,
> exeption);
> {noformat}
> The intention probably is that the third parameter is treated as an exception
> and e.g. logged with the stack trace. However, this method signature
> interprets the exception as a second argument for the message format. This
> means the exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7226) Warn messages ignoring exception parameter
Marcel Reutegger created OAK-7226:
-
Summary: Warn messages ignoring exception parameter
Key: OAK-7226
URL: https://issues.apache.org/jira/browse/OAK-7226
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob-plugins
Reporter: Marcel Reutegger
Fix For: 1.10
There are a few usages of Logger.warn() with a pattern similar to this example:
{noformat}
log.warn("Error occurred while fetching DataRecord for identifier {}", input,
exeption);
{noformat}
The intention probably is that the third parameter is treated as an exception
and e.g. logged with the stack trace. However, this method signature interprets
the exception as a second argument for the message format. This means the
exception is effectively ignored.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7221) Build Jackrabbit Oak #1212 failed
[ https://issues.apache.org/jira/browse/OAK-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346749#comment-16346749 ] Hudson commented on OAK-7221: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1214|https://builds.apache.org/job/Jackrabbit%20Oak/1214/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1214/console] > Build Jackrabbit Oak #1212 failed > - > > Key: OAK-7221 > URL: https://issues.apache.org/jira/browse/OAK-7221 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1212 has failed. > First failed run: [Jackrabbit Oak > #1212|https://builds.apache.org/job/Jackrabbit%20Oak/1212/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1212/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346743#comment-16346743 ] Marcel Reutegger commented on OAK-7223: --- The warn message you added in the cache loader will not contain the exception or stack trace. The third parameter is interpreted as the second argument for the message format. There are a few more places in oak-blob-plugin with the same usage pattern. I'll create a separate issue for those. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346741#comment-16346741 ] Amit Jain commented on OAK-7223: Thanks [~mreutegg] for the review. Yes makes sense and I am in the process of adding test for the utility method. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346726#comment-16346726 ] Marcel Reutegger edited comment on OAK-7223 at 1/31/18 12:24 PM: - I would implement the new utility method slightly different. This avoids the issue with catching all types of exceptions but then wrapping it with an IOException (even if the original one was, as in most cases, an IOException). See attached [^OAK-7223-3.patch]. It would also be good to have a test case in oak-commons that verifies the expected behaviour. was (Author: mreutegg): I would implement the new utility method slightly different. This avoids the issue with catching all types of exceptions but then wrapping it with an IOException (even if the original one was, as in most cases, an IOException). See attached [^OAK-7223-3.patch]. It would also be good to have a test case that verifies the expected behaviour. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346726#comment-16346726 ] Marcel Reutegger commented on OAK-7223: --- I would implement the new utility method slightly different. This avoids the issue with catching all types of exceptions but then wrapping it with an IOException (even if the original one was, as in most cases, an IOException). See attached [^OAK-7223-3.patch]. It would also be good to have a test case that verifies the expected behaviour. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-7223: -- Attachment: OAK-7223-3.patch > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223-3.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-6922) Azure support for the segment-tar
[
https://issues.apache.org/jira/browse/OAK-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-6922:
---
Description:
An Azure Blob Storage implementation of the segment storage, based on the
OAK-6921 work.
h3. Segment files layout
Thew new implementation doesn't use tar files. They are replaced with
directories, storing segments, named after their UUIDs. This approach has
following advantages:
* no need to call seek(), which may be expensive on a remote file system.
Rather than that we can read the whole file (=segment) at once.
* it's possible to send multiple segments at once, asynchronously, which
reduces the performance overhead (see below).
The file structure is as follows:
{noformat}
[~]$ az storage blob list -c oak --output table
Name Blob TypeBlob
TierLengthContent Type Last Modified
---
--- -
oak/data0a.tar/.ca1326d1-edf4-4d53-aef0-0f14a6d05b63 BlockBlob
192 application/octet-stream 2018-01-31T10:59:14+00:00
oak/data0a.tar/0001.c6e03426-db9d-4315-a20a-12559e6aee54 BlockBlob
262144application/octet-stream 2018-01-31T10:59:14+00:00
oak/data0a.tar/0002.b3784e27-6d16-4f80-afc1-6f3703f6bdb9 BlockBlob
262144application/octet-stream 2018-01-31T10:59:14+00:00
oak/data0a.tar/0003.5d2f9588-0c92-4547-abf7-0263ee7c37bb BlockBlob
259216application/octet-stream 2018-01-31T10:59:14+00:00
...
oak/data0a.tar/006e.7b8cf63d-849a-4120-aa7c-47c3dde25e48 BlockBlob
4368 application/octet-stream 2018-01-31T12:01:09+00:00
oak/data0a.tar/006f.93799ae9-288e-4b32-afc2-bbc676fad7e5 BlockBlob
3792 application/octet-stream 2018-01-31T12:01:14+00:00
oak/data0a.tar/0070.8b2d5ff2-6a74-4ac3-a3cc-cc439367c2aa BlockBlob
3680 application/octet-stream 2018-01-31T12:01:14+00:00
oak/data0a.tar/0071.2a1c49f0-ce33-4777-a042-8aa8a704d202 BlockBlob
7760 application/octet-stream 2018-01-31T12:10:54+00:00
oak/journal.log.001 AppendBlob
1010 application/octet-stream 2018-01-31T12:10:54+00:00
oak/manifest BlockBlob
46application/octet-stream 2018-01-31T10:59:14+00:00
oak/repo.lock BlockBlob
application/octet-stream 2018-01-31T10:59:14+00:00
{noformat}
For the segment files, each name is prefixed with the index number. This allows
to maintain an order, as in the tar archive. This order is normally stored in
the index files as well, but if it's missing, the recovery process needs it.
Each file contains the raw segment data, with no padding/headers. Apart from
the segment files, there are 3 special files: binary references (.brf), segment
graph (.gph) and segment index (.idx).
h3. Asynchronous writes
Normally, all the TarWriter writes are synchronous, appending the segments to
the tar file. In case of Azure Blob Storage each write involves a network
latency. That's why the SegmentWriteQueue was introduced. The segments are
added to the blocking dequeue, which is served by a number of the consumer
threads, writing the segments to the cloud. There's also a map UUID->Segment,
which allows to return the segments in case they are requested by the
readSegment() method before they are actually persisted. Segments are removed
from the map only after a successful write operation.
The flush() method blocks accepting the new segments and returns after all
waiting segments are written. The close() method waits until the current
operations are finished and stops all threads.
The asynchronous mode can be disabled by setting the number of threads to 0.
h5. Queue recovery mode
If the Azure Blob Storage write() operation fails, the segment will be re-added
and the queue is switched to an "recovery mode". In this mode, all the threads
are suspended and new segments are not accepted (active waiting). There's a
single thread which retries adding the segment with some delay. If the segment
is successfully written, the queue will back to the normal operation.
This way the unavailable remote service is not flooded by the requests and
we're not accepting the segments when we can't persist them.
The close() method finishes the recovery mode - in this case, some of the
awaiting segments won't be persisted.
h5. Consistency
The asynchronous mode isn't as reliable as the standard, synchronous case.
Following cases are possible:
* TarWriter#writeEntry() returns successfully, but the segments
[jira] [Commented] (OAK-7182) Make it possible to update Guava
[
https://issues.apache.org/jira/browse/OAK-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346685#comment-16346685
]
Robert Munteanu commented on OAK-7182:
--
Here's the list of APIs I (semi-manually) extracted from oak as of r1822291
{noformat}
org.apache.jackrabbit.oak.plugins.blob;uses:="com.google.common.base,com.google.common.cache,com.google.common.util.concrrent
org.apache.jackrabbit.oak.commons.io;version="1.0.0";uses:="com.google.common.io",
org.apache.jackrabbit.oak.commons;version="1.0.0";uses:="com.google.common.base,com.google.common.collect,
org.apache.jackrabbit.oak.spi.whiteboard;version="1.0.0";uses:="com.google.common.base,
org.apache.jackrabbit.oak.cache;version="1.0.0";uses:="com.google.common.cache,com.google.common.collect,
org.apache.jackrabbit.oak.plugins.index.property.strategy;uses:="com.google.common.base
org.apache.jackrabbit.oak.plugins.nodetype;uses:="com.google.common.base,
org.apache.jackrabbit.oak.plugins.observation.filter;uses:="com.google.common.base
org.apache.jackrabbit.oak.spi.security.authentication.credentials;version="2.1.0";uses:="com.google.common.collect
org.apache.jackrabbit.oak.spi.state;uses:="com.google.common.base,
org.apache.jackrabbit.oak.plugins.memory;uses:="com.google.common.hash,
{noformat}
> Make it possible to update Guava
>
>
> Key: OAK-7182
> URL: https://issues.apache.org/jira/browse/OAK-7182
> Project: Jackrabbit Oak
> Issue Type: Wish
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: guava.diff
>
>
> We currently rely on Guava 15, and this affects all users of Oak because they
> essentially need to use the same version.
> This is an overall issue to investigate what would need to be done in Oak in
> order to make updates possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7182) Make it possible to update Guava
[
https://issues.apache.org/jira/browse/OAK-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346672#comment-16346672
]
Robert Munteanu commented on OAK-7182:
--
I am looking into the Guava packaging and release policy, and it seems that
they have started following semantic versioning -
https://github.com/google/guava/wiki/ReleasePolicy . In particular, they will
only upgrade the major version "
If there are API removals or other incompatible API changes", and that includes
APIs annotated with {{\@Beta}} . Which in turn for us this means that once we
incorporate a new version of Guava we should be safe for a longer time. They do
have breaking changes, for instance this year they had 3 major releases ( and 6
minor ones ) but version 23 is now 23.6, which may mark a turn in how they
approach backwards compatibility.
What we can do is either "vet" certain versions, e.g. test with Guava 15 and
latest ( 23.6 ) and adjust the import versions to be [15,24). Or alternatively
hope for the test and import [15,).
> Make it possible to update Guava
>
>
> Key: OAK-7182
> URL: https://issues.apache.org/jira/browse/OAK-7182
> Project: Jackrabbit Oak
> Issue Type: Wish
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: guava.diff
>
>
> We currently rely on Guava 15, and this affects all users of Oak because they
> essentially need to use the same version.
> This is an overall issue to investigate what would need to be done in Oak in
> order to make updates possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346656#comment-16346656 ] Amit Jain commented on OAK-7223: Added a 2nd version as needed to bump the package version also. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-7223: --- Attachment: OAK-7223-2.patch > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223-2.patch, OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-6373) oak-run check should also check checkpoints
[
https://issues.apache.org/jira/browse/OAK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346637#comment-16346637
]
Michael Dürig commented on OAK-6373:
Looking at the usage output of {{check}}:
{noformat}
--checkpoints [String] checks only specified checkpoints (comma separated);
leave empty to check all (default:
/checkpoints){noformat}
I think we need to clarify this a bit: 1. not specifying anything to
{{--checkpoints}} does not work. 2. The meaning of the default {{/checkpoints}}
is not clear.
I would prefer to change {{/checkpoint}} to {{all}} and change the wording to
something that clarifies what {{all}} means.
> oak-run check should also check checkpoints
>
>
> Key: OAK-6373
> URL: https://issues.apache.org/jira/browse/OAK-6373
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
>Priority: Major
> Labels: candidate_oak_1_8, tooling
> Fix For: 1.9.0, 1.10
>
>
> {{oak-run check}} does currently *not* traverse and check the items in the
> checkpoint. I think we should change this and add an option to traverse all,
> some or none of the checkpoints. When doing this we need to keep in mind the
> interaction of this new feature with the {{filter}} option: the paths passed
> through this option need then be prefixed with {{/root}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-5506) reject item names with unpaired surrogates early
[
https://issues.apache.org/jira/browse/OAK-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346638#comment-16346638
]
Julian Reschke commented on OAK-5506:
-
results from {{java -jar target/oak-benchmarks-1.10-SNAPSHOT.jar benchmark
Oak-Segment-Tar StringWriteTest}}:
new:
{noformat}
Apache Jackrabbit Oak 1.10-SNAPSHOT
# StringWriteTest C min 10% 50% 90% max
N
Oak-Segment-Tar1 0 0 0 16 33
16726
Oak-Segment-Tar1 0 0 0 16 47
17000
Oak-Segment-Tar1 0 0 0 16 47
17739
Oak-Segment-Tar1 0 0 0 15 46
17635
{noformat}
old:
{noformat}
Apache Jackrabbit Oak 1.10-SNAPSHOT
# StringWriteTest C min 10% 50% 90% max
N
Oak-Segment-Tar1 0 0 0 16 35
16790
Oak-Segment-Tar1 0 0 0 16 47
17092
Oak-Segment-Tar1 0 0 0 16 46
16937
Oak-Segment-Tar1 0 0 0 16 47
16851
{noformat}
> reject item names with unpaired surrogates early
>
>
> Key: OAK-5506
> URL: https://issues.apache.org/jira/browse/OAK-5506
> Project: Jackrabbit Oak
> Issue Type: Wish
> Components: core, jcr, segment-tar
>Affects Versions: 1.5.18
>Reporter: Julian Reschke
>Assignee: Francesco Mari
>Priority: Minor
> Fix For: 1.10
>
> Attachments: OAK-5506-01.patch, OAK-5506-02.patch, OAK-5506-4.diff,
> OAK-5506-bench.diff, OAK-5506-name-conversion.diff, OAK-5506-segment.diff,
> OAK-5506-segment2.diff, OAK-5506.diff, ValidNamesTest.java
>
>
> Apparently, the following node name is accepted:
>{{"foo\ud800"}}
> but a subsequent {{getPath()}} call fails:
> {noformat}
> javax.jcr.InvalidItemStateException: This item [/test_node/foo?] does not
> exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.operation.ItemOperation.checkPreconditions(ItemOperation.java:34)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.getPath(ItemImpl.java:140)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getPath(NodeImpl.java:106)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.nameTest(ValidNamesTest.java:271)
> at
> org.apache.jackrabbit.oak.jcr.ValidNamesTest.testUnpairedSurrogate(ValidNamesTest.java:259)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source){noformat}
> (test case follows)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346629#comment-16346629
]
Alex Deparvu commented on OAK-7203:
---
bq. bind/unbind removed additionally
could you explain why this is needed? To me the test code now looks pretty
ugly: catch an IllegalAccessException to wrap it into a RuntimeException?
The current design:
* field is set to default so it has a value outside OSGi (tests & other
setups), if needed one can change it by calling *bind
* field is set via OSGi in which case unbind will also be called so the
reference needs to be set to _null_ for consistency, _not the default value_,
as we are not in the default case anymore
To me this seems to cover both OSGi and non-OSGi cases but the current code has
one issue: the reference is mandatory. We can just add the required tag to the
reference and be done. Why all the extra changes?
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7203) Make MountInfoProvider service in AuthorizationConfigurationImpl optional
[
https://issues.apache.org/jira/browse/OAK-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346625#comment-16346625
]
Robert Munteanu commented on OAK-7203:
--
Thinking about this a bit more, I think the current setup is correct. We should
not allow a 'default' service to be registered or default values to be used
when it's possible that the value will change. The components must only use the
'right' MountInfoProvider from the beginning, otherwise we open the door to
inconsistent behaviour.
The only thing that is sub-optimal here is that the {{oak-store-composite}}
bundle is now required. I don't think that's a huge issue, but we can work
around it moving the {{MountInfoProviderService}} class from
{{oak-store-composite}} to {{oak-core}}, so that the {{MountInfoProvider}}
service can be registered with an empty config without adding an extra bundle.
> Make MountInfoProvider service in AuthorizationConfigurationImpl optional
> -
>
> Key: OAK-7203
> URL: https://issues.apache.org/jira/browse/OAK-7203
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: authorization-cug, core
>Affects Versions: 1.8.1
>Reporter: Oliver Lietz
>Priority: Major
> Attachments: OAK-7203.patch
>
>
> While testing Sling with Oak 1.8 I've observed that
> AuthorizationConfigurationImpl gets not activated due to missing
> MountInfoProvider service:
> {noformat}
> @Reference
> private MountInfoProvider mountInfoProvider =
> Mounts.defaultMountInfoProvider();
> {noformat}
> {noformat}
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Bundleorg.apache.jackrabbit.oak-core (63)
> Implementation Class
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Default State enabled
> Activationdelayed
> Configuration Policy optional
> Service Type singleton
> Services
> org.apache.jackrabbit.oak.spi.security.authorization.AuthorizationConfiguration
> org.apache.jackrabbit.oak.spi.security.SecurityConfiguration
> PID
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> Reference mountInfoProvider Unsatisfied
> Service Name: org.apache.jackrabbit.oak.spi.mount.MountInfoProvider
> Cardinality: 1..1
> Policy: static
> Policy Option: reluctant
> No Services bound
> Propertiescomponent.id = 35
> component.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> configurationRanking = 100
> importBehavior = abort
> oak.security.name =
> org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl
> readPaths = [/jcr:system/rep:namespaces, /jcr:system/jcr:nodeTypes,
> /jcr:system/rep:privileges]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7182) Make it possible to update Guava
[ https://issues.apache.org/jira/browse/OAK-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346628#comment-16346628 ] Davide Giannella commented on OAK-7182: --- I've added a subtask about the AtomicCounter aspect and guava. While it's not strictly an API it's used during repository construction and I'm in favour to try to remove the guava dependency completely in such case. That does not exclude that Guava is not used in other places of the functionality. > Make it possible to update Guava > > > Key: OAK-7182 > URL: https://issues.apache.org/jira/browse/OAK-7182 > Project: Jackrabbit Oak > Issue Type: Wish >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Attachments: guava.diff > > > We currently rely on Guava 15, and this affects all users of Oak because they > essentially need to use the same version. > This is an overall issue to investigate what would need to be done in Oak in > order to make updates possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7225) Replace AtomicCounter Supplier
Davide Giannella created OAK-7225: - Summary: Replace AtomicCounter Supplier Key: OAK-7225 URL: https://issues.apache.org/jira/browse/OAK-7225 Project: Jackrabbit Oak Issue Type: Sub-task Components: core Affects Versions: 1.6.0, 1.4.0 Reporter: Davide Giannella Assignee: Davide Giannella In the [AtomicCounter|https://github.com/apache/jackrabbit-oak/blob/7a7aa1e5d4f53f5bfb410f58264c237b288f5c74/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/atomic/AtomicCounterEditorProvider.java#L121] we use guava's Supplier which should be trivially replaced by the JDK8 [java.util.function.Supplier|https://docs.oracle.com/javase/8/docs/api/java/util/function/Supplier.html]. In case of backports to Oak 1.4, and therefore Java7 it should be possible to workaround the FunctionalInterface with a utility class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-6373) oak-run check should also check checkpoints
[
https://issues.apache.org/jira/browse/OAK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346620#comment-16346620
]
Andrei Dulceanu commented on OAK-6373:
--
[~frm], thanks for reviewing!
Fixed in trunk at r1822791.
> oak-run check should also check checkpoints
>
>
> Key: OAK-6373
> URL: https://issues.apache.org/jira/browse/OAK-6373
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
>Priority: Major
> Labels: candidate_oak_1_8, tooling
> Fix For: 1.9.0, 1.10
>
>
> {{oak-run check}} does currently *not* traverse and check the items in the
> checkpoint. I think we should change this and add an option to traverse all,
> some or none of the checkpoints. When doing this we need to keep in mind the
> interaction of this new feature with the {{filter}} option: the paths passed
> through this option need then be prefixed with {{/root}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346619#comment-16346619 ] Amit Jain commented on OAK-7223: [~mreutegg] Could you please review the patch. > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7223) Files could be kept partially in case of disconnection from backends
[ https://issues.apache.org/jira/browse/OAK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-7223: --- Attachment: OAK-7223.patch > Files could be kept partially in case of disconnection from backends > > > Key: OAK-7223 > URL: https://issues.apache.org/jira/browse/OAK-7223 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Affects Versions: 1.8.1 >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Blocker > Fix For: 1.10, 1.8.2 > > Attachments: OAK-7223.patch > > > The FileCache#load method needs to clean the files which may have been > downloaded partially in case of errors from backend. This partially > downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-6373) oak-run check should also check checkpoints
[
https://issues.apache.org/jira/browse/OAK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346604#comment-16346604
]
Francesco Mari commented on OAK-6373:
-
[~dulceanu], the latest commit in your branch looks good to me.
> oak-run check should also check checkpoints
>
>
> Key: OAK-6373
> URL: https://issues.apache.org/jira/browse/OAK-6373
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
>Priority: Major
> Labels: candidate_oak_1_8, tooling
> Fix For: 1.9.0, 1.10
>
>
> {{oak-run check}} does currently *not* traverse and check the items in the
> checkpoint. I think we should change this and add an option to traverse all,
> some or none of the checkpoints. When doing this we need to keep in mind the
> interaction of this new feature with the {{filter}} option: the paths passed
> through this option need then be prefixed with {{/root}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7224) oak-run check should have an option to check the segments checksums
[
https://issues.apache.org/jira/browse/OAK-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346566#comment-16346566
]
Valentin Olteanu commented on OAK-7224:
---
If this is a completely different type of check, e.g. only checksum of
segments, I would create another command: {{check-segments}}. But if it's doing
both the classical check and the segments checksum, then it's ok to have a
parameter...
> oak-run check should have an option to check the segments checksums
> ---
>
> Key: OAK-7224
> URL: https://issues.apache.org/jira/browse/OAK-7224
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: tooling
> Fix For: 1.9.0, 1.10
>
>
> {{oak-run check}} does currently *not* check the checksums of the segments.
> As a consequence, there is no quick way of determining the state of the
> repository (corrupt/valid), after corrupting some random node record, as we
> currently do in {{CheckRepositoryTestBase#corruptRecord}}. To determine that,
> there needs to be an attempt to read the corrupt record as part of a
> traversal.
> An easier way would be to have a new dedicated option for this (i.e.,
> {{--segments}}) which checks by default the content of segments against the
> checksums from all the tar files in the specified location. Additionally, it
> could accept as an argument a list of tar files, the segments of which to be
> checked.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7224) oak-run check should have an option to check the segments checksums
[
https://issues.apache.org/jira/browse/OAK-7224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7224:
-
Description:
{{oak-run check}} does currently *not* check the checksums of the segments. As
a consequence, there is no quick way of determining the state of the repository
(corrupt/valid), after corrupting some random node record, as we currently do
in {{CheckRepositoryTestBase#corruptRecord}}. To determine that, there needs to
be an attempt to read the corrupt record as part of a traversal.
An easier way would be to have a new dedicated option for this (i.e.,
{{--segments}}) which checks by default the content of segments against the
checksums from all the tar files in the specified location. Additionally, it
could accept as an argument a list of tar files, the segments of which to be
checked.
was:
{{oak-run check}} does currently *not* check the checksums of the segments. As
a consequence, there is no quick way of determining the state of the repository
(corrupt/valid), after corrupting some random node record, as we currently do
in {{CheckRepositoryTestBase#corruptRecord}}. To determine that, there needs to
be an attempt to read the corrupt record as part of a traversal.
An easier way would be to have a new dedicated option for this (i.e.,
{{--segments}}) which checks by default all segments from all the tar files in
the specified location. Additionally, it could accept as an argument a list of
tar files, the segments of which to be checked.
> oak-run check should have an option to check the segments checksums
> ---
>
> Key: OAK-7224
> URL: https://issues.apache.org/jira/browse/OAK-7224
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run, segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: tooling
> Fix For: 1.9.0, 1.10
>
>
> {{oak-run check}} does currently *not* check the checksums of the segments.
> As a consequence, there is no quick way of determining the state of the
> repository (corrupt/valid), after corrupting some random node record, as we
> currently do in {{CheckRepositoryTestBase#corruptRecord}}. To determine that,
> there needs to be an attempt to read the corrupt record as part of a
> traversal.
> An easier way would be to have a new dedicated option for this (i.e.,
> {{--segments}}) which checks by default the content of segments against the
> checksums from all the tar files in the specified location. Additionally, it
> could accept as an argument a list of tar files, the segments of which to be
> checked.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7224) oak-run check should have an option to check the segments checksums
Andrei Dulceanu created OAK-7224:
Summary: oak-run check should have an option to check the segments
checksums
Key: OAK-7224
URL: https://issues.apache.org/jira/browse/OAK-7224
Project: Jackrabbit Oak
Issue Type: Improvement
Components: run, segment-tar
Reporter: Andrei Dulceanu
Assignee: Andrei Dulceanu
Fix For: 1.9.0, 1.10
{{oak-run check}} does currently *not* check the checksums of the segments. As
a consequence, there is no quick way of determining the state of the repository
(corrupt/valid), after corrupting some random node record, as we currently do
in {{CheckRepositoryTestBase#corruptRecord}}. To determine that, there needs to
be an attempt to read the corrupt record as part of a traversal.
An easier way would be to have a new dedicated option for this (i.e.,
{{--segments}}) which checks by default all segments from all the tar files in
the specified location. Additionally, it could accept as an argument a list of
tar files, the segments of which to be checked.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7162) Race condition on revisions head between compaction and scheduler could result in skipped commit
[
https://issues.apache.org/jira/browse/OAK-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346521#comment-16346521
]
Michael Dürig commented on OAK-7162:
Reopened since backport to 1.8 is still pending.
> Race condition on revisions head between compaction and scheduler could
> result in skipped commit
>
>
> Key: OAK-7162
> URL: https://issues.apache.org/jira/browse/OAK-7162
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: 1.8.0
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Blocker
> Labels: scalability
> Fix For: 1.9.0, 1.10, 1.8.2
>
> Attachments: OAK-7162-02.patch, OAK-7162-03.patch, OAK-7162.patch
>
>
> There is a race condition on {{TarRevisions#head}} between a running
> compaction trying to set the new head [0] and the scheduler doing the same
> after executing a specific commit [1]. If the compaction thread is first,
> then the head assignment in the scheduler will fail and not be re-attempted.
> IMO, the simple if statement should be changed to a while loop in which the
> head is refreshed and the commit is re-applied against the new head, before
> attempting again to set a new head in {{TarRevisions}}. This is somehow
> similar to what we previously had [2], but without the unneeded
> optimistic/pessimistic strategies involving tokens.
> [0]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/FileStore.java#L764
> [1]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/scheduler/LockBasedScheduler.java#L253
> [2]
> https://github.com/apache/jackrabbit-oak/blob/1.6/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentNodeStore.java#L686
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (OAK-7162) Race condition on revisions head between compaction and scheduler could result in skipped commit
[
https://issues.apache.org/jira/browse/OAK-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella reopened OAK-7162:
---
> Race condition on revisions head between compaction and scheduler could
> result in skipped commit
>
>
> Key: OAK-7162
> URL: https://issues.apache.org/jira/browse/OAK-7162
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: 1.8.0
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Blocker
> Labels: scalability
> Fix For: 1.9.0, 1.10, 1.8.2
>
> Attachments: OAK-7162-02.patch, OAK-7162-03.patch, OAK-7162.patch
>
>
> There is a race condition on {{TarRevisions#head}} between a running
> compaction trying to set the new head [0] and the scheduler doing the same
> after executing a specific commit [1]. If the compaction thread is first,
> then the head assignment in the scheduler will fail and not be re-attempted.
> IMO, the simple if statement should be changed to a while loop in which the
> head is refreshed and the commit is re-applied against the new head, before
> attempting again to set a new head in {{TarRevisions}}. This is somehow
> similar to what we previously had [2], but without the unneeded
> optimistic/pessimistic strategies involving tokens.
> [0]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/FileStore.java#L764
> [1]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/scheduler/LockBasedScheduler.java#L253
> [2]
> https://github.com/apache/jackrabbit-oak/blob/1.6/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentNodeStore.java#L686
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6922) Azure support for the segment-tar
[
https://issues.apache.org/jira/browse/OAK-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-6922:
---
Description:
An Azure Blob Storage implementation of the segment storage, based on the
OAK-6921 work.
h3. Segment files layout
Thew new implementation doesn't use tar files. They are replaced with
directories, storing segments, named after their UUIDs. This approach has
following advantages:
* no need to call seek(), which may be expensive on a remote file system.
Rather than that we can read the whole file (=segment) at once.
* it's possible to send multiple segments at once, asynchronously, which
reduces the performance overhead (see below).
The file structure is as follows:
{noformat}
$ hdfs dfs -ls /oak/data0a.tar
Found 517 items
-rw-r--r-- 1 rekawek supergroup192 2017-11-08 13:24
/oak/data0a.tar/.b1d032cd-266d-4fd6-acf4-9828f54e2b40
-rw-r--r-- 1 rekawek supergroup 262112 2017-11-08 13:24
/oak/data0a.tar/0001.445ca696-d5d1-4843-a04b-044f84d93663
-rw-r--r-- 1 rekawek supergroup 262144 2017-11-08 13:24
/oak/data0a.tar/0002.91ce6f93-d7ed-4d34-a383-3c3d2eea2acb
-rw-r--r-- 1 rekawek supergroup 262144 2017-11-08 13:24
/oak/data0a.tar/0003.43c09e6f-3d62-4747-ac75-de41b850862a
(...)
-rw-r--r-- 1 rekawek supergroup 191888 2017-11-08 13:32
/oak/data0a.tar/data0a.tar.brf
-rw-r--r-- 1 rekawek supergroup 823436 2017-11-08 13:32
/oak/data0a.tar/data0a.tar.gph
-rw-r--r-- 1 rekawek supergroup 17408 2017-11-08 13:32
/oak/data0a.tar/data0a.tar.idx
{noformat}
For the segment files, each name is prefixed with the index number. This allows
to maintain an order, as in the tar archive. This order is normally stored in
the index files as well, but if it's missing, the recovery process needs it.
Each file contains the raw segment data, with no padding/headers. Apart from
the segment files, there are 3 special files: binary references (.brf), segment
graph (.gph) and segment index (.idx).
h3. Asynchronous writes
Normally, all the TarWriter writes are synchronous, appending the segments to
the tar file. In case of Azure Blob Storage each write involves a network
latency. That's why the SegmentWriteQueue was introduced. The segments are
added to the blocking dequeue, which is served by a number of the consumer
threads, writing the segments to the cloud. There's also a map UUID->Segment,
which allows to return the segments in case they are requested by the
readSegment() method before they are actually persisted. Segments are removed
from the map only after a successful write operation.
The flush() method blocks accepting the new segments and returns after all
waiting segments are written. The close() method waits until the current
operations are finished and stops all threads.
The asynchronous mode can be disabled by setting the number of threads to 0.
h5. Queue recovery mode
If the Azure Blob Storage write() operation fails, the segment will be re-added
and the queue is switched to an "recovery mode". In this mode, all the threads
are suspended and new segments are not accepted (active waiting). There's a
single thread which retries adding the segment with some delay. If the segment
is successfully written, the queue will back to the normal operation.
This way the unavailable remote service is not flooded by the requests and
we're not accepting the segments when we can't persist them.
The close() method finishes the recovery mode - in this case, some of the
awaiting segments won't be persisted.
h5. Consistency
The asynchronous mode isn't as reliable as the standard, synchronous case.
Following cases are possible:
* TarWriter#writeEntry() returns successfully, but the segments are not
persisted.
* TarWriter#writeEntry() accepts a number of segments: S1, S2, S3. The S2 and
S3 are persisted, but the S1 is not.
On the other hand:
* If the TarWriter#flush() returns successfully, it means that all the accepted
segments has been persisted.
h5. Recovery
During the segment recovery (eg. if the index file is missing), the HDFS
implementation checks if there's no missing segment in the middle. If so, only
the consecutive segments are recovered. For instance, if we have S1, S2, S3,
S5, S6, S7, then the recovery process will return only the first three.
was:
A HDFS implementation of the segment storage, based on the OAK-6921 work.
h3. HDFS
HDFS is a distributed, network file system. The most popular implementation is
Apache Hadoop, but the HDFS is also available in the Amazon AWS and Microsoft
Azure clouds. Despite being a file system, it requires a custom API to be used
- it's not similar to NFS or CIFS which can be simply mounted locally.
h3. Segment files layout
Thew new implementation doesn't use tar files. They are replaced with
directories, storing segments, named after their
[jira] [Updated] (OAK-6921) Support pluggable segment storage
[
https://issues.apache.org/jira/browse/OAK-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-6921:
---
Description:
h3. Rationale
segment-tar, as names suggest, stores the segments in a bunch of tar archives,
inside the {{segmentstore}} directory on the local file system. For some cases,
especially in the cloud deployments, it may be interesting to store the
segments outside the local FS - the remote storage such as Amazon S3, Azure
Blob Storage or HDFS may be cheaper than a mounted disk, more scalable, easier
for the provisioning, etc.
h3. Storing segment in tar files
!current-state.png!
There are 3 classes responsible for handling tar files in the segment-tar:
TarFiles, TarWriter and TarReader. The TarFiles manages the {{segmentstore}}
directory, scans it for the .tars and for each one creates a TarReader. It also
creates a single TarWriter object, used to write (and also read) the most
recent tar file.
The TarWriter appends segments to the latest tar file and also serializes the
auxiliary indexes: segment index, binary references index and the segment
graph. It also takes of synchronization, as we're dealing with a mutable data
structure - tar file opened in the append mode.
The TarReader not only reads the segments from the tar file, but is also
responsible for the revision GC (mark & sweep methods) and recovering data from
files which hasn't been closed cleanly (eg. have no index).
h3. New abstraction layer - SegmentArchiveManager
!new-interfaces.png!
In order to store segments not in the tar files, but somewhere else, it'd be
possible to create own implementation of the TarFiles, TarWriter and TarReader.
However, such implementation would duplicate a lot of code, not strictly
related to the persistence - mark(), sweep(), synchronization, etc. Rather than
that, the attached patch presents a different approach: a new layer of
abstraction is injected into TarFiles, TarWriter and TarReader - it only takes
care of the segments persistence and knows nothing about the synchronization,
GC, etc. - leaving it to the upper layer.
The new abstraction layer is modelled using 3 new classes:
SegmentArchiveManager, SegmentArchiveReader and SegmentArchiveWriter. They are
strictly related to the existing Tar* classes and used by them.
SegmentArchiveManager provides a bunch of file system-style methods, like
open(), create(), delete(), exists(), etc. The open() and create() returns
instances of the SAReader and SAWriter.
SegmentArchiveReader, despite from reading segments, can also load and parse
the index, graph and binary references. The logic responsible for parsing these
structures has been already extracted, so it doesn't need to be duplicated in
the SAReader implementations. Also, SAReader needs to be aware about the index,
since it contains the segment offsets.
The SAWriter class allows to write and read the segments and also store the
indexes. It isn't thread safe - it assumes that the synchronization is already
done on the higher layers.
In the patch, I've moved the tar implementation to the new classes:
SegmentTarManager, SegmentTarReader and SegmentTarWriter.
h3. Other files
Apart from the segments, the {{segmentstore}} directory also contains following
files:
* repo.lock
* journal.log
* gc.log
* manifest
All these files are supported by the new SegmentNodeStorePersistence. Usually
there's a simple interface (RepositoryLock, JournalLogFile, etc.) for handling
the files.
h3. TODO
* The names and package locations for all the affected classes are subjects to
change - after applying the patch the TarFiles doesn't deal with the .tar files
anymore, similarly the TarReader and TarWriter delegates the low-level file
access duties to the SegmentArchiveReader and Writer. I didn't want to change
the names yet, to make it easier to understand and rebase the patch with the
trunk changes.
was:
h3. Rationale
segment-tar, as names suggest, stores the segments in a bunch of tar archives,
inside the {{segmentstore}} directory on the local file system. For some cases,
especially in the cloud deployments, it may be interesting to store the
segments outside the local FS - the remote storage such as Amazon S3, Azure
Blob Storage or HDFS may be cheaper than a mounted disk, more scalable, easier
for the provisioning, etc.
h3. Storing segment in tar files
!current-state.png!
There are 3 classes responsible for handling tar files in the segment-tar:
TarFiles, TarWriter and TarReader. The TarFiles manages the {{segmentstore}}
directory, scans it for the .tars and for each one creates a TarReader. It also
creates a single TarWriter object, used to write (and also read) the most
recent tar file.
The TarWriter appends segments to the latest tar file and also serializes the
auxiliary indexes: segment index, binary references index and the segment
graph. It also takes
[jira] [Issue Comment Deleted] (OAK-6922) Azure support for the segment-tar
[
https://issues.apache.org/jira/browse/OAK-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-6922:
---
Comment: was deleted
(was: https://github.com/trekawek/jackrabbit-oak/tree/segment-tar/hdfs)
> Azure support for the segment-tar
> -
>
> Key: OAK-6922
> URL: https://issues.apache.org/jira/browse/OAK-6922
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6922.patch
>
>
> An Azure Blob Storage implementation of the segment storage, based on the
> OAK-6921 work.
> h3. Segment files layout
> Thew new implementation doesn't use tar files. They are replaced with
> directories, storing segments, named after their UUIDs. This approach has
> following advantages:
> * no need to call seek(), which may be expensive on a remote file system.
> Rather than that we can read the whole file (=segment) at once.
> * it's possible to send multiple segments at once, asynchronously, which
> reduces the performance overhead (see below).
> The file structure is as follows:
> {noformat}
> $ hdfs dfs -ls /oak/data0a.tar
> Found 517 items
> -rw-r--r-- 1 rekawek supergroup192 2017-11-08 13:24
> /oak/data0a.tar/.b1d032cd-266d-4fd6-acf4-9828f54e2b40
> -rw-r--r-- 1 rekawek supergroup 262112 2017-11-08 13:24
> /oak/data0a.tar/0001.445ca696-d5d1-4843-a04b-044f84d93663
> -rw-r--r-- 1 rekawek supergroup 262144 2017-11-08 13:24
> /oak/data0a.tar/0002.91ce6f93-d7ed-4d34-a383-3c3d2eea2acb
> -rw-r--r-- 1 rekawek supergroup 262144 2017-11-08 13:24
> /oak/data0a.tar/0003.43c09e6f-3d62-4747-ac75-de41b850862a
> (...)
> -rw-r--r-- 1 rekawek supergroup 191888 2017-11-08 13:32
> /oak/data0a.tar/data0a.tar.brf
> -rw-r--r-- 1 rekawek supergroup 823436 2017-11-08 13:32
> /oak/data0a.tar/data0a.tar.gph
> -rw-r--r-- 1 rekawek supergroup 17408 2017-11-08 13:32
> /oak/data0a.tar/data0a.tar.idx
> {noformat}
> For the segment files, each name is prefixed with the index number. This
> allows to maintain an order, as in the tar archive. This order is normally
> stored in the index files as well, but if it's missing, the recovery process
> needs it.
> Each file contains the raw segment data, with no padding/headers. Apart from
> the segment files, there are 3 special files: binary references (.brf),
> segment graph (.gph) and segment index (.idx).
> h3. Asynchronous writes
> Normally, all the TarWriter writes are synchronous, appending the segments to
> the tar file. In case of Azure Blob Storage each write involves a network
> latency. That's why the SegmentWriteQueue was introduced. The segments are
> added to the blocking dequeue, which is served by a number of the consumer
> threads, writing the segments to the cloud. There's also a map UUID->Segment,
> which allows to return the segments in case they are requested by the
> readSegment() method before they are actually persisted. Segments are removed
> from the map only after a successful write operation.
> The flush() method blocks accepting the new segments and returns after all
> waiting segments are written. The close() method waits until the current
> operations are finished and stops all threads.
> The asynchronous mode can be disabled by setting the number of threads to 0.
> h5. Queue recovery mode
> If the Azure Blob Storage write() operation fails, the segment will be
> re-added and the queue is switched to an "recovery mode". In this mode, all
> the threads are suspended and new segments are not accepted (active waiting).
> There's a single thread which retries adding the segment with some delay. If
> the segment is successfully written, the queue will back to the normal
> operation.
> This way the unavailable remote service is not flooded by the requests and
> we're not accepting the segments when we can't persist them.
> The close() method finishes the recovery mode - in this case, some of the
> awaiting segments won't be persisted.
> h5. Consistency
> The asynchronous mode isn't as reliable as the standard, synchronous case.
> Following cases are possible:
> * TarWriter#writeEntry() returns successfully, but the segments are not
> persisted.
> * TarWriter#writeEntry() accepts a number of segments: S1, S2, S3. The S2 and
> S3 are persisted, but the S1 is not.
> On the other hand:
> * If the TarWriter#flush() returns successfully, it means that all the
> accepted segments has been persisted.
> h5. Recovery
> During the segment recovery (eg. if the index file is missing), the HDFS
> implementation checks if there's no missing segment in the middle. If so,
> only the consecutive segments are recovered. For instance, if we have
[jira] [Updated] (OAK-6922) Azure support for the segment-tar
[
https://issues.apache.org/jira/browse/OAK-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-6922:
---
Summary: Azure support for the segment-tar (was: HDFS support for the
segment-tar)
> Azure support for the segment-tar
> -
>
> Key: OAK-6922
> URL: https://issues.apache.org/jira/browse/OAK-6922
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6922.patch
>
>
> A HDFS implementation of the segment storage, based on the OAK-6921 work.
> h3. HDFS
> HDFS is a distributed, network file system. The most popular implementation
> is Apache Hadoop, but the HDFS is also available in the Amazon AWS and
> Microsoft Azure clouds. Despite being a file system, it requires a custom API
> to be used - it's not similar to NFS or CIFS which can be simply mounted
> locally.
> h3. Segment files layout
> Thew new implementation doesn't use tar files. They are replaced with
> directories, storing segments, named after their UUIDs. This approach has
> following advantages:
> * no need to call seek(), which may be expensive on a remote file system.
> Rather than that we can read the whole file (=segment) at once.
> * it's possible to send multiple segments at once, asynchronously, which
> reduces the performance overhead (see below).
> The file structure is as follows:
> {noformat}
> $ hdfs dfs -ls /oak/data0a.tar
> Found 517 items
> -rw-r--r-- 1 rekawek supergroup192 2017-11-08 13:24
> /oak/data0a.tar/.b1d032cd-266d-4fd6-acf4-9828f54e2b40
> -rw-r--r-- 1 rekawek supergroup 262112 2017-11-08 13:24
> /oak/data0a.tar/0001.445ca696-d5d1-4843-a04b-044f84d93663
> -rw-r--r-- 1 rekawek supergroup 262144 2017-11-08 13:24
> /oak/data0a.tar/0002.91ce6f93-d7ed-4d34-a383-3c3d2eea2acb
> -rw-r--r-- 1 rekawek supergroup 262144 2017-11-08 13:24
> /oak/data0a.tar/0003.43c09e6f-3d62-4747-ac75-de41b850862a
> (...)
> -rw-r--r-- 1 rekawek supergroup 191888 2017-11-08 13:32
> /oak/data0a.tar/data0a.tar.brf
> -rw-r--r-- 1 rekawek supergroup 823436 2017-11-08 13:32
> /oak/data0a.tar/data0a.tar.gph
> -rw-r--r-- 1 rekawek supergroup 17408 2017-11-08 13:32
> /oak/data0a.tar/data0a.tar.idx
> {noformat}
> For the segment files, each name is prefixed with the index number. This
> allows to maintain an order, as in the tar archive. This order is normally
> stored in the index files as well, but if it's missing, the recovery process
> needs it.
> Each file contains the raw segment data, with no padding/headers. Apart from
> the segment files, there are 3 special files: binary references (.brf),
> segment graph (.gph) and segment index (.idx).
> h3. Asynchronous writes
> Normally, all the TarWriter writes are synchronous, appending the segments to
> the tar file. In case of HDFS each write involves a network latency. That's
> why the SegmentWriteQueue was introduced. The segments are added to the
> blocking deque, which is served by a number of the consumer threads, writing
> the segments to HDFS. There's also a map UUID->Segment, which allows to
> return the segments in case they are requested by the readSegment() method
> before they are actually persisted. Segments are removed from the map only
> after a successful write operation.
> The flush() method blocks accepting the new segments and returns after all
> waiting segments are written. The close() method waits until the current
> operations are finished and stops all threads.
> The asynchronous mode can be disabled by setting the number of threads to 0.
> h5. Queue recovery mode
> If the HDFS write() operation fails, the segment will be re-added and the
> queue is switched to an "recovery mode". In this mode, all the threads are
> suspended and new segments are not accepted (active waiting). There's a
> single thread which retries adding the segment with some delay. If the
> segment is successfully written, the queue will back to the normal operation.
> This way the unavailable HDFS service is not flooded by the requests and
> we're not accepting the segments when we can't persist them.
> The close() method finishes the recovery mode - in this case, some of the
> awaiting segments won't be persisted.
> h5. Consistency
> The asynchronous mode isn't as reliable as the standard, synchronous case.
> Following cases are possible:
> * TarWriter#writeEntry() returns successfully, but the segments are not
> persisted.
> * TarWriter#writeEntry() accepts a number of segments: S1, S2, S3. The S2 and
> S3 are persisted, but the S1 is not.
> On the other hand:
> * If the TarWriter#flush() returns successfully, it means that all the
> accepted
[jira] [Created] (OAK-7223) Files could be kept partially in case of disconnection from backends
Amit Jain created OAK-7223: -- Summary: Files could be kept partially in case of disconnection from backends Key: OAK-7223 URL: https://issues.apache.org/jira/browse/OAK-7223 Project: Jackrabbit Oak Issue Type: Bug Components: blob-plugins Affects Versions: 1.8.1 Reporter: Amit Jain Assignee: Amit Jain Fix For: 1.10, 1.8.2 The FileCache#load method needs to clean the files which may have been downloaded partially in case of errors from backend. This partially downloaded file should be removed before returning from the method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
