[jira] [Created] (OAK-3872) [RDB] Updated blob still deleted even if deletion interval lower
Amit Jain created OAK-3872:
--
Summary: [RDB] Updated blob still deleted even if deletion
interval lower
Key: OAK-3872
URL: https://issues.apache.org/jira/browse/OAK-3872
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob, rdbmk
Reporter: Amit Jain
If an existing blob is uploaded again, the timestamp of the existing entry is
updated in the meta table. Subsequently if a call to delete
(RDBBlobStore#countDeleteBlobs) is made with {{maxLastModifiedTime}} parameter
of less than the updated time above, the entry in the meta table is not touched
but the data table entry is wiped out.
Refer
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/rdb/RDBBlobStore.java#L510
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-3872) [RDB] Updated blob still deleted even if deletion interval lower
[
https://issues.apache.org/jira/browse/OAK-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke reassigned OAK-3872:
---
Assignee: Julian Reschke
> [RDB] Updated blob still deleted even if deletion interval lower
>
>
> Key: OAK-3872
> URL: https://issues.apache.org/jira/browse/OAK-3872
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob, rdbmk
>Reporter: Amit Jain
>Assignee: Julian Reschke
> Attachments: OAK_3872.patch
>
>
> If an existing blob is uploaded again, the timestamp of the existing entry is
> updated in the meta table. Subsequently if a call to delete
> (RDBBlobStore#countDeleteBlobs) is made with {{maxLastModifiedTime}}
> parameter of less than the updated time above, the entry in the meta table is
> not touched but the data table entry is wiped out.
> Refer
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/rdb/RDBBlobStore.java#L510
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3809) Test failure: FacetTest
[
https://issues.apache.org/jira/browse/OAK-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095723#comment-15095723
]
Marcel Reutegger commented on OAK-3809:
---
Put the test on the list of known issues: http://svn.apache.org/r1724359
> Test failure: FacetTest
> ---
>
> Key: OAK-3809
> URL: https://issues.apache.org/jira/browse/OAK-3809
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>Assignee: Tommaso Teofili
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.FacetTest}} keeps failing on Jenkins:
> {noformat}
> testFacetRetrievalMV(org.apache.jackrabbit.oak.jcr.query.FacetTest) Time
> elapsed: 5.927 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected: (2), aem (1), apache (1), cosmetics (1), furniture (1)], tags:[repository
> (2), software (2), aem (1), apache (1), cosmetics (1), furniture (1)],
> tags:[repository (2), software (2), aem (1), apache (1), cosmetics (1),
> furniture (1)], tags:[repository (2), software (2), aem (1), apache (1),
> cosmetics (1), furniture (1)]]> but was:
> at junit.framework.Assert.assertEquals(Assert.java:100)
> at junit.framework.Assert.assertEquals(Assert.java:107)
> at junit.framework.TestCase.assertEquals(TestCase.java:269)
> at
> org.apache.jackrabbit.oak.jcr.query.FacetTest.testFacetRetrievalMV(FacetTest.java:80)
> {noformat}
> Failure seen at builds: 628, 629, 630, 633, 634, 636, 642, 643, 644, 645,
> 648, 651, 656, 659, 660, 663, 666
> See e.g.
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/634/#showFailuresLink
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3861) MapRecord reduce extra loop in MapEntry creation
[ https://issues.apache.org/jira/browse/OAK-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093548#comment-15093548 ] Michael Dürig commented on OAK-3861: +1 > MapRecord reduce extra loop in MapEntry creation > > > Key: OAK-3861 > URL: https://issues.apache.org/jira/browse/OAK-3861 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu >Priority: Trivial > Attachments: MapRecord.java.patch > > > removes unneeded extra loop and a bunch of temp arrays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093643#comment-15093643
]
Tomek Rękawek commented on OAK-3645:
Also, I wonder if we shouldn't change the time difference resolution to
seconds. We got a number of seconds*1000 from the DB and then we compare it to
number of millis from the local machine. As a result, even on a embedded
database, the time diff varies from 0 to 999 millis (as the milliseconds are
truncated on the database timestamp).
Maybe we should compare seconds and then multiply the difference by 1000 to be
consistent with the method contract? Something like:
{code}
stmt = connection.prepareStatement(sql);
long start = System.currentTimeMillis() / 1000;
rs = stmt.executeQuery();
if (rs.next()) {
long roundtrip = System.currentTimeMillis() / 1000 - start;
long serverTime = rs.getInt(1);
long roundedTime = start + roundtrip / 2;
result = roundedTime - serverTime;
String msg = String.format("instance timestamp: %d, DB timestamp: %d,
difference: %d", roundedTime, serverTime,
result);
System.err.println(msg);
if (Math.abs(result) >= 2) {
LOG.info(msg);
} else {
LOG.debug(msg);
}
} else {
throw new DocumentStoreException("failed to determine server timestamp");
}
return result * 1000;
{code}
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3863) [oak-blob-cloud] Incorrect export package
[ https://issues.apache.org/jira/browse/OAK-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-3863. Resolution: Fixed Committed changes: * trunk - http://svn.apache.org/viewvc?rev=1724186=rev * 1.0 - http://svn.apache.org/viewvc?rev=1724194=rev > [oak-blob-cloud] Incorrect export package > - > > Key: OAK-3863 > URL: https://issues.apache.org/jira/browse/OAK-3863 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.2.10, 1.3.14 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093603#comment-15093603
]
Julian Reschke commented on OAK-3645:
-
Actually, Derby doesn't work (in that it returns a value 1 hour off for me). We
need Derby for unit testing only, so if we can't make it work we should just
switch the detection off for it.
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093667#comment-15093667
]
Julian Reschke commented on OAK-3645:
-
Good point, but we probably should round System.currentTimeMillis, right?
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3809) Test failure: FacetTest
[
https://issues.apache.org/jira/browse/OAK-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-3809:
---
Description:
{{org.apache.jackrabbit.oak.jcr.query.FacetTest}} keeps failing on Jenkins:
{noformat}
testFacetRetrievalMV(org.apache.jackrabbit.oak.jcr.query.FacetTest) Time
elapsed: 5.927 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected: but was:
at junit.framework.Assert.assertEquals(Assert.java:100)
at junit.framework.Assert.assertEquals(Assert.java:107)
at junit.framework.TestCase.assertEquals(TestCase.java:269)
at
org.apache.jackrabbit.oak.jcr.query.FacetTest.testFacetRetrievalMV(FacetTest.java:80)
{noformat}
Failure seen at builds: 628, 629, 630, 633, 634, 636, 642, 643, 644, 645, 648,
651, 656, 659, 660, 663, 666
See e.g.
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/634/#showFailuresLink
was:
{{org.apache.jackrabbit.oak.jcr.query.FacetTest}} keeps failing on Jenkins:
{noformat}
testFacetRetrievalMV(org.apache.jackrabbit.oak.jcr.query.FacetTest) Time
elapsed: 5.927 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected: but was:
at junit.framework.Assert.assertEquals(Assert.java:100)
at junit.framework.Assert.assertEquals(Assert.java:107)
at junit.framework.TestCase.assertEquals(TestCase.java:269)
at
org.apache.jackrabbit.oak.jcr.query.FacetTest.testFacetRetrievalMV(FacetTest.java:80)
{noformat}
Failure seen at builds: 628, 629, 630, 633, 634, 636, 642, 643, 644, 645, 648,
651, 656, 659, 660, 663
See e.g.
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/634/#showFailuresLink
> Test failure: FacetTest
> ---
>
> Key: OAK-3809
> URL: https://issues.apache.org/jira/browse/OAK-3809
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>Assignee: Tommaso Teofili
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.FacetTest}} keeps failing on Jenkins:
> {noformat}
> testFacetRetrievalMV(org.apache.jackrabbit.oak.jcr.query.FacetTest) Time
> elapsed: 5.927 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected: (2), aem (1), apache (1), cosmetics (1), furniture (1)], tags:[repository
> (2), software (2), aem (1), apache (1), cosmetics (1), furniture (1)],
> tags:[repository (2), software (2), aem (1), apache (1), cosmetics (1),
> furniture (1)], tags:[repository (2), software (2), aem (1), apache (1),
> cosmetics (1), furniture (1)]]> but was:
> at junit.framework.Assert.assertEquals(Assert.java:100)
> at junit.framework.Assert.assertEquals(Assert.java:107)
> at junit.framework.TestCase.assertEquals(TestCase.java:269)
> at
> org.apache.jackrabbit.oak.jcr.query.FacetTest.testFacetRetrievalMV(FacetTest.java:80)
> {noformat}
> Failure seen at builds: 628, 629, 630, 633, 634, 636, 642, 643, 644, 645,
> 648, 651, 656, 659, 660, 663, 666
> See e.g.
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/634/#showFailuresLink
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093638#comment-15093638
]
Julian Reschke commented on OAK-3645:
-
For Derby,
{code}
@Override
public String getInitializationStatement() {
return "create function unixtime() returns bigint parameter style
java no sql language java external name 'java.lang.System.currentTimeMillis'";
}
@Override
public String getCurrentTimeStampInSecondsSyntax() {
return "values unixtime() / 1000";
}
{code}
would work except that we'd need something like "create function if not exists".
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3862) Move integration tests in a different Maven module
[
https://issues.apache.org/jira/browse/OAK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093567#comment-15093567
]
Francesco Mari commented on OAK-3862:
-
bq. Instead of moving all other users of NodeStoreFixture to separate module
may be we should compose NodeStoreFixture using ServiceLoader.
I already thought about this possibility. In this case, let's assume that
{{SegmentFixture}} lives in a {{oak-segment}} module, and that is loaded using
the {{ServiceLoader}} when the tests in {{oak-core}} are executed. Please note
that {{oak-segment}} has a dependency on {{oak-core}}, because the former uses
library code and APIs contained in the latter.
[~chetanm], when using the {{ServiceLoader}}, isn't it necessary to put
{{oak-segment}} in the classpath of {{oak-core}} while running integration
tests? Doesn't this create a circular dependency between {{oak-core}} and
{{oak-segment}}?
> Move integration tests in a different Maven module
> --
>
> Key: OAK-3862
> URL: https://issues.apache.org/jira/browse/OAK-3862
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4
>
>
> While moving the Segment Store and related packages into its own bundle, I
> figured out that integration tests contained in {{oak-core}} contribute to a
> cyclic dependency between the (new) {{oak-segment}} bundle and {{oak-core}}.
> The dependency is due to the usage of {{NodeStoreFixture}} to instantiate
> different implementations of {{NodeStore}} in a semi-transparent way.
> Tests depending on {{NodeStoreFixture}} are most likely integration tests. A
> clean solution to this problem would be to move those integration tests into
> a new Maven module, referencing the API and implementation modules as needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093628#comment-15093628
]
Tomek Rękawek commented on OAK-3645:
Yes, apparently Derby doesn't know anything about timezones and therefore
there's no way to get a correct value. Falling back to the default "" seems
like a good approach.
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3864) Filestore cleanup removes referenced segments
[
https://issues.apache.org/jira/browse/OAK-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093756#comment-15093756
]
Michael Dürig commented on OAK-3864:
I see 2 approaches to fixing this:
# fix the cleanup to be able to cope with cycles in the segment graph
# re-establish the invariant that segments never have forward references
While that invariant is certainly a very nice property to have, it has never
been strictly enforced. Even before OAK-1828 you could have run into this
situation by writing to the file store through multiple segment writers in a
certain way. The only reason we did not see the problem there is because we
didn't do this. With the current approach where we stripe writing segment
across various segment buffer writers it seems very hard to re-establish the
invariant.
OTOH making cleanup cope with cycles / forward references in the segment graph
probably means we need to turn this simple one pass algorithm into a two pass
mark and sweep like approach.
> Filestore cleanup removes referenced segments
> -
>
> Key: OAK-3864
> URL: https://issues.apache.org/jira/browse/OAK-3864
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Blocker
> Labels: regression
> Fix For: 1.3.14
>
>
> In some situations {{FileStore.cleanup()}} may remove segments that are still
> referenced, subsequently causing a {{SNFE}}.
> This is a regression introduced with OAK-1828.
> {{FileStore.cleanup()}} relies on the ordering of the segments in the tar
> files: later segments only reference earlier segments. As we have seen in
> other places this assumption does not hold any more (e.g. OAK-3794, OAK-3793)
> since OAK-1828.
> {{cleanup}} traverses the segments backwards maintaining a list of
> referenced ids. When a segment is not in that list, it is removed. However,
> this approach does not work with forward references as those are only seen
> later when the segment has been removed already.
> cc [~alex.parvulescu], [~frm]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke reassigned OAK-3645:
---
Assignee: Julian Reschke (was: Tomek Rękawek)
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3864) Filestore cleanup removes referenced segments
Michael Dürig created OAK-3864:
--
Summary: Filestore cleanup removes referenced segments
Key: OAK-3864
URL: https://issues.apache.org/jira/browse/OAK-3864
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
Priority: Blocker
Fix For: 1.4
In some situations {{FileStore.cleanup()}} may remove segments that are still
referenced, subsequently causing a {{SNFE}}.
This is a regression introduced with OAK-1828.
{{FileStore.cleanup()}} relies on the ordering of the segments in the tar
files: later segments only reference earlier segments. As we have seen in other
places this assumption does not hold any more (e.g. OAK-3794, OAK-3793) since
OAK-1828.
{{cleanup}} traverses the segments backwards maintaining a list of referenced
ids. When a segment is not in that list, it is removed. However, this approach
does not work with forward references as those are only seen later when the
segment has been removed already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3867) RDBDocumentStore: refactor JSON support
Julian Reschke created OAK-3867: --- Summary: RDBDocumentStore: refactor JSON support Key: OAK-3867 URL: https://issues.apache.org/jira/browse/OAK-3867 Project: Jackrabbit Oak Issue Type: Technical task Components: rdbmk Reporter: Julian Reschke Assignee: Julian Reschke Priority: Minor Refactor JSON support, so that - it can be better re-used in RDBExport utility and - unit test coverage can be improved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3866) Sorting on relative properties doesn't work doesn't work in Solr
Tommaso Teofili created OAK-3866:
Summary: Sorting on relative properties doesn't work doesn't work
in Solr
Key: OAK-3866
URL: https://issues.apache.org/jira/browse/OAK-3866
Project: Jackrabbit Oak
Issue Type: Bug
Components: solr
Affects Versions: 1.3.13
Reporter: Tommaso Teofili
Assignee: Tommaso Teofili
Fix For: 1.4
Executing a query like
{nocode}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{nocode}
would assume sorting on the children of resulting nodes' _jcr:primaryType_
property.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3862) Move integration tests in a different Maven module
[
https://issues.apache.org/jira/browse/OAK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093860#comment-15093860
]
Francesco Mari commented on OAK-3862:
-
bq. Package all test in oak-core and select those integration test to also run
in oak-segment with just SegmentFixture. Later we can do same when oak-document
is carved out. Would that work?
I thought about this suggestion, but I find this solution sub-optimal. Suppose
you have an {{oak-it}} module containing every integration test in Oak: this
module needs only to know which implementations to run the tests against. On
the other hand, suppose we apply your solution: {{oak-segment}} would need to
have a knowledge about every integration test in {{oak-core}}, with the
potential of missing some tests if we don't keep this knowledge up to date.
Moreover, I fear that your solution would make {{oak-segment}} a second-class
citizen: since {{oak-core}} comes before {{oak-segment}} in the Maven build
plan, no integration test for {{oak-segment}} will be ever run if some
integration test fails in {{oak-core}}. With a separate {{oak-it}} module,
instead, integration tests will run simultaneously for every known backend,
giving a comprehensive report about the status of every known implementation.
> Move integration tests in a different Maven module
> --
>
> Key: OAK-3862
> URL: https://issues.apache.org/jira/browse/OAK-3862
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4
>
>
> While moving the Segment Store and related packages into its own bundle, I
> figured out that integration tests contained in {{oak-core}} contribute to a
> cyclic dependency between the (new) {{oak-segment}} bundle and {{oak-core}}.
> The dependency is due to the usage of {{NodeStoreFixture}} to instantiate
> different implementations of {{NodeStore}} in a semi-transparent way.
> Tests depending on {{NodeStoreFixture}} are most likely integration tests. A
> clean solution to this problem would be to move those integration tests into
> a new Maven module, referencing the API and implementation modules as needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3865) New strategy to optimize secondary reads
Tomek Rękawek created OAK-3865: -- Summary: New strategy to optimize secondary reads Key: OAK-3865 URL: https://issues.apache.org/jira/browse/OAK-3865 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Tomek Rękawek Fix For: 1.4 *Introduction* In the current trunk we'll only read document _D_ from the secondary instance if: (1) we have the parent _P_ of document _D_ cached and (2) the parent hasn't been modified in 6 hours. The OAK-2106 tried to optimise (2) by estimating lag using MongoDB replica stats. It was unreliable, so the second approach was to read the last revisions directly from each Mongo instance. If the modification date of _P_ is before last revisions on all secondary Mongos, then secondary can be used. The main problem with this approach is that we still need to have the _P_ to be in cache. I think we need another way to optimise the secondary reading, as right now only about 3% of requests connects to the secondary, which is bad especially for the global-clustering case (Mongo and Oak instances across the globe). The optimisation provided in OAK-2106 doesn't make the things much better and may introduce some consistency issues. *Proposal* I had following constraints in mind preparing this: 1. Let's assume we have a sequence of commits with revisions _R1_, _R2_ and _R3_ modifying nodes _N1_, _N2_ and _N3_. If we already read the _N1_ from revision _R2_ then reading from a secondary shouldn't result in getting older revision (eg. _R1_). 2. If an Oak instance modifies a document, then reading from a secondary shouldn't result in getting the old version (before modification). So, let's have two maps: * _M1_ the most recent document revision read from the Mongo for each cluster id, * _M2_ the oldest last rev value for root document for each cluster id read from all the secondary instances. Maintaining _M1_: For every read from the Mongo we'll check if the lastRev for some cluster id is newer than _M1_ entry. If so, we'll update _M1_. For all writes we'll add the saved revision id with the current cluster id in _M1_. Maintaining _M2_: It should be periodically updated. Such mechanism is already prepared in the OAK-2106 patch. The method deciding whether we can read from the secondary instance should compare two maps. If all entries in _M2_ are newer than _M1_ it means that the secondary instances contains at least as new repository state as we already accessed and therefore it's safe to read from secondary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3862) Move integration tests in a different Maven module
[
https://issues.apache.org/jira/browse/OAK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093926#comment-15093926
]
Chetan Mehrotra commented on OAK-3862:
--
{quote}
I thought about this suggestion, but I find this solution sub-optimal. Suppose
you have an oak-it module containing every integration test in Oak: this module
needs only to know which implementations to run the tests against. On the other
hand, suppose we apply your solution: oak-segment would need to have a
knowledge about every integration test in oak-core, with the potential of
missing some tests if we don't keep this knowledge up to date.
{quote}
Such things can be simplified via use of
[Categories|https://github.com/junit-team/junit/wiki/Categories]. In Oak we
somewhat use a similar approach with RepositoryStub which allows test in
jackrabbit-core to be executed with various modules in Oak. See
[QueryJcrTestIT|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-lucene/src/test/java/org/apache/jackrabbit/oak/jcr/query/QueryJcrTestIT.java#L53]
which is custom suite listing all such jackrabbit-core test. This can be
improved via use of Categories.
In addition I would prefer to have test closer to the logic being tested unless
they are testing bigger usecase flows. So TreeTest should live _near_ tree
implementation logic.
{quote}
Moreover, I fear that your solution would make oak-segment a second-class
citizen: since oak-core comes before oak-segment in the Maven build plan, no
integration test for oak-segment will be ever run if some integration test
fails in oak-core.
{quote}
I do not think it makes any stuff second class. For e.g. currently oak-lucene
run same query test which are run in oak-core or oak-jcr. And that has not been
a problem. You can use [fail at
end|https://maven.apache.org/guides/mini/guide-multiple-modules.html] feature
to ensure test are run against different module even in case of failure.
For the modularization approach which impact the whole project it would be
preferable if changes are done gradually. So some approach might not be optimal
(to start with) but would allow gradual changes to reach the end goal. If
required we can carve out separate oak-it module. But then that should be done
as 3rd or 4th step say after extracting out oak-segment and oak-document.
Making it a prerequisite would make such refactoring riskier IMHO.
> Move integration tests in a different Maven module
> --
>
> Key: OAK-3862
> URL: https://issues.apache.org/jira/browse/OAK-3862
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4
>
>
> While moving the Segment Store and related packages into its own bundle, I
> figured out that integration tests contained in {{oak-core}} contribute to a
> cyclic dependency between the (new) {{oak-segment}} bundle and {{oak-core}}.
> The dependency is due to the usage of {{NodeStoreFixture}} to instantiate
> different implementations of {{NodeStore}} in a semi-transparent way.
> Tests depending on {{NodeStoreFixture}} are most likely integration tests. A
> clean solution to this problem would be to move those integration tests into
> a new Maven module, referencing the API and implementation modules as needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3866) Sorting on relative properties doesn't work in Solr
[
https://issues.apache.org/jira/browse/OAK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommaso Teofili updated OAK-3866:
-
Summary: Sorting on relative properties doesn't work in Solr (was: Sorting
on relative properties doesn't work doesn't work in Solr)
> Sorting on relative properties doesn't work in Solr
> ---
>
> Key: OAK-3866
> URL: https://issues.apache.org/jira/browse/OAK-3866
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
>Affects Versions: 1.3.13
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 1.4
>
>
> Executing a query like
> {nocode}
> /jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
> 'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
> descending
> {nocode}
> would assume sorting on the children of resulting nodes' _jcr:primaryType_
> property.
> That is currently not supported in Solr, while it is in Lucene as the latter
> supports index time aggregation.
> We should inspect if it's possible to extend support for Solr too, most
> probably via index time aggregation.
> The query should not fail but at least log a warning about that limitation
> for the time being.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093759#comment-15093759
]
Tomek Rękawek commented on OAK-3645:
I think I'm missing something - do we need to do something else than getting
System.currentTimeMillis() / 1000 and then multiplying the final result by 1000?
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Tomek Rękawek
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke resolved OAK-3645.
-
Resolution: Fixed
Fix Version/s: 1.3.14
1.2.10
1.0.26
trunk: http://svn.apache.org/r1724210
1.2: http://svn.apache.org/r1724211
1.0: http://svn.apache.org/r1724212
(thanks to [~tomek.rekawek]] who did most of the work)
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.0.26, 1.2.10, 1.3.14
>
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3842) Adjust package export declarations
[ https://issues.apache.org/jira/browse/OAK-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-3842: --- Priority: Blocker (was: Critical) > Adjust package export declarations > --- > > Key: OAK-3842 > URL: https://issues.apache.org/jira/browse/OAK-3842 > Project: Jackrabbit Oak > Issue Type: Task >Reporter: Michael Dürig >Assignee: Michael Dürig >Priority: Blocker > Labels: api, modularization, technical_debt > Fix For: 1.4 > > > We need to adjust the package export declarations such that they become > manageable with our branch / release model. > See http://markmail.org/thread/5g3viq5pwtdryapr for discussion. > I propose to remove package export declarations from all packages that we > don't consider public API / SPI beyond Oak itself. This would allow us to > evolve Oak internal stuff (e.g. things used across Oak modules) freely > without having to worry about merges to branches messing up semantic > versioning. OTOH it would force us to keep externally facing public API / SPI > reasonably stable also across the branches. Furthermore such an approach > would send the right signal to Oak API / SPI consumers regarding the > stability assumptions they can make. > An external API / SPI having a (transitive) dependency on internals might be > troublesome. In doubt I would remove the export version here until we can > make reasonable guarantees (either through decoupling the code or stabilising > the dependencies). > I would start digging through the export version and prepare an initial > proposal for further discussion. > /cc [~frm], [~chetanm], [~mmarth] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3862) Move integration tests in a different Maven module
[
https://issues.apache.org/jira/browse/OAK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093800#comment-15093800
]
Chetan Mehrotra commented on OAK-3862:
--
Okie missed the requirement of running same test against all fixture as a means
to test the fixture implementation also as I was thinking that they validate
the logic in oak-core (other than NodeStore related). Moving the testcase is
one way but that looks like quite a big change and it would also move test away
from logic being tested. Note that when a test which makes use of fixture like
{{TreeTest}} - The intention is to test both
# logic related to Tree as implemented in oak-core and
# how that logic works with various NodeStore implementation. So as to test
NodeStore contract through various usage scenarios
So having ability to run test with at least MemoryNodeStore help in checking #1
when they get executed within oak-core. These test then are like one we have in
jackrabbit-core. We package those test and also run them against Oak based
repository implementation.
Similar approach can be taken to start with - Package all test in oak-core and
select those integration test to also run in oak-segment with just
SegmentFixture. Later we can do same when oak-document is carved out. Would
that work?
This would help us meet both #1 and #2 requirement
> Move integration tests in a different Maven module
> --
>
> Key: OAK-3862
> URL: https://issues.apache.org/jira/browse/OAK-3862
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4
>
>
> While moving the Segment Store and related packages into its own bundle, I
> figured out that integration tests contained in {{oak-core}} contribute to a
> cyclic dependency between the (new) {{oak-segment}} bundle and {{oak-core}}.
> The dependency is due to the usage of {{NodeStoreFixture}} to instantiate
> different implementations of {{NodeStore}} in a semi-transparent way.
> Tests depending on {{NodeStoreFixture}} are most likely integration tests. A
> clean solution to this problem would be to move those integration tests into
> a new Maven module, referencing the API and implementation modules as needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3842) Adjust package export declarations
[
https://issues.apache.org/jira/browse/OAK-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093681#comment-15093681
]
Michael Dürig commented on OAK-3842:
Just talked to [~anchela]: we'll keep the export declaration for
{{org.apache.jackrabbit.oak.plugins.tree}} as this is needed for the security
parts and sufficiently stable. OTOH we will remove it for
{{org.apache.jackrabbit.oak.plugins.lock}}. The latter one is not stable enough
yet.
> Adjust package export declarations
> ---
>
> Key: OAK-3842
> URL: https://issues.apache.org/jira/browse/OAK-3842
> Project: Jackrabbit Oak
> Issue Type: Task
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: api, modularization, technical_debt
> Fix For: 1.4
>
>
> We need to adjust the package export declarations such that they become
> manageable with our branch / release model.
> See http://markmail.org/thread/5g3viq5pwtdryapr for discussion.
> I propose to remove package export declarations from all packages that we
> don't consider public API / SPI beyond Oak itself. This would allow us to
> evolve Oak internal stuff (e.g. things used across Oak modules) freely
> without having to worry about merges to branches messing up semantic
> versioning. OTOH it would force us to keep externally facing public API / SPI
> reasonably stable also across the branches. Furthermore such an approach
> would send the right signal to Oak API / SPI consumers regarding the
> stability assumptions they can make.
> An external API / SPI having a (transitive) dependency on internals might be
> troublesome. In doubt I would remove the export version here until we can
> make reasonable guarantees (either through decoupling the code or stabilising
> the dependencies).
> I would start digging through the export version and prepare an initial
> proposal for further discussion.
> /cc [~frm], [~chetanm], [~mmarth]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3864) Filestore cleanup removes referenced segments
[
https://issues.apache.org/jira/browse/OAK-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093742#comment-15093742
]
Michael Dürig commented on OAK-3864:
Test case at http://svn.apache.org/viewvc?rev=1724202=rev
> Filestore cleanup removes referenced segments
> -
>
> Key: OAK-3864
> URL: https://issues.apache.org/jira/browse/OAK-3864
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Blocker
> Labels: regression
> Fix For: 1.3.14
>
>
> In some situations {{FileStore.cleanup()}} may remove segments that are still
> referenced, subsequently causing a {{SNFE}}.
> This is a regression introduced with OAK-1828.
> {{FileStore.cleanup()}} relies on the ordering of the segments in the tar
> files: later segments only reference earlier segments. As we have seen in
> other places this assumption does not hold any more (e.g. OAK-3794, OAK-3793)
> since OAK-1828.
> {{cleanup}} traverses the segments backwards maintaining a list of
> referenced ids. When a segment is not in that list, it is removed. However,
> this approach does not work with forward references as those are only seen
> later when the segment has been removed already.
> cc [~alex.parvulescu], [~frm]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3864) Filestore cleanup removes referenced segments
[
https://issues.apache.org/jira/browse/OAK-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-3864:
---
Description:
In some situations {{FileStore.cleanup()}} may remove segments that are still
referenced, subsequently causing a {{SNFE}}.
This is a regression introduced with OAK-1828.
{{FileStore.cleanup()}} relies on the ordering of the segments in the tar
files: later segments only reference earlier segments. As we have seen in other
places this assumption does not hold any more (e.g. OAK-3794, OAK-3793) since
OAK-1828.
{{cleanup}} traverses the segments backwards maintaining a list of referenced
ids. When a segment is not in that list, it is removed. However, this approach
does not work with forward references as those are only seen later when the
segment has been removed already.
cc [~alex.parvulescu], [~frm]
was:
In some situations {{FileStore.cleanup()}} may remove segments that are still
referenced, subsequently causing a {{SNFE}}.
This is a regression introduced with OAK-1828.
{{FileStore.cleanup()}} relies on the ordering of the segments in the tar
files: later segments only reference earlier segments. As we have seen in other
places this assumption does not hold any more (e.g. OAK-3794, OAK-3793) since
OAK-1828.
{{cleanup}} traverses the segments backwards maintaining a list of referenced
ids. When a segment is not in that list, it is removed. However, this approach
does not work with forward references as those are only seen later when the
segment has been removed already.
> Filestore cleanup removes referenced segments
> -
>
> Key: OAK-3864
> URL: https://issues.apache.org/jira/browse/OAK-3864
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Blocker
> Labels: regression
> Fix For: 1.3.14
>
>
> In some situations {{FileStore.cleanup()}} may remove segments that are still
> referenced, subsequently causing a {{SNFE}}.
> This is a regression introduced with OAK-1828.
> {{FileStore.cleanup()}} relies on the ordering of the segments in the tar
> files: later segments only reference earlier segments. As we have seen in
> other places this assumption does not hold any more (e.g. OAK-3794, OAK-3793)
> since OAK-1828.
> {{cleanup}} traverses the segments backwards maintaining a list of
> referenced ids. When a segment is not in that list, it is removed. However,
> this approach does not work with forward references as those are only seen
> later when the segment has been removed already.
> cc [~alex.parvulescu], [~frm]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3645) RDBDocumentStore: server time detection for DB2 fails due to timezone/dst differences
[
https://issues.apache.org/jira/browse/OAK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093773#comment-15093773
]
Julian Reschke commented on OAK-3645:
-
You may be right that no rounding is needed; it depends a bit on whether the DB
rounds or doesn't (which we do not know).
> RDBDocumentStore: server time detection for DB2 fails due to timezone/dst
> differences
> -
>
> Key: OAK-3645
> URL: https://issues.apache.org/jira/browse/OAK-3645
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.3.10, 1.2.8, 1.0.24
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Attachments: OAK-3645-jr.patch, OAK-3645.patch
>
>
> We use {{CURRENT_TIMESTAMP(4)}} to ask the DB for it's system time.
> Apparently, at least with DB2, this might return a value that is off by a
> multiple of one hour (3600 * 1000ms) depending on whether the OAK instance
> and the DB run in different timezones.
> Known to work: both on the same machine.
> Known to fail: OAK in CET, DB2 in UTC, in which case we're getting a
> timestamp one hour in the past.
> At this time it's not clear whether the same problem occurs for other
> databases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3869) Refactor RecordWriter.write to always return a RecordId
[
https://issues.apache.org/jira/browse/OAK-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094832#comment-15094832
]
Michael Dürig commented on OAK-3869:
bq. losing some type safety
Right. Though we never had the that kind of type safety for the other record
types (i.e. list records) neither. So to address the type safety aspect we
could still return a {{RecordId}} from the {{write}} method but parametrise
{{RecordWriter}} on the record types (i.e. MapRecord, ListRecord, etc.). Will
give it a shot.
bq. what the change is actually for
It would have helped me to address OAK-3864 in a certain way. However I gave up
on this. See comments there. OTOH it might also help to simplify tracking free
segments in OAK-3348 (See
https://github.com/mduerig/jackrabbit-oak/commits/OAK-3348). This is still
early draft state so there is no hurry here.
> Refactor RecordWriter.write to always return a RecordId
> ---
>
> Key: OAK-3869
> URL: https://issues.apache.org/jira/browse/OAK-3869
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
>
> I think it would be cleaner if {{RecordId.write}} would always return a
> {{RecordId}} instead of depending on its type parametrisation and would like
> to refactor it to that respect..
> This is also a pre-requisite for my work on OAK-3348 and might also be for
> OAK-3864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3869) Refactor RecordWriter.write to always return a RecordId
[
https://issues.apache.org/jira/browse/OAK-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094883#comment-15094883
]
Michael Dürig commented on OAK-3869:
See
https://github.com/mduerig/jackrabbit-oak/commit/35f6ada085046a46f061788776917d53915ad6c1
for an attempt to regain type safety. I don't like it too much though as it
doesn't work for all record types. See e.g. [{{newValueWriter()}} |
https://github.com/mduerig/jackrabbit-oak/blob/35f6ada085046a46f061788776917d53915ad6c1/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/RecordWriters.java#L107],
which I had to parametrise with just {{Record}} as a value might represent an
entry in a list, a property, a value, etc..
> Refactor RecordWriter.write to always return a RecordId
> ---
>
> Key: OAK-3869
> URL: https://issues.apache.org/jira/browse/OAK-3869
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
>
> I think it would be cleaner if {{RecordId.write}} would always return a
> {{RecordId}} instead of depending on its type parametrisation and would like
> to refactor it to that respect..
> This is also a pre-requisite for my work on OAK-3348 and might also be for
> OAK-3864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3866) Sorting on relative properties doesn't work in Solr
[
https://issues.apache.org/jira/browse/OAK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommaso Teofili updated OAK-3866:
-
Description:
Executing a query like
{noformat}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{noformat}
would assume sorting on the _jcr:primaryType_ property of resulting nodes'
children.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
was:
Executing a query like
{noformat}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{noformat}
would assume sorting on the children of resulting nodes' _jcr:primaryType_
property.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
> Sorting on relative properties doesn't work in Solr
> ---
>
> Key: OAK-3866
> URL: https://issues.apache.org/jira/browse/OAK-3866
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
>Affects Versions: 1.3.13
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 1.4
>
>
> Executing a query like
> {noformat}
> /jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
> 'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
> descending
> {noformat}
> would assume sorting on the _jcr:primaryType_ property of resulting nodes'
> children.
> That is currently not supported in Solr, while it is in Lucene as the latter
> supports index time aggregation.
> We should inspect if it's possible to extend support for Solr too, most
> probably via index time aggregation.
> The query should not fail but at least log a warning about that limitation
> for the time being.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3866) Sorting on relative properties doesn't work in Solr
[
https://issues.apache.org/jira/browse/OAK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommaso Teofili updated OAK-3866:
-
Description:
Executing a query like
{noformat}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{noformat}
would assume sorting on the _jcr:primaryType_ property of resulting nodes'
_jcr:content_ children.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
was:
Executing a query like
{noformat}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{noformat}
would assume sorting on the _jcr:primaryType_ property of resulting nodes'
children.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
> Sorting on relative properties doesn't work in Solr
> ---
>
> Key: OAK-3866
> URL: https://issues.apache.org/jira/browse/OAK-3866
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
>Affects Versions: 1.3.13
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 1.4
>
>
> Executing a query like
> {noformat}
> /jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
> 'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
> descending
> {noformat}
> would assume sorting on the _jcr:primaryType_ property of resulting nodes'
> _jcr:content_ children.
> That is currently not supported in Solr, while it is in Lucene as the latter
> supports index time aggregation.
> We should inspect if it's possible to extend support for Solr too, most
> probably via index time aggregation.
> The query should not fail but at least log a warning about that limitation
> for the time being.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3858) Review slow running tests
[
https://issues.apache.org/jira/browse/OAK-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094042#comment-15094042
]
Marcel Reutegger commented on OAK-3858:
---
Similar fix for CheckpointsTest in trunk: http://svn.apache.org/r1724258
> Review slow running tests
> -
>
> Key: OAK-3858
> URL: https://issues.apache.org/jira/browse/OAK-3858
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Francesco Mari
>
> Some of the tests executed during a normal {{mvn clean test}} execution seem
> to be very slow if compared with the rest of the suite. On my machine, some
> problematic tests are:
> {noformat}
> Running org.apache.jackrabbit.oak.spi.blob.FileBlobStoreTest
> Tests run: 18, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 10.982 sec
> Running org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.961 sec
> Running org.apache.jackrabbit.oak.plugins.document.BulkCreateOrUpdateTest
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.076 sec
> Running org.apache.jackrabbit.oak.plugins.document.ConcurrentDocumentStoreTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.054 sec
> Running
> org.apache.jackrabbit.oak.plugins.document.DocumentDiscoveryLiteServiceTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 982.526 sec
> Running org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreTest
> Tests run: 53, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.132 sec
> Running org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgentTest
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.068 sec
> Running
> org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStorePerformanceTest
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 10.006 sec
> Running org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStoreTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.017 sec
> Running org.apache.jackrabbit.oak.plugins.document.VersionGCWithSplitTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.128 sec
> Running
> org.apache.jackrabbit.oak.security.authentication.ldap.LdapLoginStandaloneTest
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.96 sec
> {noformat}
> These tests should be analyzed for potential errors or moved to the
> integration test phase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3869) Refactor RecordWriter.write to always return a RecordId
[
https://issues.apache.org/jira/browse/OAK-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094049#comment-15094049
]
Francesco Mari commented on OAK-3869:
-
I like the API of the class after your change. It looks more uniform to me.
> Refactor RecordWriter.write to always return a RecordId
> ---
>
> Key: OAK-3869
> URL: https://issues.apache.org/jira/browse/OAK-3869
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
>
> I think it would be cleaner if {{RecordId.write}} would always return a
> {{RecordId}} instead of depending on its type parametrisation and would like
> to refactor it to that respect..
> This is also a pre-requisite for my work on OAK-3348 and might also be for
> OAK-3864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3866) Sorting on relative properties doesn't work in Solr
[
https://issues.apache.org/jira/browse/OAK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommaso Teofili updated OAK-3866:
-
Description:
Executing a query like
{noformat}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{noformat}
would assume sorting on the children of resulting nodes' _jcr:primaryType_
property.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
was:
Executing a query like
{nocode}
/jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
descending
{nocode}
would assume sorting on the children of resulting nodes' _jcr:primaryType_
property.
That is currently not supported in Solr, while it is in Lucene as the latter
supports index time aggregation.
We should inspect if it's possible to extend support for Solr too, most
probably via index time aggregation.
The query should not fail but at least log a warning about that limitation for
the time being.
> Sorting on relative properties doesn't work in Solr
> ---
>
> Key: OAK-3866
> URL: https://issues.apache.org/jira/browse/OAK-3866
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
>Affects Versions: 1.3.13
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 1.4
>
>
> Executing a query like
> {noformat}
> /jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
> 'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
> descending
> {noformat}
> would assume sorting on the children of resulting nodes' _jcr:primaryType_
> property.
> That is currently not supported in Solr, while it is in Lucene as the latter
> supports index time aggregation.
> We should inspect if it's possible to extend support for Solr too, most
> probably via index time aggregation.
> The query should not fail but at least log a warning about that limitation
> for the time being.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3868) Move createSegmentWriter() from FileStore to SegmentTracker
Michael Dürig created OAK-3868:
--
Summary: Move createSegmentWriter() from FileStore to
SegmentTracker
Key: OAK-3868
URL: https://issues.apache.org/jira/browse/OAK-3868
Project: Jackrabbit Oak
Issue Type: Technical task
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
I think it makes sense to move said method. This simplifies the code in various
places as it somewhat decouples the concern "writing segments" from an
implementation ({{FileStore}}).
Also this is somewhat a prerequisite for my current work on OAK-3348.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3858) Review slow running tests
[
https://issues.apache.org/jira/browse/OAK-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094018#comment-15094018
]
Marcel Reutegger commented on OAK-3858:
---
Fixed the DocumentNodeStoreTest in trunk: http://svn.apache.org/r1724254 (down
to 7s on my machine).
> Review slow running tests
> -
>
> Key: OAK-3858
> URL: https://issues.apache.org/jira/browse/OAK-3858
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Francesco Mari
>
> Some of the tests executed during a normal {{mvn clean test}} execution seem
> to be very slow if compared with the rest of the suite. On my machine, some
> problematic tests are:
> {noformat}
> Running org.apache.jackrabbit.oak.spi.blob.FileBlobStoreTest
> Tests run: 18, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 10.982 sec
> Running org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.961 sec
> Running org.apache.jackrabbit.oak.plugins.document.BulkCreateOrUpdateTest
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.076 sec
> Running org.apache.jackrabbit.oak.plugins.document.ConcurrentDocumentStoreTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.054 sec
> Running
> org.apache.jackrabbit.oak.plugins.document.DocumentDiscoveryLiteServiceTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 982.526 sec
> Running org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreTest
> Tests run: 53, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.132 sec
> Running org.apache.jackrabbit.oak.plugins.document.LastRevRecoveryAgentTest
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.068 sec
> Running
> org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStorePerformanceTest
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 10.006 sec
> Running org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStoreTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.017 sec
> Running org.apache.jackrabbit.oak.plugins.document.VersionGCWithSplitTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.128 sec
> Running
> org.apache.jackrabbit.oak.security.authentication.ldap.LdapLoginStandaloneTest
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.96 sec
> {noformat}
> These tests should be analyzed for potential errors or moved to the
> integration test phase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3826) Lucene index augmentation doesn't work in Osgi environment
[
https://issues.apache.org/jira/browse/OAK-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094074#comment-15094074
]
Chetan Mehrotra commented on OAK-3826:
--
Approach looks fine. Some minor comments
# Omit metatype - As for now this component does not have any config to be
edited
{code}
+@Component(metatype = true, label = "Apache Jackrabbit Oak
IndexAugmentorFactory")
{code}
# You can omit bind/unbind as the _The default value is the name created by
appending the reference name to the string unbind_
{code}
+@Reference(name = "IndexFieldProvider",
+policy = ReferencePolicy.DYNAMIC,
+cardinality = ReferenceCardinality.OPTIONAL_MULTIPLE,
+referenceInterface = IndexFieldProvider.class,
+bind = "bindIndexFieldProviderService",
+unbind = "unbindIndexFieldProviderService")
{code}
# Visibility of state in {{IndexAugmentorFactory}} - Keep them provide. And
provide a method to enable assert if they are all reset to empty
# Use {{com.google.common.collect.Sets#newIdentityHashSet}} which would be
safer and not depend on equals of service impls
{code}
+public IndexAugmentorFactory() {
+indexFieldProviderSet = Sets.newHashSet();
+fulltextQueryTermsProviderSet = Sets.newHashSet();
{code}
# Instead of directly assigning {{tempMap}} to {{indexFieldProviderMap}} use
{{com.google.common.collect.ImmutableMap#copyOf}}. This provides immutable
variant and fast access for single entry setup!
{code}
Map tempMap = Maps.newHashMap();
for (String nodeType : indexFieldProviderListMultimap.keySet()) {
List providers =
indexFieldProviderListMultimap.get(nodeType);
CompositeIndexFieldProvider compositeIndexFieldProvider =
new CompositeIndexFieldProvider(nodeType, providers);
tempMap.put(nodeType, compositeIndexFieldProvider);
}
indexFieldProviderMap = tempMap;
{code}
# CompositeIndexFieldProvider - It might be safer to make use of ImmutableList
to copy the list. Not sure the semantics of list from a listMultiMap (may be a
view?)
> Lucene index augmentation doesn't work in Osgi environment
> --
>
> Key: OAK-3826
> URL: https://issues.apache.org/jira/browse/OAK-3826
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
>Priority: Minor
> Fix For: 1.4
>
> Attachments: OAK-3826-v2.patch, OAK-3826.patch
>
>
> OAK-3576 introduced a way to hook SPI to provide extra fields and query terms
> for a lucene index.
> In Osgi world, due to OAK-3815, {{LuceneIndexProviderService}} registered
> references to SPI and pinged {{IndexAugmentFactory}} to update its map. But,
> it seems bind/unbind methods get called ahead of time as compared to the
> information Tracker contains. This leads to wrong set of services captured by
> {{IndexAugmentFactory}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3869) Refactor RecordWriter.write to always return a RecordId
[
https://issues.apache.org/jira/browse/OAK-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094084#comment-15094084
]
Alex Parvulescu commented on OAK-3869:
--
looks good, although you are some type safety which is not ideal.
after this refactoring you could pass in any kind of _RecordWriter_ to
_writeMapRecord_ and wrap the resulting id into a _MapRecord_. ie. what would
stop me from passing in a _newNodeStateWriter(ids)_ for example?
> Refactor RecordWriter.write to always return a RecordId
> ---
>
> Key: OAK-3869
> URL: https://issues.apache.org/jira/browse/OAK-3869
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
>
> I think it would be cleaner if {{RecordId.write}} would always return a
> {{RecordId}} instead of depending on its type parametrisation and would like
> to refactor it to that respect..
> This is also a pre-requisite for my work on OAK-3348 and might also be for
> OAK-3864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3869) Refactor RecordWriter.write to always return a RecordId
Michael Dürig created OAK-3869:
--
Summary: Refactor RecordWriter.write to always return a RecordId
Key: OAK-3869
URL: https://issues.apache.org/jira/browse/OAK-3869
Project: Jackrabbit Oak
Issue Type: Technical task
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
I think it would be cleaner if {{RecordId.write}} would always return a
{{RecordId}} instead of depending on its type parametrisation and would like to
refactor it to that respect..
This is also a pre-requisite for my work on OAK-3348 and might also be for
OAK-3864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-3869) Refactor RecordWriter.write to always return a RecordId
[
https://issues.apache.org/jira/browse/OAK-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094084#comment-15094084
]
Alex Parvulescu edited comment on OAK-3869 at 1/12/16 3:46 PM:
---
looks good, although you are losing some type safety which is not ideal.
after this refactoring you could pass in any kind of _RecordWriter_ to
_writeMapRecord_ and wrap the resulting id into a _MapRecord_. ie. what would
stop me from passing in a _newNodeStateWriter(ids)_ for example?
I'd like to know more about what the change is actually for, as it stands now,
it looks purely cosmetic, so a more detailed explanation would be nice :)
was (Author: alex.parvulescu):
looks good, although you are some type safety which is not ideal.
after this refactoring you could pass in any kind of _RecordWriter_ to
_writeMapRecord_ and wrap the resulting id into a _MapRecord_. ie. what would
stop me from passing in a _newNodeStateWriter(ids)_ for example?
> Refactor RecordWriter.write to always return a RecordId
> ---
>
> Key: OAK-3869
> URL: https://issues.apache.org/jira/browse/OAK-3869
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
>
> I think it would be cleaner if {{RecordId.write}} would always return a
> {{RecordId}} instead of depending on its type parametrisation and would like
> to refactor it to that respect..
> This is also a pre-requisite for my work on OAK-3348 and might also be for
> OAK-3864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3868) Move createSegmentWriter() from FileStore to SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094122#comment-15094122
]
Alex Parvulescu commented on OAK-3868:
--
looks ok, one further refactoring step is to call the new method in the
SegmentTracker constructor itself.
[0]
https://github.com/mduerig/jackrabbit-oak/blob/942be4d0ba227fad3965ac8b849ca56e0211aed3/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentTracker.java#L129
> Move createSegmentWriter() from FileStore to SegmentTracker
> ---
>
> Key: OAK-3868
> URL: https://issues.apache.org/jira/browse/OAK-3868
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
>
> I think it makes sense to move said method. This simplifies the code in
> various places as it somewhat decouples the concern "writing segments" from
> an implementation ({{FileStore}}).
> Also this is somewhat a prerequisite for my current work on OAK-3348.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3867) RDBDocumentStore: refactor JSON support
[ https://issues.apache.org/jira/browse/OAK-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3867: Attachment: OAK-3867.diff Proposed change. > RDBDocumentStore: refactor JSON support > --- > > Key: OAK-3867 > URL: https://issues.apache.org/jira/browse/OAK-3867 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Attachments: OAK-3867.diff > > > Refactor JSON support, so that > - it can be better re-used in RDBExport utility and > - unit test coverage can be improved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3866) Sorting on relative properties doesn't work in Solr
[
https://issues.apache.org/jira/browse/OAK-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommaso Teofili updated OAK-3866:
-
Affects Version/s: 1.0.22
> Sorting on relative properties doesn't work in Solr
> ---
>
> Key: OAK-3866
> URL: https://issues.apache.org/jira/browse/OAK-3866
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
>Affects Versions: 1.0.22, 1.3.13
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 1.4
>
>
> Executing a query like
> {noformat}
> /jcr:root/content/foo//*[(@sling:resourceType = 'x' or @sling:resourceType =
> 'y') and jcr:contains(., 'bar*~')] order by jcr:content/@jcr:primaryType
> descending
> {noformat}
> would assume sorting on the _jcr:primaryType_ property of resulting nodes'
> _jcr:content_ children.
> That is currently not supported in Solr, while it is in Lucene as the latter
> supports index time aggregation.
> We should inspect if it's possible to extend support for Solr too, most
> probably via index time aggregation.
> The query should not fail but at least log a warning about that limitation
> for the time being.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3870) Support for XPATH queries using wildcards in property names in indexes
Tom Blackford created OAK-3870:
--
Summary: Support for XPATH queries using wildcards in property
names in indexes
Key: OAK-3870
URL: https://issues.apache.org/jira/browse/OAK-3870
Project: Jackrabbit Oak
Issue Type: Improvement
Components: query
Reporter: Tom Blackford
At present, it's not possible to optimise the XPATH query below using an index:
{code}
/jcr:root/content//*
[jcr:content/relationship/*/@relationshipName="exampleRelationshipName"]
{code}
...due to the wildcard in the relative property path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3868) Move createSegmentWriter() from FileStore to SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-3868.
Resolution: Fixed
Fix Version/s: 1.3.14
Applied patch including Alex' suggestion at
http://svn.apache.org/viewvc?rev=1724289=rev
> Move createSegmentWriter() from FileStore to SegmentTracker
> ---
>
> Key: OAK-3868
> URL: https://issues.apache.org/jira/browse/OAK-3868
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
> Fix For: 1.3.14
>
>
> I think it makes sense to move said method. This simplifies the code in
> various places as it somewhat decouples the concern "writing segments" from
> an implementation ({{FileStore}}).
> Also this is somewhat a prerequisite for my current work on OAK-3348.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3844) Better support for versionable nodes without version histories
[
https://issues.apache.org/jira/browse/OAK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3844:
---
Attachment: BrokenVersionableTest.java
> Better support for versionable nodes without version histories
> --
>
> Key: OAK-3844
> URL: https://issues.apache.org/jira/browse/OAK-3844
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Affects Versions: 1.3.13
>Reporter: Tomek Rękawek
>Assignee: Julian Sedding
> Fix For: 1.4
>
> Attachments: BrokenVersionableTest.java, OAK-3844.patch
>
>
> One of the customers reported following exception that has been thrown during
> the migration:
> {noformat}
> Caused by: java.lang.IllegalStateException: This builder does not exist:
> 95a5253f-d37b-4e88-a4b4-0721530344fc
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:150)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:506)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:522)
> at
> org.apache.jackrabbit.oak.upgrade.version.VersionableEditor.setVersionablePath(VersionableEditor.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:226)
> ...
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:486)
> {noformat}
> It seems that the node with reported UUID has primary type inheriting from
> {{mix:versionable}}, but there is no appropriate version history in the
> version storage.
> Obviously this means that there's something wrong with the repository.
> However, I think that the migration process shouldn't fail, but proceed
> silently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3228) Delayed visibility of new groups when using PrincipalManager
[
https://issues.apache.org/jira/browse/OAK-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094360#comment-15094360
]
Konrad Windszus commented on OAK-3228:
--
Seems related to OAK-938.
> Delayed visibility of new groups when using PrincipalManager
> -
>
> Key: OAK-3228
> URL: https://issues.apache.org/jira/browse/OAK-3228
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.2.2
>Reporter: Georg Henzler
>
> PrincipalManager does not show groups that were just created (this is causing
> problems in our code). As workaround we use now
> UserManager.getAuthorizable().getPrincipal() which curiously works
> immediately after saving a group. Also it does not seem to be an index
> problem, as a query {{SELECT * FROM [rep:Authorizable] WHERE
> [rep:principalName] = "mygroup"}} also immediately shows a new group.
> See the following servlet snippet for easy reproduction:
> {code}
> ...
> @SlingServlet(paths = "/bin/CreateGroupAndRetrievePrincipal", methods = "GET")
> public class CreateGroupAndRetrievePrincipalServlet extends
> SlingSafeMethodsServlet {
> private static final long serialVersionUID = 1L;
> private static final Logger LOG =
> LoggerFactory.getLogger(CreateGroupAndRetrievePrincipalServlet.class);
> @Reference
> private SlingRepository repository;
> @Override
> @SuppressWarnings("deprecation")
> protected void doGet(final SlingHttpServletRequest request, final
> SlingHttpServletResponse response) throws ServletException,
> IOException {
> response.setContentType("text/plain");
> PrintWriter out = response.getWriter();
> final String groupName = request.getParameter("g");
> LOG.debug("test");
> Session session = null;
> try {
> session = repository.loginAdministrative(null);
> UserManager usermanager = ((JackrabbitSession)
> session).getUserManager();
> Group newGroup = usermanager.createGroup(new
> java.security.Principal() {
> @Override
> public String getName() {
> return groupName;
> }
> }, "principaltest");
> out.println("Created Group " + newGroup);
> session.save();
> } catch (Exception e) {
> throw new ServletException(e.getMessage(), e);
> } finally {
> if (session != null) {
> session.logout();
> session = null;
> }
> }
> out.println();
> try {
> session = repository.loginAdministrative(null);
> // No 1: PrincipalManager
> PrincipalManager principalManager = ((JackrabbitSession)
> session).getPrincipalManager();
> Principal principal = principalManager.getPrincipal(groupName);
> out.println("PrincipalManager: principal: " + principal);
> // No 2: UserManager
> UserManager usermanager = ((JackrabbitSession)
> session).getUserManager();
> Authorizable authorizable =
> usermanager.getAuthorizable(groupName);
> out.println("UserManager: authorizable: " + authorizable);
> // No 3: query
> QueryManager queryManager =
> session.getWorkspace().getQueryManager();
> final Query query = queryManager.createQuery("SELECT * FROM
> [rep:Authorizable] WHERE [rep:principalName] = \"" + groupName
> + "\"", Query.JCR_SQL2);
> QueryResult result = query.execute();
> NodeIterator nodes = result.getNodes();
> if (!nodes.hasNext()) {
> out.println("QUERY: group not found: " + groupName);
> }
> while (nodes.hasNext()) {
> Node resultNode = (Node) nodes.next();
> out.println("QUERY: node " + resultNode.getPath() + "
> property rep:principalName="
> +
> resultNode.getProperty("rep:principalName").getString());
> }
> query.execute();
> } catch (Exception e) {
> throw new ServletException(e.getMessage(), e);
> } finally {
> if (session != null) {
> session.logout();
> session = null;
> }
> }
> }
> }
> {code}
> returns (using AEM 6.1)
> {code}
> Created Group Group 'my-test-group'
> PrincipalManager: principal: null
> UserManager: authorizable: Group 'my-test-group'
> QUERY: node /home/groups/principaltest/qb2WDGrrC0q9bE8jaObH property
> rep:principalName=my-test-group
> {code}
> Potentially the problem is that the principal manager holds its own session
> (even though it was retrieved by {{((JackrabbitSession)
>
[jira] [Updated] (OAK-3844) Better support for versionable nodes without version histories
[
https://issues.apache.org/jira/browse/OAK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3844:
---
Attachment: (was: BrokenVersionableTest.java)
> Better support for versionable nodes without version histories
> --
>
> Key: OAK-3844
> URL: https://issues.apache.org/jira/browse/OAK-3844
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Affects Versions: 1.3.13
>Reporter: Tomek Rękawek
>Assignee: Julian Sedding
> Fix For: 1.4
>
> Attachments: OAK-3844.patch
>
>
> One of the customers reported following exception that has been thrown during
> the migration:
> {noformat}
> Caused by: java.lang.IllegalStateException: This builder does not exist:
> 95a5253f-d37b-4e88-a4b4-0721530344fc
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:150)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:506)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:522)
> at
> org.apache.jackrabbit.oak.upgrade.version.VersionableEditor.setVersionablePath(VersionableEditor.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:226)
> ...
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:486)
> {noformat}
> It seems that the node with reported UUID has primary type inheriting from
> {{mix:versionable}}, but there is no appropriate version history in the
> version storage.
> Obviously this means that there's something wrong with the repository.
> However, I think that the migration process shouldn't fail, but proceed
> silently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3844) Better support for versionable nodes without version histories
[
https://issues.apache.org/jira/browse/OAK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3844:
---
Attachment: (was: OAK-3844.patch)
> Better support for versionable nodes without version histories
> --
>
> Key: OAK-3844
> URL: https://issues.apache.org/jira/browse/OAK-3844
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Affects Versions: 1.3.13
>Reporter: Tomek Rękawek
>Assignee: Julian Sedding
> Fix For: 1.4
>
> Attachments: OAK-3844.patch
>
>
> One of the customers reported following exception that has been thrown during
> the migration:
> {noformat}
> Caused by: java.lang.IllegalStateException: This builder does not exist:
> 95a5253f-d37b-4e88-a4b4-0721530344fc
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:150)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:506)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:522)
> at
> org.apache.jackrabbit.oak.upgrade.version.VersionableEditor.setVersionablePath(VersionableEditor.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:226)
> ...
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:486)
> {noformat}
> It seems that the node with reported UUID has primary type inheriting from
> {{mix:versionable}}, but there is no appropriate version history in the
> version storage.
> Obviously this means that there's something wrong with the repository.
> However, I think that the migration process shouldn't fail, but proceed
> silently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3844) Better support for versionable nodes without version histories
[
https://issues.apache.org/jira/browse/OAK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3844:
---
Attachment: OAK-3844.patch
> Better support for versionable nodes without version histories
> --
>
> Key: OAK-3844
> URL: https://issues.apache.org/jira/browse/OAK-3844
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Affects Versions: 1.3.13
>Reporter: Tomek Rękawek
>Assignee: Julian Sedding
> Fix For: 1.4
>
> Attachments: OAK-3844.patch
>
>
> One of the customers reported following exception that has been thrown during
> the migration:
> {noformat}
> Caused by: java.lang.IllegalStateException: This builder does not exist:
> 95a5253f-d37b-4e88-a4b4-0721530344fc
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:150)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:506)
> at
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:522)
> at
> org.apache.jackrabbit.oak.upgrade.version.VersionableEditor.setVersionablePath(VersionableEditor.java:148)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:226)
> ...
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:486)
> {noformat}
> It seems that the node with reported UUID has primary type inheriting from
> {{mix:versionable}}, but there is no appropriate version history in the
> version storage.
> Obviously this means that there's something wrong with the repository.
> However, I think that the migration process shouldn't fail, but proceed
> silently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
