Re: [PR] OAK-10458 - Set LZ4 as the default compression algorithm for the indexing job. [jackrabbit-oak]
dalbani commented on code in PR #1136:
URL: https://github.com/apache/jackrabbit-oak/pull/1136#discussion_r1362822701
##
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/indexstore/IndexStoreUtils.java:
##
@@ -39,6 +40,23 @@
public class IndexStoreUtils {
public static final String METADATA_SUFFIX = ".metadata";
+public static final String OAK_INDEXER_USE_ZIP = "oak.indexer.useZip";
+public static final String OAK_INDEXER_USE_LZ4 = "oak.indexer.useLZ4";
+
+public static boolean compressionEnabled() {
Review Comment:
> But that is a bigger change, as we would have to update helm charts and
bootstrap rules.
Hum, interesting, it's the first that see a _Helm chart_ being mentioned in
relation with Oak.
I'm curious: is there a such a chart which is publicly available?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10464: Use Testcontainers instead of com.arakelian:docker-junit-rule [jackrabbit-oak]
reschke commented on code in PR #1143:
URL: https://github.com/apache/jackrabbit-oak/pull/1143#discussion_r1362470833
##
oak-blob-cloud-azure/pom.xml:
##
@@ -191,10 +191,10 @@
mockito-core
test
-
-com.arakelian
-docker-junit-rule
+org.testcontainers
+testcontainers
+${testcontainers.version}
Review Comment:
I don't get that. We define the dependency with version in the parent POM,
and then use it version-less in all subprojects. That's IMHO the right
approach, and no variable is needed at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] OAK-10499: Node bundling may resurrect node [jackrabbit-oak]
mreutegg opened a new pull request, #1160: URL: https://github.com/apache/jackrabbit-oak/pull/1160 Add ignored test that reproduces the issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10473 do not throw an IAE on invalid names [jackrabbit-oak]
anchela commented on code in PR #1152:
URL: https://github.com/apache/jackrabbit-oak/pull/1152#discussion_r1362384562
##
oak-core/src/test/java/org/apache/jackrabbit/oak/security/user/UserImporterMembershipIgnoreTest.java:
##
@@ -65,7 +65,7 @@ public void before() throws Exception {
assertTrue(importer.start(memberRefList));
}
-@Test(expected = IllegalArgumentException.class)
Review Comment:
if the IdentifierManager no longer verifies the validity of an given uuid
(as they are used for storing group membership, we might consider doing so in
the UserImporter.
reason: there exist 3 different import modes
- ABORT: must fail with ConstraintViolationException if the member uuid
cannot be resolved to a valid user
- IGNORE: ignore the uuids that cannot be resolved
- BESTEFFORT: assume the member does just not yet exist and add the value.
that latter is the one that bothers me here: if the validity of the uuid
format is no longer verified, we will end up with garbage in the rep:members
properties that might later on fail the resolution (i haven't checked). the
goal of BESTEFFORT is to not mandate that members exist already but
obviously storing completely illegal values should be avoided.
i am not sure yet what the best option is.
throwing RepositoryException in the IdentifierManager instead of
illegalargumentexception would have the same problem btw as the UserImporter is
catching them (and then proceeding as described above).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10464: Use Testcontainers instead of com.arakelian:docker-junit-rule [jackrabbit-oak]
smiroslav commented on PR #1143: URL: https://github.com/apache/jackrabbit-oak/pull/1143#issuecomment-1766591419 Got an error running the test in oak-segment-azure ``` > mvn clean install -Dtest=AzureRepositoryLockTest ... [INFO] T E S T S [INFO] --- [INFO] Running org.apache.jackrabbit.oak.segment.azure.AzureRepositoryLockTest [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.893 s <<< FAILURE! - in org.apache.jackrabbit.oak.segment.azure.AzureRepositoryLockTest [ERROR] testLeaseRefreshUnsuccessful(org.apache.jackrabbit.oak.segment.azure.AzureRepositoryLockTest) Time elapsed: 0.545 s <<< ERROR! org.mockito.exceptions.base.MockitoException: Mockito cannot mock this class: class com.microsoft.azure.storage.blob.CloudBlockBlob. Can not mock final classes with the following settings : - explicit serialization (e.g. withSettings().serializable()) - extra interfaces (e.g. withSettings().extraInterfaces(...)) You are seeing this disclaimer because Mockito is configured to create inlined mocks. You can learn about inline mocks and their limitations under item #39 of the Mockito class javadoc. Underlying exception : org.mockito.exceptions.base.MockitoException: Could not modify all classes [interface com.microsoft.azure.storage.blob.ListBlobItem, class com.microsoft.azure.storage.blob.CloudBlob, class java.lang.Object, class com.microsoft.azure.storage.blob.CloudBlockBlob] at org.apache.jackrabbit.oak.segment.azure.AzureRepositoryLockTest.testLeaseRefreshUnsuccessful(AzureRepositoryLockTest.java:94) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule$1.evaluate(AzuriteDockerRule.java:84) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) Caused by: org.mockito.exceptions.base.MockitoException: Could not modify all classes [interface com.microsoft.azure.storage.blob.ListBlobItem, class com.microsoft.azure.storage.blob.CloudBlob, class java.lang.Object, class com.microsoft.azure.storage.blob.CloudBlockBlob] at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:168) at net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:399) at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:190) at net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:410) ... 30 more Caused by: java.lang.IllegalStateException: Byte Buddy could not instrument all classes within the mock's type hierarchy
Re: [PR] OAK-10497 Properties order in FFS can be different across runs: always sort [jackrabbit-oak]
sonarcloud[bot] commented on PR #1159: URL: https://github.com/apache/jackrabbit-oak/pull/1159#issuecomment-1766461047 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_coverage&view=list) [80.0% Coverage](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_coverage&view=list) [](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_duplicated_lines_density&view=list)  The version of Java (11.0.20.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10497 Properties order in FFS can be different across runs: always sort [jackrabbit-oak]
sonarcloud[bot] commented on PR #1159: URL: https://github.com/apache/jackrabbit-oak/pull/1159#issuecomment-1766234678 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_coverage&view=list) [100.0% Coverage](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_coverage&view=list) [](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1159&metric=new_duplicated_lines_density&view=list)  The version of Java (11.0.20.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Oak-10497 | Properties order in FFS can be different across runs [jackrabbit-oak]
nit0906 commented on PR #1158: URL: https://github.com/apache/jackrabbit-oak/pull/1158#issuecomment-1766200177 imo, we could actually hold off on this change and simply handle in the client verification script instead. Changing the behaviour of the public asJson method doesn’t seem correct to me and the other alternative would be make it configurable for all sort strategies (which would probably require another system property). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Oak-10497 | Properties order in FFS can be different across runs [jackrabbit-oak]
nit0906 closed pull request #1158: Oak-10497 | Properties order in FFS can be different across runs URL: https://github.com/apache/jackrabbit-oak/pull/1158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-9459: ConstraintViolationException in VersionManagerImplRestore w… [jackrabbit-oak]
reschke commented on code in PR #1146:
URL: https://github.com/apache/jackrabbit-oak/pull/1146#discussion_r1361934632
##
oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/version/VersionableState.java:
##
@@ -285,8 +287,32 @@ private void restoreFrozen(@NotNull NodeBuilder frozen,
@NotNull NodeBuilder dest,
@NotNull VersionSelector selector)
throws RepositoryException, CommitFailedException {
-// 15.7.2 Restoring Type and Identifier
-restoreFrozenTypeAndUUID(frozen, dest);
+//OAK-9459: if the NodeState is not empty, retrieve OPVs before
restoring types to avoid constraint violations
+boolean frozenTypeRestored = false;
+if (!dest.hasProperty(JCR_PRIMARYTYPE)) {
+// empty NodeState
+// 15.7.2 Restoring Type and Identifier
+restoreFrozenTypeAndUUID(frozen, dest);
+frozenTypeRestored = true;
+}
+HashMap opvs = new HashMap<>();
Review Comment:
```suggestion
Map opvs = new HashMap<>();
```
##
oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/version/VersionableState.java:
##
@@ -285,8 +287,32 @@ private void restoreFrozen(@NotNull NodeBuilder frozen,
@NotNull NodeBuilder dest,
@NotNull VersionSelector selector)
throws RepositoryException, CommitFailedException {
-// 15.7.2 Restoring Type and Identifier
-restoreFrozenTypeAndUUID(frozen, dest);
+//OAK-9459: if the NodeState is not empty, retrieve OPVs before
restoring types to avoid constraint violations
+boolean frozenTypeRestored = false;
+if (!dest.hasProperty(JCR_PRIMARYTYPE)) {
+// empty NodeState
+// 15.7.2 Restoring Type and Identifier
+restoreFrozenTypeAndUUID(frozen, dest);
+frozenTypeRestored = true;
+}
+HashMap opvs = new HashMap<>();
+for (PropertyState p : dest.getProperties()) {
+String propName = p.getName();
+if (BASIC_PROPERTIES.contains(propName)) {
+continue;
Review Comment:
Can we change the logic not to use "continue"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10497 Properties order in FFS can be different across runs: always sort [jackrabbit-oak]
nfsantos commented on code in PR #1159:
URL: https://github.com/apache/jackrabbit-oak/pull/1159#discussion_r1361924260
##
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/NodeStateEntryWriter.java:
##
@@ -86,10 +86,6 @@ public String toString(List pathElements, String
nodeStateAsJson) {
}
public String asJson(NodeState nodeState) {
Review Comment:
I don't think we should change the behavior of this method, as this is
breaking the public contract. The API already had a method for returning node
with sorted properties and another unsorted, so there is no need to change the
behavior of the methods. Instead, it should be the API users that switch to
using the method that returns the properties sorted by name.
The Pipelined strategy is using this method and if you change the behavior,
this will affect its performance characteristics and may have other undesirable
effects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10497 Properties order in FFS can be different across runs: always sort [jackrabbit-oak]
nfsantos commented on PR #1159:
URL: https://github.com/apache/jackrabbit-oak/pull/1159#issuecomment-1766166956
I rerun the tests with 20 properties per node and with larger values in the
property, which I think is a more realistic scenario. The sorted version is
slower, around 15% slower.
I think the impact in performance will be measurable but maybe not very
significant. In the case of the Pipelined strategy, the overhead from sorting
will happen in the transform threads, which can be easily scaled up. The Mongo
download thread is the main bottleneck, as this stage is single threaded and
there is no good way to parallelize it, so I would resist adding overhead in
the work done by this thread, but I don't have objections to slightly
increasing the work of the transform threads.
I would anyway suggest having a configuration setting to enable/disable
sorting of the properties when writing to the FFS.
```java
package org.apache.jackrabbit.oak.index.indexer.document.flatfile;
import java.util.ArrayList;
import org.apache.commons.lang3.RandomStringUtils;
import org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState;
import org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder;
import org.apache.jackrabbit.oak.spi.blob.BlobStore;
import org.apache.jackrabbit.oak.spi.blob.MemoryBlobStore;
import org.apache.jackrabbit.oak.spi.state.NodeBuilder;
import org.apache.jackrabbit.oak.spi.state.NodeState;
import org.junit.Test;
public class MicroBenchmark {
public void test() {
BlobStore blobStore = new MemoryBlobStore();
NodeStateEntryWriter entryWriter = new
NodeStateEntryWriter(blobStore);
ArrayList list = new ArrayList<>();
for (int j = 0; j < 100; j++) {
NodeBuilder b = new MemoryNodeBuilder(EmptyNodeState.EMPTY_NODE);
for (int i = 0; i < 20; i++) {
b.setProperty("p" + i, RandomStringUtils.random(40, true,
true));
}
NodeState ns = b.getNodeState();
list.add(ns);
}
// Profiler prof = new Profiler().startCollecting();
for(int test = 0; test < 10; test++) {
long start = System.currentTimeMillis();
int len = 0;
for (NodeState ns : list) {
len += entryWriter.asJson(ns).length();
}
long time = System.currentTimeMillis() - start;
System.out.println(time + " ms; string length " + len + "
unsorted");
start = System.currentTimeMillis();
len = 0;
for (NodeState ns : list) {
len += entryWriter.asSortedJson(ns).length();
}
time = System.currentTimeMillis() - start;
System.out.println(time + " ms; string length " + len + "
sorted");
System.out.println();
}
// System.out.println(prof.getTop(10));
}
public static void main(String[] args) {
new MicroBenchmark().test();
}
}
```
```
4418 ms; string length 97100 unsorted
2399 ms; string length 97100 sorted
1886 ms; string length 97100 unsorted
2156 ms; string length 97100 sorted
1760 ms; string length 97100 unsorted
2039 ms; string length 97100 sorted
1667 ms; string length 97100 unsorted
2000 ms; string length 97100 sorted
1665 ms; string length 97100 unsorted
2000 ms; string length 97100 sorted
1665 ms; string length 97100 unsorted
1999 ms; string length 97100 sorted
1667 ms; string length 97100 unsorted
2000 ms; string length 97100 sorted
1664 ms; string length 97100 unsorted
2001 ms; string length 97100 sorted
1663 ms; string length 97100 unsorted
1999 ms; string length 97100 sorted
1667 ms; string length 97100 unsorted
1999 ms; string length 97100 sorted
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10497 Properties order in FFS can be different across runs: always sort [jackrabbit-oak]
thomasmueller commented on PR #1159:
URL: https://github.com/apache/jackrabbit-oak/pull/1159#issuecomment-1766130468
Micro-benchmark and results:
```
package org.apache.jackrabbit.oak.index.indexer.document.flatfile;
import java.util.ArrayList;
import org.apache.jackrabbit.oak.commons.Profiler;
import org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState;
import org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder;
import org.apache.jackrabbit.oak.spi.blob.BlobStore;
import org.apache.jackrabbit.oak.spi.blob.MemoryBlobStore;
import org.apache.jackrabbit.oak.spi.state.NodeBuilder;
import org.apache.jackrabbit.oak.spi.state.NodeState;
import org.junit.Test;
public class MicroBench {
@Test
public void test() {
BlobStore blobStore = new MemoryBlobStore();
NodeStateEntryWriter entryWriter = new
NodeStateEntryWriter(blobStore);
ArrayList list = new ArrayList<>();
for (int j = 0; j < 100; j++) {
NodeBuilder b = new MemoryNodeBuilder(EmptyNodeState.EMPTY_NODE);
for (int i = 0; i < 5; i++) {
b.setProperty("p" + i, "Hello, World");
}
NodeState ns = b.getNodeState();
list.add(ns);
}
// Profiler prof = new Profiler().startCollecting();
for(int test = 0; test < 10; test++) {
long start = System.currentTimeMillis();
int len = 0;
for (NodeState ns : list) {
len += entryWriter.asUnsortedJson(ns).length();
}
long time = System.currentTimeMillis() - start;
System.out.println(time + " ms; string length " + len + "
unsorted");
start = System.currentTimeMillis();
len = 0;
for (NodeState ns : list) {
len += entryWriter.asJson(ns).length();
}
time = System.currentTimeMillis() - start;
System.out.println(time + " ms; string length " + len + "
sorted");
System.out.println();
}
// System.out.println(prof.getTop(10));
}
}
1069 ms; string length 10100 unsorted
610 ms; string length 10100 sorted
401 ms; string length 10100 unsorted
425 ms; string length 10100 sorted
439 ms; string length 10100 unsorted
612 ms; string length 10100 sorted
411 ms; string length 10100 unsorted
514 ms; string length 10100 sorted
356 ms; string length 10100 unsorted
428 ms; string length 10100 sorted
362 ms; string length 10100 unsorted
371 ms; string length 10100 sorted
417 ms; string length 10100 unsorted
380 ms; string length 10100 sorted
387 ms; string length 10100 unsorted
381 ms; string length 10100 sorted
355 ms; string length 10100 unsorted
542 ms; string length 10100 sorted
611 ms; string length 10100 unsorted
621 ms; string length 10100 sorted
```
It is initially slower due to JVM warmup.
Afterwards, sorting is a bit slower, but not much.
For 100 MB of data, sorting overhead is about 0.1 seconds at most, which is
insignificant in my view. (If we want to speed up things, we should not use
JSON.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10464: Use Testcontainers instead of com.arakelian:docker-junit-rule [jackrabbit-oak]
t-rana commented on code in PR #1143:
URL: https://github.com/apache/jackrabbit-oak/pull/1143#discussion_r1361878571
##
oak-blob-cloud-azure/pom.xml:
##
@@ -191,10 +191,10 @@
mockito-core
test
-
-com.arakelian
-docker-junit-rule
+org.testcontainers
+testcontainers
+${testcontainers.version}
Review Comment:
Defining a version in multi module project is always beneficial. The version
which is mentioned in the child's pom always received precedence over any
dependency conflicts, in this case the version is defined in parent pom which
in turn is used by all child modules (this makes sure that the same version of
test containers is used in oak).
I don't know why the same has not been done in other cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10464: Use Testcontainers instead of com.arakelian:docker-junit-rule [jackrabbit-oak]
t-rana commented on code in PR #1143:
URL: https://github.com/apache/jackrabbit-oak/pull/1143#discussion_r1361878571
##
oak-blob-cloud-azure/pom.xml:
##
@@ -191,10 +191,10 @@
mockito-core
test
-
-com.arakelian
-docker-junit-rule
+org.testcontainers
+testcontainers
+${testcontainers.version}
Review Comment:
Defining a version in multi module project is always beneficial. The version
which is mentioned in the child's pom always received precedence over any
dependency conflicts, in this case the version is defined in parent pom which
in turn is used by all child modules.
I don't know why the same has not been done in other cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] OAK-10497 Properties order in FFS can be different across runs: always sort [jackrabbit-oak]
thomasmueller opened a new pull request, #1159: URL: https://github.com/apache/jackrabbit-oak/pull/1159 I think it's easier to always sort. Performance-wise, it doesn't have a big impact, according to a micro-branchmark I wrote (a few percent slower per serialized node). If we want to speed up things, we should not use JSON. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10464: Use Testcontainers instead of com.arakelian:docker-junit-rule [jackrabbit-oak]
t-rana commented on code in PR #1143:
URL: https://github.com/apache/jackrabbit-oak/pull/1143#discussion_r1361859235
##
oak-segment-azure/src/test/resources/logback-test.xml:
##
@@ -21,11 +21,19 @@
%d{-MM-dd HH:mm:ss.SSS} [%thread] %-5level
%logger{36} - %msg%n
-
+
Review Comment:
this change is reverted now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] OAK-10464: Use Testcontainers instead of com.arakelian:docker-junit-rule [jackrabbit-oak]
t-rana commented on code in PR #1143:
URL: https://github.com/apache/jackrabbit-oak/pull/1143#discussion_r1361859235
##
oak-segment-azure/src/test/resources/logback-test.xml:
##
@@ -21,11 +21,19 @@
%d{-MM-dd HH:mm:ss.SSS} [%thread] %-5level
%logger{36} - %msg%n
-
+
Review Comment:
reverted back this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] Oak-10497 | Properties order in FFS can be different across runs [jackrabbit-oak]
nit0906 opened a new pull request, #1158: URL: https://github.com/apache/jackrabbit-oak/pull/1158 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (JCR-4954) Problem with WebDav Client and Repository Server Deployment Model
[ https://issues.apache.org/jira/browse/JCR-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776114#comment-17776114 ] Manfred Baedke commented on JCR-4954: - [~c_koell], could you share the configuration files of the web applications? Then we can quickly set up a test environment. > Problem with WebDav Client and Repository Server Deployment Model > - > > Key: JCR-4954 > URL: https://issues.apache.org/jira/browse/JCR-4954 > Project: Jackrabbit Content Repository > Issue Type: Bug > Components: jackrabbit-jcr-server, jackrabbit-webdav >Reporter: Claus Köll >Assignee: Manfred Baedke >Priority: Major > Fix For: 2.22 > > Attachments: Call-Hierarchy.txt > > > We have one WebApp where we have deployed the SimpleWebDavServlet > (WebDav-WebApp). From there we connect to a other WebApp where we have > exposed a Repository through DavEx > (org.apache.jackrabbit.server.remoting.davex.JcrRemotingServlet) > and also the RMI-Layer by the RMI Registry (Repository-WebApp) > Until now we connected over RMI but now we tried to switch to DavEx. The > Problem is, that we are now unable to unlock a opened WebDav Document. > The Lock Request in the Repository-WebApp adds a Reference (LockToken) to the > DavSession so a future Requests can obtain the same DavSession to perform the > unlock. > https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-jcr-server/src/main/java/org/apache/jackrabbit/webdav/jcr/DefaultItemCollection.java#L700 > The Reference (Locktoken) looks like > opaquelocktoken:dccce564-412e-11e1-b969-00059a3c7a00:0089610c-02e5-43cd-bfec-3a90361350f4-0 > It will be generated by the LockTokenMapper > https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-jcr-server/src/main/java/org/apache/jackrabbit/webdav/jcr/lock/LockTokenMapper.java#L53 > In the WebDav-WebApp the HeaderLockToken will be stored in this format > opaquelocktoken:dccce564-412e-11e1-b969-00059a3c7a00:opaquelocktoken%3a4403ef44-4124-11e1-b965-00059a3c7a00%3a0089610c-02e5-43cd-bfec-3a90361350f4 > As you can see it will be double wrapped. That's not really a Problem because > on unlock the WebDav-WebApp removes the prefix > opaquelocktoken:dccce564-412e-11e1-b969-00059a3c7a00: > Finally this locktoken will be send from the WebDav-App to the > Repository-WebApp > opaquelocktoken:4403ef44-4124-11e1-b965-00059a3c7a00:0089610c-02e5-43cd-bfec-3a90361350f4 > On the Repository-WebApp the JCRWebdavServer will look for a DavSession in > the Cache based on the lockToken > https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-jcr-server/src/main/java/org/apache/jackrabbit/server/jcr/JCRWebdavServer.java#L210 > So the problem is that the DavSession whitch have created the Lock, is stored > with a lockToken that do not match with the incoming lockToken > Cache-Key-Token: > opaquelocktoken:dccce564-412e-11e1-b969-00059a3c7a00:0089610c-02e5-43cd-bfec-3a90361350f4-0 > Incoming-Token: > opaquelocktoken:4403ef44-4124-11e1-b965-00059a3c7a00:0089610c-02e5-43cd-bfec-3a90361350f4 > The DavSession-Cache Key is taken from the LockInfo (LockTokenCheckDigit is > present) > https://github.com/apache/jackrabbit/blob/trunk/jackrabbit-core/src/main/java/org/apache/jackrabbit/core/lock/LockInfo.java#L118 > Maybe someone which is familiar with the code of LockTokenMapper can explain > the two different Scopes (SESSIONSCOPED/OPENSCOPED) > One possible solution could be to change the code in the LockTokenMapper to > always return the Node Identifier with the same SCOPE-PREFIX? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (JCR-4954) Problem with WebDav Client and Repository Server Deployment Model
[
https://issues.apache.org/jira/browse/JCR-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776110#comment-17776110
]
Manfred Baedke commented on JCR-4954:
-
I do not understand this call stack from server B:
{code:java}
bstractWebdavServlet.doPropFind -> LOCKDISCOVERY
org.apache.jackrabbit.webdav.jcr.AbstractResource.initProperties
properties.add(new
LockDiscovery(getLocks()));
org.apache.jackrabbit.webdav.jcr.AbstractResource.getLocks
// write lock (either
exclusive or session-scoped).
getLock(Type.WRITE,
Scope.EXCLUSIVE);
org.apache.jackrabbit.webdav.jcr.DefaultItemCollection.getLock
jcrLock ->
org.apache.jackrabbit.core.lock.XALockImpl
ActiveLock lock
= new JcrActiveLock(jcrLock);
org.apache.jackrabbit.webdav.jcr.lock.JcrActiveLock.toXml
org.apache.jackrabbit.webdav.jcr.lock.JcrActiveLock.getToken
Re: [PR] OAK-10495: Ignore DocumentStoreIndexerIT#parallelReindex* tests [jackrabbit-oak]
amit-jain merged PR #1157: URL: https://github.com/apache/jackrabbit-oak/pull/1157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10495: Ignore DocumentStoreIndexerIT#parallelReindex* tests [jackrabbit-oak]
sonarcloud[bot] commented on PR #1157: URL: https://github.com/apache/jackrabbit-oak/pull/1157#issuecomment-1765996409 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=1157&metric=new_duplicated_lines_density&view=list)  The version of Java (11.0.20.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10494 - Add cache to reduce number of remote blobstore calls. [jackrabbit-oak]
amit-jain commented on PR #1155: URL: https://github.com/apache/jackrabbit-oak/pull/1155#issuecomment-1765997786 @ahanikel the PR looks good but I am not sure if the invalidation also has to happen somewhere. The tests are in CachingDataStoreTest for this class. Overall, I think the call need not go to the backend in case the the corresponding file is available in the cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10471 Implement ConflictHandler for UserPrincipalProvider Cache - fix tests [jackrabbit-oak]
anchela merged PR #1154: URL: https://github.com/apache/jackrabbit-oak/pull/1154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10494 - Add cache to reduce number of remote blobstore calls. [jackrabbit-oak]
ahanikel commented on PR #1155: URL: https://github.com/apache/jackrabbit-oak/pull/1155#issuecomment-1765933840 @amit-jain Could you have a look when you have time? Thanks a lot! The test is kinda stupid but I couldn't think of a better one... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] OAK-10495: Ignore DocumentStoreIndexerIT#parallelReindex* tests [jackrabbit-oak]
amit-jain opened a new pull request, #1157: URL: https://github.com/apache/jackrabbit-oak/pull/1157 Ignore relevant tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] OAK-10491 - Indexing: pass a MongoDatabase instance instead of MongoConnection to indexing logic [jackrabbit-oak]
fabriziofortino merged PR #1151: URL: https://github.com/apache/jackrabbit-oak/pull/1151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@jackrabbit.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
