[
https://issues.apache.org/jira/browse/JCRVLT-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Hoh updated JCRVLT-809:
-----------------------------
Description:
*NOTE*: I leave this initial idea as is, but the improvement is done
differently; see the comments below and the Github PR.
In the case of Sling distribution the packages being created and imported are
controlled by the framework, and IIUC protected properties are not part of the
package; for that reason {{DocViewImporter.logIgnoredProtectedProperties}} will
not write any WARN message.
When checking the details of the import of a complex package (with almost 100k
nodes and low to medium amount of properties per node), I came frequently
across this type of stack, where it's checked if properties are protected.
Commenting the 2 calls to {{DocViewImporter.logIgnoredProtectedProperties}}
gave me an improvement of the import time in the range of 10%.
For that I think that we should provide a chance to disable these checks,
preferably via the API, so that it can configured per package I want to import.
Disabling it globally might not be helpful, as we might have cases, where we
cannot give the guarantee about no protected properties being part of the
package(s).
{noformat}
at
org.apache.commons.collections4.iterators.FilterIterator.hasNext(FilterIterator.java:105)
at
org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getDeclaredPropertyDefinitions(NodeTypeImpl.java:189)
at
org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.internalGetPropertyDefinitions(NodeTypeImpl.java:521)
at
org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getPropertyDefinitions(NodeTypeImpl.java:316)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType.lambda$getApplicablePropertyDefinition$1(EffectiveNodeType.java:111)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType$$Lambda/0x000000080194cb00.apply(Unknown
Source)
at
java.util.stream.ReferencePipeline$7$1.accept([email protected]/ReferencePipeline.java:273)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining([email protected]/ArrayList.java:1708)
at
java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential([email protected]/ReduceOps.java:921)
at
java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect([email protected]/ReferencePipeline.java:682)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:111)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:107)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.isPropertyProtected(DocViewImporter.java:1235)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.lambda$logIgnoredProtectedProperties$7(DocViewImporter.java:1217)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter$$Lambda/0x000000080194c688.accept(Unknown
Source)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept([email protected]/ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$2$1.accept([email protected]/ReferencePipeline.java:179)
at
java.util.Iterator.forEachRemaining([email protected]/Iterator.java:133)
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining([email protected]/Spliterators.java:1939)
at
java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential([email protected]/ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential([email protected]/ForEachOps.java:174)
at
java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach([email protected]/ReferencePipeline.java:596)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.logIgnoredProtectedProperties(DocViewImporter.java:1215)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.createNewNode(DocViewImporter.java:1181)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.addNode(DocViewImporter.java:931)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.startDocViewNode(DocViewImporter.java:410)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXHandler.startElement(DocViewSAXHandler.java:353)
{noformat}
Alternatively I would also be happy about an optimized version of
"DocViewImporter.isPropertyProtected", which does not need resolve every
property all the time from scratch.
Note: Optimizing {{isPropertyProtected}} might be the prefered way, as it is
also invoked via the {{DocViewImporter.setUnprotectedProperties}} code path,
which I also see very frequently in my dumps.
was:
In the case of Sling distribution the packages being created and imported are
controlled by the framework, and IIUC protected properties are not part of the
package; for that reason {{DocViewImporter.logIgnoredProtectedProperties}} will
not write any WARN message.
When checking the details of the import of a complex package (with almost 100k
nodes and low to medium amount of properties per node), I came frequently
across this type of stack, where it's checked if properties are protected.
Commenting the 2 calls to {{DocViewImporter.logIgnoredProtectedProperties}}
gave me an improvement of the import time in the range of 10%.
For that I think that we should provide a chance to disable these checks,
preferably via the API, so that it can configured per package I want to import.
Disabling it globally might not be helpful, as we might have cases, where we
cannot give the guarantee about no protected properties being part of the
package(s).
{noformat}
at
org.apache.commons.collections4.iterators.FilterIterator.hasNext(FilterIterator.java:105)
at
org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getDeclaredPropertyDefinitions(NodeTypeImpl.java:189)
at
org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.internalGetPropertyDefinitions(NodeTypeImpl.java:521)
at
org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getPropertyDefinitions(NodeTypeImpl.java:316)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType.lambda$getApplicablePropertyDefinition$1(EffectiveNodeType.java:111)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType$$Lambda/0x000000080194cb00.apply(Unknown
Source)
at
java.util.stream.ReferencePipeline$7$1.accept([email protected]/ReferencePipeline.java:273)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining([email protected]/ArrayList.java:1708)
at
java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential([email protected]/ReduceOps.java:921)
at
java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect([email protected]/ReferencePipeline.java:682)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:111)
at
org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:107)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.isPropertyProtected(DocViewImporter.java:1235)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.lambda$logIgnoredProtectedProperties$7(DocViewImporter.java:1217)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter$$Lambda/0x000000080194c688.accept(Unknown
Source)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept([email protected]/ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$2$1.accept([email protected]/ReferencePipeline.java:179)
at
java.util.Iterator.forEachRemaining([email protected]/Iterator.java:133)
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining([email protected]/Spliterators.java:1939)
at
java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential([email protected]/ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential([email protected]/ForEachOps.java:174)
at
java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach([email protected]/ReferencePipeline.java:596)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.logIgnoredProtectedProperties(DocViewImporter.java:1215)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.createNewNode(DocViewImporter.java:1181)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.addNode(DocViewImporter.java:931)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.startDocViewNode(DocViewImporter.java:410)
at
org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXHandler.startElement(DocViewSAXHandler.java:353)
{noformat}
Alternatively I would also be happy about an optimized version of
"DocViewImporter.isPropertyProtected", which does not need resolve every
property all the time from scratch.
Note: Optimizing {{isPropertyProtected}} might be the prefered way, as it is
also invoked via the {{DocViewImporter.setUnprotectedProperties}} code path,
which I also see very frequently in my dumps.
> DocViewImporter: logIgnoredProtectedProperties can slow down package import
> ---------------------------------------------------------------------------
>
> Key: JCRVLT-809
> URL: https://issues.apache.org/jira/browse/JCRVLT-809
> Project: Jackrabbit FileVault
> Issue Type: Improvement
> Components: Packaging
> Reporter: Joerg Hoh
> Assignee: Joerg Hoh
> Priority: Major
> Fix For: 4.0.0
>
>
> *NOTE*: I leave this initial idea as is, but the improvement is done
> differently; see the comments below and the Github PR.
> In the case of Sling distribution the packages being created and imported are
> controlled by the framework, and IIUC protected properties are not part of
> the package; for that reason
> {{DocViewImporter.logIgnoredProtectedProperties}} will not write any WARN
> message.
> When checking the details of the import of a complex package (with almost
> 100k nodes and low to medium amount of properties per node), I came
> frequently across this type of stack, where it's checked if properties are
> protected. Commenting the 2 calls to
> {{DocViewImporter.logIgnoredProtectedProperties}} gave me an improvement of
> the import time in the range of 10%.
> For that I think that we should provide a chance to disable these checks,
> preferably via the API, so that it can configured per package I want to
> import. Disabling it globally might not be helpful, as we might have cases,
> where we cannot give the guarantee about no protected properties being part
> of the package(s).
> {noformat}
> at
> org.apache.commons.collections4.iterators.FilterIterator.hasNext(FilterIterator.java:105)
> at
> org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getDeclaredPropertyDefinitions(NodeTypeImpl.java:189)
> at
> org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.internalGetPropertyDefinitions(NodeTypeImpl.java:521)
> at
> org.apache.jackrabbit.oak.plugins.nodetype.NodeTypeImpl.getPropertyDefinitions(NodeTypeImpl.java:316)
> at
> org.apache.jackrabbit.vault.util.EffectiveNodeType.lambda$getApplicablePropertyDefinition$1(EffectiveNodeType.java:111)
> at
> org.apache.jackrabbit.vault.util.EffectiveNodeType$$Lambda/0x000000080194cb00.apply(Unknown
> Source)
> at
> java.util.stream.ReferencePipeline$7$1.accept([email protected]/ReferencePipeline.java:273)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining([email protected]/ArrayList.java:1708)
> at
> java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential([email protected]/ReduceOps.java:921)
> at
> java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect([email protected]/ReferencePipeline.java:682)
> at
> org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:111)
> at
> org.apache.jackrabbit.vault.util.EffectiveNodeType.getApplicablePropertyDefinition(EffectiveNodeType.java:107)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.isPropertyProtected(DocViewImporter.java:1235)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.lambda$logIgnoredProtectedProperties$7(DocViewImporter.java:1217)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter$$Lambda/0x000000080194c688.accept(Unknown
> Source)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept([email protected]/ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept([email protected]/ReferencePipeline.java:179)
> at
> java.util.Iterator.forEachRemaining([email protected]/Iterator.java:133)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining([email protected]/Spliterators.java:1939)
> at
> java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential([email protected]/ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential([email protected]/ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach([email protected]/ReferencePipeline.java:596)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.logIgnoredProtectedProperties(DocViewImporter.java:1215)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.createNewNode(DocViewImporter.java:1181)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.addNode(DocViewImporter.java:931)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewImporter.startDocViewNode(DocViewImporter.java:410)
> at
> org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXHandler.startElement(DocViewSAXHandler.java:353)
> {noformat}
> Alternatively I would also be happy about an optimized version of
> "DocViewImporter.isPropertyProtected", which does not need resolve every
> property all the time from scratch.
> Note: Optimizing {{isPropertyProtected}} might be the prefered way, as it is
> also invoked via the {{DocViewImporter.setUnprotectedProperties}} code path,
> which I also see very frequently in my dumps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
