[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774276#comment-17774276
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
olamy merged PR #91:
URL: https://github.com/apache/maven-build-cache-extension/pull/91
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17771350#comment-17771350
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1744305773
Re-opening the PR for merge.
It's now a mix between the very first implementation and the one with globs,
and it's combining better with the input section mechanism.
I feel much more at ease with this version. We can use only path
declaration, only glob declaration, or combine both if we want the best
trade-off between performances and functionalities.
Here a small list of improvements:
- the glob interpretation can be configured to better handle any specific
case (= less code to handle the specific cases ... :p )
- easier to test the exclusion mechanism with a unit test (added one)
- unit + IT test about unix/dos path style (and tested on Windows and Debian)
Here is the documentation presenting it :
And the updated "project property" page.
Looking forward to your feedbacks. :)
Especially in the project property section, I'm undecided about the naming
of the exclusion value : "maven.build.cache.exclude.value.my-exclusion" versus
"maven.build.cache.exclude.my-exclusion". (currently the first option, simpler
syntax / readability with the additional properties like
"maven.build.cache.exclude.glob.my-exclusion")
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770030#comment-17770030
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1739027006
About this PR, I'm still working on it. Even if I couldn't find the time the
last past weeks.
I re-put it in draft @maximilian-novikov-db et @AlexanderAshitkin, I think
I'll be able to submit a more consensual version in the next days.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760948#comment-17760948
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311669985
##
src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.maven.buildcache.checksum;
+
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+
+/**
+ * A path matcher with some extra info
+ */
+public class TreeWalkerPathMatcher implements PathMatcher {
+
+/**
+ * True if the matching should stop exploring the directory tree further
away
+ */
+private final boolean matchingSkipSubtree;
+
+/**
+ * Wrapped regular path matcher
+ */
+private final PathMatcher pathMatcher;
Review Comment:
There might be room for error here. For example, the same file with
back/forward slashes is not matched on my workstation:
```java
String linux = "dir/file.txt";
String win = "dir\\file.txt";
Path linuxPath = Paths.get(linux);
Path winPath = Paths.get(win);
System.out.println("linuxPath: " + linuxPath);
System.out.println("winPath: " + winPath);
PathMatcher linuxMatcher =
FileSystems.getDefault().getPathMatcher("glob:" + linuxPath);
System.out.println(linuxMatcher.matches(winPath));
System.out.println(linuxMatcher.matches(linuxPath));
--OUT
linuxPath: dir/file.txt
winPath: dir\file.txt
false
true
```
I have no objections against the matcher as such. But I wouldn't rely on it
without a test demonstrating the same results on Windows and Mac in the
presence of exclusions.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760947#comment-17760947
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311669985
##
src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.maven.buildcache.checksum;
+
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+
+/**
+ * A path matcher with some extra info
+ */
+public class TreeWalkerPathMatcher implements PathMatcher {
+
+/**
+ * True if the matching should stop exploring the directory tree further
away
+ */
+private final boolean matchingSkipSubtree;
+
+/**
+ * Wrapped regular path matcher
+ */
+private final PathMatcher pathMatcher;
Review Comment:
There might be room for error here. For example, the same file with
back/forward slashes is not matched on my workstation:
```java
String linux = "dir/file.txt";
String win = "dir\\file.txt";
Path linuxPath = Paths.get(linux);
Path winPath = Paths.get(win);
System.out.println("linuxPath: " + linuxPath);
System.out.println("winPath: " + winPath);
PathMatcher linuxMatcher =
FileSystems.getDefault().getPathMatcher("glob:" + linuxPath);
System.out.println(linuxMatcher.matches(winPath));
System.out.println(linuxMatcher.matches(linuxPath));
--
false
true
```
I have no objections against the matcher as such. But I wouldn't rely on it
without a test demonstrating the same results on Windows and Mac in the
presence of exclusions.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760935#comment-17760935
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311038794
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlob : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+// In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+// "files + directories"
+inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+// Globs ending with "**" should end any
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760698#comment-17760698
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311038794
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlob : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+// In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+// "files + directories"
+inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+// Globs ending with "**" should end any
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760697#comment-17760697
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311044724
##
src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.maven.buildcache.checksum;
+
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+
+/**
+ * A path matcher with some extra info
+ */
+public class TreeWalkerPathMatcher implements PathMatcher {
+
+/**
+ * True if the matching should stop exploring the directory tree further
away
+ */
+private final boolean matchingSkipSubtree;
+
+/**
+ * Wrapped regular path matcher
+ */
+private final PathMatcher pathMatcher;
+
+public TreeWalkerPathMatcher(String pathMatcherGlob, boolean
matchingSkipSubtree) {
+pathMatcher = FileSystems.getDefault().getPathMatcher(pathMatcherGlob);
+this.matchingSkipSubtree = matchingSkipSubtree;
+}
+
+@Override
+public boolean matches(Path path) {
Review Comment:
Paths could be expressed differently, and before matching, it should be
normalized - placeholders resolved, etc.
```
PathMatcher pathMatcher =
FileSystems.getDefault().getPathMatcher("glob:./file.txt");
System.out.println("./file.txt -> "+
Paths.get("./file.txt").normalize().toAbsolutePath());
System.out.println("./dir/../file.txt -> "+
Paths.get("dir/../file.txt").normalize().toAbsolutePath());
System.out.println(pathMatcher.matches(Paths.get("dir/../file.txt")));
Out---
./file.txt -> /sandbox/file.txt
./dir/../file.txt -> /sandbox/file.txt
false
```
The matcher might be required to normalize paths to match reliably. Just
taking any path as an input might not work. It's better to have clear semantics
here.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760696#comment-17760696
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311044724
##
src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.maven.buildcache.checksum;
+
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+
+/**
+ * A path matcher with some extra info
+ */
+public class TreeWalkerPathMatcher implements PathMatcher {
+
+/**
+ * True if the matching should stop exploring the directory tree further
away
+ */
+private final boolean matchingSkipSubtree;
+
+/**
+ * Wrapped regular path matcher
+ */
+private final PathMatcher pathMatcher;
+
+public TreeWalkerPathMatcher(String pathMatcherGlob, boolean
matchingSkipSubtree) {
+pathMatcher = FileSystems.getDefault().getPathMatcher(pathMatcherGlob);
+this.matchingSkipSubtree = matchingSkipSubtree;
+}
+
+@Override
+public boolean matches(Path path) {
Review Comment:
Paths could be expressed differently, and before matching, it should be
normalized - placeholders resolved, etc.
```
PathMatcher pathMatcher =
FileSystems.getDefault().getPathMatcher("glob:./file.txt");
System.out.println("./file.txt -> "+
Paths.get("./file.txt").normalize().toAbsolutePath());
System.out.println("./dir/../file.txt -> "+
Paths.get("dir/../file.txt").normalize().toAbsolutePath());
System.out.println(pathMatcher.matches(Paths.get("dir/../file.txt")));
Out---
./file.txt -> /sandbox/file.txt
./dir/../file.txt -> /sandbox/file.txt
false
```
The matcher might be required to normalize paths to match reliably. Just
taking any path as an input might not work. It's better to have clear semantics
here.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760691#comment-17760691
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311038794
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlob : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+// In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+// "files + directories"
+inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+// Globs ending with "**" should end any
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760689#comment-17760689
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311038794
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlob : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+// In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+// "files + directories"
+inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+// Globs ending with "**" should end any
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760681#comment-17760681
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311033263
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +185,101 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.projectGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
-org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+this.baseDirectoryGlobPrefix = baseDirPath.toString().replace("\\",
"/") + "/";
+
+addDefaultExcludeSection(project);
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private void addDefaultExcludeSection(MavenProject project) {
+
+Build build = project.getBuild();
+Path buildDirectoryPath = normalizedPath(build.getDirectory());
+Path outputDirectoryPath = normalizedPath(build.getOutputDirectory());
+Path testOutputDirectoryPath =
normalizedPath(build.getTestOutputDirectory());
+addToExcludedSection(
+
convertToPathMatcherFileSeperator(buildDirectoryPath.toString()) +
GLOB_SX_FULL_SUB_TREE,
+false); // target by default
+
+if (!outputDirectoryPath.startsWith(buildDirectoryPath)) {
+addToExcludedSection(
+
convertToPathMatcherFileSeperator(outputDirectoryPath.toString()) +
GLOB_SX_FULL_SUB_TREE,
+false); // target/classes by default
+}
+if (!testOutputDirectoryPath.startsWith(buildDirectoryPath)) {
+addToExcludedSection(
+
convertToPathMatcherFileSeperator(testOutputDirectoryPath.toString()) +
GLOB_SX_FULL_SUB_TREE,
+false); // target/test-classes by default
+}
+}
+
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlobPrefix : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.s
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760679#comment-17760679
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311031579
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +185,101 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.projectGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
-org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+this.baseDirectoryGlobPrefix = baseDirPath.toString().replace("\\",
"/") + "/";
+
+addDefaultExcludeSection(project);
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private void addDefaultExcludeSection(MavenProject project) {
+
+Build build = project.getBuild();
+Path buildDirectoryPath = normalizedPath(build.getDirectory());
+Path outputDirectoryPath = normalizedPath(build.getOutputDirectory());
+Path testOutputDirectoryPath =
normalizedPath(build.getTestOutputDirectory());
+addToExcludedSection(
+
convertToPathMatcherFileSeperator(buildDirectoryPath.toString()) +
GLOB_SX_FULL_SUB_TREE,
+false); // target by default
+
+if (!outputDirectoryPath.startsWith(buildDirectoryPath)) {
+addToExcludedSection(
+
convertToPathMatcherFileSeperator(outputDirectoryPath.toString()) +
GLOB_SX_FULL_SUB_TREE,
+false); // target/classes by default
+}
+if (!testOutputDirectoryPath.startsWith(buildDirectoryPath)) {
+addToExcludedSection(
+
convertToPathMatcherFileSeperator(testOutputDirectoryPath.toString()) +
GLOB_SX_FULL_SUB_TREE,
+false); // target/test-classes by default
+}
+}
+
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
Review Comment:
Why need the `addProjectBaseDir` parameter? Intuitively, the project dir
should always be prepended. Otherwise, the `exclude` could be applied to an
unexpected subtree (calculated not from the root but to a nested one)
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL:
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760643#comment-17760643
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1310900630
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -526,6 +603,10 @@ public FileVisitResult visitFileFailed(Path path,
IOException exc) throws IOExce
});
}
+private boolean entryMustBeSkipped(Path path) {
+return inputExcludePathMatchers.stream().anyMatch(pathMatcher ->
pathMatcher.matches(path));
Review Comment:
Right, iteration is over the matchers, not paths 🤦
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -526,6 +603,10 @@ public FileVisitResult visitFileFailed(Path path,
IOException exc) throws IOExce
});
}
+private boolean entryMustBeSkipped(Path path) {
+return inputExcludePathMatchers.stream().anyMatch(pathMatcher ->
pathMatcher.matches(path));
Review Comment:
Right, iteration is over the matchers, not paths 🤦
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759573#comment-17759573
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1695606322
> I'm starting to wonder if the use of the JDK `PathMatcher` is sufficient
here. If we want to optimize the file system access and not traverse _all_
files / subdirectories, we'll have to use
[`Files.walkFileTree`](https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#walkFileTree-java.nio.file.Path-java.nio.file.FileVisitor-)
as it has built-in support for skipping subtrees. However, I'm not sure there
is any good implementation of glob syntax that would skip subtree when known to
not match.
Well, good news, we are already using `Files#walkFileTree`. One of the
discussion is precisely how I managed to keep as well as possible the capacity
to skip subtree scanning (it is a key point for the projects I want to use this
extension on).
@All : Seems this PR draw attention to details about this functionality but
I would suggest to not go too far with it.
I already agreed to change it from a bugfix to an evolution. Maybe the
current state is good enough for now. And if good ideas emerge they can be
implemented in another PR. I have the feeling that in term of functionality /
performance / test and documentation, this PR brings already a lot to the
table. And people will be easily capable after merging to change some details
they would have seen differently.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758841#comment-17758841
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
gnodet commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1692814388
I'm starting to wonder if the use of the JDK `PathMatcher` is sufficient
here. If we want to optimize the file system access and not traverse _all_
files / subdirectories, we'll have to use
[`Files.walkFileTree`](https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#walkFileTree-java.nio.file.Path-java.nio.file.FileVisitor-)
as it has built-in support for skipping subtrees. However, I'm not sure there
is any good implementation of glob syntax that would skip subtree when known to
not match.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758260#comment-17758260
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303583940
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -633,13 +714,7 @@ private static boolean isReadable(Path entry) throws
IOException {
}
private boolean isFilteredOutSubpath(Path path) {
-Path normalized = path.normalize();
-for (Path filteredOutDir : filteredOutPaths) {
-if (normalized.startsWith(filteredOutDir)) {
-return true;
-}
-}
-return false;
+return inputExcludeDirectoryPathMatchers.stream().anyMatch(matcher ->
matcher.stopTreeWalking(path));
Review Comment:
I'm not following. Could you give me some code snippets to illustrate your
reasoning?
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
Review Comment:
Good call. Added it. I did a naive implementation to keep the code simple.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758259#comment-17758259
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303583781
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -526,6 +603,10 @@ public FileVisitResult visitFileFailed(Path path,
IOException exc) throws IOExce
});
}
+private boolean entryMustBeSkipped(Path path) {
+return inputExcludePathMatchers.stream().anyMatch(pathMatcher ->
pathMatcher.matches(path));
Review Comment:
I can't, "matches" takes a parameter (or you refer to a syntax I don't know).
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758258#comment-17758258
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303583503
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlob : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+// In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+// "files + directories"
+inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+// Globs ending with "**" should end any sub-dire
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758256#comment-17758256
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303583332
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+String pathMatcherGlob = GLOB_PX
++
+// Add the base directory to any input directly coming from
user configuration
+(addProjectBaseDir ? baseDirectoryGlob : "")
++
+// If the glob start with "/", we remove it since it's already
added in the added basedir glob
+(excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
Review Comment:
As the glob is converted under the hood to a regex, a PatternSyntaxException
can be immediately thrown (like the InvalidPathException for a path conversion).
And if the pattern is valid but not corresponding to any visited
file/directory, I'm not sure we can do something about it.
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758254#comment-17758254
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303582683
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
Review Comment:
This constant is used as a prefix and concatenated with other values.
Renamed it to `baseDirectoryGlobPrefix `to improve the readability.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758250#comment-17758250
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303582142
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -127,6 +121,14 @@ public class MavenProjectInput {
* Flag to control if we should check values from plugin configs as file
system objects
*/
private static final String CACHE_PROCESS_PLUGINS =
"maven.build.cache.processPlugins";
+/**
+ * Glob prefix for path matchers
+ */
+private static final String GLOB_PX = "glob:";
+/**
+ * Glob suffix meaning "all files in this directory or any sub-directory"
+ */
+private static final String GLOB_SX_ALL_SUB_FILES = "/**";
Review Comment:
Good call, fixed
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758255#comment-17758255
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303582929
##
src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.maven.buildcache.checksum;
+
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+
+/**
+ * A path matcher with some extra info
+ */
+public class TreeWalkerPathMatcher implements PathMatcher {
+
+/**
+ * True if the matching should stop exploring the directory tree further
away
+ */
+private final boolean matchingSkipSubtree;
+
+/**
+ * Wrapped regular path matcher
+ */
+private final PathMatcher pathMatcher;
+
+public TreeWalkerPathMatcher(String pathMatcherGlob, boolean
matchingSkipSubtree) {
+pathMatcher = FileSystems.getDefault().getPathMatcher(pathMatcherGlob);
+this.matchingSkipSubtree = matchingSkipSubtree;
+}
+
+@Override
+public boolean matches(Path path) {
Review Comment:
Do you mean in case we decide to do some extra work about OS specific
filesystems? Or anyway?
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758253#comment-17758253
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303582683
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
Review Comment:
This constant is used as a prefix and concatenated with other values.
Renamed it to 'baseDirectoryGlobPrefix' to improve the readability.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758252#comment-17758252
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303582443
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
Review Comment:
Good point, changed by the suggested name.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758249#comment-17758249
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1303582003
##
src/main/java/org/apache/maven/buildcache/checksum/TreeWalkerPathMatcher.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.maven.buildcache.checksum;
+
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+
+/**
+ * A path matcher with some extra info
+ */
+public class TreeWalkerPathMatcher implements PathMatcher {
+
+/**
+ * True if the matching should stop exploring the directory tree further
away
+ */
+private final boolean matchingSkipSubtree;
+
+/**
+ * Wrapped regular path matcher
+ */
+private final PathMatcher pathMatcher;
Review Comment:
It should not, the underlying conversion from glob to regex pattern
(sun.nio.fs.Globs#toRegexPattern) used by different FS pathmatcher
implementations (ex : sun.nio.fs.WindowsFileSystem#getPathMatcher) seems to
handle FS specific syntax conversion since java 8.
(https://github.com/adoptium/jdk8u)
But PathMatching is not something I've dealt with a lot. Do you think we
should take a belt and braces approach ? (french expression, but it looks like
existing in English too ^^)
PS : On a side note, I'm always testing on Windows (but not all jvm ...)
since I primarily work on Windows those days.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758248#comment-17758248
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1690697801
> @kbuntrock nicely done, it would be useful to update the sample config as
well
https://github.com/apache/maven-build-cache-extension/blob/master/src/site/resources/maven-build-cache-config.xml
Done it. Wasn't comfortable with this file since I found it confusing when I
discovered this extension + it was still valid with my changes. Added some
comments on some related other parameters around. Is it ok for you
@maximilian-novikov-db ?
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757035#comment-17757035
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1298793252
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
Review Comment:
For consistency, this should probably be normalized to a `/**` format,
or the variable should be renamed `*Path`.
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
-filteredOutPaths.add(Paths.get(excludePath.getValue()));
+addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
-Path path = Paths.get(propertyValue);
-filteredOutPaths.add(path);
+addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
-"Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
-propertyName,
-propertyValue,
-path);
+"Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
-LOGGER,
-filteredOutPaths,
-"List of excluded paths (checked either by fileName or by
startsWith prefix)",
-"Path entry");
+LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+private String convertToPathMatcherFileSeperator(String path) {
+return path.replace("\\", "/");
+}
+
+/**
+ * Add a value from the excluded section list to the directories and/or
the filename
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17756704#comment-17756704
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
maximilian-novikov-db commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1685792860
> Changed the exclude mechanism to use path matchers. A bit more than a
bugfix now, feedbacks appreciated if you see any lack, implementation detail
you disagree with or extra test case to add.
@kbuntrock nicely done, it would be useful to update the sample config as
well
https://github.com/apache/maven-build-cache-extension/blob/master/src/site/resources/maven-build-cache-config.xml
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751751#comment-17751751
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1668279033
Changed the exclude mechanism to use path matchers. A bit more than a bugfix
now, feedbacks appreciated if you see any lack, implementation detail you
disagree with or extra test case to add.
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751749#comment-17751749
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1286165121
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -108,11 +107,6 @@ public class MavenProjectInput {
* property name to pass glob value. The glob to be used to list directory
files in plugins scanning
*/
private static final String CACHE_INPUT_GLOB_NAME =
"maven.build.cache.input.glob";
-/**
- * default glob, bbsdk/abfx specific
- */
-public static final String DEFAULT_GLOB =
"{*.java,*.groovy,*.yaml,*.svcd,*.proto,*assembly.xml,assembly"
Review Comment:
Was not used except in a UT.
##
src/test/java/org/apache/maven/buildcache/checksum/MavenProjectInputTest.java:
##
@@ -47,52 +47,23 @@
public class MavenProjectInputTest {
-private static final String GLOB = "{*-pom.xml}";
+private static final String DEFAULT_GLOB = "*";
@Test
-public void testAddInputsRelativePath() {
-//MavenProjectInput inputs = new MavenProjectInput(config, new
ArrayList(),
-// Paths.get("src\\test\\resources\\org"), GLOB);
-//ArrayList files = new ArrayList<>();
-//inputs.listDirOrFile("../../resources", inputs.dirGlob,
files, new HashSet());
-//assertEquals(4, files.size());
-}
-
-@Test
-public void testAddInputsAbsolutePath() {
-//Path baseDirPath = Paths.get("src\\test\\resources\\org");
-//MavenProjectInput inputs = new MavenProjectInput(config, new
ArrayList(), baseDirPath, GLOB);
-//ArrayList files = new ArrayList<>();
-//Path candidatePath =
baseDirPath.resolve("../../resources").normalize().toAbsolutePath();
-//inputs.listDirOrFile(candidatePath.toString(),
inputs.dirGlob, files, new HashSet());
-//assertEquals(4, files.size()); // pom is filtered out by
hardcoded if
-}
-
-@Test
-public void testGlobInput() {
-//Path baseDirPath = Paths.get("src\\test\\resources");
-//MavenProjectInput inputs = new MavenProjectInput(config, new
ArrayList(), baseDirPath, GLOB);
-//ArrayList files = new ArrayList<>();
-//inputs.tryAddInputs("*.java", files, new HashSet());
-//assertEquals(0, files.size()); // pom is filtered out by
hardcoded if
-}
-
-@Test
-public void testGetDirectoryFiles() {
-List directoryFiles = new ArrayList<>();
+public void testCollectFilteredFiles() {
Review Comment:
Cleaned dead code and updated a test with a wrong assumption. Not the most
useful test but better than before.
##
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
-this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
-filteredOutPaths = new ArrayList<>(Arrays.asList(
-normalizedPath(build.getDirectory()), // target by default
-normalizedPath(build.getOutputDirectory()),
-normalizedPath(build.getTestOutputDirectory(;
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/classes by default
+addToExcludedSection(
+convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
++ GLOB_SX_ALL_SUB_FILES,
+false); // target/test-classes by default
List e
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751181#comment-17751181
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1665985477
> I'm a bit skeptical about the way which has been chosen to filter files. I
think it would be more intuitive to use a glob `PathMatcher` rather than a
`startsWith()` call on the path. This seems inconsistent with what is usually
done in the maven land, for example
https://maven.apache.org/plugins/maven-jar-plugin/examples/include-exclude.html
Sure, I can do this. It will also bring a bit more flexibility. I suggest
that I stay focused on the exclude section and I don't touch the input section.
PS : Since I'm new there, hello everyone and great job! This project is a
really good idea. I'm dying to be able to use it on my projects. :)
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750922#comment-17750922
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
olamy commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1664857827
LGTM
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Assignee: Olivier Lamy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750884#comment-17750884
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock commented on PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#issuecomment-1664633394
Miss-clicked on the sync button on my repo for this branch (I only wanted to
sync master ...).
So I finished to rebase it on master to keep the tree clean. But it reseted
the workflow, sorry. :s
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Priority: Major
> Labels: pull-request-available
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750361#comment-17750361
]
Kevin Buntrock commented on MBUILDCACHE-64:
---
I updated the description of the pull request for a better understanding
(https://github.com/apache/maven-build-cache-extension/pull/91)
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Priority: Major
> Labels: pull-request-available
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (MBUILDCACHE-64) Apply global exclusions to folder names
[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750014#comment-17750014
]
ASF GitHub Bot commented on MBUILDCACHE-64:
---
kbuntrock opened a new pull request, #91:
URL: https://github.com/apache/maven-build-cache-extension/pull/91
### Description of this pull request
Fix a bug regarding the exclusion mechanism not working properly on folders.
It also add an integration test on this functionality (global + per project).
Finally add some documentation on these aspects.
Here a screenshot of the new "exclude" element in the documentation to
understand how this fix intend to work.
I did not implemented the following sentence in the Jira issue since I did
not got the exact meaning :
"That function could be enhanced with an additional criterion like in
MavenProjectInput.java#L510"
Nevertheless, I think this MR is already a huge improvement on the exclusion
topic since it's a critical feature and it was not possible to use it on
folders before (pretty hard to use it in a project with a "node_module" folder
for example).
Following this checklist to help us incorporate your
contribution quickly and easily:
- [x] Make sure there is a [MBUILDCACHE JIRA
issue](https://issues.apache.org/jira/browse/MBUILDCACHE) filed
for the change (usually before you start working on it). Trivial
changes like typos do not
require a JIRA issue. Your pull request should address just this
issue, without
pulling in other changes.
- [x] Each commit in the pull request should have a meaningful subject line
and body.
- [x] Format the pull request title like `[MBUILDCACHE-XXX] - Fixes bug in
ApproximateQuantiles`,
where you replace `MBUILDCACHE-XXX` with the appropriate JIRA issue.
Best practice
is to use the JIRA issue title in the pull request title and in the
first line of the
commit message.
- [x] Write a pull request description that is detailed enough to
understand what the pull request does, how, and why.
- [x] Run `mvn clean verify` to make sure basic checks pass. A more
thorough check will
be performed on your pull request automatically.
- [x] You have run the [Core IT][core-its] successfully.
If your pull request is about ~20 lines of code you don't need to sign an
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the [Apache License Version 2.0, January
2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [ ] I hereby declare this contribution to be licenced under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [x] In any other case, please file an [Apache Individual Contributor
License Agreement](https://www.apache.org/licenses/icla.pdf).
[core-its]: https://maven.apache.org/core-its/core-it-suite/
> Apply global exclusions to folder names
> ---
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Frank Wagner
>Priority: Major
>
> It is currently not possible to exclude folders by their name, like
> {quote}
>
>
> node_modules
> dist
> build
>
>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
