[
https://issues.apache.org/jira/browse/MRESOLVER-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611848#comment-17611848
]
ASF GitHub Bot commented on MRESOLVER-269:
------------------------------------------
michael-o commented on code in PR #199:
URL: https://github.com/apache/maven-resolver/pull/199#discussion_r985076081
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that
use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local"
layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local
repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will
factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
Review Comment:
Call me stupid, but why don't we use the `LocalPathPrefixComposer` for this.
It will behave identically to `EnhancedLocalRepositoryManager`?
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/FileTrustedChecksumsSourceSupport.java:
##########
@@ -0,0 +1,154 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.checksums.TrustedChecksumsSource;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.ConfigUtils;
+import org.eclipse.aether.util.DirectoryUtils;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Support class for implementing {@link TrustedChecksumsSource} backed by
local filesystem. It implements basic support
+ * like basedir calculation, "enabled" flag and "originAware" flag.
+ * <p>
+ * The configuration keys supported:
+ * <ul>
+ * <li><pre>aether.trustedChecksumsSource.${name}.enabled</pre> (boolean)
must be explicitly set to "true"
+ * to become enabled</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.basedir</pre> (string,
path) directory from where implementation
+ * can use files. May be relative path (the is resolved from local
repository basedir) or absolute. If unset,
+ * default value is ".checksums" and is resolved from local repository
basedir.</li>
+ * <li><pre>aether.trustedChecksumsSource.${name}.originAware</pre>
(boolean) whether to make implementation
+ * "originAware", to factor in origin repository ID as well or not.</li>
+ * </ul>
+ * <p>
+ * This implementation ensures that implementations have "name" property, used
in configuration properties above.
+ *
+ * @since TBD
+ */
+abstract class FileTrustedChecksumsSourceSupport
+ implements TrustedChecksumsSource
+{
+ private static final String CONFIG_PROP_PREFIX =
"aether.trustedChecksumsSource.";
+
+ private static final String CONF_NAME_ENABLED = "enabled";
+
+ private static final String CONF_NAME_BASEDIR = "basedir";
+
+ private static final String CONF_NAME_ORIGIN_AWARE = "originAware";
+
+ /**
+ * Visible for testing.
+ */
+ static final String LOCAL_REPO_PREFIX = ".checksums";
Review Comment:
This should be rather `LOCAL_REPO_PREFIX_DIR` since it stands as a dir name .
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,162 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that
use specified directory as base
+ * directory, where it expects a "summary" file named as
"checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The
format supports comments "#" (hash) and
+ * empty lines (both are ignored).
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will
factor in origin repository ID as well into
+ * file name (for example "checksums-central.sha1").
+ * <p>
+ * The checksums file once loaded are cached in session, so in-flight file
changes during lifecycle of session are NOT
+ * noticed.
+ * <p>
+ * The name of this implementation is "file-compact".
+ *
+ * @see ArtifactIdUtils#toId(Artifact)
+ * @since TBD
+ */
+@Singleton
+@Named( CompactFileTrustedChecksumsSource.NAME )
+public final class CompactFileTrustedChecksumsSource
+ extends FileTrustedChecksumsSourceSupport
+{
+ public static final String NAME = "file-compact";
+
+ private static final String CHECKSUMS_FILE_PREFIX = "checksums";
+
+ private static final String CHECKSUMS_CACHE_KEY =
CompactFileTrustedChecksumsSource.class.getName() + ".checksums";
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(
CompactFileTrustedChecksumsSource.class );
+
+ public CompactFileTrustedChecksumsSource()
+ {
+ super( NAME );
+ }
+
+ @SuppressWarnings( "unchecked" )
+ @Override
+ protected Map<String, String> performLookup( RepositorySystemSession
session,
+ Path basedir,
+ Artifact artifact,
+ ArtifactRepository
artifactRepository,
+
List<ChecksumAlgorithmFactory> checksumAlgorithmFactories )
+ {
+ final String prefix;
+ if ( isOriginAware( session ) )
+ {
+ prefix = CHECKSUMS_FILE_PREFIX + "-" + artifactRepository.getId()
+ ".";
+ }
+ else
+ {
+ prefix = CHECKSUMS_FILE_PREFIX + ".";
+ }
+
+ final ConcurrentHashMap<String, ConcurrentHashMap<String, String>>
basedirProvidedChecksums =
+ (ConcurrentHashMap<String, ConcurrentHashMap<String, String>>)
session.getData()
+ .computeIfAbsent( CHECKSUMS_CACHE_KEY,
ConcurrentHashMap::new );
+
+ final HashMap<String, String> checksums = new HashMap<>();
+ for ( ChecksumAlgorithmFactory checksumAlgorithmFactory :
checksumAlgorithmFactories )
+ {
+ ConcurrentHashMap<String, String> algorithmChecksums =
basedirProvidedChecksums.computeIfAbsent(
+ checksumAlgorithmFactory.getName(),
+ algName -> loadProvidedChecksums(
+ basedir.resolve( prefix +
checksumAlgorithmFactory.getFileExtension() )
+ )
+ );
+ String checksum = algorithmChecksums.get( ArtifactIdUtils.toId(
artifact ) );
+ if ( checksum != null )
+ {
+ checksums.put( checksumAlgorithmFactory.getName(), checksum );
+ }
+ }
+ return checksums;
+ }
+
+ private ConcurrentHashMap<String, String> loadProvidedChecksums( Path
checksumFile )
+ {
+ ConcurrentHashMap<String, String> result = new ConcurrentHashMap<>();
+ try
+ {
+ try ( BufferedReader reader = Files.newBufferedReader(
checksumFile, StandardCharsets.UTF_8 ) )
+ {
+ LOGGER.debug( "Loading provided checksums file '{}'",
checksumFile );
+ String line;
+ while ( ( line = reader.readLine() ) != null )
+ {
+ if ( !line.startsWith( "#" ) && !line.isEmpty() )
+ {
+ String[] parts = line.split( " ", 2 );
+ if ( parts.length == 2 )
+ {
+ String old = result.put( parts[0], parts[1] );
+ if ( old != null )
+ {
+ LOGGER.warn( "Checksums file '{}' contains
duplicate checksums for artifact {}: "
+ + "old '{}' replaced by new '{}'",
checksumFile, parts[0], old, parts[1] );
+ }
+ }
+ else
+ {
+ LOGGER.warn( "Checksums file '{}' ignored
malformed line '{}'", checksumFile, line );
+ }
+ }
+ }
+ }
+ }
+ catch ( NoSuchFileException e )
+ {
+ // ignore, will return empty result
+ LOGGER.debug( "Checksums file '{}' not found", checksumFile );
Review Comment:
This is an antipattern according to Joshua Bloch. Test for existance instead
of reyling on such exceptions.
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/SparseFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,125 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.internal.impl.LocalPathComposer;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.spi.io.FileProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that
use specified directory as base
+ * directory, where it expects artifacts checksums on standard Maven2 "local"
layout. This implementation uses Artifact
+ * coordinates solely to form path from basedir, pretty much as Maven local
repository does.
+ * <p>
+ * The source may be configured to be "origin aware", in that case it will
factor in origin repository ID as well into
+ * base directory name (for example ".checksums/central/...").
+ * <p>
+ * The name of this implementation is "file-sparse".
+ *
+ * @see LocalPathComposer
+ * @since TBD
+ */
+@Singleton
+@Named( SparseFileTrustedChecksumsSource.NAME )
+public final class SparseFileTrustedChecksumsSource
Review Comment:
I know what sparse means here and I think it is used incorrectly. Your
understanding: A spare directory tree where only a limited amount of files
exist for artifacts required. Meaning of sparse file:
https://en.wikipedia.org/wiki/Sparse_file
I think the term is abused.
It should be `SparseDirectoryFileTrustedChecksumsSource` or
`SparseDirectoryTrustedChecksumsSource`.
See impl of a sparse dir tree:
https://svnbook.red-bean.com/en/1.7/svn.advanced.sparsedirs.html
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/checksum/CompactFileTrustedChecksumsSource.java:
##########
@@ -0,0 +1,162 @@
+package org.eclipse.aether.internal.impl.checksum;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import javax.inject.Named;
+import javax.inject.Singleton;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.NoSuchFileException;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.repository.ArtifactRepository;
+import org.eclipse.aether.spi.connector.checksum.ChecksumAlgorithmFactory;
+import org.eclipse.aether.util.artifact.ArtifactIdUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Compact file {@link FileTrustedChecksumsSourceSupport} implementation that
use specified directory as base
+ * directory, where it expects a "summary" file named as
"checksums.${checksumExt}" for each checksum algorithm, and
+ * file format is artifact ID and checksum separated by space per line. The
format supports comments "#" (hash) and
Review Comment:
Stupid question: If you say yourself that it uses a summary file, why just
not call it so? `SummaryFileTrustedChecksumsSource`, DI name `summary-file`
> Allow more compact storage of provided checksums
> ------------------------------------------------
>
> Key: MRESOLVER-269
> URL: https://issues.apache.org/jira/browse/MRESOLVER-269
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
> Reporter: Rafael Winterhalter
> Assignee: Tamás Cservenák
> Priority: Major
> Fix For: resolver-next
>
>
> While the repository layout makes sense for storage outside of a project, it
> would be more convenient to store checksums in a single file (per algorithm)
> when keeping checksums along when storing these checksums within a project.
> This makes the storage easier to version control and avoids the overhead of
> storing a lot of files in version control what often creates some overhead.
> Ideally, Maven could support such files out of the box by shipping a provider
> for such files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
