junzhong qin created FLINK-32809:
------------------------------------
Summary:
YarnClusterDescriptor.isarchiveonlyincludedinShipArchiveFiles dose not work as
expected
Key: FLINK-32809
URL: https://issues.apache.org/jira/browse/FLINK-32809
Project: Flink
Issue Type: Bug
Components: Deployment / YARN
Affects Versions: 1.17.1, 1.16.2, 1.18.0
Reporter: junzhong qin
YarnClusterDescriptor.isarchiveonlyincludedinShipArchiveFiles(List<File>
shipFiles) check wether the shipFiles are all archive files, but it dose not
work as expected.
{code:java}
public static boolean isArchiveOnlyIncludedInShipArchiveFiles(List<File>
shipFiles) {
return shipFiles.stream()
.filter(File::isFile)
.map(File::getName)
.map(String::toLowerCase)
.allMatch(
name ->
name.endsWith(".tar.gz")
|| name.endsWith(".tar")
|| name.endsWith(".tgz")
|| name.endsWith(".dst")
|| name.endsWith(".jar")
|| name.endsWith(".zip"));
} {code}
When we pass a directory and an archive file it should return false but it
returns true.
{code:java}
// dir1 is a directory and archive.zip is an archive file
List<File> files = Arrays.asList(new File("/tmp/dir1"), new
File("/tmp/archive.zip"));
boolean result = isArchiveOnlyIncludedInShipArchiveFiles(files);
System.out.println(result); // Print true but is should print false{code}
If flink user want to add directory as ship file, they should use
configuration: yarn.ship-files .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
