[
https://issues.apache.org/jira/browse/BEAM-14232?focusedWorklogId=753157&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753157
]
ASF GitHub Bot logged work on BEAM-14232:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Apr/22 00:30
Start Date: 06/Apr/22 00:30
Worklog Time Spent: 10m
Work Description: robertwb commented on code in PR #17249:
URL: https://github.com/apache/beam/pull/17249#discussion_r843366367
##########
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/External.java:
##########
@@ -263,9 +268,11 @@ public OutputT expand(InputT input) {
return toOutputCollection(outputMapBuilder.build());
}
- private RunnerApi.Components resolveArtifacts(RunnerApi.Components
components) {
- if (components.getEnvironmentsMap().values().stream()
- .allMatch(env -> env.getDependenciesCount() == 0)) {
+ private RunnerApi.Components resolveArtifacts(
+ RunnerApi.Components components, Set<String> originalEnvIds) {
Review Comment:
It would be a cleaner API to pass the set of environments that should be
expanded than pass in the set that should not be. (Even better might be to
simply take a Map<string, Environment> rather than the whole surrounding
Components object.)
##########
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/External.java:
##########
@@ -71,6 +72,7 @@
@Experimental(Kind.PORTABILITY)
@SuppressWarnings({
"rawtypes", // TODO(https://issues.apache.org/jira/browse/BEAM-10556)
+ "keyfor",
Review Comment:
Can we scope this more locally?
Issue Time Tracking
-------------------
Worklog Id: (was: 753157)
Time Spent: 1h (was: 50m)
> Only resolve artifacts in expanded environments for Java External transform
> ---------------------------------------------------------------------------
>
> Key: BEAM-14232
> URL: https://issues.apache.org/jira/browse/BEAM-14232
> Project: Beam
> Issue Type: Bug
> Components: cross-language
> Reporter: Heejong Lee
> Assignee: Heejong Lee
> Priority: P2
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Only resolve artifacts in expanded environments for Java External transform.
>
> We can't assume that any expansion service resolves any artifact information.
> For example, Java artifacts returned from Java expansion service cannot be
> resolved (+downloaded) with Python expansion service. Also, one Python
> expansion service may returns Python artifacts which are unknown to other
> Python expansion service. We need to skip pre-existing artifacts and only
> resolves artifacts in new environments from the expansion service at the
> moment.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
