[ https://issues.apache.org/jira/browse/BEAM-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neville Li resolved BEAM-3247. ------------------------------ Resolution: Fixed Fix Version/s: 2.2.0 > Sample.any memory constraint > ---------------------------- > > Key: BEAM-3247 > URL: https://issues.apache.org/jira/browse/BEAM-3247 > Project: Beam > Issue Type: Improvement > Components: sdk-java-core > Affects Versions: 2.1.0 > Reporter: Neville Li > Assignee: Neville Li > Priority: Minor > Fix For: 2.2.0 > > > Right now {{Sample.any}} converts the collection to an iterable view and take > first n in a side input. This may require materializing the entire collection > to disk and is potentially inefficient. > https://github.com/apache/beam/blob/v2.1.0/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java#L74 > It can be fixed by applying a truncating `DoFn` first, then a combine into > `List<T>` which limits the list size, and finally flattening the list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)