[ https://issues.apache.org/jira/browse/BEAM-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neville Li reassigned BEAM-3247: -------------------------------- Assignee: Neville Li (was: Kenneth Knowles) > Sample.any memory constraint > ---------------------------- > > Key: BEAM-3247 > URL: https://issues.apache.org/jira/browse/BEAM-3247 > Project: Beam > Issue Type: Improvement > Components: sdk-java-core > Affects Versions: 2.1.0 > Reporter: Neville Li > Assignee: Neville Li > Priority: Minor > > Right now {{Sample.any}} converts the collection to an iterable view and take > first n in a side input. This may require materializing the entire collection > to disk and is potentially inefficient. > https://github.com/apache/beam/blob/v2.1.0/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java#L74 > It can be fixed by applying a truncating `DoFn` first, then a combine into > `List<T>` which limits the list size, and finally flattening the list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)