Vladimir Sitnikov created CALCITE-4480: ------------------------------------------
Summary: Make EnumerableDefaults#union a non-blocking operation Key: CALCITE-4480 URL: https://issues.apache.org/jira/browse/CALCITE-4480 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Vladimir Sitnikov Currently, EnumerableDefaults#union buffers all the rows before it returns the first of them Pros: 1) Faster iteration in case enumerable is queried multiple times Cons: 1) The implementation does not work with infinite streams 2) Keeps memory even after iteration is finished --- An alternative might be something like {code:java} public static <TSource> Enumerable<TSource> union(Enumerable<TSource> source0, Enumerable<TSource> source1) { Enumerable<TSource> unionAll = concat(source0, source1); return new AbstractEnumerable<TSource>() { @Override public Enumerator<TSource> enumerator() { Set<TSource> set = new HashSet<>(); return EnumerableDefaults.where(unionAll, set::add).enumerator(); } }; } {code} Pros: 1) Supports infinite streams 2) In theory, it could reset hashSet after iteration finishes Cons: 1) Slower iteration in case enumerable is queried multiple times (hashSet is rebuilt every time) 2) concat+abstractenumerable might const CPU cycles -- This message was sent by Atlassian Jira (v8.3.4#803005)