sasmith-14 opened a new pull request, #885:
URL: https://github.com/apache/wayang/pull/885
Fixes #884
### What changes were proposed in this pull request?
`JavaReservoirSampleOperator` returned the **entire input** instead of a
sample whenever the dataset size was unknown. The "return all" shortcut
```java
if (sampleSize >= datasetSize) { //return all
```
is always taken when `datasetSize` is `UNKNOWN_DATASET_SIZE` (`0`, the
default), so `sampleSize >= 0` always holds.
This guards the shortcut with `isDataSetSizeKnown()` — the same check
`JavaRandomSampleOperator` already uses:
```java
if (this.isDataSetSizeKnown() && sampleSize >= datasetSize) { //return all
```
When the size is unknown the operator now runs the reservoir algorithm,
which already handles inputs smaller than the sample size.
### How was this patch tested?
Added `testExecutionWithUnknownDatasetSize` (no `setDatasetSize` call): it
fails on `main` (`expected: <5> but was: <10>`) and passes with this change.
Full `wayang-java` suite: 56 run, 0 failures, 0 errors, 5 skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]