sasmith-14 opened a new issue, #884:
URL: https://github.com/apache/wayang/issues/884

   **`JavaReservoirSampleOperator` returns the entire input instead of a sample 
whenever the dataset size is unknown.**
   
   ### The bug
   
   `evaluate()` takes a "return all" shortcut at 
[`JavaReservoirSampleOperator.java:84`](https://github.com/apache/wayang/blob/713faa7d8bf4726536f6b0920ebb73053160274c/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/operators/JavaReservoirSampleOperator.java#L84):
   
   ```java
   if (sampleSize >= datasetSize) { // return all
   ```
   
   `datasetSize` defaults to `SampleOperator.UNKNOWN_DATASET_SIZE` (`0`). So 
when the size isn't known, `sampleSize >= 0` is always true and the operator 
emits **every element**. The sibling `JavaRandomSampleOperator` already guards 
this with `isDataSetSizeKnown()` — the reservoir operator doesn't.
   
   ### Reproduce
   
   Sample 5 from 10 elements, without setting the dataset size (the normal 
streaming case):
   
   ```java
   JavaReservoirSampleOperator<Integer> op = new JavaReservoirSampleOperator<>(
           it -> 5, DataSetType.createDefaultUnchecked(Integer.class), it -> 
42L);
   // no setDatasetSize(...)  ->  size is UNKNOWN
   // input: 1..10
   ```
   
   |          | Result                   |
   | -------- | ------------------------ |
   | Expected | 5 elements               |
   | Actual   | 10 elements (all of them) |
   
   The existing `JavaReservoirSampleOperatorTest` always calls 
`setDatasetSize(10)`, so it never hits this path.
   
   ### Environment
   
   `wayang-platforms/wayang-java` · `main` (`713faa7d`) and `1.1.1`.
   
   ### Fix
   
   A one-line guard (`isDataSetSizeKnown() && …`) plus a regression test — the 
streaming reservoir algorithm already handles inputs smaller than the sample 
size. I have it ready and can open a PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to