ramitg254 commented on code in PR #6946:
URL: https://github.com/apache/hive/pull/6946#discussion_r4560997480


##########
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFEvaluatorNTile.java:
##########
@@ -44,34 +44,41 @@
  * evaluator keeps O(1) state regardless of partition size, and a batch that 
falls entirely within
  * one bucket is emitted as a repeating vector.
  *
- * <p>The number of buckets must be a positive integer constant (e.g. {@code 
ntile(4)}); the
- * Vectorizer leaves any other form to row mode.
+ * <p>The number of buckets is either a positive integer constant (e.g. {@code 
ntile(4)}) or any
+ * other int expression (e.g. {@code ntile(col)}), which, as in {@code 
GenericUDAFNTile}, is read
+ * from the first row of each partition. The Vectorizer leaves a non-positive 
constant to row mode.
  */
 public class VectorPTFEvaluatorNTile extends VectorPTFEvaluatorBase {
 
-  private final long numBuckets;
+  // Number of buckets of a constant argument, or 0 when it is read from the 
partition's first row.
+  private final long constantNumBuckets;
 
-  // Per-partition bucket layout, set up by addStreamingGroupResults.
+  // Per-partition bucket layout, set up by setUpBuckets once the first batch 
is replayed.
   private int bucketSize;
   private int largerBucketCount;
 
-  // Replay position within the partition.
+  // Replay position within the partition; 0 until the bucket layout is set up.
   private int currentBucket;
   private int rowsLeftInBucket;
 
   public VectorPTFEvaluatorNTile(WindowFrameDef windowFrameDef,
       VectorExpression numBucketsExpression, int outputColumnNum) {
-    super(windowFrameDef, outputColumnNum);
-    if (!(numBucketsExpression instanceof ConstantVectorExpression 
constantExpr)) {
-      throw new RuntimeException(
-          "NTILE expects a constant integer expression for the number of 
buckets, got: "
-              + (numBucketsExpression == null ? "null"
-                  : numBucketsExpression.getClass().getName()));
+    // A constant needs no input column; any other expression is evaluated 
before batches are
+    // buffered, and its result column is read back when the partition's first 
batch is replayed.
+    super(windowFrameDef,
+        numBucketsExpression instanceof ConstantVectorExpression ? null : 
numBucketsExpression,
+        outputColumnNum);
+    if (numBucketsExpression == null) {
+      throw new RuntimeException("NTILE expects an expression for the number 
of buckets");

Review Comment:
   can we throw exception more earlier before calling super?



##########
ql/src/test/queries/clientnegative/vector_ptf_ntile_invalid_buckets.q:
##########
@@ -0,0 +1,31 @@
+set hive.vectorized.execution.enabled=true;

Review Comment:
   these properties are true by defulat, we can avoid setting it explicitly



##########
ql/src/test/queries/clientnegative/vector_ptf_ntile_invalid_buckets.q:
##########
@@ -0,0 +1,31 @@
+set hive.vectorized.execution.enabled=true;
+set hive.vectorized.execution.ptf.enabled=true;
+

Review Comment:
   can we add explain vectorization detail queries in this q file for 
validation?



##########
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFEvaluatorNTile.java:
##########
@@ -86,18 +93,56 @@ public void addStreamingGroupResults(List<Integer> 
groupRowCounts) throws HiveEx
     if (partitionSize <= 0) {
       throw new HiveException("Partition size must be set before computing 
ntile");
     }
-    // Peer groups are irrelevant: buckets depend only on row position and 
partition size.
-    // With more buckets than rows every row gets its own bucket, which is 
ntile(partitionSize);
-    // capping the bucket count keeps every bucket non-empty (and fits it in 
an int).
-    final int buckets = (int) Math.min(numBuckets, partitionSize);
+    // Peer groups are irrelevant: buckets depend only on row position and 
partition size. The
+    // layout is set up on the first replayed batch, which holds the row a 
non-constant number of
+    // buckets is read from.
+    currentBucket = 0;
+  }
+
+  /**
+   * Mirrors {@code GenericUDAFNTile}, which does not validate the number of 
buckets: wherever row
+   * mode produces a result this produces the same one, and wherever row mode 
fails this fails fast.
+   */
+  private void setUpBuckets(long numBuckets) throws HiveException {
+    final int buckets;
+    if (numBuckets > 0) {
+      // With more buckets than rows every row gets its own bucket, which is 
ntile(partitionSize);
+      // capping the bucket count keeps every bucket non-empty (and fits it in 
an int).
+      buckets = (int) Math.min(numBuckets, partitionSize);
+    } else if (numBuckets < -partitionSize) {
+      // Row mode's truncating division yields an empty base bucket size, so 
every row gets its own
+      // bucket, as for a positive count larger than the partition.
+      buckets = partitionSize;
+    } else {
+      // Row mode divides by zero, or for any other negative count loops until 
it runs out of
+      // memory.
+      throw new HiveException("NTILE requires a positive number of buckets, 
got: " + numBuckets);

Review Comment:
   this error message seems more or less redundant to the one in 
`readNumBuckets` and `VectorPTFEvaluatorNTile constructor `, can it be moved to 
some kind of single reference for consistent updation later on or some kind of 
earlier throw in the code flow introduced rather than checking everywhere?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to