This is an automated email from the ASF dual-hosted git repository.

zhztheplayer pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git


The following commit(s) were added to refs/heads/main by this push:
     new d893a86435 [GLUTEN-10215][VL] Delta write: Native top-level NOT NULL 
checks (#12647)
d893a86435 is described below

commit d893a86435039d6be3e5382d0f006bdfe3231e79
Author: Mohammad Linjawi <[email protected]>
AuthorDate: Wed Nov 11 20:59:58 2026 +0300

    [GLUTEN-10215][VL] Delta write: Native top-level NOT NULL checks (#12647)
    
    * [GLUTEN-10215][VL] Delta write: Offload top-level NOT NULL checks
    
    * [GLUTEN-10215][VL] Address review of native NOT NULL checks
    
    - Find nulls by decoding each constrained column and counting its nulls
      a word at a time, instead of trusting the cached null count: it can
      be stale, and on a dictionary or constant wrapping it misses the nulls
      of the wrapped values. The scan moves into
      VeloxColumnarBatch::firstNullColumnIndex, with unit tests.
    - Fail the write if a native invariant checker reaches the WriteFilesExec
      branch of GlutenDeltaFileFormatWriter, which cannot apply it.
    - Keep the checker's column ordinals and constraints in parallel arrays.
    - Move GlutenDeltaInvariantChecker into the shared src-delta tree, so
      that every Delta version enforces the constraints the same way.
    - Undo the reindentation of the spotless:off src-delta4 writer.
    - Test the checker without a native library, and NOT NULL violations in
      data read from Parquet and with optimized writes.
    
    * [GLUTEN-10215][VL] Test partitioned NOT NULL violations and update the 
Delta docs
    
    A partitioned native write sorts the rows natively before writing them.
    The native NOT NULL check runs after that sort, on the JVM, so violations
    keep Delta's DeltaInvariantViolationException instead of being wrapped in
    a GlutenException. Test it on Spark 3.5 and 4.x, including an empty string
    partition value that Delta turns into a null, and update VeloxDelta.md,
    which still described the row-based check for every NOT NULL write.
    
    * [GLUTEN-10215][VL] Choose the native NOT NULL checker in one shared place
    
    Both Delta transactions picked the checker with the same lines. Move them
    into GlutenDeltaInvariantChecker.forWrite, which also keeps Delta's row
    checker when 
spark.databricks.delta.constraints.convertEmptyToNullForStringPartitionCol
    is off and the table has a string partition column. Delta then checks the
    empty strings themselves, while the native checker would see the nulls the
    writer turns them into, and would fail writes that Delta accepts.
    
    * [GLUTEN-10215][VL] Trim the native NOT NULL tests
    
    Move the NOT NULL tests into the existing DeltaNativeWriteSuite of each
    Delta tree, which drops DeltaNativeWriteInvariantSuite and the helpers
    it copied. Each tree keeps three tests:
    - a native write with neither Delta's row checker nor C2R, and its
      violation, with and without optimized writes;
    - CHECK constraints keep DeltaInvariantCheckerExec;
    - violations on a partitioned table.
    
    Drop the Parquet-source test. VeloxColumnarBatchTest covers dictionary
    encodings directly, and the end-to-end test could not ensure them.
    Shorten the checker's doc comment, which repeated the PR description.
    
    * [GLUTEN-10215][VL] Check NOT NULL in a columnar DeltaInvariantCheckerExec
    
    Plan GlutenDeltaInvariantCheckerExec where Delta plans its row-based
    DeltaInvariantCheckerExec, instead of wrapping the Velox writer's task
    iterator. The writer no longer takes a checker.
    
    When native code pulls the node's batches (the writer's sort on
    partitioned writes, the optimized-write shuffle), it keeps only the
    violation's text. The transaction then finds that text in the failure
    and rethrows Delta's DeltaInvariantViolationException, until #13561
    keeps Java exceptions across native code.
    
    Co-Authored-By: Claude Opus 5.5 <[email protected]>
    
    * [GLUTEN-10215][VL] Create the native NOT NULL checker in 
glutenPhysicalPlan
    
    glutenPhysicalPlan already gets the constraints, so it now creates the
    checker itself and its arguments are back to what they are on main.
    
    The unwrap moves into a catch around the native write. It takes the
    checker from the plan's GlutenDeltaInvariantCheckerExec and leaves
    failures that Delta's InnerInvariantViolationException already handles
    to Delta's catch.
    
    Co-Authored-By: Claude Opus 5.5 <[email protected]>
    
    * [GLUTEN-10215][VL] Make findViolation static
    
    findViolation no longer needs the checker's constraints. It finds the
    text of a NOT NULL violation in the failure, takes the column name from
    it and rebuilds Delta's exception with the constructor Delta uses. Both
    transactions call it directly, so the transaction no longer looks up the
    checker in the plan.
    
    NOT NULL violations from Delta's row checker whose text reaches the
    driver only inside a GlutenException now keep Delta's exception type
    too. CHECK violations still do not.
    
    Co-Authored-By: Claude Opus 5.5 <[email protected]>
    
    ---------
    
    Co-authored-by: Mohammad Linjawi <[email protected]>
    Co-authored-by: Claude Opus 5.5 <[email protected]>
---
 .../constraints/GlutenDeltaInvariantChecker.scala  | 140 +++++++++++++++++++++
 .../GlutenDeltaInvariantCheckerExec.scala          |  49 ++++++++
 .../GlutenDeltaInvariantCheckerSuite.scala         |  71 +++++++++++
 .../sql/delta/GlutenOptimisticTransaction.scala    |  17 ++-
 .../delta/files/GlutenDeltaFileFormatWriter.scala  |   6 +
 .../spark/sql/delta/DeltaNativeWriteSuite.scala    | 101 ++++++++++++++-
 .../sql/delta/GlutenOptimisticTransaction.scala    |  51 +++++---
 .../delta/files/GlutenDeltaFileFormatWriter.scala  |   6 +
 .../spark/sql/delta/DeltaNativeWriteSuite.scala    |  99 +++++++++++++++
 .../VeloxColumnarBatchJniWrapper.java              |   2 +
 .../gluten/columnarbatch/VeloxColumnarBatches.java |  13 ++
 cpp/velox/jni/VeloxJniWrapper.cc                   |  22 ++++
 cpp/velox/memory/VeloxColumnarBatch.cc             |  36 ++++++
 cpp/velox/memory/VeloxColumnarBatch.h              |   4 +
 cpp/velox/tests/VeloxColumnarBatchTest.cc          |  46 +++++++
 docs/backends/velox/get-started/VeloxDelta.md      |   6 +-
 16 files changed, 643 insertions(+), 26 deletions(-)

diff --git 
a/backends-velox/src-delta/main/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantChecker.scala
 
b/backends-velox/src-delta/main/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantChecker.scala
new file mode 100644
index 0000000000..25152eb589
--- /dev/null
+++ 
b/backends-velox/src-delta/main/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantChecker.scala
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.delta.constraints
+
+import org.apache.gluten.columnarbatch.VeloxColumnarBatches
+
+import org.apache.spark.sql.catalyst.expressions.Attribute
+import org.apache.spark.sql.delta.constraints.Constraints.NotNull
+import org.apache.spark.sql.delta.schema.{DeltaInvariantViolationException, 
SchemaUtils}
+import org.apache.spark.sql.vectorized.ColumnarBatch
+
+import org.apache.commons.lang3.exception.ExceptionUtils
+
+import scala.collection.JavaConverters._
+
+/**
+ * Checks top-level NOT NULL constraints on Velox batches, for 
[[GlutenDeltaInvariantCheckerExec]].
+ * Nulls are found natively, but a violation is thrown on the JVM as Delta's 
own
+ * [[DeltaInvariantViolationException]]: an exception raised inside Velox 
would reach Spark only as
+ * a GlutenException.
+ *
+ * @param columnOrdinals
+ *   the ordinal in the checked plan's output of the column each constraint 
checks
+ * @param constraints
+ *   the constraints, parallel to `columnOrdinals`
+ */
+private[delta] class GlutenDeltaInvariantChecker private (
+    columnOrdinals: Array[Int],
+    constraints: Array[NotNull])
+  extends Serializable {
+
+  def check(batch: ColumnarBatch): Unit = {
+    val failedConstraintIndex = 
VeloxColumnarBatches.firstNullColumnIndex(batch, columnOrdinals)
+    if (failedConstraintIndex >= 0) {
+      throw 
DeltaInvariantViolationException(constraints(failedConstraintIndex))
+    }
+  }
+
+  override def toString: String = constraints.map(_.column.head).mkString("NOT 
NULL [", ", ", "]")
+}
+
+private[delta] object GlutenDeltaInvariantChecker {
+
+  /**
+   * Returns a checker for `constraints` if all of them are top-level NOT NULL 
constraints on
+   * columns of `output`, or None if any of them needs Delta's row-based 
DeltaInvariantCheckerExec.
+   */
+  def create(
+      output: Seq[Attribute],
+      constraints: Seq[Constraint]): Option[GlutenDeltaInvariantChecker] = {
+    if (constraints.isEmpty) {
+      return None
+    }
+
+    val topLevelNotNullConstraints = constraints.collect {
+      case constraint: NotNull if constraint.column.length == 1 => constraint
+    }
+    if (topLevelNotNullConstraints.size != constraints.size) {
+      return None
+    }
+
+    val columnOrdinals = topLevelNotNullConstraints.map {
+      constraint =>
+        output.indexWhere {
+          attribute => SchemaUtils.DELTA_COL_RESOLVER(attribute.name, 
constraint.column.head)
+        }
+    }
+    if (columnOrdinals.contains(-1)) {
+      return None
+    }
+    Some(new GlutenDeltaInvariantChecker(
+      columnOrdinals.toArray,
+      topLevelNotNullConstraints.toArray))
+  }
+
+  /**
+   * Returns the NOT NULL violation whose text is in the messages of 
`failure`, the failure of a
+   * write, if there is one.
+   *
+   * Native code that pulls the checked batches, such as the Velox writer's 
sort on a partitioned
+   * write, keeps only the text of a Java exception. The write then fails with 
a GlutenException
+   * whose message holds the violation's text. The returned violation is 
rebuilt from the column
+   * name in that text and carries `failure` as a suppressed exception.
+   *
+   * TODO: Remove once native code keeps Java exceptions
+   * (https://github.com/apache/gluten/pull/13561).
+   */
+  def findViolation(failure: Throwable): 
Option[DeltaInvariantViolationException] = {
+    ExceptionUtils
+      .getThrowableList(failure)
+      .asScala
+      .iterator
+      .flatMap(throwable => Option(throwable.getMessage))
+      .map(violatedColumn)
+      .collectFirst { case Some(column) => column }
+      .map {
+        column =>
+          val violation = notNullViolation(column)
+          violation.addSuppressed(failure)
+          violation
+      }
+  }
+
+  private def notNullViolation(column: String): 
DeltaInvariantViolationException =
+    new DeltaInvariantViolationException("DELTA_NOT_NULL_CONSTRAINT_VIOLATED", 
Array(column))
+
+  // The text of a NOT NULL violation before and after its column name.
+  private lazy val notNullText: Option[(String, String)] = {
+    val column = "glutenColumnName"
+    val text = notNullViolation(column).toString
+    val columnStart = text.indexOf(column)
+    if (columnStart < 0) {
+      None
+    } else {
+      Some((text.substring(0, columnStart), text.substring(columnStart + 
column.length)))
+    }
+  }
+
+  /** The column name in the text of a NOT NULL violation in `message`, if 
there is one. */
+  private def violatedColumn(message: String): Option[String] = 
notNullText.flatMap {
+    case (before, after) =>
+      val start = message.indexOf(before)
+      val columnEnd = if (start < 0) -1 else message.indexOf(after, start + 
before.length)
+      if (columnEnd < 0) None else Some(message.substring(start + 
before.length, columnEnd))
+  }
+}
diff --git 
a/backends-velox/src-delta/main/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantCheckerExec.scala
 
b/backends-velox/src-delta/main/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantCheckerExec.scala
new file mode 100644
index 0000000000..10df6402e8
--- /dev/null
+++ 
b/backends-velox/src-delta/main/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantCheckerExec.scala
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.delta.constraints
+
+import org.apache.gluten.backendsapi.velox.VeloxBatchType
+import org.apache.gluten.execution.ColumnarToColumnarExec
+import org.apache.gluten.extension.columnar.transition.Convention
+
+import org.apache.spark.sql.execution.SparkPlan
+import org.apache.spark.sql.vectorized.ColumnarBatch
+
+/**
+ * Gluten's columnar version of Delta's DeltaInvariantCheckerExec, for writes 
whose constraints are
+ * all top-level NOT NULL constraints. It checks the Velox batches in place, 
so the write plan needs
+ * no conversion to rows and back around it.
+ */
+case class GlutenDeltaInvariantCheckerExec(
+    override val child: SparkPlan,
+    checker: GlutenDeltaInvariantChecker)
+  extends ColumnarToColumnarExec(child) {
+
+  override protected def mapIterator(in: Iterator[ColumnarBatch]): 
Iterator[ColumnarBatch] =
+    in.map {
+      batch =>
+        checker.check(batch)
+        batch
+    }
+
+  override def batchType(): Convention.BatchType = VeloxBatchType
+
+  override def rowType(): Convention.RowType = Convention.RowType.None
+
+  override protected def withNewChildInternal(newChild: SparkPlan): SparkPlan =
+    copy(child = newChild)
+}
diff --git 
a/backends-velox/src-delta/test/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantCheckerSuite.scala
 
b/backends-velox/src-delta/test/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantCheckerSuite.scala
new file mode 100644
index 0000000000..bd0ab7cb3b
--- /dev/null
+++ 
b/backends-velox/src-delta/test/scala/org/apache/spark/sql/delta/constraints/GlutenDeltaInvariantCheckerSuite.scala
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.delta.constraints
+
+import org.apache.gluten.exception.GlutenException
+
+import org.apache.spark.{SparkException, SparkFunSuite}
+import org.apache.spark.sql.catalyst.expressions.{AttributeReference, Literal}
+import org.apache.spark.sql.delta.constraints.Constraints.{Check, NotNull}
+import org.apache.spark.sql.delta.schema.DeltaInvariantViolationException
+import org.apache.spark.sql.types.LongType
+
+import org.apache.commons.lang3.exception.ExceptionUtils
+
+/** Checks of GlutenDeltaInvariantChecker that need no native library. */
+class GlutenDeltaInvariantCheckerSuite extends SparkFunSuite {
+
+  private val output = Seq("a", "b", "c").map(AttributeReference(_, 
LongType)())
+
+  test("only top-level NOT NULL constraints on output columns are checked 
natively") {
+    assert(GlutenDeltaInvariantChecker.create(output, Nil).isEmpty)
+    assert(GlutenDeltaInvariantChecker.create(output, 
Seq(NotNull(Seq("a")))).isDefined)
+    // Delta resolves column names case-insensitively.
+    assert(GlutenDeltaInvariantChecker.create(output, 
Seq(NotNull(Seq("B")))).isDefined)
+
+    // Any other constraint keeps Delta's DeltaInvariantCheckerExec for the 
whole write.
+    val check = Check("positive", Literal(true))
+    assert(GlutenDeltaInvariantChecker.create(output, Seq(NotNull(Seq("a")), 
check)).isEmpty)
+    assert(GlutenDeltaInvariantChecker.create(output, Seq(NotNull(Seq("a", 
"x")))).isEmpty)
+    assert(GlutenDeltaInvariantChecker.create(output, 
Seq(NotNull(Seq("missing")))).isEmpty)
+  }
+
+  test("finds a NOT NULL violation that native code turned into text") {
+    // The message quotes this column name as `a.b`.
+    val violation = DeltaInvariantViolationException(NotNull(Seq("a.b")))
+    // Native code keeps the text cpp/core/jni/JniCommon.h builds from the 
Java exception.
+    val taskFailure = new SparkException(
+      "[TASK_WRITE_FAILED] Task failed while writing rows",
+      new GlutenException(
+        "Operator::getOutput failed: Error during calling Java code from 
native code: " +
+          ExceptionUtils.getStackTrace(violation))
+    )
+    // The driver gets the task's exception as the cause of the job's failure, 
and its stack
+    // trace in the message, which is all it has when it cannot deserialize 
the exception.
+    val withCause = new SparkException("Job aborted due to stage failure", 
taskFailure)
+    val textOnly = new SparkException(
+      "Job aborted due to stage failure: " + 
ExceptionUtils.getStackTrace(taskFailure))
+
+    for (failure <- Seq(withCause, textOnly)) {
+      val found = GlutenDeltaInvariantChecker.findViolation(failure).get
+      assert(found.getMessage == violation.getMessage)
+      assert(found.getSuppressed.toSeq == Seq(failure))
+    }
+    val otherFailure = new SparkException("Job aborted", new 
GlutenException("Out of memory"))
+    assert(GlutenDeltaInvariantChecker.findViolation(otherFailure).isEmpty)
+  }
+}
diff --git 
a/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
 
b/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
index 8af546935c..8e53d53b2c 100644
--- 
a/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
+++ 
b/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
@@ -21,7 +21,7 @@ import 
org.apache.gluten.extension.columnar.transition.Transitions
 
 import org.apache.spark.sql.{AnalysisException, Dataset}
 import org.apache.spark.sql.delta.actions.{AddFile, FileAction}
-import org.apache.spark.sql.delta.constraints.{Constraint, Constraints, 
DeltaInvariantCheckerExec}
+import org.apache.spark.sql.delta.constraints.{Constraint, Constraints, 
DeltaInvariantCheckerExec, GlutenDeltaInvariantChecker, 
GlutenDeltaInvariantCheckerExec}
 import org.apache.spark.sql.delta.files.{GlutenDeltaFileFormatWriter, 
TransactionalWrite}
 import org.apache.spark.sql.delta.hooks.AutoCompact
 import org.apache.spark.sql.delta.perf.{DeltaOptimizedWriterExec, 
GlutenDeltaOptimizedWriterExec}
@@ -35,6 +35,7 @@ import org.apache.spark.sql.util.ScalaExtensions.OptionExt
 import org.apache.spark.util.SerializableConfiguration
 
 import scala.collection.mutable.ListBuffer
+import scala.util.control.NonFatal
 
 class GlutenOptimisticTransaction(delegate: OptimisticTransaction)
   extends OptimisticTransaction(
@@ -103,7 +104,14 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
         case aqe: AdaptiveSparkPlanExec =>
           assert(!aqe.isFinalPlan)
           aqe.copy(supportsColumnar = true)
-        case _ => Transitions.toBatchPlan(maybeCheckInvariants, VeloxBatchType)
+        case _ => Transitions.toBatchPlan(plan, VeloxBatchType)
+      }
+      // When every constraint is a top-level NOT NULL, check them on the 
Velox batches.
+      val nativeInvariantChecker =
+        GlutenDeltaInvariantChecker.create(empty2NullPlan.output, constraints)
+      val veloxPlan = nativeInvariantChecker match {
+        case Some(checker) => 
GlutenDeltaInvariantCheckerExec(toVeloxPlan(empty2NullPlan), checker)
+        case None => toVeloxPlan(maybeCheckInvariants)
       }
       // No need to plan optimized write if the write command is OPTIMIZE, 
which aims to produce
       // evenly-balanced data files already.
@@ -115,7 +123,6 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
           // We uniformly convert the query plan to a columnar plan. If
           // the further write operation turns out to be non-offload-able, the
           // columnar plan will be converted back to a row-based plan.
-          val veloxPlan = toVeloxPlan(maybeCheckInvariants)
           try {
             val glutenWriterExec =
               GlutenDeltaOptimizedWriterExec(veloxPlan, 
metadata.partitionColumns, deltaLog)
@@ -136,7 +143,6 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
               DeltaOptimizedWriterExec(maybeCheckInvariants, 
metadata.partitionColumns, deltaLog)
           }
         } else {
-          val veloxPlan = toVeloxPlan(maybeCheckInvariants)
           veloxPlan
         }
 
@@ -191,6 +197,9 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
         case InnerInvariantViolationException(violationException) =>
           // Pull an InvariantViolationException up to the top level if it was 
the root cause.
           throw violationException
+        // Native code that pulls the checked batches keeps only a violation's 
text.
+        case NonFatal(e) =>
+          throw GlutenDeltaInvariantChecker.findViolation(e).getOrElse(e)
       }
       statsTrackers.foreach {
         case tracker: BasicWriteJobStatsTracker =>
diff --git 
a/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
 
b/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
index 2bcd7de021..1fcaa01a09 100644
--- 
a/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
+++ 
b/backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
@@ -73,6 +73,12 @@ object GlutenDeltaFileFormatWriter extends LoggingShims {
   /** A variable used in tests to check the final executed plan. */
   private var executedPlan: Option[SparkPlan] = None
 
+  private[delta] def getExecutedPlanForTesting: Option[SparkPlan] = 
executedPlan
+
+  private[delta] def clearExecutedPlanForTesting(): Unit = {
+    executedPlan = None
+  }
+
   // scalastyle:off argcount
   /**
    * Basic work flow of this command is:
diff --git 
a/backends-velox/src-delta33/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
 
b/backends-velox/src-delta33/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
index 2620571186..41260daad9 100644
--- 
a/backends-velox/src-delta33/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
+++ 
b/backends-velox/src-delta33/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
@@ -19,10 +19,13 @@ package org.apache.spark.sql.delta
 import org.apache.gluten.config.GlutenConfig
 import org.apache.gluten.config.VeloxDeltaConfig
 
-import org.apache.spark.sql.DataFrame
+import org.apache.spark.sql.{DataFrame, Row}
 import org.apache.spark.sql.catalyst.TableIdentifier
 import org.apache.spark.sql.delta.actions.AddFile
 import org.apache.spark.sql.delta.commands.optimize.OptimizeMetrics
+import org.apache.spark.sql.delta.constraints.{DeltaInvariantCheckerExec, 
GlutenDeltaInvariantCheckerExec}
+import org.apache.spark.sql.delta.files.GlutenDeltaFileFormatWriter
+import org.apache.spark.sql.delta.schema.InvariantViolationException
 import org.apache.spark.sql.delta.sources.DeltaSQLConf
 import org.apache.spark.sql.delta.test.DeltaSQLCommandTest
 import org.apache.spark.sql.execution.SparkPlan
@@ -284,4 +287,100 @@ class DeltaNativeWriteSuite extends DeltaSQLCommandTest {
       }
     }
   }
+
+  private def appendRows(path: String, rows: (java.lang.Long, String)*): Unit 
= {
+    rows.toDF("id", "s").write.format("delta").mode("append").save(path)
+  }
+
+  /** The plan of the last write by the Velox writer. */
+  private def writerPlan: SparkPlan = 
GlutenDeltaFileFormatWriter.getExecutedPlanForTesting
+    .getOrElse(fail("Expected a write by the Velox writer"))
+
+  test("native delta write checks top-level NOT NULL constraints natively") {
+    for (optimizeWrite <- Seq(false, true)) {
+      withNativeWriteOffloadConf {
+        withSQLConf(DeltaSQLConf.DELTA_OPTIMIZE_WRITE_ENABLED.key -> 
optimizeWrite.toString) {
+          withTempDir {
+            dir =>
+              val path = dir.getCanonicalPath
+              sql(s"CREATE TABLE delta.`$path` (id BIGINT NOT NULL, s STRING) 
USING delta")
+              GlutenDeltaFileFormatWriter.clearExecutedPlanForTesting()
+              appendRows(path, (1L, "a"))
+              assert(
+                
writerPlan.find(_.isInstanceOf[GlutenDeltaInvariantCheckerExec]).isDefined,
+                writerPlan)
+              
assert(writerPlan.find(_.isInstanceOf[DeltaInvariantCheckerExec]).isEmpty, 
writerPlan)
+              assert(!writerPlan.treeString.contains("ColumnarToRow"), 
writerPlan)
+
+              val e = intercept[InvariantViolationException] {
+                appendRows(path, (2L, "b"), (null, "c"))
+              }
+              assert(e.getMessage.contains("NOT NULL constraint violated for 
column: id"))
+              // An optimized write of several input partitions shuffles the 
checked batches
+              // natively, which keeps only the text of the violation.
+              val shuffled = intercept[InvariantViolationException] {
+                spark
+                  .range(0, 1000, 1, 4)
+                  .repartition(4)
+                  .selectExpr("IF(id = 777, CAST(NULL AS BIGINT), id) AS id", 
"'x' AS s")
+                  .write
+                  .format("delta")
+                  .mode("append")
+                  .save(path)
+              }
+              assert(shuffled.getMessage.contains("NOT NULL constraint 
violated for column: id"))
+              assert(spark.read.format("delta").load(path).collect().toSeq == 
Seq(Row(1L, "a")))
+          }
+        }
+      }
+    }
+  }
+
+  test("native delta write keeps DeltaInvariantCheckerExec for CHECK 
constraints") {
+    withNativeWriteOffloadConf {
+      withTempDir {
+        dir =>
+          val path = dir.getCanonicalPath
+          sql(
+            s"CREATE TABLE delta.`$path` (id BIGINT NOT NULL, s STRING) USING 
delta " +
+              "TBLPROPERTIES ('delta.constraints.positive' = 'id > 0')")
+          GlutenDeltaFileFormatWriter.clearExecutedPlanForTesting()
+          appendRows(path, (1L, "a"))
+          
assert(writerPlan.find(_.isInstanceOf[DeltaInvariantCheckerExec]).isDefined, 
writerPlan)
+          assert(
+            
writerPlan.find(_.isInstanceOf[GlutenDeltaInvariantCheckerExec]).isEmpty,
+            writerPlan)
+      }
+    }
+  }
+
+  test("native delta write checks NOT NULL constraints on a partitioned 
table") {
+    withNativeWriteOffloadConf {
+      withTempDir {
+        dir =>
+          val path = dir.getCanonicalPath
+          sql(
+            s"CREATE TABLE delta.`$path` (id BIGINT NOT NULL, s STRING NOT 
NULL) USING delta " +
+              "PARTITIONED BY (s)")
+          appendRows(path, (1L, "a"))
+          // The writer's native sort pulls the checked batches, and keeps 
only the text of the
+          // violation. The write still fails with Delta's exception.
+          val dataViolation = intercept[InvariantViolationException] {
+            appendRows(path, (2L, "a"), (null, "b"))
+          }
+          assert(dataViolation.getMessage.contains("NOT NULL constraint 
violated for column: id"))
+          // Delta turns an empty partition string into a null before checking 
it.
+          val partitionViolation = intercept[InvariantViolationException] {
+            appendRows(path, (3L, ""))
+          }
+          assert(
+            partitionViolation.getMessage.contains("NOT NULL constraint 
violated for column: s"))
+          assert(spark.read.format("delta").load(path).collect().toSeq == 
Seq(Row(1L, "a")))
+          // With this conf off, Delta checks the empty string itself, which 
is not null.
+          
withSQLConf(DeltaSQLConf.CONVERT_EMPTY_TO_NULL_FOR_STRING_PARTITION_COL.key -> 
"false") {
+            appendRows(path, (4L, ""))
+          }
+      }
+    }
+  }
 }
diff --git 
a/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
 
b/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
index 4b48b71fc1..7d5fc113fb 100644
--- 
a/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
+++ 
b/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/GlutenOptimisticTransaction.scala
@@ -22,7 +22,7 @@ import 
org.apache.gluten.extension.columnar.transition.Transitions
 import org.apache.spark.sql.{AnalysisException, Dataset, SparkSession}
 import org.apache.spark.sql.catalyst.types.DataTypeUtils
 import org.apache.spark.sql.delta.actions.{AddFile, FileAction}
-import org.apache.spark.sql.delta.constraints.{Constraint, Constraints, 
DeltaInvariantCheckerExec}
+import org.apache.spark.sql.delta.constraints.{Constraint, Constraints, 
DeltaInvariantCheckerExec, GlutenDeltaInvariantChecker, 
GlutenDeltaInvariantCheckerExec}
 import org.apache.spark.sql.delta.files.{DeltaFileFormatWriter, 
GlutenDeltaFileFormatWriter, TransactionalWrite}
 import org.apache.spark.sql.delta.hooks.AutoCompact
 import org.apache.spark.sql.delta.perf.{DeltaOptimizedWriterExec, 
GlutenDeltaOptimizedWriterExec}
@@ -38,6 +38,7 @@ import org.apache.spark.sql.util.ScalaExtensions.OptionExt
 import org.apache.spark.util.SerializableConfiguration
 
 import scala.collection.mutable.ListBuffer
+import scala.util.control.NonFatal
 
 /**
  * An [[OptimisticTransaction]] whose `writeFiles` writes with the Velox 
writer.
@@ -173,19 +174,27 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
           partitioningColumns,
           metadata)
         if (nativeUnsupportedReason.isEmpty) {
-          GlutenDeltaFileFormatWriter.write(
-            sparkSession = spark,
-            plan = glutenPhysicalPlan(spark, empty2NullPlan, constraints, 
isOptimize, writeOptions),
-            fileFormat = new GlutenDeltaParquetFileFormat(protocol, metadata),
-            committer = committer,
-            outputSpec = outputSpec,
-            hadoopConf = jobHadoopConf,
-            partitionColumns = partitioningColumns,
-            bucketSpec = None,
-            statsTrackers = optionalStatsTracker.toSeq
-              ++ statsTrackers
-              ++ identityTrackerOpt.toSeq,
-            options = options)
+          try {
+            GlutenDeltaFileFormatWriter.write(
+              sparkSession = spark,
+              plan =
+                glutenPhysicalPlan(spark, empty2NullPlan, constraints, 
isOptimize, writeOptions),
+              fileFormat = new GlutenDeltaParquetFileFormat(protocol, 
metadata),
+              committer = committer,
+              outputSpec = outputSpec,
+              hadoopConf = jobHadoopConf,
+              partitionColumns = partitioningColumns,
+              bucketSpec = None,
+              statsTrackers = optionalStatsTracker.toSeq
+                ++ statsTrackers
+                ++ identityTrackerOpt.toSeq,
+              options = options)
+          } catch {
+            // Native code that pulls the checked batches keeps only a 
violation's text. Delta's
+            // own catch below handles the rest.
+            case NonFatal(e) if 
InnerInvariantViolationException.unapply(e).isEmpty =>
+              throw GlutenDeltaInvariantChecker.findViolation(e).getOrElse(e)
+          }
         } else {
           logInfo(s"${nativeUnsupportedReason.get}; writing with the vanilla 
Delta writer.")
         // GLUTEN-SPLICE-END
@@ -269,7 +278,8 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
 
   /**
    * The plan the Velox writer consumes: Delta's plan made columnar. The 
invariant checker is left
-   * out when there are no constraints, to avoid the transitions around it.
+   * out when there are no constraints, to avoid the transitions around it. 
When every constraint is
+   * a top-level NOT NULL, [[GlutenDeltaInvariantCheckerExec]] checks them on 
the Velox batches.
    */
   private def glutenPhysicalPlan(
       spark: SparkSession,
@@ -286,7 +296,13 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
       case aqe: AdaptiveSparkPlanExec =>
         assert(!aqe.isFinalPlan)
         aqe.copy(supportsColumnar = true)
-      case _ => Transitions.toBatchPlan(maybeCheckInvariants, VeloxBatchType)
+      case _ => Transitions.toBatchPlan(plan, VeloxBatchType)
+    }
+    val nativeInvariantChecker =
+      GlutenDeltaInvariantChecker.create(empty2NullPlan.output, constraints)
+    val veloxPlan = nativeInvariantChecker match {
+      case Some(checker) => 
GlutenDeltaInvariantCheckerExec(toVeloxPlan(empty2NullPlan), checker)
+      case None => toVeloxPlan(maybeCheckInvariants)
     }
     // No need to plan optimized write if the write command is OPTIMIZE, which 
aims to produce
     // evenly-balanced data files already.
@@ -294,7 +310,6 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
       // We uniformly convert the query plan to a columnar plan. If
       // the further write operation turns out to be non-offload-able, the
       // columnar plan will be converted back to a row-based plan.
-      val veloxPlan = toVeloxPlan(maybeCheckInvariants)
       try {
         val glutenWriterExec =
           GlutenDeltaOptimizedWriterExec(veloxPlan, metadata.partitionColumns, 
deltaLog)
@@ -315,7 +330,7 @@ class GlutenOptimisticTransaction(delegate: 
OptimisticTransaction)
           DeltaOptimizedWriterExec(maybeCheckInvariants, 
metadata.partitionColumns, deltaLog)
       }
     } else {
-      toVeloxPlan(maybeCheckInvariants)
+      veloxPlan
     }
   }
 
diff --git 
a/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
 
b/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
index 7763a3b969..c159bd4e1f 100644
--- 
a/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
+++ 
b/backends-velox/src-delta4/main/scala/org/apache/spark/sql/delta/files/GlutenDeltaFileFormatWriter.scala
@@ -83,6 +83,12 @@ object GlutenDeltaFileFormatWriter extends Logging {
    */
   private var executedPlan: Option[SparkPlan] = None
 
+  private[delta] def getExecutedPlanForTesting: Option[SparkPlan] = 
executedPlan
+
+  private[delta] def clearExecutedPlanForTesting(): Unit = {
+    executedPlan = None
+  }
+
   // scalastyle:off argcount
   /**
    * Basic work flow of this command is:
diff --git 
a/backends-velox/src-delta4/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
 
b/backends-velox/src-delta4/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
index c955eb7d17..855f599577 100644
--- 
a/backends-velox/src-delta4/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
+++ 
b/backends-velox/src-delta4/test/scala/org/apache/spark/sql/delta/DeltaNativeWriteSuite.scala
@@ -23,6 +23,9 @@ import org.apache.spark.sql.{DataFrame, Row}
 import org.apache.spark.sql.catalyst.TableIdentifier
 import org.apache.spark.sql.delta.actions.AddFile
 import org.apache.spark.sql.delta.commands.optimize.OptimizeMetrics
+import org.apache.spark.sql.delta.constraints.{DeltaInvariantCheckerExec, 
GlutenDeltaInvariantCheckerExec}
+import org.apache.spark.sql.delta.files.GlutenDeltaFileFormatWriter
+import org.apache.spark.sql.delta.schema.InvariantViolationException
 import org.apache.spark.sql.delta.sources.DeltaSQLConf
 import org.apache.spark.sql.delta.test.DeltaSQLCommandTest
 import org.apache.spark.sql.execution.QueryExecution
@@ -517,4 +520,100 @@ class DeltaNativeWriteSuite extends DeltaSQLCommandTest {
       }
     }
   }
+
+  private def appendRows(path: String, rows: (java.lang.Long, String)*): Unit 
= {
+    rows.toDF("id", "s").write.format("delta").mode("append").save(path)
+  }
+
+  /** The plan of the last write by the Velox writer. */
+  private def writerPlan: SparkPlan = 
GlutenDeltaFileFormatWriter.getExecutedPlanForTesting
+    .getOrElse(fail("Expected a write by the Velox writer"))
+
+  test("native delta write checks top-level NOT NULL constraints natively") {
+    for (optimizeWrite <- Seq(false, true)) {
+      withNativeWriteOffloadConf {
+        withSQLConf(DeltaSQLConf.DELTA_OPTIMIZE_WRITE_ENABLED.key -> 
optimizeWrite.toString) {
+          withTempDir {
+            dir =>
+              val path = dir.getCanonicalPath
+              sql(s"CREATE TABLE delta.`$path` (id BIGINT NOT NULL, s STRING) 
USING delta")
+              GlutenDeltaFileFormatWriter.clearExecutedPlanForTesting()
+              appendRows(path, (1L, "a"))
+              assert(
+                
writerPlan.find(_.isInstanceOf[GlutenDeltaInvariantCheckerExec]).isDefined,
+                writerPlan)
+              
assert(writerPlan.find(_.isInstanceOf[DeltaInvariantCheckerExec]).isEmpty, 
writerPlan)
+              assert(!writerPlan.treeString.contains("ColumnarToRow"), 
writerPlan)
+
+              val e = intercept[InvariantViolationException] {
+                appendRows(path, (2L, "b"), (null, "c"))
+              }
+              assert(e.getMessage.contains("NOT NULL constraint violated for 
column: id"))
+              // An optimized write of several input partitions shuffles the 
checked batches
+              // natively, which keeps only the text of the violation.
+              val shuffled = intercept[InvariantViolationException] {
+                spark
+                  .range(0, 1000, 1, 4)
+                  .repartition(4)
+                  .selectExpr("IF(id = 777, CAST(NULL AS BIGINT), id) AS id", 
"'x' AS s")
+                  .write
+                  .format("delta")
+                  .mode("append")
+                  .save(path)
+              }
+              assert(shuffled.getMessage.contains("NOT NULL constraint 
violated for column: id"))
+              assert(spark.read.format("delta").load(path).collect().toSeq == 
Seq(Row(1L, "a")))
+          }
+        }
+      }
+    }
+  }
+
+  test("native delta write keeps DeltaInvariantCheckerExec for CHECK 
constraints") {
+    withNativeWriteOffloadConf {
+      withTempDir {
+        dir =>
+          val path = dir.getCanonicalPath
+          sql(
+            s"CREATE TABLE delta.`$path` (id BIGINT NOT NULL, s STRING) USING 
delta " +
+              "TBLPROPERTIES ('delta.constraints.positive' = 'id > 0')")
+          GlutenDeltaFileFormatWriter.clearExecutedPlanForTesting()
+          appendRows(path, (1L, "a"))
+          
assert(writerPlan.find(_.isInstanceOf[DeltaInvariantCheckerExec]).isDefined, 
writerPlan)
+          assert(
+            
writerPlan.find(_.isInstanceOf[GlutenDeltaInvariantCheckerExec]).isEmpty,
+            writerPlan)
+      }
+    }
+  }
+
+  test("native delta write checks NOT NULL constraints on a partitioned 
table") {
+    withNativeWriteOffloadConf {
+      withTempDir {
+        dir =>
+          val path = dir.getCanonicalPath
+          sql(
+            s"CREATE TABLE delta.`$path` (id BIGINT NOT NULL, s STRING NOT 
NULL) USING delta " +
+              "PARTITIONED BY (s)")
+          appendRows(path, (1L, "a"))
+          // The writer's native sort pulls the checked batches, and keeps 
only the text of the
+          // violation. The write still fails with Delta's exception.
+          val dataViolation = intercept[InvariantViolationException] {
+            appendRows(path, (2L, "a"), (null, "b"))
+          }
+          assert(dataViolation.getMessage.contains("NOT NULL constraint 
violated for column: id"))
+          // Delta turns an empty partition string into a null before checking 
it.
+          val partitionViolation = intercept[InvariantViolationException] {
+            appendRows(path, (3L, ""))
+          }
+          assert(
+            partitionViolation.getMessage.contains("NOT NULL constraint 
violated for column: s"))
+          assert(spark.read.format("delta").load(path).collect().toSeq == 
Seq(Row(1L, "a")))
+          // With this conf off, Delta checks the empty string itself, which 
is not null.
+          
withSQLConf(DeltaSQLConf.CONVERT_EMPTY_TO_NULL_FOR_STRING_PARTITION_COL.key -> 
"false") {
+            appendRows(path, (4L, ""))
+          }
+      }
+    }
+  }
 }
diff --git 
a/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatchJniWrapper.java
 
b/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatchJniWrapper.java
index fb2a5625c6..4886971fee 100644
--- 
a/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatchJniWrapper.java
+++ 
b/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatchJniWrapper.java
@@ -36,6 +36,8 @@ public class VeloxColumnarBatchJniWrapper implements 
RuntimeAware {
 
   public native long slice(long veloxBatchHandle, int offset, int limit);
 
+  public native int firstNullColumnIndex(long veloxBatchHandle, int[] 
columnOrdinals);
+
   public native long repeatedThenCompose(
       long repeatedBatch, long nonRepeatedBatch, int[] rowId2RowNums);
 
diff --git 
a/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatches.java
 
b/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatches.java
index 41c10324dc..e8320dda00 100644
--- 
a/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatches.java
+++ 
b/backends-velox/src/main/java/org/apache/gluten/columnarbatch/VeloxColumnarBatches.java
@@ -142,6 +142,19 @@ public final class VeloxColumnarBatches {
     }
   }
 
+  public static int firstNullColumnIndex(ColumnarBatch batch, int[] 
columnOrdinals) {
+    if (columnOrdinals.length == 0 || batch.numRows() == 0) {
+      return -1;
+    }
+    ColumnarBatches.checkOffloaded(batch);
+    Runtime runtime =
+        Runtimes.contextInstance(
+            BackendsApiManager.getBackendName(), 
"VeloxColumnarBatches#firstNullColumnIndex");
+    long nativeHandle = 
ColumnarBatches.getNativeHandle(BackendsApiManager.getBackendName(), batch);
+    return VeloxColumnarBatchJniWrapper.create(runtime)
+        .firstNullColumnIndex(nativeHandle, columnOrdinals);
+  }
+
   /**
    * repeat batch1 using the array `rowId2RowNums` passed in and then compose 
with batch2.
    * rowId2RowNums records the number of each row after repeated.
diff --git a/cpp/velox/jni/VeloxJniWrapper.cc b/cpp/velox/jni/VeloxJniWrapper.cc
index ab44a97445..79f19f7383 100644
--- a/cpp/velox/jni/VeloxJniWrapper.cc
+++ b/cpp/velox/jni/VeloxJniWrapper.cc
@@ -717,6 +717,28 @@ JNIEXPORT jlong JNICALL 
Java_org_apache_gluten_columnarbatch_VeloxColumnarBatchJ
   JNI_METHOD_END(kInvalidObjectHandle)
 }
 
+JNIEXPORT jint JNICALL 
Java_org_apache_gluten_columnarbatch_VeloxColumnarBatchJniWrapper_firstNullColumnIndex(
 // NOLINT
+    JNIEnv* env,
+    jobject wrapper,
+    jlong veloxBatchHandle,
+    jintArray columnOrdinals) {
+  JNI_METHOD_START
+  auto ctx = getRuntime(env, wrapper);
+  auto runtime = dynamic_cast<VeloxRuntime*>(ctx);
+  auto batch = ObjectStore::retrieve<ColumnarBatch>(veloxBatchHandle);
+  auto safeColumnOrdinals = getIntArrayElementsSafe(env, columnOrdinals);
+
+  if (safeColumnOrdinals.length() == 0 || batch->numRows() == 0) {
+    return -1;
+  }
+
+  auto veloxBatch = 
VeloxColumnarBatch::from(runtime->memoryManager()->getLeafMemoryPool().get(), 
batch);
+  const std::vector<int32_t> columnIndices(
+      safeColumnOrdinals.elems(), safeColumnOrdinals.elems() + 
safeColumnOrdinals.length());
+  return veloxBatch->firstNullColumnIndex(columnIndices);
+  JNI_METHOD_END(-1)
+}
+
 JNIEXPORT void JNICALL 
Java_org_apache_gluten_monitor_VeloxMemoryProfiler_start( // NOLINT
     JNIEnv* env,
     jclass) {
diff --git a/cpp/velox/memory/VeloxColumnarBatch.cc 
b/cpp/velox/memory/VeloxColumnarBatch.cc
index d54221c2b0..ee4e387a61 100644
--- a/cpp/velox/memory/VeloxColumnarBatch.cc
+++ b/cpp/velox/memory/VeloxColumnarBatch.cc
@@ -19,6 +19,7 @@
 #include "utils/VeloxArrowUtils.h"
 #include "velox/row/UnsafeRowFast.h"
 #include "velox/type/Type.h"
+#include "velox/vector/DecodedVector.h"
 #include "velox/vector/FlatVector.h"
 
 namespace gluten {
@@ -158,6 +159,41 @@ std::shared_ptr<VeloxColumnarBatch> 
VeloxColumnarBatch::select(
   return std::make_shared<VeloxColumnarBatch>(rowVector);
 }
 
+int32_t VeloxColumnarBatch::firstNullColumnIndex(const std::vector<int32_t>& 
columnIndices) const {
+  const auto numRows = rowVector_->size();
+  const auto numColumns = static_cast<int32_t>(rowVector_->childrenSize());
+  for (const auto index : columnIndices) {
+    GLUTEN_CHECK(index >= 0 && index < numColumns, "Column index out of range 
when looking for nulls");
+  }
+  if (columnIndices.empty()) {
+    return -1;
+  }
+
+  // The cached null count is not used: it can be stale, and on a dictionary 
or constant wrapping it misses the nulls
+  // of the wrapped values. The nulls buffers are scanned a word at a time 
instead.
+  // A RowVector is flat, so its own nulls buffer is indexed by row.
+  const auto* rowNulls = rowVector_->rawNulls();
+  if (rowNulls != nullptr && bits::countNulls(rowNulls, 0, numRows) > 0) {
+    return 0;
+  }
+
+  const SelectivityVector allRows(numRows);
+  for (size_t i = 0; i < columnIndices.size(); ++i) {
+    const auto& child = rowVector_->childAt(columnIndices[i]);
+    VELOX_CHECK_NOT_NULL(child, "Column vector is null when looking for 
nulls.");
+    // Decoding combines the nulls of every wrapping with the nulls of the 
base values.
+    DecodedVector decoded(*child, allRows);
+    if (!decoded.mayHaveNulls()) {
+      continue;
+    }
+    const auto* nulls = decoded.nulls(&allRows);
+    if (nulls != nullptr && bits::countNulls(nulls, 0, numRows) > 0) {
+      return static_cast<int32_t>(i);
+    }
+  }
+  return -1;
+}
+
 std::vector<char> VeloxColumnarBatch::toUnsafeRow(int32_t rowId) const {
   auto fast = 
std::make_unique<facebook::velox::row::UnsafeRowFast>(rowVector_);
   auto size = fast->rowSize(rowId);
diff --git a/cpp/velox/memory/VeloxColumnarBatch.h 
b/cpp/velox/memory/VeloxColumnarBatch.h
index 4b83d302a0..7f82358204 100644
--- a/cpp/velox/memory/VeloxColumnarBatch.h
+++ b/cpp/velox/memory/VeloxColumnarBatch.h
@@ -59,6 +59,10 @@ class VeloxColumnarBatch final : public ColumnarBatch {
   facebook::velox::RowVectorPtr getRowVector() const;
   facebook::velox::RowVectorPtr getFlattenedRowVector();
 
+  /// Returns the position in 'columnIndices' of the first listed column that 
has a null in this batch, or -1 if none
+  /// has. A null row counts as a null in every column.
+  int32_t firstNullColumnIndex(const std::vector<int32_t>& columnIndices) 
const;
+
  private:
   void ensureFlattened();
 
diff --git a/cpp/velox/tests/VeloxColumnarBatchTest.cc 
b/cpp/velox/tests/VeloxColumnarBatchTest.cc
index 8f7eb544d3..45a86ada1b 100644
--- a/cpp/velox/tests/VeloxColumnarBatchTest.cc
+++ b/cpp/velox/tests/VeloxColumnarBatchTest.cc
@@ -62,4 +62,50 @@ TEST_F(VeloxColumnarBatchTest, flattenTruncatedVector) {
   ASSERT_NO_THROW(batchOfMap->getFlattenedRowVector());
 }
 
+TEST_F(VeloxColumnarBatchTest, firstNullColumnIndex) {
+  auto noNulls = makeFlatVector<int64_t>({1, 2, 3});
+  auto withNull = makeNullableFlatVector<int64_t>({1, std::nullopt, 3});
+  auto batch = std::make_shared<VeloxColumnarBatch>(makeRowVector({noNulls, 
withNull, noNulls}));
+
+  ASSERT_EQ(batch->firstNullColumnIndex({}), -1);
+  ASSERT_EQ(batch->firstNullColumnIndex({0, 2}), -1);
+  ASSERT_EQ(batch->firstNullColumnIndex({1}), 0);
+  ASSERT_EQ(batch->firstNullColumnIndex({0, 2, 1}), 2);
+  ASSERT_ANY_THROW(batch->firstNullColumnIndex({3}));
+  ASSERT_ANY_THROW(batch->firstNullColumnIndex({-1}));
+}
+
+TEST_F(VeloxColumnarBatchTest, firstNullColumnIndexIgnoresCachedNullCount) {
+  // A stale cached null count of 0 must not hide a null in the nulls buffer.
+  auto staleCount = makeNullableFlatVector<int64_t>({1, std::nullopt, 3});
+  staleCount->setNullCount(0);
+  
ASSERT_EQ(std::make_shared<VeloxColumnarBatch>(makeRowVector({staleCount}))->firstNullColumnIndex({0}),
 0);
+
+  // A dictionary without nulls of its own, and with a null count of 0, over 
base values with a null.
+  auto base = makeNullableFlatVector<int64_t>({std::nullopt, 1, 2});
+  auto dictionary = wrapInDictionary(makeIndices({1, 0, 2}), 3, base);
+  dictionary->setNullCount(0);
+  
ASSERT_EQ(std::make_shared<VeloxColumnarBatch>(makeRowVector({dictionary}))->firstNullColumnIndex({0}),
 0);
+
+  // The same base values, with indices that skip the null.
+  auto dictionaryWithoutNull = wrapInDictionary(makeIndices({1, 2, 1}), 3, 
base);
+  ASSERT_EQ(
+      
std::make_shared<VeloxColumnarBatch>(makeRowVector({dictionaryWithoutNull}))->firstNullColumnIndex({0}),
 -1);
+
+  auto nullConstant = makeNullConstant(TypeKind::BIGINT, 3);
+  
ASSERT_EQ(std::make_shared<VeloxColumnarBatch>(makeRowVector({nullConstant}))->firstNullColumnIndex({0}),
 0);
+}
+
+TEST_F(VeloxColumnarBatchTest, 
firstNullColumnIndexRowNullsAndTruncatedChildren) {
+  // A null row is a null in every column.
+  auto rowVector = makeRowVector({makeFlatVector<int64_t>({1, 2, 3})});
+  rowVector->setNull(1, true);
+  
ASSERT_EQ(std::make_shared<VeloxColumnarBatch>(rowVector)->firstNullColumnIndex({0}),
 0);
+
+  // A null past the end of the batch, in a child longer than the batch, is 
not in the batch.
+  auto child = makeNullableFlatVector<int64_t>({1, 2, std::nullopt});
+  auto truncated = std::make_shared<RowVector>(pool(), ROW({BIGINT()}), 
nullptr, 2, std::vector<VectorPtr>{child});
+  
ASSERT_EQ(std::make_shared<VeloxColumnarBatch>(truncated)->firstNullColumnIndex({0}),
 -1);
+}
+
 } // namespace gluten
diff --git a/docs/backends/velox/get-started/VeloxDelta.md 
b/docs/backends/velox/get-started/VeloxDelta.md
index d2dc359195..37f47944fb 100644
--- a/docs/backends/velox/get-started/VeloxDelta.md
+++ b/docs/backends/velox/get-started/VeloxDelta.md
@@ -339,7 +339,7 @@ The Read column covers Spark 3.5 to 4.2. It covers Spark 
3.4 only where the row
 | Column mapping, id mode | Offload, also on Spark 3.4. ResultMismatch when 
matching by field ID would select no column (seen on 4.1) or a different column 
(from code). | ResultMismatch on Spark 3.5, Exception when Spark's reader reads 
the files. Fallback on Spark 4.x. Both are from code. | Gluten matches columns 
by physical name. See [Known problems](#known-problems). |
 | Identity columns | Offload | Offload | Seen on 4.1. High-water marks are 
correct. The projection that generates identity values runs on Spark (from 
code). |
 | Generated columns | Offload | Offload | Seen on 4.1. No Gluten test covers 
generated columns. Velox computes the generated values when it supports the 
expression (from code). Partition filters derived from them reach the native 
scan. `DATE_FORMAT` results for years above 9999 differ (see [Write 
problems](#write-problems)). |
-| CHECK constraints, NOT NULL | Offload | Offload | Delta checks the rows on 
the JVM before Velox writes them. Gluten's tests cover NOT NULL on Spark 3.5 
and 4.0. Seen on 4.1. |
+| CHECK constraints, NOT NULL | Offload | Offload | When every constraint of 
the table is a NOT NULL on a top-level column, Gluten checks the Velox batches 
for nulls before Velox writes them, with no conversion to rows. A CHECK 
constraint, a generated column or a NOT NULL on a nested field makes Delta 
check all the constraints on the JVM, row by row, before Velox writes the rows. 
Both are from code. Gluten's tests cover NOT NULL on Spark 3.5 and 4.0. Seen on 
4.1. |
 | Row tracking | Offload, Exception for `_metadata.base_row_id` filters, 
ResultMismatch for `_metadata.row_id` when older files lack base row IDs | 
Offload | Seen on 4.1. See [Read problems](#read-problems). |
 | Liquid clustering | Offload | PartialOffload | Reads use the same native 
scan as other tables. Native commands write to clustered tables. `OPTIMIZE`, 
which clusters the data, uses Delta's writer. All three are from code. |
 | UniForm (IcebergCompatV1, V2) | | ResultMismatch on Spark 3.5, Exception 
when partitioned. Fallback on Spark 4.x. | The write statuses are from code. No 
test reads a UniForm table. |
@@ -476,13 +476,13 @@ These operations fail as in Delta Lake, with a different 
exception type or error
 
 | Failure | Delta Lake raises | Gluten raises |
 |---|---|---|
-| CHECK, NOT NULL or generated-column violation on a partitioned native write. 
The CHECK and NOT NULL cases are from code. | 
`DeltaInvariantViolationException` | `SparkException` (`TASK_WRITE_FAILED`) 
wrapping `GlutenException` |
+| CHECK or generated-column violation on a partitioned native write or an 
optimized write of several input partitions. The CHECK case is from code. | 
`DeltaInvariantViolationException` | `SparkException` wrapping 
`GlutenException` (`TASK_WRITE_FAILED` on partitioned writes) |
 | `replaceWhere` violation on a partitioned native write | 
`DeltaAnalysisException` (`DELTA_REPLACE_WHERE_MISMATCH`) | `SparkException` 
(`TASK_WRITE_FAILED`) wrapping `GlutenException` |
 | Store-assignment cast overflow in a MERGE `WHEN MATCHED THEN UPDATE` clause 
| `DeltaArithmeticException` (`DELTA_CAST_OVERFLOW_IN_TABLE_WRITE`) | 
`SparkException` wrapping `GlutenException` |
 | ANSI cast overflow evaluated in Velox. A MERGE that casts overflowing ARRAY 
or MAP values can complete without an error (see [MERGE, UPDATE and DELETE 
problems](#merge-update-and-delete-problems)). | `SparkArithmeticException` | 
`SparkException` wrapping `GlutenException` (`VeloxUserError`) |
 | Missing data file | `SparkException` (`FAILED_READ_FILE.FILE_NOT_EXIST`) on 
Spark 4.x | `SparkException` wrapping `GlutenException` (`FILE_NOT_FOUND`) |
 
-For errors raised on the JVM, the message of the exception raised in the task 
stays in the `GlutenException` message. For a `replaceWhere` violation, that is 
the CHECK constraint message. The `DELTA_REPLACE_WHERE_MISMATCH` line that 
Delta Lake puts before it, `Written data does not conform to partial table 
overwrite condition or constraint`, is missing. The lost exception types are 
tracked in [#11912](https://github.com/apache/gluten/issues/11912). On 
unpartitioned `save(path)` writes, C [...]
+For errors raised on the JVM, the message of the exception raised in the task 
stays in the `GlutenException` message. For a `replaceWhere` violation, that is 
the CHECK constraint message. The `DELTA_REPLACE_WHERE_MISMATCH` line that 
Delta Lake puts before it, `Written data does not conform to partial table 
overwrite condition or constraint`, is missing. The lost exception types are 
tracked in [#11912](https://github.com/apache/gluten/issues/11912). On 
unpartitioned `save(path)` writes wi [...]
 
 ## Configuration
 ### Gluten Options


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to