GitHub user rdblue opened a pull request:

    https://github.com/apache/spark/pull/21118

    SPARK-23325: Use InternalRow when reading with DataSourceV2.

    ## What changes were proposed in this pull request?
    
    This updates the DataSourceV2 API to use InternalRow instead of Row for the 
default case with no scan mix-ins. Because the API is changing significantly in 
the same places, this also renames ReaderFactory back to ReadTask.
    
    Support for readers that produce Row is added through 
SupportsDeprecatedScanRow, which matches the previous API. Readers that used 
Row now implement this class and should be migrated to InternalRow.
    
    Readers that previously implemented SupportsScanUnsafeRow have been 
migrated to use no SupportsScan mix-ins and produce InternalRow.
    
    ## How was this patch tested?
    
    This uses existing tests.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rdblue/spark 
SPARK-23325-datasource-v2-internal-row

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21118.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21118
    
----
commit eddd049ed78970dda0796a37462e52f53bfeacc4
Author: Ryan Blue <blue@...>
Date:   2018-04-20T20:15:58Z

    SPARK-23325: Use InternalRow when reading with DataSourceV2.
    
    This updates the DataSourceV2 API to use InternalRow instead of Row for
    the default case with no scan mix-ins. Because the API is changing
    significantly in the same places, this also renames ReaderFactory back
    to ReadTask.
    
    Support for readers that produce Row is added through
    SupportsDeprecatedScanRow, which matches the previous API. Readers that
    used Row now implement this class and should be migrated to InternalRow.
    
    Readers that previously implemented SupportsScanUnsafeRow have been
    migrated to use no SupportsScan mix-ins and produce InternalRow.

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to