[
https://issues.apache.org/jira/browse/SPARK-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-7186:
-------------------------------
Description:
Currently, we use o.a.s.sql.Row both internally and externally. The external
interface is wider than what the internal needs because it is designed to
facilitate end-user programming. This design has proven to be very error prone
and cumbersome for internal Row implementations.
As a first step, we should just create an InternalRow interface in the catalyst
module, which is identical to the current Row interface. And we should switch
all internal operators to use this InternalRow instead. When we need to expose
Row, we convert the InternalRow implementation into Row for users.
After this, we can start removing methods that don't make sense for InternalRow
(in a separate ticket). This is probably one of the most important refactoring
in Spark 1.5.
was:
Currently, we use o.a.s.sql.Row both internally and externally. The external
interface is wider than what the internal needs because it is designed to
facilitate end-user programming. This design has proven to be very error prone
and cumbersome for internal Row implementations.
As a first step, we should just create an InternalRow interface in the catalyst
module, which is identical to the current Row interface. And we should switch
all internal operators to use this InternalRow instead. When we need to expose
Row, we convert the InternalRow implementation into Row for users.
After this, we can start removing methods that don't make sense for
InternalRow. This is probably one of the most important refactoring in Spark
1.5.
> Decouple internal Row from external Row
> ---------------------------------------
>
> Key: SPARK-7186
> URL: https://issues.apache.org/jira/browse/SPARK-7186
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Reporter: Reynold Xin
>
> Currently, we use o.a.s.sql.Row both internally and externally. The external
> interface is wider than what the internal needs because it is designed to
> facilitate end-user programming. This design has proven to be very error
> prone and cumbersome for internal Row implementations.
> As a first step, we should just create an InternalRow interface in the
> catalyst module, which is identical to the current Row interface. And we
> should switch all internal operators to use this InternalRow instead. When we
> need to expose Row, we convert the InternalRow implementation into Row for
> users.
> After this, we can start removing methods that don't make sense for
> InternalRow (in a separate ticket). This is probably one of the most
> important refactoring in Spark 1.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
