[
https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taoufik DACHRAOUI updated SPARK-27388:
--------------------------------------
Description:
Add an expression encoder for objects defined by properties (ie. I call them
Beans). A property of an object is defined by a _setter_ and a _getter_
functions where the _getter_ return type is equal to the _setter_ unique
parameter type and the _getter_ and _setter_ functions have the same name or
the _getter_ name is prefixed by "get" and the _setter_ name is prefixed by
"set"
An example of a Bean:
{code:java}
class Bar {
...
def age():Int = {...}
def age(v:Int):Unit {...}
...
}
class Foo extends Bar {
...
def setName(n:String) {...}
def getName():String = {...}
...
}
{code}
```
The class _Foo in the example above_ has 2 properties _age_ and _Name._
Also, in this new feature, we added support for _java.util.List_,
_java.util.Map_ and java _Enums_
Avro objects are beans and thus we can create an expression encoder for avro
objects with the current addition. All avro types, including fixed types, and
excluding complex union types, are suppported by this addition.
Currently complex avro unions are not supported because a complex union is
declared as Object and there cannot be an expression encoder for Object type
(need to use a custom serializer like kryo for example)
was:
Add an expression encoder for objects defined by properties (ie. I call them
Beans). A property of an object is defined by a _setter_ and a _getter_
functions where the _getter_ return type is equal to the _setter_ unique
parameter type and the _getter_ and _setter_ functions have the same name or
the _getter_ name is prefixed by "get" and the _setter_ name is prefixed by
"set"
An example of a Bean:
{code:java}
class Bar {
...
def age():Int = {...}
def age(v:Int):Unit {...}
...
}
class Foo extends Bar {
...
def setName(n:String) {...}
def getName():String = {...}
...
}
{code}
```
The class _Foo in the example above_ has 2 properties _age_ and _Name._
Also, in this new feature, we added support for _java.util.List_,
_java.util.Map_ and java _Enums_
Avro objects are beans and thus we can create an expression encoder with the
current addition. And all avro types, including fixed types, and excluding
complex union types, are suppported by this addition.
Currently complex avro unions are not supported because a complex union is
declared as Object and there cannot be an expression encoder for Object type
(need to use a custom serializer like kryo for example)
> expression encoder for avro like objects
> ----------------------------------------
>
> Key: SPARK-27388
> URL: https://issues.apache.org/jira/browse/SPARK-27388
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 2.4.1
> Reporter: Taoufik DACHRAOUI
> Priority: Major
>
> Add an expression encoder for objects defined by properties (ie. I call them
> Beans). A property of an object is defined by a _setter_ and a _getter_
> functions where the _getter_ return type is equal to the _setter_ unique
> parameter type and the _getter_ and _setter_ functions have the same name or
> the _getter_ name is prefixed by "get" and the _setter_ name is prefixed by
> "set"
>
> An example of a Bean:
> {code:java}
> class Bar {
> ...
> def age():Int = {...}
> def age(v:Int):Unit {...}
> ...
> }
> class Foo extends Bar {
> ...
> def setName(n:String) {...}
> def getName():String = {...}
> ...
> }
> {code}
> ```
> The class _Foo in the example above_ has 2 properties _age_ and _Name._
>
> Also, in this new feature, we added support for _java.util.List_,
> _java.util.Map_ and java _Enums_
> Avro objects are beans and thus we can create an expression encoder for avro
> objects with the current addition. All avro types, including fixed types, and
> excluding complex union types, are suppported by this addition.
>
> Currently complex avro unions are not supported because a complex union is
> declared as Object and there cannot be an expression encoder for Object type
> (need to use a custom serializer like kryo for example)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
