[
https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taoufik DACHRAOUI updated SPARK-27388:
--------------------------------------
Description:
Add an expression encoder for objects (ie. Avro) defined by properties (ie. I
call them Beans)
A property of an object is defined by a setter and a getter functions where the
getter return type is equal to the setter unique parameter type and the getter
and setter functions have the same name of the getter is prefixed by "get" and
the setter is prefixed by "set"
An example of a Bean:
{{```}}
class Bar {
...
def age():Int = \{...}
def age(v:Int):Unit \{...}
...
}
class Foo extends Bar {
...
def setName(n:String) \{...}
def getName():String = \{...}
...
}
{{```}}
The class `Foo` has 2 properties `age` and `Name`
also added support for java.util.List, java.util.Map and java Enums
Currently complex avro unions are not supported; indeed a complex union is
declared as Object and there cannot be an expression encoder for Object type
(need to use a custom serializer like kryo for example)
was:
Add an expression encoder for objects (ie. Avro) defined by properties (ie. I
call them Beans)
A property of an object is defined by a setter and a getter functions where the
getter return type is equal to the setter unique parameter type and the getter
and setter functions have the same name of the getter is prefixed by "get" and
the setter is prefixed by "set"
An example of a Bean:
{{```}}
{{class Bar { ... }}
{{ def age():Int = \{...} }}
{{ def age(v:Int):Unit \{...} }}
{{... } }}
{{class Foo extends Bar { ... }}
{{ def setName(n:String) \{...} }}
{{ def getName():String = \{...} }}
{{... }}}
{{```}}
also added support for java.util.List, java.util.Map and java Enums
Currently complex avro unions are not supported; indeed a complex union is
declared as Object and there cannot be an expression encoder for Object type
(need to use a custom serializer like kryo for example)
> expression encoder for avro like objects
> ----------------------------------------
>
> Key: SPARK-27388
> URL: https://issues.apache.org/jira/browse/SPARK-27388
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 2.4.1
> Reporter: Taoufik DACHRAOUI
> Priority: Major
>
> Add an expression encoder for objects (ie. Avro) defined by properties (ie. I
> call them Beans)
>
> A property of an object is defined by a setter and a getter functions where
> the getter return type is equal to the setter unique parameter type and the
> getter and setter functions have the same name of the getter is prefixed by
> "get" and the setter is prefixed by "set"
>
> An example of a Bean:
> {{```}}
> class Bar {
> ...
> def age():Int = \{...}
> def age(v:Int):Unit \{...}
> ...
> }
> class Foo extends Bar {
> ...
> def setName(n:String) \{...}
> def getName():String = \{...}
> ...
> }
> {{```}}
> The class `Foo` has 2 properties `age` and `Name`
>
> also added support for java.util.List, java.util.Map and java Enums
>
> Currently complex avro unions are not supported; indeed a complex union is
> declared as Object and there cannot be an expression encoder for Object type
> (need to use a custom serializer like kryo for example)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
