Re: Unsubscribe

2018-05-13 Thread Lefty Leverenz
Rahul, to unsubscribe please send a message to user-unsubscr...@hive.apache.org as described here: Mailing Lists . Thanks. -- Lefty On Mon, May 7, 2018 at 4:22 PM Rahul Channe wrote: >

Re: Unsubscribe

2018-05-13 Thread Lefty Leverenz
Beth, to unsubscribe please send a message to user-unsubscr...@hive.apache.org as described here: Mailing Lists . Thanks. -- Lefty On Mon, May 7, 2018 at 4:52 PM Beth Lee wrote: > >

Re: Unsubscribe

2018-05-13 Thread Lefty Leverenz
Roger, to unsubscribe please send a message to user-unsubscr...@hive.apache.org as described here: Mailing Lists . Thanks. -- Lefty On Tue, May 8, 2018 at 12:49 AM Roger Baatjes wrote: > >

Re: Unsubscribe

2018-05-13 Thread Lefty Leverenz
Dheena, to unsubscribe please send a message to user-unsubscr...@hive.apache.org as described here: Mailing Lists . Thanks. -- Lefty On Wed, May 9, 2018 at 2:19 AM Dheena Dhayalan wrote: > >

Re: What does the ORC SERDE do

2018-05-13 Thread Lefty Leverenz
Jörn, please do update the wiki, we really need better SerDe documentation. Getting write access is easy: About This Wiki -- How to get permission to edit -- Lefty On Sun, May 13, 2018 at

Re: What does the ORC SERDE do

2018-05-13 Thread Jörn Franke
You have in AbstractSerde a method to return very basic stats related to your fileformat (mostly size of the data and number of rows etc): https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/SerDeStats.java In method initialize of your Serde you can retrieve

Re: What does the ORC SERDE do

2018-05-13 Thread Elliot West
Hi Jörn, I’m curious to know how the SerDe framework provides the means to deal with partitions, table properties, and statistics? I was under the impression that these were in the domain of the metastore and I’ve not found anything in the SerDe interface related to these. I would appreciate if

Re: What does the ORC SERDE do

2018-05-13 Thread Jörn Franke
Yes this was what I did when writing the Hive part of the HadoopOffice / HadoopCryptoledger library. Be aware that Orc uses also some internal Hive APIs/ Extended the existing ones (eg Vectorizedserde) I don’t have access to the Hive Wiki otherwise I could update it a little bit. > On 13. May

Re: What does the ORC SERDE do

2018-05-13 Thread 侯宗田
Thank you, it makes the concept clearer to me. I think I need to look up the source code for some details. > 在 2018年5月13日,下午10:42,Jörn Franke 写道: > > In detail you can check the source code, but a Serde needs to translate an > object to a Hive object and vice versa.

Re: What does the ORC SERDE do

2018-05-13 Thread Jörn Franke
In detail you can check the source code, but a Serde needs to translate an object to a Hive object and vice versa. Usually this is very simple (simply passing the object or create A HiveDecimal etc). It also provides an ObjectInspector that basically describes an object in more detail (eg to be

What does the ORC SERDE do

2018-05-13 Thread 侯宗田
Hello,everyone I know the json serde turn fields in a row to a json format, csv serde turn it to csv format with their serdeproperties. But I wonder what the orc serde does when I choose to stored as orc file format. And why is there still escaper, separator in orc serdeproperties. Also with