Re: Which SerDe for Custom Binary Data.
Currently we have data in NFS and we have proprietery tools to access the data. We are planning to move the data into HDFS and use HiveQL for accessing the data and run batch jobs. So looking out for the custom SerDe(assuming the existing SerDe's will not be able to read the underlying data) to read the data using hive. On Fri, Mar 13, 2015 at 10:33 AM, Mich Talebzadeh m...@peridale.co.uk wrote: Hive as I use it is particularly useful for getting data out of relational tables and more importantly query that data using HiveQL (a variation of transact sql) . If your data is in binary format and assuming that you manage to store it in HDFS, how are you intending to access the data. At the consumer level what tools are you going to use? Do you a propriety tool with the correct drivers to access the data? HTH Mich Talebzadeh http://talebzadehmich.wordpress.com *Publications due shortly:* *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache* NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. *From:* karthik maddala [mailto:karthikmaddal...@gmail.com] *Sent:* 13 March 2015 15:56 *To:* user@hive.apache.org *Subject:* Which SerDe for Custom Binary Data. I want to set up a DW based on Hive. However, my data does not come as handy csv files but as binary files in a proprietary format. The binary file consists of serialized data using C language. Could you please suggest which input format to be used and how to write a custom SerDe for the above mentioned data. Thanks, Karthik Maddala
RE: Which SerDe for Custom Binary Data.
Hive as I use it is particularly useful for getting data out of relational tables and more importantly query that data using HiveQL (a variation of transact sql) . If your data is in binary format and assuming that you manage to store it in HDFS, how are you intending to access the data. At the consumer level what tools are you going to use? Do you a propriety tool with the correct drivers to access the data? HTH Mich Talebzadeh http://talebzadehmich.wordpress.com Publications due shortly: Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: karthik maddala [mailto:karthikmaddal...@gmail.com] Sent: 13 March 2015 15:56 To: user@hive.apache.org Subject: Which SerDe for Custom Binary Data. I want to set up a DW based on Hive. However, my data does not come as handy csv files but as binary files in a proprietary format. The binary file consists of serialized data using C language. Could you please suggest which input format to be used and how to write a custom SerDe for the above mentioned data. Thanks, Karthik Maddala
Which SerDe for Custom Binary Data.
I want to set up a DW based on Hive. However, my data does not come as handy csv files but as binary files in a proprietary format. The binary file consists of serialized data using C language. Could you please suggest which input format to be used and how to write a custom SerDe for the above mentioned data. Thanks, Karthik Maddala
Fwd: Which SerDe for Custom Binary Data.
I want to set up a DW based on Hive. However, my data does not come as handy csv files but as binary files in a proprietary format. The binary file consists of serialized data using C language. Could you please suggest which input format to be used and how to write a custom SerDe for the above mentioned data. Thanks, Karthik Maddala
Re: Which SerDe for Custom Binary Data.
https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-HowtoWriteYourOwnSerDe Daniel On 13 במרץ 2015, at 17:56, karthik maddala karthikmaddal...@gmail.com wrote: I want to set up a DW based on Hive. However, my data does not come as handy csv files but as binary files in a proprietary format. The binary file consists of serialized data using C language. Could you please suggest which input format to be used and how to write a custom SerDe for the above mentioned data. Thanks, Karthik Maddala
