RE: Stopping HiveServer2
Typed ‘hiveserver2’ in my terminal From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Wednesday, April 29, 2015 2:01 PM To: user@hive.apache.org Subject: Re: Stopping HiveServer2 how did you start it ? On Wed, Apr 29, 2015 at 4:26 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, How can I stop hiveserver2? I am not able to find the command. Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
Hive columns
Hello, Is it possible to create an external table without specifying the columns? In fact, I am creating an external table that points to a directory that contains 3 text file, and each text file has different number of columns. Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Hive columns
Can you please specify what this means? From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Thursday, September 04, 2014 4:00 PM To: user@hive.apache.org Subject: Re: Hive columns If those are text files you can create the table with single column and then process them line by line On Thu, Sep 4, 2014 at 6:13 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Is it possible to create an external table without specifying the columns? In fact, I am creating an external table that points to a directory that contains 3 text file, and each text file has different number of columns. Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Hive columns
thanks From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Thursday, September 04, 2014 4:23 PM To: user@hive.apache.org Subject: Re: Hive columns it means you will need to define atleast one column in hive or build your fileformat which can handle reading the files and giving data back to hive when i say atleast one column, by default hive uses \n as record terminator that means you can define an entire row as a column and then process it the way you want this is just a suggestion and it would be really tedious to keep the mapping. Instead I would suggest use pig to create proper tables from these files and then use hive to do more deeper analytics On Thu, Sep 4, 2014 at 6:35 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Can you please specify what this means? From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Thursday, September 04, 2014 4:00 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: Hive columns If those are text files you can create the table with single column and then process them line by line On Thu, Sep 4, 2014 at 6:13 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Is it possible to create an external table without specifying the columns? In fact, I am creating an external table that points to a directory that contains 3 text file, and each text file has different number of columns. Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
External table
Hello, Is it possible to create an external table and point it to a file instead of a directory? thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
Where does hive store its data?
I can see that Hive stores the tables in hdfs/user/hive/warehouse, but I need to access the data from the disk. Where does Hive stores the data on the disk? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
Distribution of Data
This question is not directly related to Hive, but: I configured 3 datanodes on my Linux machine. In my configuration, I configured the number of replication to be 1. I am submitting a file to the hdfs, and found that the file has 3 copies on each datanodes (I checked it from the browser) Isn't right that I should only see the file on 1 datanodes and on 1 replica? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Distribution of Data
What does it mean my replication have set as final? From: hadoop hive [mailto:hadooph...@gmail.com] Sent: Tuesday, August 26, 2014 4:36 PM To: user@hive.apache.org Subject: Re: Distribution of Data Can you check if your replication have set as final or not else it won't be effected. On Aug 26, 2014 6:46 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: This question is not directly related to Hive, but: I configured 3 datanodes on my Linux machine. In my configuration, I configured the number of replication to be 1. I am submitting a file to the hdfs, and found that the file has 3 copies on each datanodes (I checked it from the browser) Isn't right that I should only see the file on 1 datanodes and on 1 replica? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Distribution of Data
I marked it as final, and tried again, still facing the same issue. From: hadoop hive [mailto:hadooph...@gmail.com] Sent: Tuesday, August 26, 2014 5:19 PM To: user@hive.apache.org Subject: RE: Distribution of Data If in your core-site.xml or hdfs-site.XML .. Property is Marked as final it would not be overwrite by your code On Aug 26, 2014 7:07 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: What does it mean my replication have set as final? From: hadoop hive [mailto:hadooph...@gmail.commailto:hadooph...@gmail.com] Sent: Tuesday, August 26, 2014 4:36 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: Distribution of Data Can you check if your replication have set as final or not else it won't be effected. On Aug 26, 2014 6:46 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: This question is not directly related to Hive, but: I configured 3 datanodes on my Linux machine. In my configuration, I configured the number of replication to be 1. I am submitting a file to the hdfs, and found that the file has 3 copies on each datanodes (I checked it from the browser) Isn't right that I should only see the file on 1 datanodes and on 1 replica? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
Data in Hive
My target is to perform a SELECT query using Hive When I have a small data on a single machine (namenode), I start by: 1-Creating a table that contains this data: create table table1 (int col1, string col2) 2-Loading the data from a file path: load data local inpath 'path' into table table1; 3-Perform my SELECT query: select * from table1 where col10 I have huge data, of 10 millions rows that doesn't fit into a single machine. Lets assume Hadoop divided my data into for example 10 datanodes and each datanode contains 1 million row. Retrieving the data to a single computer is impossible due to its huge size or would take alot of time in case it is possible. Will Hive create a table at each datanode and perform the SELECT query or will Hive move all the data a one location (datanode) and create one table? (which is inefficient) *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
Distributed data
Hello, Using Hive, we know that we should specify the file path to read data from a specific location. If the data is distributed on many computers, how can we read it? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Distributed data
Yes I mean the data is on hdfs like filesystem From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Tuesday, August 12, 2014 3:26 PM To: user@hive.apache.org Subject: Re: Distributed data what do you mean the data is distributed on many computers? are you saying the data is on hdfs like filesystem ? On Tue, Aug 12, 2014 at 5:51 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Using Hive, we know that we should specify the file path to read data from a specific location. If the data is distributed on many computers, how can we read it? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Distributed data
First of all, thank you, the information is very helpful. Can you please provide me more details about “If your hadoop is setup with same filesystem as hdfs, hive will take care of it “ ? Thanks From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Tuesday, August 12, 2014 3:50 PM To: user@hive.apache.org Subject: Re: Distributed data If your hadoop is setup with same filesystem as hdfs, hive will take care of it If your hdfs is totally different than where the file resides, then you need to get the file from that filesystem and then push it to hive using load if that filesystem supports import/export with tools like sqoop then you can use them as well On Tue, Aug 12, 2014 at 5:58 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Yes I mean the data is on hdfs like filesystem From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Tuesday, August 12, 2014 3:26 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: Distributed data what do you mean the data is distributed on many computers? are you saying the data is on hdfs like filesystem ? On Tue, Aug 12, 2014 at 5:51 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Using Hive, we know that we should specify the file path to read data from a specific location. If the data is distributed on many computers, how can we read it? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Distributed data
Hello, Please explain to me : “If your hadoop is setup with same filesystem as hdfs, hive will take care of it “ From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Tuesday, August 12, 2014 3:50 PM To: user@hive.apache.org Subject: Re: Distributed data If your hadoop is setup with same filesystem as hdfs, hive will take care of it If your hdfs is totally different than where the file resides, then you need to get the file from that filesystem and then push it to hive using load if that filesystem supports import/export with tools like sqoop then you can use them as well On Tue, Aug 12, 2014 at 5:58 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Yes I mean the data is on hdfs like filesystem From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Tuesday, August 12, 2014 3:26 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: Distributed data what do you mean the data is distributed on many computers? are you saying the data is on hdfs like filesystem ? On Tue, Aug 12, 2014 at 5:51 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Using Hive, we know that we should specify the file path to read data from a specific location. If the data is distributed on many computers, how can we read it? Thanks *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
Input
Hello, I am using Hive and trying to read from a txt file. I have an input like the following: string;string;integer. First, I specified that the row fields are delimited by a semi-column. Is it possible to read the integer without the quotations? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Input
I know it will be a complete string. I am trying to “integer” without the quotations. From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Thursday, July 31, 2014 1:28 PM To: user@hive.apache.org Subject: Re: Input if you specified ; as your delimiter then abc will be complete string and not abc only. Take a look at csv fileformat if you want to use proper comma delimited feature On Thu, Jul 31, 2014 at 3:44 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am using Hive and trying to read from a txt file. I have an input like the following: “string”;”string”;”integer”. First, I specified that the row fields are delimited by a semi-column. Is it possible to read the integer without the quotations? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
SELECT specific data
Hello, I am interested in selecting specific data from a source and loading it to a table. For example, if I have 5 columns in my dataset, I want to load 3 columns of it. Is it possible to do it without create a second table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: SELECT specific data
Hello, Thank you for your reply. Consider we have data divided into 5 columns (col1, col2, col3, col4, col5). So I can’t load directly col1, col3 and col5? If I can’t do it directly, can you provide me with an alternate solution? Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 11:37 AM To: user@hive.apache.org Subject: Re: SELECT specific data you mean just by writing query then I think no. But if you want to read only first 3 columns of the data then it would work with just a single table and load data into On Wed, Jul 30, 2014 at 1:47 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in selecting specific data from a source and loading it to a table. For example, if I have 5 columns in my dataset, I want to load 3 columns of it. Is it possible to do it without create a second table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: SELECT specific data
I am only using Hive and hadoop, nothing more. From: Devopam Mittra [mailto:devo...@gmail.com] Sent: Wednesday, July 30, 2014 12:15 PM To: user@hive.apache.org Subject: Re: SELECT specific data Are you using any tool to load data ? If yes, then the ETL tool will provide you such options. If not, then please explore unix file processing/external table route. On Wed, Jul 30, 2014 at 2:09 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Thank you for your reply. Consider we have data divided into 5 columns (col1, col2, col3, col4, col5). So I can’t load directly col1, col3 and col5? If I can’t do it directly, can you provide me with an alternate solution? Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 11:37 AM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data you mean just by writing query then I think no. But if you want to read only first 3 columns of the data then it would work with just a single table and load data into On Wed, Jul 30, 2014 at 1:47 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in selecting specific data from a source and loading it to a table. For example, if I have 5 columns in my dataset, I want to load 3 columns of it. Is it possible to do it without create a second table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Devopam Mittra Life and Relations are not binary *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: SELECT specific data
“With hive, without creating a table with full data, you can do intermediate processing like select only few columns and write into another table”. How can I do this process? Thank you alot! From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 12:37 PM To: user@hive.apache.org Subject: Re: SELECT specific data sorry hit send too soon .. I mean without creating intermediate tables, in hive you can process the file directly On Wed, Jul 30, 2014 at 3:06 PM, Nitin Pawar nitinpawar...@gmail.commailto:nitinpawar...@gmail.com wrote: With hive, without creating a table with full data, you can do intermediate processing like select only few columns and write into another table, If this is something one time then you can take a look at awk or cut commands in linux and generate those files only. On Wed, Jul 30, 2014 at 2:49 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: I am only using Hive and hadoop, nothing more. From: Devopam Mittra [mailto:devo...@gmail.commailto:devo...@gmail.com] Sent: Wednesday, July 30, 2014 12:15 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data Are you using any tool to load data ? If yes, then the ETL tool will provide you such options. If not, then please explore unix file processing/external table route. On Wed, Jul 30, 2014 at 2:09 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Thank you for your reply. Consider we have data divided into 5 columns (col1, col2, col3, col4, col5). So I can’t load directly col1, col3 and col5? If I can’t do it directly, can you provide me with an alternate solution? Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 11:37 AM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data you mean just by writing query then I think no. But if you want to read only first 3 columns of the data then it would work with just a single table and load data into On Wed, Jul 30, 2014 at 1:47 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in selecting specific data from a source and loading it to a table. For example, if I have 5 columns in my dataset, I want to load 3 columns of it. Is it possible to do it without create a second table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Devopam Mittra Life and Relations are not binary *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: SELECT specific data
Thank you very much, your respond was very helpful From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 12:53 PM To: user@hive.apache.org Subject: Re: SELECT specific data Please check another mail i sent right after that. my bad had hit send button too soon without reading the mail. I will rephrase In hive to process the data, you will need the table created and data loaded to the table. You can not process a file without loading it into a table. If you want to do that and do not want to create a temporary table in hive with full columns from file then options available to you are 1) simple unix tools like awk or sed or cut 2) write a pig script 3) write your own mapreduce code On Wed, Jul 30, 2014 at 3:09 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: “With hive, without creating a table with full data, you can do intermediate processing like select only few columns and write into another table”. How can I do this process? Thank you alot! From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 12:37 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data sorry hit send too soon .. I mean without creating intermediate tables, in hive you can process the file directly On Wed, Jul 30, 2014 at 3:06 PM, Nitin Pawar nitinpawar...@gmail.commailto:nitinpawar...@gmail.com wrote: With hive, without creating a table with full data, you can do intermediate processing like select only few columns and write into another table, If this is something one time then you can take a look at awk or cut commands in linux and generate those files only. On Wed, Jul 30, 2014 at 2:49 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: I am only using Hive and hadoop, nothing more. From: Devopam Mittra [mailto:devo...@gmail.commailto:devo...@gmail.com] Sent: Wednesday, July 30, 2014 12:15 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data Are you using any tool to load data ? If yes, then the ETL tool will provide you such options. If not, then please explore unix file processing/external table route. On Wed, Jul 30, 2014 at 2:09 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Thank you for your reply. Consider we have data divided into 5 columns (col1, col2, col3, col4, col5). So I can’t load directly col1, col3 and col5? If I can’t do it directly, can you provide me with an alternate solution? Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 11:37 AM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data you mean just by writing query then I think no. But if you want to read only first 3 columns of the data then it would work with just a single table and load data into On Wed, Jul 30, 2014 at 1:47 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in selecting specific data from a source and loading it to a table. For example, if I have 5 columns in my dataset, I want to load 3 columns of it. Is it possible to do it without create a second table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Devopam Mittra Life and Relations are not binary *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any
RE: SELECT specific data
Thank you From: Devopam Mittra [mailto:devo...@gmail.com] Sent: Wednesday, July 30, 2014 2:57 PM To: user@hive.apache.org Subject: Re: SELECT specific data If you have a defined table, then loading partial columns will be easiest handled with inserting the rest columns with NULL value after mapping your partial column file as an external table. regards Devopam On Wed, Jul 30, 2014 at 2:49 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: I am only using Hive and hadoop, nothing more. From: Devopam Mittra [mailto:devo...@gmail.commailto:devo...@gmail.com] Sent: Wednesday, July 30, 2014 12:15 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data Are you using any tool to load data ? If yes, then the ETL tool will provide you such options. If not, then please explore unix file processing/external table route. On Wed, Jul 30, 2014 at 2:09 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, Thank you for your reply. Consider we have data divided into 5 columns (col1, col2, col3, col4, col5). So I can’t load directly col1, col3 and col5? If I can’t do it directly, can you provide me with an alternate solution? Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.commailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 11:37 AM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: SELECT specific data you mean just by writing query then I think no. But if you want to read only first 3 columns of the data then it would work with just a single table and load data into On Wed, Jul 30, 2014 at 1:47 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in selecting specific data from a source and loading it to a table. For example, if I have 5 columns in my dataset, I want to load 3 columns of it. Is it possible to do it without create a second table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Devopam Mittra Life and Relations are not binary *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Devopam Mittra Life and Relations are not binary *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Hive Data
Till now I don’t have a file. I am willing to search online for a sample dataset that contains at least 1 million rows. If you know any link to a sample file, it would be very much appreciated. Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 3:33 PM To: user@hive.apache.org Subject: Re: Hive Data hive reads the files by the input format defined by the table schema. By default it reads the TextFile in which columns are separated by CTRL+A key if you have a csv file then you can use a csv serde. there are lots of such file formats. what does your file look like? On Wed, Jul 30, 2014 at 5:54 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in testing Hive with a huge sample data. Does Hive read all data types? Should the file be a table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
RE: Hive Data
Thank you From: Devopam Mittra [mailto:devo...@gmail.com] Sent: Wednesday, July 30, 2014 5:54 PM To: user@hive.apache.org Subject: Re: Hive Data You may please give dbpedia dataset a try - I am sure you won't be disappointed :) regards Dev +91 958 305 9899 On Jul 30, 2014, at 6:05 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Till now I don’t have a file. I am willing to search online for a sample dataset that contains at least 1 million rows. If you know any link to a sample file, it would be very much appreciated. Thank you. From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Wednesday, July 30, 2014 3:33 PM To: user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: Hive Data hive reads the files by the input format defined by the table schema. By default it reads the TextFile in which columns are separated by CTRL+A key if you have a csv file then you can use a csv serde. there are lots of such file formats. what does your file look like? On Wed, Jul 30, 2014 at 5:54 PM, CHEBARO Abdallah abdallah.cheb...@murex.commailto:abdallah.cheb...@murex.com wrote: Hello, I am interested in testing Hive with a huge sample data. Does Hive read all data types? Should the file be a table? Thank you *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. -- Nitin Pawar *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system. *** This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
ERROR in JDBC
Hello Hive Community,
I am trying to run the JDBC (from cwiki.apache.org), using HiveServer2.
Everything in the Java code (attached above) runs well except for the last
query: sql = select * from + tableName;
Attached is the complete log file of several runs. I have noticed the following
error:
ERROR mr.ExecDriver (MapRedTask.java:execute(304)) - Exception: Cannot run
program /usr/local/bin/hadoop-2.2.0\bin\hadoop.cmd (in directory
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin): CreateProcess error=2, The
system cannot find the file specified
I have tried to navigate to the following path and manually add the path along
with the hadoop.cmd script but this did not work. So I had {
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin\usr\local\bin\hadoop-2.2.0\bin\hadoop.cmd)
as a path but still it failed to find it.
I have tried to debug the Apache Java code to try and figure out the problem. I
have noticed that TStatusCode.fingByValue returns ERROR_STATUS which was only
the case in the last query mentioned earlier. This flag is eventually checked
and results in throwing the exception below:
Exception in thread main java.sql.SQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:165)
at
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:153)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:300)
at
murex.pop.hadoop.connector.HiveJdbcClient.main(HiveJdbcClient.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
My questions are:
1- Why isn't my Hiveserver2 finding the hadoop.cmd script? In general
everything during the installation was set to default and no extra
configuration is done
2- Why is the exception thrown specifically when trying this query? Do you
have any ideas about what might the glitch be?
Any help would be much appreciated
Abdallah Chebaro
***
This e-mail contains information for the intended recipient only. It may
contain proprietary material or confidential information. If you are not the
intended recipient you are not authorised to distribute, copy or use this
e-mail or any attachment to it. Murex cannot guarantee that it is virus free
and accepts no responsibility for any loss or damage arising from its use. If
you have received this e-mail in error please notify immediately the sender and
delete the original email received, any attachments and all copies from your
system.
HiveJdbcClient .java
Description: HiveJdbcClient .java
hive.log
Description: hive.log
ERROR in JDBC
Hello Hive Community,
I am trying to run the JDBC (from cwiki.apache.org), using HiveServer2.
Everything in the Java code (attached above) runs well except for the last
query: sql = select * from + tableName;
Attached is the complete log file of several runs. I have noticed the following
error:
ERROR mr.ExecDriver (MapRedTask.java:execute(304)) - Exception: Cannot run
program /usr/local/bin/hadoop-2.2.0\bin\hadoop.cmd (in directory
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin): CreateProcess error=2, The
system cannot find the file specified
I have tried to navigate to the following path and manually add the path along
with the hadoop.cmd script but this did not work. So I had {
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin\usr\local\bin\hadoop-2.2.0\bin\hadoop.cmd)
as a path but still it failed to find it.
I have tried to debug the Apache Java code to try and figure out the problem. I
have noticed that TStatusCode.fingByValue returns ERROR_STATUS which was only
the case in the last query mentioned earlier. This flag is eventually checked
and results in throwing the exception below:
Exception in thread main java.sql.SQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:165)
at
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:153)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:300)
at
murex.pop.hadoop.connector.HiveJdbcClient.main(HiveJdbcClient.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
My questions are:
1- Why isn't my Hiveserver2 finding the hadoop.cmd script? In general
everything during the installation was set to default and no extra
configuration is done
2- Why is the exception thrown specifically when trying this query? Do you
have any ideas about what might the glitch be?
Any help would be much appreciated
Abdallah Chebaro
***
This e-mail contains information for the intended recipient only. It may
contain proprietary material or confidential information. If you are not the
intended recipient you are not authorised to distribute, copy or use this
e-mail or any attachment to it. Murex cannot guarantee that it is virus free
and accepts no responsibility for any loss or damage arising from its use. If
you have received this e-mail in error please notify immediately the sender and
delete the original email received, any attachments and all copies from your
system.
HiveJdbcClient .java
Description: HiveJdbcClient .java
hive.log
Description: hive.log
ERROR in JDBC
Hello Hive Community,
I am trying to run the JDBC (from cwiki.apache.org), using HiveServer2.
Everything in the Java code (attached above) runs well except for the last
query: sql = select * from + tableName;
Attached is the complete log file of several runs. I have noticed the following
error:
ERROR mr.ExecDriver (MapRedTask.java:execute(304)) - Exception: Cannot run
program /usr/local/bin/hadoop-2.2.0\bin\hadoop.cmd (in directory
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin): CreateProcess error=2, The
system cannot find the file specified
I have tried to navigate to the following path and manually add the path along
with the hadoop.cmd script but this did not work. So I had {
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin\usr\local\bin\hadoop-2.2.0\bin\hadoop.cmd)
as a path but still it failed to find it.
I have tried to debug the Apache Java code to try and figure out the problem. I
have noticed that TStatusCode.fingByValue returns ERROR_STATUS which was only
the case in the last query mentioned earlier. This flag is eventually checked
and results in throwing the exception below:
Exception in thread main java.sql.SQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:165)
at
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:153)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:300)
at
murex.pop.hadoop.connector.HiveJdbcClient.main(HiveJdbcClient.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
My questions are:
1- Why isn't my Hiveserver2 finding the hadoop.cmd script? In general
everything during the installation was set to default and no extra
configuration is done
2- Why is the exception thrown specifically when trying this query? Do you
have any ideas about what might the glitch be?
Any help would be much appreciated
Abdallah Chebaro
***
This e-mail contains information for the intended recipient only. It may
contain proprietary material or confidential information. If you are not the
intended recipient you are not authorised to distribute, copy or use this
e-mail or any attachment to it. Murex cannot guarantee that it is virus free
and accepts no responsibility for any loss or damage arising from its use. If
you have received this e-mail in error please notify immediately the sender and
delete the original email received, any attachments and all copies from your
system.
HiveJdbcClient .java
Description: HiveJdbcClient .java
hive.log
Description: hive.log
ERROR in JDBC
Hello Hive Community,
I am trying to run the JDBC (from cwiki.apache.org), using HiveServer2.
Everything in the Java code (attached above) runs well except for the last
query: sql = select * from + tableName;
Attached is the complete log file of several runs. I have noticed the following
error:
ERROR mr.ExecDriver (MapRedTask.java:execute(304)) - Exception: Cannot run
program /usr/local/bin/hadoop-2.2.0\bin\hadoop.cmd (in directory
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin): CreateProcess error=2, The
system cannot find the file specified
I have tried to navigate to the following path and manually add the path along
with the hadoop.cmd script but this did not work. So I had {
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin\usr\local\bin\hadoop-2.2.0\bin\hadoop.cmd)
as a path but still it failed to find it.
I have tried to debug the Apache Java code to try and figure out the problem. I
have noticed that TStatusCode.fingByValue returns ERROR_STATUS which was only
the case in the last query mentioned earlier. This flag is eventually checked
and results in throwing the exception below:
Exception in thread main java.sql.SQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:165)
at
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:153)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:300)
at
murex.pop.hadoop.connector.HiveJdbcClient.main(HiveJdbcClient.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
My questions are:
1- Why isn't my Hiveserver2 finding the hadoop.cmd script? In general
everything during the installation was set to default and no extra
configuration is done
2- Why is the exception thrown specifically when trying this query? Do you
have any ideas about what might the glitch be?
Any help would be much appreciated
Abdallah Chebaro
***
This e-mail contains information for the intended recipient only. It may
contain proprietary material or confidential information. If you are not the
intended recipient you are not authorised to distribute, copy or use this
e-mail or any attachment to it. Murex cannot guarantee that it is virus free
and accepts no responsibility for any loss or damage arising from its use. If
you have received this e-mail in error please notify immediately the sender and
delete the original email received, any attachments and all copies from your
system.
HiveJdbcClient .java
Description: HiveJdbcClient .java
hive.log
Description: hive.log
ERROR in JDBC
Hello Hive Community,
I am trying to run the JDBC (from cwiki.apache.org), using HiveServer2.
Everything in the Java code (attached above) runs well except for the last
query: sql = select * from + tableName;
Attached is the complete log file of several runs. I have noticed the following
error:
ERROR mr.ExecDriver (MapRedTask.java:execute(304)) - Exception: Cannot run
program /usr/local/bin/hadoop-2.2.0\bin\hadoop.cmd (in directory
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin): CreateProcess error=2, The
system cannot find the file specified
I have tried to navigate to the following path and manually add the path along
with the hadoop.cmd script but this did not work. So I had {
C:\cygwin64\usr\local\bin\hive-0.12.0-bin\bin\usr\local\bin\hadoop-2.2.0\bin\hadoop.cmd)
as a path but still it failed to find it.
I have tried to debug the Apache Java code to try and figure out the problem. I
have noticed that TStatusCode.fingByValue returns ERROR_STATUS which was only
the case in the last query mentioned earlier. This flag is eventually checked
and results in throwing the exception below:
Exception in thread main java.sql.SQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:165)
at
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:153)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:300)
at
murex.pop.hadoop.connector.HiveJdbcClient.main(HiveJdbcClient.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
My questions are:
1- Why isn't my Hiveserver2 finding the hadoop.cmd script? In general
everything during the installation was set to default and no extra
configuration is done
2- Why is the exception thrown specifically when trying this query? Do you
have any ideas about what might the glitch be?
Any help would be much appreciated
Abdallah Chebaro
***
This e-mail contains information for the intended recipient only. It may
contain proprietary material or confidential information. If you are not the
intended recipient you are not authorised to distribute, copy or use this
e-mail or any attachment to it. Murex cannot guarantee that it is virus free
and accepts no responsibility for any loss or damage arising from its use. If
you have received this e-mail in error please notify immediately the sender and
delete the original email received, any attachments and all copies from your
system.
HiveJdbcClient .java
Description: HiveJdbcClient .java
hive.log
Description: hive.log
