[jira] [Created] (HIVE-14543) Create Druid table without specifying data source

2016-08-16 Thread Jesus Camacho Rodriguez (JIRA) 
Jesus Camacho Rodriguez created HIVE-14543:
--

 Summary: Create Druid table without specifying data source
 Key: HIVE-14543
 URL: https://issues.apache.org/jira/browse/HIVE-14543
 Project: Hive
  Issue Type: Sub-task
  Components: Druid integration
Affects Versions: 2.2.0
Reporter: Jesus Camacho Rodriguez


We should be able to omit the Druid datasource from the TBLPROPERTIES. In that 
case, the Druid datasource name should match the Hive table name.

{code:sql}
CREATE TABLE druid_table_1
STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler';
TBLPROPERTIES ("druid.address" = "localhost");
{code}

For instance, the statement above creates a table that references the Druid 
datasource "druid_table_1".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14544) LOAD DATA statement appends .0 to the partition name

2016-08-16 Thread Oleksiy Sayankin (JIRA) 
Oleksiy Sayankin created HIVE-14544:
---

 Summary: LOAD DATA statement appends .0 to the partition name
 Key: HIVE-14544
 URL: https://issues.apache.org/jira/browse/HIVE-14544
 Project: Hive
  Issue Type: Bug
Reporter: Oleksiy Sayankin
Assignee: Oleksiy Sayankin


*STEP 1. Create file with data:*

{noformat}
echo 1 > /tmp/data.file
{noformat}

*STEP 2. Create table in hive:*

{noformat}
CREATE TABLE `issue` (`id` INT) PARTITIONED BY (`ts` TIMESTAMP);
{noformat}

*STEP 3. Insert data into table:*

{noformat}
SET hive.exec.dynamic.partition.mode=nonstrict;
INSERT INTO TABLE `issue` PARTITION (`ts`) VALUES (1,'1970-01-01 
00:00:00'),(2,'1980-01-01 00:00:00'),(3,'1990-01-01 00:00:00');
{noformat}

*STEP 4. Load data into table using hive:*

{noformat}
LOAD DATA LOCAL INPATH '/tmp/data.file' OVERWRITE INTO TABLE `issue` PARTITION 
(`ts`='2000-01-01 00:00:00');
{noformat}

*STEP 5. Run show partitions query:*

{noformat}
SHOW PARTITIONS `issue`;
{noformat}

*EXPECTED RESULT:*

{noformat}
ts=1970-01-01 00%3A00%3A00
ts=1980-01-01 00%3A00%3A00
ts=1990-01-01 00%3A00%3A00
ts=2000-01-01 00%3A00%3A00
{noformat}
*ACTUAL RESULT*

We've gotten partitions with different precision
{noformat}
ts=1970-01-01 00%3A00%3A00
ts=1980-01-01 00%3A00%3A00
ts=1990-01-01 00%3A00%3A00
ts=2000-01-01 00%3A00%3A00.0
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: YourKit open source license

2016-08-16 Thread calvin hung 


Thanks for your response, Rui.

I don't have an apache email account. 

It looks like only committer can get an email account according to this page 
http://www.apache.org/dev/committers.html

Does it mean that only Hive committers can get YourKit free licenses for Hive 
performance profiling?





 On Tue, 16 Aug 2016 13:33:34 +0800 Rui Li 
wrote  




If I remember correctly, I just contacted the sales of Yourkit and they 

sent me the license by email. You'd better send your email using your 

apache email account, in order to convince them you're a developer of Hive. 

 

On Tue, Aug 16, 2016 at 2:51 AM, calvin hung  

wrote: 

 

> Hi Rui and Alan, 

> 

> Could you or any nice guy share more detail steps of getting a Yourkit 

> license for Hive? 

> I've searched the full Hive dev mail archive but got no exact steps to get 

> one. 

> Thanks! 

> 

> Calvin 

> From: "Li, Rui" 

> Date: Tue, 31 Mar 2015 01:22:51 + 

> To: "dev@hive.apache.org" 

> 

> - Contents - 

> 

> Thanks Alan! But I don’t see Hive in the sponsored open source project 

> list. I’ll contact them anyway. 

> 

> 

> 

> Cheers, 

> 

> Rui Li 

> 

> 

> 

> From: Alan Gates [mailto:alanfga...@gmail.com] 

> Sent: Tuesday, March 31, 2015 1:02 AM 

> To: dev@hive.apache.org 

> Subject: Re: YourKit open source license 

> 

> 

> 

> Seehttps://www.yourkit.com/customers/. 

> 

> Alan. 

> 

> 

> 

> 

> 

> Li, Rui 

> 

> March 30, 2015 at 0:54 

> 

> Hi guys, 

> 

> I want to use YourKit to profile hive performance. According to the 
wiki< 

> https://cwiki.apache.org/confluence/display/Hive/Performance>; hive has 

> been granted open source license. Could anybody tell me how I can get the 

> license? Thanks! 

> 

> Cheers, 

> Rui Li 

 

 

 

 

-- 

Best regards! 

Rui Li 

Cell: (+86) 13564950210

[jira] [Created] (HIVE-14545) HiveServer2 with http transport mode spending too much time just creating configs

2016-08-16 Thread Rajesh Balamohan (JIRA) 
Rajesh Balamohan created HIVE-14545:
---

 Summary: HiveServer2 with http transport mode spending too much 
time just creating configs
 Key: HIVE-14545
 URL: https://issues.apache.org/jira/browse/HIVE-14545
 Project: Hive
  Issue Type: Improvement
  Components: HiveServer2
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan
Priority: Minor


Env: Hive master with LLAP with Q-55 running via beeline mode continuously.  
Query itself completes in 0.6 - 0.7 seconds. HiveServer2 is configured with  
{{hive.server2.transport.mode=http}} and 
{{hive.server2.long.polling.timeout=1000}}
In such cases, HiverServer2 consistently runs with high CPU usage and profiling 
shows spending too much time just for creating hiveConf.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14546) TestMiniTezCliDriver plan diffs

2016-08-16 Thread Hari Sankar Sivarama Subramaniyan (JIRA) 
Hari Sankar Sivarama Subramaniyan created HIVE-14546:


 Summary: TestMiniTezCliDriver plan diffs
 Key: HIVE-14546
 URL: https://issues.apache.org/jira/browse/HIVE-14546
 Project: Hive
  Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan


The following tests are failing consistently for the last some days in 
precommit runs:
{code}
 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]   
56 sec  5
 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]   
35 sec  19
 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]   
58 sec  19
 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]  
2.8 sec 19
 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]  
2.4 sec 19
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14547) [Umbrella] Test case fixes / improvements

2016-08-16 Thread Siddharth Seth (JIRA) 
Siddharth Seth created HIVE-14547:
-

 Summary: [Umbrella] Test case fixes / improvements
 Key: HIVE-14547
 URL: https://issues.apache.org/jira/browse/HIVE-14547
 Project: Hive
  Issue Type: Task
Reporter: Siddharth Seth


Splitting HIVE-13503. HIVE-13503 tracks general test infrastructure 
improvements. This one tracks specific test fixes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14548) Add a copy of the build configuration file to the hive repo

2016-08-16 Thread Siddharth Seth (JIRA) 
Siddharth Seth created HIVE-14548:
-

 Summary: Add a copy of the build configuration file to the hive 
repo
 Key: HIVE-14548
 URL: https://issues.apache.org/jira/browse/HIVE-14548
 Project: Hive
  Issue Type: Task
Reporter: Siddharth Seth
Assignee: Siddharth Seth


The property files which drive a ptest run are present on the build box. These 
should be somewhere in the repo - at least as a reference.

Ideally, there should be a way to pull the files from the branch, and have the 
test specific data like host names, passwords picked up from the host running 
ptest.

At the moment, at least check in the files as reference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14549) HiveServer2: Improve HiveServer2 JDBC/ODBC ResultSet performance - part2

2016-08-16 Thread Vaibhav Gumashta (JIRA) 
Vaibhav Gumashta created HIVE-14549:
---

 Summary: HiveServer2: Improve HiveServer2 JDBC/ODBC ResultSet 
performance - part2
 Key: HIVE-14549
 URL: https://issues.apache.org/jira/browse/HIVE-14549
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, JDBC, ODBC
Reporter: Vaibhav Gumashta






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14550) HiveServer2: enable ThriftJDBCBinarySerde use by default

2016-08-16 Thread Vaibhav Gumashta (JIRA) 
Vaibhav Gumashta created HIVE-14550:
---

 Summary: HiveServer2: enable ThriftJDBCBinarySerde use by default
 Key: HIVE-14550
 URL: https://issues.apache.org/jira/browse/HIVE-14550
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2, JDBC, ODBC
Affects Versions: 2.1.0
Reporter: Vaibhav Gumashta


We've covered all items in HIVE-12427 and created HIVE-14549 for part2 of the 
effort. Before closing the umbrella jira, we should enable this feature by 
default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14551) HiveServer2: Use vectorized data whenever available for writing final results

2016-08-16 Thread Vaibhav Gumashta (JIRA) 
Vaibhav Gumashta created HIVE-14551:
---

 Summary: HiveServer2: Use vectorized data whenever available for 
writing final results 
 Key: HIVE-14551
 URL: https://issues.apache.org/jira/browse/HIVE-14551
 Project: Hive
  Issue Type: Sub-task
  Components: HiveServer2
Affects Versions: 2.1.0
Reporter: Vaibhav Gumashta


In ThriftJDBCBinarySerde, which we are using in FileSinkOperator to write final 
results, we buffer rows and store them into typed columns before writing a 
batch of rows to the result file. However, when vectorized rows batches are 
available from higher level operators, we should try to use them and avoid the 
extra penalty of converting from vector --> non-vector single row --> buffered 
thrift columns (equivalent to vector).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14552) TestParseNegative fix

2016-08-16 Thread Hari Sankar Sivarama Subramaniyan (JIRA) 
Hari Sankar Sivarama Subramaniyan created HIVE-14552:


 Summary: TestParseNegative fix
 Key: HIVE-14552
 URL: https://issues.apache.org/jira/browse/HIVE-14552
 Project: Hive
  Issue Type: Sub-task
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan


1300s runtime.
Straggler towards the end of the build.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14553) Remove tez golden files after HIVE-14502

2016-08-16 Thread Prasanth Jayachandran (JIRA) 
Prasanth Jayachandran created HIVE-14553:


 Summary: Remove tez golden files after HIVE-14502
 Key: HIVE-14553
 URL: https://issues.apache.org/jira/browse/HIVE-14553
 Project: Hive
  Issue Type: Sub-task
  Components: Test
Affects Versions: 2.2.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


Follow up for HIVE-14502 to remove all the tez golden files that are not 
required anymore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: YourKit open source license

2016-08-16 Thread Rui Li 
Our wiki doesn't mention it's only for committers. Anyway I suggest you
contact YourKit sales to figure out.

On Tue, Aug 16, 2016 at 8:38 PM, calvin hung 
wrote:

>
>
> Thanks for your response, Rui.
>
> I don't have an apache email account.
>
> It looks like only committer can get an email account according to this
> page http://www.apache.org/dev/committers.html
>
> Does it mean that only Hive committers can get YourKit free licenses for
> Hive performance profiling?
>
>
>
>
>
>  On Tue, 16 Aug 2016 13:33:34 +0800 Rui Li  >wrote 
>
>
>
>
> If I remember correctly, I just contacted the sales of Yourkit and they
>
> sent me the license by email. You'd better send your email using your
>
> apache email account, in order to convince them you're a developer of Hive.
>
>
>
> On Tue, Aug 16, 2016 at 2:51 AM, calvin hung  gt;
>
> wrote:
>
>
>
> > Hi Rui and Alan,
>
> >
>
> > Could you or any nice guy share more detail steps of getting a Yourkit
>
> > license for Hive?
>
> > I've searched the full Hive dev mail archive but got no exact steps
> to get
>
> > one.
>
> > Thanks!
>
> >
>
> > Calvin
>
> > From: "Li, Rui"
>
> > Date: Tue, 31 Mar 2015 01:22:51 +
>
> > To: "dev@hive.apache.org"
>
> >
>
> > - Contents -
>
> >
>
> > Thanks Alan! But I don’t see Hive in the sponsored open source project
>
> > list. I’ll contact them anyway.
>
> >
>
> >
>
> >
>
> > Cheers,
>
> >
>
> > Rui Li
>
> >
>
> >
>
> >
>
> > From: Alan Gates [mailto:alanfga...@gmail.com]
>
> > Sent: Tuesday, March 31, 2015 1:02 AM
>
> > To: dev@hive.apache.org
>
> > Subject: Re: YourKit open source license
>
> >
>
> >
>
> >
>
> > Seehttps://www.yourkit.com/customers/.
>
> >
>
> > Alan.
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > Li, Rui
>
> >
>
> > March 30, 2015 at 0:54
>
> >
>
> > Hi guys,
>
> >
>
> > I want to use YourKit to profile hive performance. According to the
> wiki<
>
> > https://cwiki.apache.org/confluence/display/Hive/Performance>;
> hive has
>
> > been granted open source license. Could anybody tell me how I can get
> the
>
> > license? Thanks!
>
> >
>
> > Cheers,
>
> > Rui Li
>
>
>
>
>
>
>
>
>
> --
>
> Best regards!
>
> Rui Li
>
> Cell: (+86) 13564950210
>
>
>
>
>
>
>


-- 
Best regards!
Rui Li
Cell: (+86) 13564950210

[jira] [Created] (HIVE-14554) Hive ptest should delete the itests/thirdparty directory everytime it builds hive

2016-08-16 Thread JIRA 
Sergio Peña created HIVE-14554:
--

 Summary: Hive ptest should delete the itests/thirdparty directory 
everytime it builds hive
 Key: HIVE-14554
 URL: https://issues.apache.org/jira/browse/HIVE-14554
 Project: Hive
  Issue Type: Task
  Components: Testing Infrastructure
Reporter: Sergio Peña


The {{itests/thridparty}} directory is created by hive on spark when 
downloading the spark-assembly file. Hive ptest should delete this directory 
everytime it runs a new set of tests to avoid conflicts when a new spark 
tarball is submitted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14555) JDBC:ClassNotFoundException when executing a map join query with UDF

2016-08-16 Thread hizero (JIRA) 
hizero created HIVE-14555:
-

 Summary: JDBC:ClassNotFoundException when executing a map join 
query with UDF
 Key: HIVE-14555
 URL: https://issues.apache.org/jira/browse/HIVE-14555
 Project: Hive
  Issue Type: Bug
  Components: Query Planning
Affects Versions: 1.1.0
Reporter: hizero
Assignee: hizero
 Fix For: 1.1.0


when I submit a map join query with UDF using JDBC  and sometimes it throws:
Error while compiling statement: FAILED: SemanticException Generate Map Join 
Task Error: Unable to find class: com.kingnetdc.hive.udf.FilterByMap 
Serialization trace: genericUDF 
(org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) colExprMap 
(org.apache.hadoop.hive.ql.exec.SelectOperator) childOperators 
(org.apache.hadoop.hive.ql.exec.FilterOperator) childOperators 
(org.apache.hadoop.hive.ql.exec.JoinOperator) reducer 
(org.apache.hadoop.hive.ql.plan.ReduceWork) reduceWork 
(org.apache.hadoop.hive.ql.plan.MapredWork)

 I have found the fact that it fails at cloning plan when invoking 
Utilities.deserializePlan.
An existing thread deals with the query and its static threadlocal 
variable,cloningQueryPlanKryo has been initialed at most once per thread.When 
this thread registered UDF setting in aux_jar_paths  it wont reinitialize the 
cloningQueryPlanKryo.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)