Re: column names from Object Inspector in serialize() method of custom serde

[Ashutosh Chauhan]

Thanks John and Arvind. Your explanation make sense. Early days of
getting used to "hive way" of doing things in the world of serdes,
storage-handler, meta-hook, object-inspectors etc. :)

Ashutosh

On Thu, May 27, 2010 at 10:56, Arvind Prabhakar  wrote:
> John, Ashutosh,
>
> I agree with John's evaluation on this. Consider the case of writing to a
> partition of a table. Clearly, the columns being written to will not be the
> same as what are defined in the metadata for the entire table. Moreover,
> there are cases where intermediate tables (files) may be produced during a
> particular operation which are not defined by the user. In such cases you
> are dealing with either a subset of columns of a table or columns of an
> intermediate transient table. And since Struct OIs insist on having names
> for fields, it follows that to cover the general case we can use any unique
> names where necessary.
>
> The actual data pipeline underneath the Hive query is already semantically
> verified to fit the appropriate type definitions and hence adding the column
> names would not add any value to the runtime. It will add to the overall
> processing overhead.
>
> Arvind
>
> On Wed, May 26, 2010 at 6:29 PM, John Sichi  wrote:
>
>> Hey Ashutosh,
>>
>> You're right, currently the target table column names come in via
>> initialize in the Properties parameter, e.g.
>> props.getProperty(Constants.LIST_COLUMNS), whereas the object inspector gets
>> _col1, _col2, _col3.  (And of course, if you have a custom mapping string
>> like HBase, then that comes in through the initialize Properties parameter
>> via your own private property name.)
>>
>> I haven't looked into the details of why this is, but probably the object
>> inspector references an internally produced row from whatever was upstream
>> (rather than being derived from the target table itself, although the number
>> of columns has to match).  I'm not sure this is a bug per se, just something
>> to be aware of.  In general, you should try to precompute any data
>> structures needed during initialize so that serialize can be as lean as
>> possible, meaning you probably don't want to be looking at the field names
>> in there anyway.
>>
>> Opinions from other hive devs?
>>
>> JVS
>>
>> On May 21, 2010, at 12:22 PM, Ashutosh Chauhan wrote:
>>
>> > Hi,
>> >
>> > I am writing my own custom serde to write data to an external table.
>> > In serialize() method of my serde I am handed over an object and an
>> > object Inspector. Since this object represents a row, I make an
>> > assumption that object Inspector is of type StructObjectInspector and
>> > then I get fields out of this struct using struct Object inspector.
>> > When I do field.getFieldName() on it I expect it will give me the real
>> > column name as contained in my table schema in metastore. But, instead
>> > I get names like _col1, _col2, _col3 ..
>> >
>> > Now the workaround for it is to store the column names in a list in
>> > initialize() method and then use that list to get names in
>> > serialize(). This is what I am doing now and it works. It seems hbase
>> > serde is also doing similar thing. But, it was counter intuitive to me
>> > not to expect to get the real column names in getFieldName() but
>> > rather some random made up names. If this is not the expected behavior
>> > then potentially I am doing something wrong in my serde.. if so I will
>> > appreciate if some one confirms that.. But if this is how things are
>> > implemented currently.. then I think its a bug and I will open a jira
>> > for it..
>> >
>> > Thanks,
>> > Ashutosh
>> >
>> > PS: I am posting it on dev-list But if folks think its more
>> > appropriate for user-list, feel free to move it there, while replying
>> > to it.
>>
>>
>

[jira] Created: (HIVE-1375) dynamic partitions should not create some of the partitions if the query fails

[Namit Jain (JIRA)]

dynamic partitions should not create some of the partitions if the query fails
--

 Key: HIVE-1375
 URL: https://issues.apache.org/jira/browse/HIVE-1375
 Project: Hadoop Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Ning Zhang
 Fix For: 0.6.0


Currently, if a bad row exists, which cannot be part of a partitioning column, 
it fails - but some of the partitions may already have been created

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1335) DataNucleus should use connection pooling

[Paul Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872379#action_12872379
 ] 

Paul Yang commented on HIVE-1335:
-

This patch makes ant 1.7.1 or higher required. I'll send you an email on how to 
update our internal build.

> DataNucleus should use connection pooling
> -
>
> Key: HIVE-1335
> URL: https://issues.apache.org/jira/browse/HIVE-1335
> Project: Hadoop Hive
>  Issue Type: Improvement
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: commons-dbcp-1.2.2.jar, commons-dbcp.LICENSE, 
> commons-pool-1.2.jar, commons-pool.LICENSE, 
> datanucleus-connectionpool-1.0.2.jar, datanucleus-connectionpool.LICENSE, 
> hive-1335-1.patch.txt, hive-1335-2.patch.txt, hive-1335-3.patch.txt, 
> hive-1335.patch.txt
>
>
> Currently each Data Nucleus operation disconnects and reconnects to the 
> MetaStore over jdbc. Queries fail to even explain properly in cases where a 
> table has many partitions. This is fixed by enabling one parameter and 
> including several jars.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1335) DataNucleus should use connection pooling

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872378#action_12872378
 ] 

John Sichi commented on HIVE-1335:
--

@Yongqiang:  Per Paul's instructions, you need to upgrade to ant 1.8.1


> DataNucleus should use connection pooling
> -
>
> Key: HIVE-1335
> URL: https://issues.apache.org/jira/browse/HIVE-1335
> Project: Hadoop Hive
>  Issue Type: Improvement
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: commons-dbcp-1.2.2.jar, commons-dbcp.LICENSE, 
> commons-pool-1.2.jar, commons-pool.LICENSE, 
> datanucleus-connectionpool-1.0.2.jar, datanucleus-connectionpool.LICENSE, 
> hive-1335-1.patch.txt, hive-1335-2.patch.txt, hive-1335-3.patch.txt, 
> hive-1335.patch.txt
>
>
> Currently each Data Nucleus operation disconnects and reconnects to the 
> MetaStore over jdbc. Queries fail to even explain properly in cases where a 
> table has many partitions. This is fixed by enabling one parameter and 
> including several jars.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1335) DataNucleus should use connection pooling

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872376#action_12872376
 ] 

He Yongqiang commented on HIVE-1335:


It seems this introduced an error msg?


heyongqiangMBP:Hive-Index heyongqiang$ ant test -Dtestcase=TestCliDriver
Buildfile: build.xml

BUILD FAILED
/Users/heyongqiang/Documents/workspace/Hive-Index/build.xml:112: fileset 
doesn't support the "erroronmissingdir" attribute



> DataNucleus should use connection pooling
> -
>
> Key: HIVE-1335
> URL: https://issues.apache.org/jira/browse/HIVE-1335
> Project: Hadoop Hive
>  Issue Type: Improvement
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: commons-dbcp-1.2.2.jar, commons-dbcp.LICENSE, 
> commons-pool-1.2.jar, commons-pool.LICENSE, 
> datanucleus-connectionpool-1.0.2.jar, datanucleus-connectionpool.LICENSE, 
> hive-1335-1.patch.txt, hive-1335-2.patch.txt, hive-1335-3.patch.txt, 
> hive-1335.patch.txt
>
>
> Currently each Data Nucleus operation disconnects and reconnects to the 
> MetaStore over jdbc. Queries fail to even explain properly in cases where a 
> table has many partitions. This is fixed by enabling one parameter and 
> including several jars.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1374) Query compile-only option

[Paul Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Yang updated HIVE-1374:


Description: A compile-only option might be useful for helping users 
quickly prototype queries, fix errors, and do test runs. The proposed change 
would be adding a -c switch that behaves like -e but only compiles the 
specified query.  (was: A compile-only option might be useful for helping users 
quickly prototype queries and fix syntax errors. Possible improvements could be 
adding a -c switch that behaves like -e but only compiles the specified query, 
and a way to switch between compile-only and compile-and-execute modes in the 
interactive CLI.)

> Query compile-only option
> -
>
> Key: HIVE-1374
> URL: https://issues.apache.org/jira/browse/HIVE-1374
> Project: Hadoop Hive
>  Issue Type: New Feature
>Affects Versions: 0.6.0
>Reporter: Paul Yang
>Assignee: Paul Yang
>
> A compile-only option might be useful for helping users quickly prototype 
> queries, fix errors, and do test runs. The proposed change would be adding a 
> -c switch that behaves like -e but only compiles the specified query.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1374) Query compile-only option

[Paul Yang (JIRA)]

Query compile-only option
-

 Key: HIVE-1374
 URL: https://issues.apache.org/jira/browse/HIVE-1374
 Project: Hadoop Hive
  Issue Type: New Feature
Affects Versions: 0.6.0
Reporter: Paul Yang
Assignee: Paul Yang


A compile-only option might be useful for helping users quickly prototype 
queries and fix syntax errors. Possible improvements could be adding a -c 
switch that behaves like -e but only compiles the specified query, and a way to 
switch between compile-only and compile-and-execute modes in the interactive 
CLI.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1373) Missing connection pool plugin in Eclipse classpath

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872337#action_12872337
 ] 

Edward Capriolo commented on HIVE-1373:
---

I was thinking to move everything that came from ivy to build lib. I see the 
benefit, but I saw this technique adding more copies and moves into the ant 
process. Try different approaches. I  found none of them were better then the 
next. All involved doing more work hear and less there, or changing this 
classpath insteasd of putting a file into X folder. I was kinda confused on the 
"best way" to handle that. I would be interested to swee what you come up with.

> Missing connection pool plugin in Eclipse classpath
> ---
>
> Key: HIVE-1373
> URL: https://issues.apache.org/jira/browse/HIVE-1373
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
> Environment: Eclipse, Linux
>Reporter: Vinithra Varadharajan
>Assignee: Vinithra Varadharajan
>Priority: Minor
> Attachments: HIVE-1373.patch
>
>
> In a recent checkin, connection pool dependency was introduced but eclipse 
> .classpath file was not updated.  This causes launch configurations from 
> within Eclipse to fail.
> {code}
> hive> show tables;
> show tables;
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parsing command: show tables
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parse Completed
> 10/05/26 14:59:46 INFO ql.Driver: Semantic Analysis Completed
> 10/05/26 14:59:46 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from 
> deserializer)], properties:null)
> 10/05/26 14:59:46 INFO ql.Driver: query plan = 
> file:/tmp/vinithra/hive_2010-05-26_14-59-46_058_1636674338194744357/queryplan.xml
> 10/05/26 14:59:46 INFO ql.Driver: Starting command: show tables
> 10/05/26 14:59:46 INFO metastore.HiveMetaStore: 0: Opening raw store with 
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/05/26 14:59:46 INFO metastore.ObjectStore: ObjectStore, initialize called
> FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Error 
> creating transactional connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> 10/05/26 14:59:47 ERROR exec.DDLTask: FAILED: Error in metadata: 
> javax.jdo.JDOFatalInternalException: Error creating transactional connection 
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> javax.jdo.JDOFatalInternalException: Error creating transactional connection 
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
>   at org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:491)
>   at 
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:472)
>   at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:458)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:504)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:176)
>   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>   at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
> Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional 
> connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
>   at 
> org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:395)
>   at 
> org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:547)
>   at 
> org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:175)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
>   at 
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImp

Build failed in Hudson: Hive-trunk-h0.20 #276

[Apache Hudson Server]

See 

Changes:

[namit] HIVE-1371. Bug in rcfilecat
(He Yongqiang via namit)

--
[...truncated 14236 lines...]
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2

Re: column names from Object Inspector in serialize() method of custom serde

[Arvind Prabhakar]

John, Ashutosh,

I agree with John's evaluation on this. Consider the case of writing to a
partition of a table. Clearly, the columns being written to will not be the
same as what are defined in the metadata for the entire table. Moreover,
there are cases where intermediate tables (files) may be produced during a
particular operation which are not defined by the user. In such cases you
are dealing with either a subset of columns of a table or columns of an
intermediate transient table. And since Struct OIs insist on having names
for fields, it follows that to cover the general case we can use any unique
names where necessary.

The actual data pipeline underneath the Hive query is already semantically
verified to fit the appropriate type definitions and hence adding the column
names would not add any value to the runtime. It will add to the overall
processing overhead.

Arvind

On Wed, May 26, 2010 at 6:29 PM, John Sichi  wrote:

> Hey Ashutosh,
>
> You're right, currently the target table column names come in via
> initialize in the Properties parameter, e.g.
> props.getProperty(Constants.LIST_COLUMNS), whereas the object inspector gets
> _col1, _col2, _col3.  (And of course, if you have a custom mapping string
> like HBase, then that comes in through the initialize Properties parameter
> via your own private property name.)
>
> I haven't looked into the details of why this is, but probably the object
> inspector references an internally produced row from whatever was upstream
> (rather than being derived from the target table itself, although the number
> of columns has to match).  I'm not sure this is a bug per se, just something
> to be aware of.  In general, you should try to precompute any data
> structures needed during initialize so that serialize can be as lean as
> possible, meaning you probably don't want to be looking at the field names
> in there anyway.
>
> Opinions from other hive devs?
>
> JVS
>
> On May 21, 2010, at 12:22 PM, Ashutosh Chauhan wrote:
>
> > Hi,
> >
> > I am writing my own custom serde to write data to an external table.
> > In serialize() method of my serde I am handed over an object and an
> > object Inspector. Since this object represents a row, I make an
> > assumption that object Inspector is of type StructObjectInspector and
> > then I get fields out of this struct using struct Object inspector.
> > When I do field.getFieldName() on it I expect it will give me the real
> > column name as contained in my table schema in metastore. But, instead
> > I get names like _col1, _col2, _col3 ..
> >
> > Now the workaround for it is to store the column names in a list in
> > initialize() method and then use that list to get names in
> > serialize(). This is what I am doing now and it works. It seems hbase
> > serde is also doing similar thing. But, it was counter intuitive to me
> > not to expect to get the real column names in getFieldName() but
> > rather some random made up names. If this is not the expected behavior
> > then potentially I am doing something wrong in my serde.. if so I will
> > appreciate if some one confirms that.. But if this is how things are
> > implemented currently.. then I think its a bug and I will open a jira
> > for it..
> >
> > Thanks,
> > Ashutosh
> >
> > PS: I am posting it on dev-list But if folks think its more
> > appropriate for user-list, feel free to move it there, while replying
> > to it.
>
>

[jira] Assigned: (HIVE-80) Allow Hive Server to run multiple queries simulteneously

[Arvind Prabhakar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arvind Prabhakar reassigned HIVE-80:


Assignee: Arvind Prabhakar  (was: Neil Conway)

> Allow Hive Server to run multiple queries simulteneously
> 
>
> Key: HIVE-80
> URL: https://issues.apache.org/jira/browse/HIVE-80
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Server Infrastructure
>Reporter: Raghotham Murthy
>Assignee: Arvind Prabhakar
>Priority: Critical
> Attachments: hive_input_format_race-2.patch
>
>
> Can use one driver object per query.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1198) When checkstyle is activated for Hive in Eclipse environment, it shows all checkstyle problems as errors.

[Arvind Prabhakar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872296#action_12872296
 ] 

Arvind Prabhakar commented on HIVE-1198:


Ning - I have attached an updated patch - (hive-1198-2.patch). The key 
difference in this patch is that it does not activate checkstyle by default. 
When you import the hive project and you wish to activate checkstyle, you will 
have to right click on the project  and select Checkstyle > Activate Checkstyle 
from the context menu.

So in case checkstyle is causing problems on your workbench, you can choose to 
not activate it. The steps to activate checkstyle plugin in eclipse are also 
documented in the README.txt file, right below the section on setting up 
Eclipse. 

Can you give this patch a try and see if it resolves the problem you were 
facing?


> When checkstyle is activated for Hive in Eclipse environment, it shows all 
> checkstyle problems as errors.
> -
>
> Key: HIVE-1198
> URL: https://issues.apache.org/jira/browse/HIVE-1198
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
> Environment: Mac OS X (10.6.2), Eclipse 3.5.1.R35, Checkstyle Plugin 
> 5.1.0.201002232103 (latest eclipse and checkstyle build as of 02/2010)
>Reporter: Arvind Prabhakar
>Assignee: Arvind Prabhakar
>Priority: Minor
> Attachments: HIVE-1198-1.patch, HIVE-1198-2.patch, HIVE-1198.patch
>
>
> As of now, checkstyle plugin reports all problems as errors. This causes an 
> overwhelming number of errors to show up (3000+) which masks real errors that 
> might be there. Since all the checkstyle violations are not going to be fixed 
> in one shot, it is desirable to lower the severity of checkstyle violations 
> to warnings so that the plugin can be kept enabled. This will encourage 
> developers to spot checkstyle violations in the files they touch and 
> potentially fix them as they go along, along with pointing out violations as 
> they code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-80) Allow Hive Server to run multiple queries simulteneously

[Neil Conway (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872294#action_12872294
 ] 

Neil Conway commented on HIVE-80:
-

Arvind, I'm not actively working on it, so please go ahead.

> Allow Hive Server to run multiple queries simulteneously
> 
>
> Key: HIVE-80
> URL: https://issues.apache.org/jira/browse/HIVE-80
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Server Infrastructure
>Reporter: Raghotham Murthy
>Assignee: Neil Conway
>Priority: Critical
> Attachments: hive_input_format_race-2.patch
>
>
> Can use one driver object per query.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1198) When checkstyle is activated for Hive in Eclipse environment, it shows all checkstyle problems as errors.

[Arvind Prabhakar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arvind Prabhakar updated HIVE-1198:
---

Attachment: HIVE-1198-2.patch

> When checkstyle is activated for Hive in Eclipse environment, it shows all 
> checkstyle problems as errors.
> -
>
> Key: HIVE-1198
> URL: https://issues.apache.org/jira/browse/HIVE-1198
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
> Environment: Mac OS X (10.6.2), Eclipse 3.5.1.R35, Checkstyle Plugin 
> 5.1.0.201002232103 (latest eclipse and checkstyle build as of 02/2010)
>Reporter: Arvind Prabhakar
>Assignee: Arvind Prabhakar
>Priority: Minor
> Attachments: HIVE-1198-1.patch, HIVE-1198-2.patch, HIVE-1198.patch
>
>
> As of now, checkstyle plugin reports all problems as errors. This causes an 
> overwhelming number of errors to show up (3000+) which masks real errors that 
> might be there. Since all the checkstyle violations are not going to be fixed 
> in one shot, it is desirable to lower the severity of checkstyle violations 
> to warnings so that the plugin can be kept enabled. This will encourage 
> developers to spot checkstyle violations in the files they touch and 
> potentially fix them as they go along, along with pointing out violations as 
> they code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Hive-trunk-h0.19 #453

[Apache Hudson Server]

See 

Changes:

[namit] HIVE-1371. Bug in rcfilecat
(He Yongqiang via namit)

--
[...truncated 14094 lines...]
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2

[jira] Commented: (HIVE-80) Allow Hive Server to run multiple queries simulteneously

[Arvind Prabhakar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872282#action_12872282
 ] 

Arvind Prabhakar commented on HIVE-80:
--

This sounds like a good plan. If Neil is not actively working on this issue, I 
can move this to my queue and start working on it. 

> Allow Hive Server to run multiple queries simulteneously
> 
>
> Key: HIVE-80
> URL: https://issues.apache.org/jira/browse/HIVE-80
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Server Infrastructure
>Reporter: Raghotham Murthy
>Assignee: Neil Conway
>Priority: Critical
> Attachments: hive_input_format_race-2.patch
>
>
> Can use one driver object per query.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Hive-trunk-h0.18 #454

[Apache Hudson Server]

See 

Changes:

[namit] HIVE-1371. Bug in rcfilecat
(He Yongqiang via namit)

--
[...truncated 14059 lines...]
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2

[jira] Commented: (HIVE-80) Allow Hive Server to run multiple queries simulteneously

[Ning Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872254#action_12872254
 ] 

Ning Zhang commented on HIVE-80:


Yes we should add more test cases for parallel execution. There is an open 
issue HIVE-1019 for parallel execution. The HIVE_PLAN* file name need to be 
unique rather than relying on timestamp. 

> Allow Hive Server to run multiple queries simulteneously
> 
>
> Key: HIVE-80
> URL: https://issues.apache.org/jira/browse/HIVE-80
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Server Infrastructure
>Reporter: Raghotham Murthy
>Assignee: Neil Conway
>Priority: Critical
> Attachments: hive_input_format_race-2.patch
>
>
> Can use one driver object per query.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-802) Bug in DataNucleus prevents Hive from building if inside a dir with '+' in it

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872244#action_12872244
 ] 

Edward Capriolo commented on HIVE-802:
--

I just did a patch that adds connection pooling to DataNucleas. (Sorry that I 
jumped ahead of you). It should be easy to update now just bump the versions in 
metastore/ivy.xml. Please make sure the version you pick works with the connect 
pooling libs, as ivy fetches versions and dependants that do not work well 
together.

> Bug in DataNucleus prevents Hive from building if inside a dir with '+' in it
> -
>
> Key: HIVE-802
> URL: https://issues.apache.org/jira/browse/HIVE-802
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Todd Lipcon
>Assignee: Arvind Prabhakar
>
> There's a bug in DataNucleus that causes this issue:
> http://www.jpox.org/servlet/jira/browse/NUCCORE-371
> To reproduce, simply put your hive source tree in a directory that contains a 
> '+' character.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-80) Allow Hive Server to run multiple queries simulteneously

[Ashish Thusoo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872240#action_12872240
 ] 

Ashish Thusoo commented on HIVE-80:
---

yes I think what Ning is saying is correct. We should however add a test case 
to the unit tests to check that. I am not sure that we added a test case for 
the parallel execution stuff.

> Allow Hive Server to run multiple queries simulteneously
> 
>
> Key: HIVE-80
> URL: https://issues.apache.org/jira/browse/HIVE-80
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Server Infrastructure
>Reporter: Raghotham Murthy
>Assignee: Neil Conway
>Priority: Critical
> Attachments: hive_input_format_race-2.patch
>
>
> Can use one driver object per query.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-802) Bug in DataNucleus prevents Hive from building if inside a dir with '+' in it

[Ashish Thusoo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872238#action_12872238
 ] 

Ashish Thusoo commented on HIVE-802:


Should we just mark this as a duplicate of 1176 in that case?

> Bug in DataNucleus prevents Hive from building if inside a dir with '+' in it
> -
>
> Key: HIVE-802
> URL: https://issues.apache.org/jira/browse/HIVE-802
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Todd Lipcon
>Assignee: Arvind Prabhakar
>
> There's a bug in DataNucleus that causes this issue:
> http://www.jpox.org/servlet/jira/browse/NUCCORE-371
> To reproduce, simply put your hive source tree in a directory that contains a 
> '+' character.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1373) Missing connection pool plugin in Eclipse classpath

[Ashish Thusoo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872237#action_12872237
 ] 

Ashish Thusoo commented on HIVE-1373:
-

+1. Looks good to me. I think in future we should move all the lib dependencies 
in the eclipse files to come from build/dist/lib as that will help us migrate 
more stuff over to ivy.

Will run tests and commit once the tests pass.

> Missing connection pool plugin in Eclipse classpath
> ---
>
> Key: HIVE-1373
> URL: https://issues.apache.org/jira/browse/HIVE-1373
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
> Environment: Eclipse, Linux
>Reporter: Vinithra Varadharajan
>Assignee: Vinithra Varadharajan
>Priority: Minor
> Attachments: HIVE-1373.patch
>
>
> In a recent checkin, connection pool dependency was introduced but eclipse 
> .classpath file was not updated.  This causes launch configurations from 
> within Eclipse to fail.
> {code}
> hive> show tables;
> show tables;
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parsing command: show tables
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parse Completed
> 10/05/26 14:59:46 INFO ql.Driver: Semantic Analysis Completed
> 10/05/26 14:59:46 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from 
> deserializer)], properties:null)
> 10/05/26 14:59:46 INFO ql.Driver: query plan = 
> file:/tmp/vinithra/hive_2010-05-26_14-59-46_058_1636674338194744357/queryplan.xml
> 10/05/26 14:59:46 INFO ql.Driver: Starting command: show tables
> 10/05/26 14:59:46 INFO metastore.HiveMetaStore: 0: Opening raw store with 
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/05/26 14:59:46 INFO metastore.ObjectStore: ObjectStore, initialize called
> FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Error 
> creating transactional connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> 10/05/26 14:59:47 ERROR exec.DDLTask: FAILED: Error in metadata: 
> javax.jdo.JDOFatalInternalException: Error creating transactional connection 
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> javax.jdo.JDOFatalInternalException: Error creating transactional connection 
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
>   at org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:491)
>   at 
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:472)
>   at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:458)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:504)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:176)
>   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>   at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
> Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional 
> connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
>   at 
> org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:395)
>   at 
> org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:547)
>   at 
> org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:175)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
>   at 
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
>   at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
>   at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
>   at 
> org.apache.hadoop.hive.metastore.O

Build failed in Hudson: Hive-trunk-h0.17 #451

[Apache Hudson Server]

See 

Changes:

[namit] HIVE-1371. Bug in rcfilecat
(He Yongqiang via namit)

--
[...truncated 11388 lines...]
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-08, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=11)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition (ds=2008-04-09, hr=12)
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[j

[jira] Assigned: (HIVE-1373) Missing connection pool plugin in Eclipse classpath

[Ashish Thusoo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Thusoo reassigned HIVE-1373:
---

Assignee: Vinithra Varadharajan

Have added you to the contributors so you should be able to assign things to 
yourself now.

Thx.

> Missing connection pool plugin in Eclipse classpath
> ---
>
> Key: HIVE-1373
> URL: https://issues.apache.org/jira/browse/HIVE-1373
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
> Environment: Eclipse, Linux
>Reporter: Vinithra Varadharajan
>Assignee: Vinithra Varadharajan
>Priority: Minor
> Attachments: HIVE-1373.patch
>
>
> In a recent checkin, connection pool dependency was introduced but eclipse 
> .classpath file was not updated.  This causes launch configurations from 
> within Eclipse to fail.
> {code}
> hive> show tables;
> show tables;
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parsing command: show tables
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parse Completed
> 10/05/26 14:59:46 INFO ql.Driver: Semantic Analysis Completed
> 10/05/26 14:59:46 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from 
> deserializer)], properties:null)
> 10/05/26 14:59:46 INFO ql.Driver: query plan = 
> file:/tmp/vinithra/hive_2010-05-26_14-59-46_058_1636674338194744357/queryplan.xml
> 10/05/26 14:59:46 INFO ql.Driver: Starting command: show tables
> 10/05/26 14:59:46 INFO metastore.HiveMetaStore: 0: Opening raw store with 
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/05/26 14:59:46 INFO metastore.ObjectStore: ObjectStore, initialize called
> FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Error 
> creating transactional connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> 10/05/26 14:59:47 ERROR exec.DDLTask: FAILED: Error in metadata: 
> javax.jdo.JDOFatalInternalException: Error creating transactional connection 
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> javax.jdo.JDOFatalInternalException: Error creating transactional connection 
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
>   at org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:491)
>   at 
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:472)
>   at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:458)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:504)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:176)
>   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>   at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
> Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional 
> connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
>   at 
> org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:395)
>   at 
> org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:547)
>   at 
> org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:175)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
>   at 
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
>   at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
>   at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
>   at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:191)
>   at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:208)
>