Re: HiveServer2 & Kerberos

[Loïc Chanel]

You were right ! Thanks a lot, I didn't checked this property as I thought
Ambari set it to true when enabling Kerberos.
Thanks again,

Loïc

Loïc CHANEL
Engineering student at TELECOM Nancy
Trainee at Worldline - Villeurbanne

2015-09-09 19:53 GMT+02:00 Takahiko Saito :

> Hi Loic,
>
> One possible solution is if hive.server2.enable.doAs is set false in
> hive-site.xml, you can change it to true and restart HiveServer2. And then
> try to connect via beeline.
>
> Cheers,
>
> On Wed, Sep 9, 2015 at 8:02 AM, Loïc Chanel 
> wrote:
>
>> Hi guys !
>>
>> Sorry to interrupt but I need to go back to the first reason of this
>> thread : I can't connect to hive anymore.
>> I upgraded my cluster to HDP 2.3, and I saw that the way to connect to
>> Hive via Beeline & Kerberos hasn't changed, but the exact command that
>> worked before doesn't work anymore.
>> Instead of connecting, Beeline returns me :
>> Error: Failed to open new session: java.lang.RuntimeException:
>> java.lang.RuntimeException:
>> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
>> User: hive/hiveserverh...@example.com is not allowed to impersonate
>> testUser (state=,code=0)
>>
>> The logs are not more explicit, as there is an exception with the same
>> conclusion : Caused by:
>> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
>> User: hive/hiveserverh...@example.com is not allowed to impersonate
>> testUser
>>
>> Do any of you have an idea about where this could come from ?
>>
>>
>>
>> Loïc CHANEL
>> Engineering student at TELECOM Nancy
>> Trainee at Worldline - Villeurbanne
>>
>> 2015-08-31 13:51 GMT+02:00 Lars Francke :
>>
>>> That said, +1 to adding a check that we are using kerberos and skipping
 the prompt if we are. I think we probably don't even need to parse the URL
 to detect that. Just checking on the auth type property(
 hive.server2.authentication) is KERBEROS or not should do the trick.

>>>
>>> I have not looked into this at all but Beeline being a generic client
>>> does it even use that property? I mean I could connect to any server,
>>> right? Will try to take a look.
>>>
>>>
 [1]
 https://github.com/apache/hive/blob/3991dba30c5068cac296f32e24e97cf87efa266c/jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java#L450-L455

 On Wed, Aug 26, 2015 at 5:40 PM, Lars Francke 
 wrote:

>
> On Wed, Aug 26, 2015 at 4:53 PM, kulkarni.swar...@gmail.com <
> kulkarni.swar...@gmail.com> wrote:
>
>> > my understanding is that after using kerberos authentication, you
>> probably don’t need the password.
>>
>> That is not an accurate statement. Beeline is a JDBC client as
>> compared to Hive CLI which is a thrift client to talk to HIveServer2. So 
>> it
>> would need the password to establish that JDBC connection. If you look at
>> the beeline console code[1], it actually first tries to read the
>> "javax.jdo.option.ConnectionUserName" and
>> "javax.jdo.option.ConnectionPassword" property which is the same username
>> and password that you have setup your backing metastore DB with. If it is
>> MySWL, it would be the password you set MySQL with or empty if you
>> haven't(or are using derby). Kerberos is merely a tool for you to
>> authenticate yourself so that you cannot impersonate yourself as someone
>> else.
>>
>
> I don't think what you're saying is accurate.
>
> 1) Hive CLI does not talk to HiveServer2
>
> 2) Beeline talks to HiveServer2 and needs some way to authenticate
> itself depending on the configuration of HS2.
>
> HS2 can be configured to authenticate in one of these ways if I'm up
> to date:
>
> * NOSASL: no password needed
> * KERBEROS (SASL): no password needed
> * NONE (SASL) using the AnonymousAuthenticationProviderImpl: no
> password needed
> * LDAP (SASL) using the LdapAuthenticationProviderImpl: username and
> password required
> * PAM (SASL) using the PamAuthenticationProviderImpl: username and
> password required
> * CUSTOM (SASL) using the CustomAuthenticationProviderImpl: username
> and password required
>
> By tar the most common configurations are NONE (default I think) and
> KERBEROS. Both don't need a username and password provided so it does not
> make sense to ask for one every time.
>
> The only good reason I can think of to ask for a password is so that
> it doesn't appear in a shell/beeline history and/or on screen. I'm sure
> there are others?
> The username can be safely provided in the URL if needed so I don't
> think asking for that every time is reasonable either.
>
> What would be a good way to deal with this? I'm tempted to just rip
> out those prompts. The 

Querying hive without providing schema during create table for JSON data

[Jeetendra G]

Hi All,

I have JSON data Can I create table without schema? and query that data,
since all my data os JSON data so I was thinking there should be some way.

Currently I am doing something like below, but I want to remove Schema from
this, because I have i ma having thousand of event type and all of them i
cant really create the schema, Highlighted part I want to remove and let
the hive decide the schema.


create external table if not exists impressions_data
(
*city string,*
*service string,*
*id int,*
*type string,*
*listings int,*
*visible_markers int,*
*visible_as_markers int,*
*city_id int,*
*location_type string,*
*polygon_id string,*
*polygon_name string,*
*BuyListViewCount string,*
*AutoSuggest string,*
*LocalityExpert string,*
*BuyerExperiment string,*
*SellerExperiment string,*
*PaidAdProducts string,*
*index_view_type string,*
*page_type string,*
*clid string,*
*device string,*
*results_by_profile string,*
*listing_index int,*
*huid string,*
*uid string,*
*sid int,*
*count int,*
*category string,*
*action string,*
*url string,*
*`timeStamp` bigint)*
partitioned by (`date` string)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 'hdfs://
hadoop01.housing.com:8020/housing/events/validated/category=impressions/';

MSCK REPAIR TABLE impressions_data;
SHOW PARTITIONS impressions_data;
select * from impressions_data limit 1;

Re: Starting Hive Thrift Server

[Jason Dere]

Pretty sure those are just warning messages: 
http://stackoverflow.com/questions/22896243/maven-slf4j-class-path-contains-multiple-slf4j-bindings


Try to connect to your server and see if it's working.

?



From: Paulo Moniz 
Sent: Wednesday, September 09, 2015 5:13 AM
To: user@hive.apache.org
Subject: Starting Hive Thrift Server

Hi, I am beginner and may possibly even find my naive questions regarding when 
trying to connect to the server hive.
So I ask you to help.

After I run the command: ~ $ $ HIVE_HOME / bin / hive --service hiveserver and 
appear the messages as shown above, should appear a message indicating that 
connection hiveserver is ok?

How do I know if the connection is ok, when the last line is as follows:
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]


 I am using Apache Hadoop 2.2.0 and Hadoop HIVE 0.12.0 in Ubuntu 14.04

I run the command ($ HIVE_HOME / bin / hive --service hiveserver) it shows the 
following message below:



hduser@master-VirtualBox:~$ $HIVE_HOME/bin/hive --service hiveserver

Starting Hive Thrift Server

15/09/08 09:32:38 INFO Configuration.deprecation: mapred.input.dir.recursive is 
deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive

15/09/08 09:32:38 INFO Configuration.deprecation: mapred.max.split.size is 
deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize

15/09/08 09:32:38 INFO Configuration.deprecation: mapred.min.split.size is 
deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize

15/09/08 09:32:38 INFO Configuration.deprecation: 
mapred.min.split.size.per.rack is deprecated. Instead, use 
mapreduce.input.fileinputformat.split.minsize.per.rack

15/09/08 09:32:38 INFO Configuration.deprecation: 
mapred.min.split.size.per.node is deprecated. Instead, use 
mapreduce.input.fileinputformat.split.minsize.per.node

15/09/08 09:32:38 INFO Configuration.deprecation: mapred.reduce.tasks is 
deprecated. Instead, use mapreduce.job.reduces

15/09/08 09:32:38 INFO Configuration.deprecation: 
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use 
mapreduce.reduce.speculative

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in 
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in 
[jar:file:/usr/local/hive/hive-0.12.0/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]



Immensely like to get help from you, because I've done numerous and can not 
move forward because I believe that is missing something to connect to the 
server.


I thank you.

Paul Moniz.

Create table using Parquet metadata

[Michał Warecki]

Hi guys!

I'm using Spark SQL 1.3 on Hive with HDFS and Parquet. I've configured Hive
Metastore and I'd like to start using it.
It is possible to create table in Hive Metastore based on metadata stored
in Parquet tables? Stored tables contains lots of columns (sometimes
volatile) and I can't specify all of them.

I see that Impala added sql keyword "LIKE PARQUET 'path'" but it looks like
it's not working in pure Hive.

Thanks

Why do we need the gloval "X" permission to run INSERT in Hive-HBase integration?

[Han-Cheol Cho]

p{margin-top:0px;margin-bottom:0px;} Hi, Hive user mailing list memebers,
 
 
I am testing Hive-HBase integration on a Hadoop cluster.
The cluster uses CDH5.4.4 with Kerberos enabled.
 
Following the documentation of Cloudera and Apache HiveHBaseIntegration, I 
installed
hive-hbase package on a client machine.
Then, I tried some commands as follows:
 
1) login as "testuser" and run kinit
2) run hive
3) add jars
ADD JAR /usr/lib/hive/lib/zookeeper.jar;
ADD JAR /usr/lib/hive/lib/hive-hbase-handler.jar
ADD JAR /usr/lib/hive/lib/guava-14.0.1.jar;
ADD JAR /usr/lib/hive/lib/hbase-client.jar;
ADD JAR /usr/lib/hive/lib/hbase-common.jar;
ADD JAR /usr/lib/hive/lib/hbase-hadoop-compat.jar;
ADD JAR /usr/lib/hive/lib/hbase-hadoop2-compat.jar;
ADD JAR /usr/lib/hive/lib/hbase-protocol.jar;
ADD JAR /usr/lib/hive/lib/hbase-server.jar;
ADD JAR /usr/lib/hive/lib/htrace-core.jar;
4) create a table
CREATE TABLE hbase_table_1(key int, value string) 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "test:xyz");
5) insert some data
INSERT OVERWRITE TABLE hbase_table_1 SELECT 1, 'first_row' FROM t1; 
 
I already created a namespace "test" and gave full permission (RWXCA) to the 
"testuser" account.
 
However, the step 5) fails with the following error messages, 
uery ID = testuser_20150910164848_d22bfbac-107a-45d9-acfb-63a983806e46
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.RuntimeException: 
org.apache.hadoop.hbase.security.AccessDeniedException: 
org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient 
permissions (user=testu...@mykerberos.com, scope=hbase:meta, 
params=[table=hbase:meta],action=EXEC)
at 
org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:428)
at 
org.apache.hadoop.hbase.security.access.AccessController.preEndpointInvocation(AccessController.java:2069)


While digging the cause of this error, I found that giving the global execute 
permission solves this problem.
hbase grant 'testuser', 'X'
 
But, I can't find this solution from any document (please notify me if there is 
some reference about this).
Why should I need to give any global permission while I gave the full 
permission on the namespace where 
the table belongs to?
 
 
Best wishes,
HanCheol
 
 
 趙漢哲  CHO, HAN-CHEOL (Ph.D) 
 データ研究室   / 社員 
  --> 〒105-6322 東京都港区虎ノ門１－23-1 虎ノ門ヒルズ森タワー22階
Email  hancheol@nhn-playart.com   Messenger   

NHN PlayArt 株式会社
 

Re: hiveserver2 hangs

[Sanjeev Verma]

Thanks sanjeev for your help.

BTW I try to increase the Heap Size of HS2 but seeing the same
exception.from where this exception has originated, it looks like
originated from the thrift client.any idea what operation it is doing with
the given stack.

Local Variable: org.apache.thrift.TByteArrayOutputStream#42
Local Variable: byte[]#5378
at org.apache.thrift.transport.TSaslTransport.write(TSaslTransport.java:446)
at org.apache.thrift.transport.TSaslServerTransport.write(TSasl
ServerTransport.java:41)
at org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryP
rotocol.java:163)
at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBina
ryProtocol.java:186)
Local Variable: byte[]#2
at org.apache.hive.service.cli.thrift.TStringColumn$TStringColu
mnStandardScheme.write(TStringColumn.java:490)
Local Variable: java.util.ArrayList$Itr#1
at org.apache.hive.service.cli.thrift.TStringColumn$TStringColu
mnStandardScheme.write(TStringColumn.java:433)
Local Variable: org.apache.hive.service.cli.thrift.TStringColumn$TStringColu
mnStandardScheme#1
at org.apache.hive.service.cli.thrift.TStringColumn.write(TStri
ngColumn.java:371)
at org.apache.hive.service.cli.thrift.TColumn.standardSchemeWri
teValue(TColumn.java:381)
Local Variable: org.apache.hive.service.cli.thrift.TColumn#504
Local Variable: org.apache.hive.service.cli.thrift.TStringColumn#453
at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.java:244)
at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.java:213)
at org.apache.thrift.TUnion.write(TUnion.java:152)

On Wed, Sep 9, 2015 at 8:19 AM, kulkarni.swar...@gmail.com <
kulkarni.swar...@gmail.com> wrote:

> Sanjeev,
>
> I am going off this exception in the stacktrace that you posted.
>
> "at java.lang.OutOfMemoryError.(OutOfMemoryError.java:48)"
>
> which def. indicates that it's not very happy memory wise. I would def.
> recommend to bump up the memory and see if it helps. If not, we can debug
> further from there.
>
> On Tue, Sep 8, 2015 at 12:17 PM, Sanjeev Verma 
> wrote:
>
>> What this exception implies here? how to identify the problem here.
>> Thanks
>>
>> On Tue, Sep 8, 2015 at 10:44 PM, Sanjeev Verma > > wrote:
>>
>>> We have 8GB HS2 java heap, we have not tried any bumping.
>>>
>>> On Tue, Sep 8, 2015 at 8:14 PM, kulkarni.swar...@gmail.com <
>>> kulkarni.swar...@gmail.com> wrote:
>>>
 How much memory have you currently provided to HS2? Have you tried
 bumping that up?

 On Mon, Sep 7, 2015 at 1:09 AM, Sanjeev Verma <
 sanjeev.verm...@gmail.com> wrote:

> *I am getting the following exception when the HS2 is crashing, any
> idea why it has happening*
>
> "pool-1-thread-121" prio=4 tid=19283 RUNNABLE
> at java.lang.OutOfMemoryError.(OutOfMemoryError.java:48)
> at java.util.Arrays.copyOf(Arrays.java:2271)
> Local Variable: byte[]#1
> at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
> at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutput
> Stream.java:93)
> at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
> Local Variable: org.apache.thrift.TByteArrayOutputStream#42
> Local Variable: byte[]#5378
> at org.apache.thrift.transport.TSaslTransport.write(TSaslTransp
> ort.java:446)
> at org.apache.thrift.transport.TSaslServerTransport.write(TSasl
> ServerTransport.java:41)
> at org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryP
> rotocol.java:163)
> at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBina
> ryProtocol.java:186)
> Local Variable: byte[]#2
> at org.apache.hive.service.cli.thrift.TStringColumn$TStringColu
> mnStandardScheme.write(TStringColumn.java:490)
> Local Variable: java.util.ArrayList$Itr#1
> at org.apache.hive.service.cli.thrift.TStringColumn$TStringColu
> mnStandardScheme.write(TStringColumn.java:433)
> Local Variable: org.apache.hive.service.cli.th
> rift.TStringColumn$TStringColumnStandardScheme#1
> at org.apache.hive.service.cli.thrift.TStringColumn.write(TStri
> ngColumn.java:371)
> at org.apache.hive.service.cli.thrift.TColumn.standardSchemeWri
> teValue(TColumn.java:381)
> Local Variable: org.apache.hive.service.cli.thrift.TColumn#504
> Local Variable: org.apache.hive.service.cli.thrift.TStringColumn#453
> at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.j
> ava:244)
> at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.j
> ava:213)
> at org.apache.thrift.TUnion.write(TUnion.java:152)
>
>
>
> On Fri, Aug 21, 2015 at 6:16 AM, kulkarni.swar...@gmail.com <
> kulkarni.swar...@gmail.com> wrote:
>
>> Sanjeev,
>>
>> One possibility is that you are running into[1] which affects hive
>> 0.13. Is it possible for you to apply the patch on [1] and see if it 
>> fixes
>> your problem?
>>

RE: Accumulo Storage Manager

[peter.mar...@baesystems.com]

Hi Josh,

At this stage I don't know whether there's anything wrong with Hive or it's 
just user error.
Perhaps if I go through what I have done you can see where the error lies.
Unfortunately this is going to be wordy. Apologies in advance for the long 
email.

So I created a "normal" table in HDFS with a variety of column types like this:

CREATE TABLE employees4 (
 rowid STRING,
 flag BOOLEAN,
 number INT,
 bignum BIGINT,
 name STRING,
 salary FLOAT,
 bigsalary DOUBLE,
 numbers ARRAY,
 floats ARRAY,
 subordinates ARRAY,
 deductions MAP,
 namedNumbers MAP,
 address STRUCT);

And I put some data into it and I can see the data:

hive> SELECT * FROM employees4;
OK
row1true100 7   John Doe10.010.0
[13,23,-1,1001] [3.14159,2.71828,-1.1,1001.0]   ["Mary Smith","Todd Jones"] 
{"Federal Taxes":0.2,"State Taxes":0.05,"Insurance":0.1} {"nameOne":123,"Name 
Two":49,"The Third Man":-1}{"street":"1 Michigan 
Ave.","city":"Chicago","state":"IL","zip":60600}
row2false   7   100 Mary Smith  10.08.0 
[13,23,-1,1001] [3.14159,2.71828,-1.1,1001.0,1001.0]["Bill King"]   
{"Federal Taxes":0.2,"State Taxes":0.05,"Insurance":0.1}{"nameOne":123,"Name 
Two":49,"The Third Man":-1} {"street":"100 Ontario 
St.","city":"Chicago","state":"IL","zip":60601}
row3false   3245877878  Todd Jones  10.07.0 
[13,23,-1,1001] [3.14159,2.71828,-1.1,1001.0,2.0]   []  {"Federal 
Taxes":0.15,"State Taxes":0.03,"Insurance":0.1}   {"nameOne":123,"Name 
Two":49,"The Third Man":-1} {"street":"200 Chicago Ave.","city":"Oak 
Park","state":"IL","zip":60700}
row4true877878  3245Bill King   10.06.0 
[13,23,-1,1001] [3.14159,2.71828,-1.1,1001.0,1001.0,1001.0,1001.0]  []  
{"Federal Taxes":0.15,"State Taxes":0.03,"Insurance":0.1}
{"nameOne":123,"Name Two":49,"The Third Man":-1}{"street":"300 Obscure 
Dr.","city":"Obscuria","state":"IL","zip":60100}
Time taken: 0.535 seconds, Fetched: 4 row(s)

Everything looks fine.
Now I create a Hive table stored in Accumulo:

DROP TABLE IF EXISTS accumulo_table4;
CREATE TABLE accumulo_table4 (
 rowid STRING,
 flag BOOLEAN,
 number INT,
 bignum BIGINT,
 name STRING,
 salary FLOAT,
 bigsalary DOUBLE,
 numbers ARRAY,
 floats ARRAY,
 subordinates ARRAY,
 deductions MAP,
 namednumbers MAP,
 address STRUCT)
STORED BY 'org.apache.hadoop.hive.accumulo.AccumuloStorageHandler'
WITH SERDEPROPERTIES('accumulo.columns.mapping' = 
':rowid,person:flag#binary,person:number#binary,person:bignum#binary,person:name,person:salary#binary,person:bigsalary#binary,person:numbers#binary,person:floats,person:subordinates,deductions:*,namednumbers:*,person:address');

(Note that I am only really interested in storing the values in "binary".)
Now I can load the Accumulo table from the normal table:

INSERT OVERWRITE TABLE accumulo_table4 SELECT * FROM employees4;

And I can query the data from the Accumulo table.

hive> SELECT * FROM accumulo_table4;
OK
row1true100 7   John Doe10.010.0
[null]  [null]  ["Mary Smith\u0003Todd Jones"]  {"Federal 
Taxes":0.2,"Insurance":0.1,"State Taxes":0.05}{"Name Two":49,"The Third 
Man":-1,"nameOne":123} {"street":"1 Michigan 
Ave.\u0003Chicago\u0003IL\u000360600","city":null,"state":null,"zip":null}
row2false   7   100 Mary Smith  10.08.0 [null]  
[null]  ["Bill King"]   {"Federal Taxes":0.2,"Insurance":0.1,"State 
Taxes":0.05}{"Name Two":49,"The Third Man":-1,"nameOne":123} 
{"street":"100 Ontario 
St.\u0003Chicago\u0003IL\u000360601","city":null,"state":null,"zip":null}
row3false   3245877878  Todd Jones  10.07.0 [null]  
[null]  []  {"Federal Taxes":0.15,"Insurance":0.1,"State Taxes":0.03}   
{"Name Two":49,"The Third Man":-1,"nameOne":123} {"street":"200 Chicago 
Ave.\u0003Oak Park\u0003IL\u000360700","city":null,"state":null,"zip":null}
row4true877878  3245Bill King   10.06.0 [null]  
[null]  []  {"Federal Taxes":0.15,"Insurance":0.1,"State Taxes":0.03}   
{"Name Two":49,"The Third Man":-1,"nameOne":123} {"street":"300 Obscure 
Dr.\u0003Obscuria\u0003IL\u000360100","city":null,"state":null,"zip":null}
Time taken: 0.109 seconds, Fetched: 4 row(s)

Notice that the columns with type ARRAYand ARRAY are empty.
I assume that this means that there is something wrong and the Hive Storage 
Handler is returning a null?
When I use the accumulo shell to look 

Re: Querying hive without providing schema during create table for JSON data

[Jeetendra G]

anybody here? can somebody guide me on this?

On Thu, Sep 10, 2015 at 2:36 PM, Jeetendra G 
wrote:

> Hi All,
>
> I have JSON data Can I create table without schema? and query that data,
> since all my data os JSON data so I was thinking there should be some way.
>
> Currently I am doing something like below, but I want to remove Schema
> from this, because I have i ma having thousand of event type and all of
> them i cant really create the schema, Highlighted part I want to remove and
> let the hive decide the schema.
>
>
> create external table if not exists impressions_data
> (
> *city string,*
> *service string,*
> *id int,*
> *type string,*
> *listings int,*
> *visible_markers int,*
> *visible_as_markers int,*
> *city_id int,*
> *location_type string,*
> *polygon_id string,*
> *polygon_name string,*
> *BuyListViewCount string,*
> *AutoSuggest string,*
> *LocalityExpert string,*
> *BuyerExperiment string,*
> *SellerExperiment string,*
> *PaidAdProducts string,*
> *index_view_type string,*
> *page_type string,*
> *clid string,*
> *device string,*
> *results_by_profile string,*
> *listing_index int,*
> *huid string,*
> *uid string,*
> *sid int,*
> *count int,*
> *category string,*
> *action string,*
> *url string,*
> *`timeStamp` bigint)*
> partitioned by (`date` string)
> ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
> LOCATION 'hdfs://
> hadoop01.housing.com:8020/housing/events/validated/category=impressions/';
>
> MSCK REPAIR TABLE impressions_data;
> SHOW PARTITIONS impressions_data;
> select * from impressions_data limit 1;
>

Fw: important

[Steve Loughran]

Hello!

 

Important message, visit 
http://www.innovative-electrical.co.uk/general.php?erfr8

 

Steve Loughran

Fw: important

[adults-bounces]

Hello!

 

Important message, visit http://lalanicg.com/pleasure.php?xcb

 

adults-boun...@troop775.org

Fw: important

[Tim Wintle]

Hello!

 

Important message, visit http://git.weywang.com/began.php?x1x

 

Tim Wintle

Fw: important

[Kristene Chappell]

Hello!

 

Important message, visit http://sayurpremium.com/farther.php?qyiy

 

Kristene Chappell

Fw: important

[Allen Wittenauer]

Hello!

 

Important message, visit http://abcbordados.com.br/family.php?wj28o

 

Allen Wittenauer

Fw: important

[Peter Wojciechowski]

Hello!

 

Important message, visit http://rosseden.com/eat.php?p

 

Peter Wojciechowski

Fw: important

[Lefty Leverenz]

Hello!

 

Important message, visit http://aldersgatencsc.org/need.php?1ef

 

Lefty Leverenz

Fw: important

[Simone Leo]

Hello!

 

Important message, visit http://www.innovative-electrical.co.uk/general.php?wn

 

Simone Leo

Fw: important

[Christa Nonnemaker]

Hello!

 

Important message, visit http://adixion.net/comfort.php?rf

 

Christa Nonnemaker

Fw: important

[Nathan Loring]

Hello!

 

Important message, visit http://idansanthaus.com/king.php?x0q

 

Nathan Loring

Fw: important

[Elena Radosavcev]

Hello!

 

Important message, visit http://carcrazydirect.com/line.php?d

 

Elena Radosavcev

Fw: important

[Tsuyoshi OZAWA]

Hello!

 

Important message, visit http://stephenlong.ca/history.php?iqv

 

Tsuyoshi OZAWA

Fw: important

[Joann Beatty]

Hello!

 

Important message, visit http://santinoristorante.com/degree.php?n5

 

Joann Beatty

Fw: important

[Schuda Robert]

Hello!

 

Important message, visit http://izing.stronazen.pl/mine.php?a6

 

Schuda Robert

Fw: important

[Allen Wittenauer]

Hello!

 

Important message, visit http://cjmirra.com/town.php?07m95

 

Allen Wittenauer

Fw: important

[Vinithra Varadharajan]

Hello!

 

Important message, visit http://optionsbinairesbroker.com/let.php?mdd

 

Vinithra Varadharajan

Fw: important

[Kristene Chappell]

Hello!

 

Important message, visit http://www.innovative-electrical.co.uk/weather.php?v3pl

 

Kristene Chappell

Fw: important

[Member Solutions]

Hello!

 

Important message, visit http://www.newspostz.com/let.php?l

 

Member Solutions

Fw: important

[Lisa Krijger]

Hello!

 

Important message, visit http://carcrazydirect.com/become.php?tep

 

Lisa Krijger

Fw: important

[Brooke Gabbey]

Hello!

 

Important message, visit http://www.bazarnetwork.com/known.php?dhb

 

Brooke Gabbey

Fw: important

[Debbie Thomas]

Hello!

 

Important message, visit http://stpsoccer.com/believe.php?xzo

 

Debbie Thomas

Fw: important

[xiaoshanfang]

Hello!

 

Important message, visit http://chairfilms.org/own.php?d686

 

xiaoshanfang

Fw: important

[xiaoshanfang]

Hello!

 

Important message, visit http://7del7group.com/carriage.php?jdw7

 

xiaoshanfang

Fw: important

[Macon McIntyre]

Hello!

 

Important message, visit http://bigobjectbase.com/however.php?1508n

 

Macon McIntyre

Fw: important

[Tsuyoshi OZAWA]

Hello!

 

Important message, visit http://git.weywang.com/mean.php?tdwt

 

Tsuyoshi OZAWA

Re: Querying hive without providing schema during create table for JSON data

[Muthu Pandi]

You can use Apache Drill to query Json data without creating the schema but
you cant in Hive.

http://drill.apache.org/docs/querying-json-files/


*RegardsMuthupandi.K*

*muthupandi.strikingly.com *

 Think before you print.



On Thu, Sep 10, 2015 at 9:16 PM, Jeetendra G 
wrote:

> anybody here? can somebody guide me on this?
>
> On Thu, Sep 10, 2015 at 2:36 PM, Jeetendra G 
> wrote:
>
>> Hi All,
>>
>> I have JSON data Can I create table without schema? and query that data,
>> since all my data os JSON data so I was thinking there should be some way.
>>
>> Currently I am doing something like below, but I want to remove Schema
>> from this, because I have i ma having thousand of event type and all of
>> them i cant really create the schema, Highlighted part I want to remove and
>> let the hive decide the schema.
>>
>>
>> create external table if not exists impressions_data
>> (
>> *city string,*
>> *service string,*
>> *id int,*
>> *type string,*
>> *listings int,*
>> *visible_markers int,*
>> *visible_as_markers int,*
>> *city_id int,*
>> *location_type string,*
>> *polygon_id string,*
>> *polygon_name string,*
>> *BuyListViewCount string,*
>> *AutoSuggest string,*
>> *LocalityExpert string,*
>> *BuyerExperiment string,*
>> *SellerExperiment string,*
>> *PaidAdProducts string,*
>> *index_view_type string,*
>> *page_type string,*
>> *clid string,*
>> *device string,*
>> *results_by_profile string,*
>> *listing_index int,*
>> *huid string,*
>> *uid string,*
>> *sid int,*
>> *count int,*
>> *category string,*
>> *action string,*
>> *url string,*
>> *`timeStamp` bigint)*
>> partitioned by (`date` string)
>> ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
>> LOCATION 'hdfs://
>> hadoop01.housing.com:8020/housing/events/validated/category=impressions/
>> ';
>>
>> MSCK REPAIR TABLE impressions_data;
>> SHOW PARTITIONS impressions_data;
>> select * from impressions_data limit 1;
>>
>
>

Fw: important

[Julie Affronti]

Hello!

 

Important message, visit http://beetheadventure.com/not.php?q6m

 

Julie Affronti

Fw: important

[Renato Marroqun Mogrovejo]

Hello!

 

Important message, visit http://ridingfarm.co.uk/evening.php?xa

 

Renato Marroqun Mogrovejo

Fw: important

[Jon and Teresa Sutcliffe]

Hello!

 

Important message, visit http://jcautostyles.com/seen.php?ul

 

Jon and Teresa Sutcliffe