I think we should use IRC to supplement JIRA discussions. We already do have
some email conversations whose conclusions are put on corresponding JIRA. We
should do the same for IRC.
From: Johan Oskarsson
Reply-To:
Date: Mon, 5 Jan 2009 10:49:21 -0800
To:
Subje
You can generate ruby bindings by adding '-gen rb' to thriftif target of
service/build.xml
From: Josh Ferguson
Reply-To:
Date: Tue, 6 Jan 2009 21:03:50 -0800
To:
Subject: rb-gen love
Can we get some ruby loving in the default service/src dir?
Josh
Hi Jeremy,
They are used in junit testcases. To run input1.q you have to execute 'ant test
-Dtestcase=TestCliDriver -Dqfile=input1.q'. I don't think they can be run from
command line directly since some initial tables need to be created and loaded
before most of .q files can be run.
Prasad
__
I have never used 'int' partition keys.
You might want to try inserting the transform output into a HDFS file using
'insert overwrite directory' and then use 'load into table' to load that file
into the correct partition?
From: Josh Ferguson
Reply-To:
Date: M
I think it is the same problem as Jeremy Chow's (the name configured in
hadoop-site.xml and hive-default.xml seem to be different).
From: Ashish Thusoo
Reply-To:
Date: Tue, 13 Jan 2009 11:30:18 -0800
To:
Subject: RE: INSERT OVERWRITE not working with map/reduc
Jeremy, could post the namenode and the metastore.warehouse variables that are
in hadoop-site.xml and hive-default.xml files. I have not seen this problem in
my setup so I want to see why it is different in my set up.
Thanks,
Prasad
From: Joydeep Sen Sarma
Rep
e.xml
fs.default.name
hdfs://localhost:9000
hive-default.xml
hive.metastore.warehouse.dir
hdfs://localhost:9000/user/hive/warehouse
location of default database for the warehouse
From: Prasad Chakka
Date: Wed, 14 Jan 2009 10:01:24 -0800
To:
Conversation
What is the value of config param fs.default.name?
From: Joydeep Sen Sarma
Reply-To:
Date: Fri, 23 Jan 2009 08:39:56 -0800
To:
Subject: RE: Problem loading data from local file
There was a small change to the Load command a couple of days back (to fix a
diffe
There could be many reasons for it. Could you post the stacktrace for the
JDOFatal?
Some reasons could be
That mysql client libs are in the classpath.
That there is a database called 'hive' in mysql
That there is schema already created in that database (otherwise set
autoCreateSchema to true) et
attern(Hive.java:444)
... 16 more
On Jan 24, 2009, at 12:58 PM, Prasad Chakka wrote:
There could be many reasons for it. Could you post the stacktrace for the
JDOFatal?
Some reasons could be
That mysql client libs are in the classpath.
That there is a database called 'hive' in mysql
Down load the jar from here http://dev.mysql.com/downloads/connector/j/5.1.html
From: Prasad Chakka
Reply-To:
Date: Sat, 24 Jan 2009 13:13:35 -0800
To:
Subject: Re: Still can't get metastore working with mysql
JPOX/Hive doesn't yet download th
ive.ql.exec.DDLTask
In my hive-default.xml I turned on autoCreateTables and autoCreateSchema and
it's still throwing this error back to me? Is there something else I need to do?
Josh Ferguson
On Jan 24, 2009, at 1:28 PM, Prasad Chakka wrote:
Down load the jar from here http://dev.mysql.com/downloads/c
an 24, 2009, at 2:26 PM, Prasad Chakka wrote:
Try setting fixedDataStore to false.
BTB, where did you get the initial values from the config you posted? These
values are for the store that is already running in production and you don't
expect any changes to the schema.
__
On the other hand, you can generate SQL schema using "generate-schem" ant
target in hive/metastore and use the config parameters that Joydeep sent.
____
From: Prasad Chakka
Date: Sat, 24 Jan 2009 15:00:00 -0800
To:
Conversation: Still can't get me
as
making the shift from just prototyping on your local machine to a production
deployment can be a difficult step.
Josh F.
On Jan 24, 2009, at 3:00 PM, Prasad Chakka wrote:
Well, if you had just used the defaults that are in the hive source and changed
just the JDBC params, that is what will h
Hive client is invoked through hadoop client code so hadoop jar is needed
Prasad from mobile so the brevity
- Original Message -
From: Josh Ferguson
To: hive-user@hadoop.apache.org
Sent: Sat Jan 24 19:23:47 2009
Subject: Hive w/o hadoop installation
Is it possible to run hive without a
: Re: Hive w/o hadoop installation
Just hadoop-*-core.jar?
I'm trying to figure out if I can just package up everything hive
needs into a single package.
Josh Ferguson
On Jan 24, 2009, at 10:01 PM, Prasad Chakka wrote:
> Hive client is invoked through hadoop client code so hadoop jar is
There is bug in the code. Please apply the patch for Hive-248.
From: Josh Ferguson
Reply-To:
Date: Sun, 25 Jan 2009 21:12:15 -0800
To:
Subject: Load local data
I'm using hive trunk and the following breaks:
hive -e "LOAD DATA LOCAL INPATH '/Users/Josh/tempora
Normally in hive, a table or partition is loaded by a single job/process at
once. Once it is loaded you can't append or insert any more data into that
table (only if you do it manually by moving data to that directory) So you can
most probably easier to enforce the constraints in that loading pr
You can change the last line bin/hive script to print the classpath, may we can
figure out from that.
From: Shirley Cohen
Reply-To:
Date: Thu, 29 Jan 2009 11:04:23 -0800
To:
Subject: classpath issue
Hi,
I'm finally getting around to installing hive. I was ab
og4j-1.2.15.jar:/users/scohen/hive/build/dist/bin/../lib/stringtemplate-3.1b1.jar:/users/scohen/hive/build/dist/bin/../lib/velocity-1.5.jar:
>
> Please let me know your thoughts.
>
> Thanks,
>
> Shirley
>
> Prasad Chakka wrote:
>
>> You can change
What if we want to serve metadata to non-Hive clients or the machine on which
Hive is running may not have access to the database? There are many scenarios
in which having a stand alone metastore server will be the only alternative.
From: Yongqiang He
Reply-To:
69(O)
Fax：86-10-626000900
Key Laboratory of Network Science and Technology
Research Center for Grid and Service Computing Institute of Computing
Technology, Chinese Academy of Sciences
No.3 Kexueyuan South Road
Beijing 100190, China
On 09-2-7 上午11:56, "Prasad Chakka" wrote:
What if we
che.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:94)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:625)
What I missed?
Thanks very much.
On 09-2-7 下午1:47, "Prasad Chakka" wrote:
What kind of data do you want to add?
You can add a
?
Thanks, here are what I did:
1)check the chuck
2) ant package (should I use ant deploy here?)
3) modify the conf file with remote metastore
4) start hive
On 09-2-9 上午10:36, "Prasad Chakka" wrote:
It is trying to find jpox enhanced classes and it is not finding any or the
classes weren’
start the remotestore
server.
On 09-2-9 上午10:43, "Prasad Chakka" wrote:
Where are you getting this error? In the Hive’s log or Hive Metastor’s log? Did
you start the remote metastore server?
From: Yongqiang He
Reply-To:
Date: Sun, 8 Feb 2009 18:40:5
Were you able to load non-partitioned table?
From: Yongqiang He
Reply-To:
Date: Sun, 8 Feb 2009 23:04:17 -0800
To:
Subject: Error with partition
Hi,all
I am trying to run a simple tutorial example with partition. But one error
occurs. What this error mean
following pattern:
38^Aval_238
...
On 09-2-10 上午2:57, "Prasad Chakka" wrote:
Were you able to load non-partitioned table?
From: Yongqiang He
Reply-To:
Date: Sun, 8 Feb 2009 23:04:17 -0800
To:
Subject: Error with partition
Hi,all
I am tr
Ant package should copy all required jars into build/dist directory.
From: Sancar Gazi
Reply-To:
Date: Fri, 13 Feb 2009 05:31:21 -0800
To:
Subject: AW: Missing hive execution jar
Ok I saw the two libraries have been included in the ql/lib and cli/lib.
Now I wo
Try below,
SELECT a.sender_id,
b.actor_id,
a.version,
b.reference_id
FROM communications a
LEFT OUTER JOIN (select conversions.actor_id,
conversions.reference_id from conversions where
conversions.application='reference') b
ON a.re
Mainly for performance and for ability to query/modify easily. But you can
plug-in a xml store by implementing a new MetaStore. I can help you if you want
to do it.
From: Min Zhou
Reply-To:
Date: Sat, 21 Feb 2009 02:52:40 -0800
To:
Subject: why hive use jpox r
You can use custom mapper and reducer scripts using TRANSFORM/MAP/REDUCE
facilities. Check the wiki on how to use them. Or do you want something
different?
From: Min Zhou
Reply-To:
Date: Sun, 22 Feb 2009 19:42:50 -0800
To:
Subject: How to simplify our devel
Can you use the properties from here?
http://wiki.apache.org/hadoop/Hive/AdminManual/MetastoreAdmin
Default JPOX options with only changes from above should work since unit tests
do create the database and schema.
Also there is an ant target to create schema in metastore/build.xml that you
can
doing another build install. I'll probably futz
around a little more and then start looking into the code base. If anyone's
been sucessful at getting this going, would love to hear how you made that
happen.
Ryan
On Mon, Feb 23, 2009 at 5:37 PM, Prasad Chakka wrote:
Can you use the prope
ast gotten further, so it's likely I'm just overlooking
something.
I attempted the metastore/build.xml but that didn't work either. In the end
though, I'll need this to be possible from the install directory. I'm using
Hadoop 18.3 with the 0.2 branch build if that is helpful
Most probably you have an utf-8 database and PARAM_KEY size is multiplied by 3
which is 768 and exceeds the mysql's prescribed max length.
Reducing this to 255 may cause problems to users with existing tables. We
should certainly change this next time we do an upgrade of schema.
___
Can you use ThriftDeserializer? Look at Complex class to see how it is used.
Prasad
From: Stephen Corona
Reply-To:
Date: Fri, 6 Mar 2009 16:02:02 -0800
To:
Subject: RE: Querying JSON/Thrift data?
From: Stephen Corona
I am assuming he is using the same code as MetaStore server. AFAIK,
TThreadPoolServer is supposed to use a new thread for each connection.
From: Joydeep Sen Sarma
Reply-To:
Date: Mon, 9 Mar 2009 20:16:22 -0700
To:
Subject: RE: thread cofinement session state
s and TThreadPoolServer.serve())
From: Prasad Chakka [mailto:pra...@facebook.com]
Sent: Monday, March 09, 2009 8:23 PM
To: hive-user@hadoop.apache.org
Subject: Re: thread cofinement session state
I am assuming he is using the same code as MetaStore server.
How do you get this error? While starting HiveServer?
From: Min Zhou
Reply-To:
Date: Tue, 10 Mar 2009 18:35:48 -0700
To:
Subject: python client import error
Hi list,
It's seems Hive hasnot published some modules python client relies on. e.g.
ImportError:
Hi Kevin,
There is no such facility right now but it is a good functionality to have. It
can be added in HiveMetaStore and exposed using a new Hive QL command.
One workaround is to use a vip but I am not sure it can be done in EC2.
Thanks,
Prasad
From: Kevin Ep
The query exactly as below is not supported but you can covert that query to a
join and run it
Prasad from bberry
From: javateck javateck
To: hive-user@hadoop.apache.org
Sent: Mon Mar 30 16:04:35 2009
Subject: hive sub query support
Hi,
I want to check if hi
, 2009 at 4:19 PM, Prasad Chakka wrote:
The query exactly as below is not supported but you can covert that query to a
join and run it
Prasad from bberry
From: javateck javateck
To: hive-user@hadoop.apache.org
Sent: Mon Mar 30 16:04:35 2009
Subject: hive sub
Hi Frederick,
Select sum(col1), key from tab1 group by key having sum(col1) > 5
Can be written in Hive as
Select a.*
>From (select sum(col1) as s, key from tab1 group by key) a
Where a.s > 5
Prasad
From: Frederick Oko
Reply-To:
Date: Wed, 8 Apr 2009 19:43:13
I highly doubt this is the case but If there are too many partitions in the
metadata db, the db query that gets all the partitions for a table slows down.
You might want to check whether this is causing the issue by doing `show
partitions `. Ofcourse this won't be more than few secs in any case
That is strange... Does below also fail?
select m.description, o_buyers.num , count(1) as total
from clickstream_output o
join merchandise m on (o.merchandise_id = m.merchandise_id)
left outer join ( select o1.merchandise_id, count(distinct o1.consumer_id) as
num from clickstream_output o1
I think it will be very useful to put in Hive (or rather a contrib in hive) but
I don't think GPL is compatible with Apache. So it will be a no go putting the
maxmind code/data in Hive.
Prasad
From: Edward Capriolo
Reply-To:
Date: Tue, 28 Apr 2009 07:42:48 -0
By having partition level metadata and not deriving it, quite a few things can
be achieved
* partitions can be external (i.e. Location of a partitions need not be
relative to the table)
* partitions level schemas so that a table's schema can evolve
* partitions can be disabled without
> I copy and paste pieces into the CLI.
FYI, there hive -e "hive ql command" or hive -f "file name" options. I suppose
you want more but you can use these write a simple framework that patches up
all the different steps in your program.
From: John Warden
Reply-T
Are these speculative execution maps?
From: Neil Conway
Reply-To:
Date: Tue, 5 May 2009 22:18:25 -0700
To:
Subject: Tasks killed with "Filesystem closed"
I'm seeing tasks killed, with the following exception in the log:
2009-05-06 04:58:20,799 WARN org.apache
Hive doesn't support distinct on more than one column. I don't think any one is
working on that right now. The only work around I could think of is two issue
separate queries
Or you could try something like this
from actions
insert overwrite local directory 'a'
select count(distinct user)
i
HiveServer is also a HiveMetastoreServer in usual configuration. But I think it
is possible make HiveServer talk to a different server for metadata by changing
hive-site.xml that HiveServer loads. I am not sure why it would be useful
though. Though I didn't try this configuration.
Prasad
_
Hive doesn't do any transformation while loading, atleast not yet. You can load
data into a temporary field and then do a 'insert overwrite select * from
where '
From: Manhee Jo
Reply-To:
Date: Wed, 20 May 2009 23:05:03 -0700
To:
Subject: Can Hive recognize
Do you have the jdbc driver in tha path? If you can paste the full chain of
exceptions, I may be able to tell you exactly what is missing.
Thanks,
Prasad
From: Bill Graham
Reply-To: ,
Date: Wed, 3 Jun 2009 12:57:38 -0700
To:
Subject: Re: hive jdbc client usag
Most probably metastore died. What were you doing when this happened and also
what is the log on metastore side?
Prasad from bberry
- Original Message -
From: Bill Craig
To: hive-user@hadoop.apache.org
Sent: Thu Jun 04 12:21:30 2009
Subject: Hive JDBC
After I got a seemingly good conne
I am not sure how to handle permanent storage in EC2 but you can run derby as a
networked database similar to that of mysql. You can run derby server on the
same node as jobtracker and when you are done with the cluster, you can copy
the directory on which derby is running to a permanent place.
I think you need to try using HiveServer or a JDBC client.
From: David Lerman
Reply-To:
Date: Sun, 28 Jun 2009 09:07:22 -0700
To:
Subject: getting the field types of a query result
I'm trying to build a tool which runs a query and writes the results into an
au
-user-h...@hadoop.apache.org"
wrote:
>
>
> hive-user Digest of: get
>
> Topics (messages 947 through 950):
>
> getting the field types of a query result
> 947 by: David Lerman
> 948 by: Prasad Chakka
> 949 by: Min Zhou
> 950 by: H
Haritha,
Welcome to Hive. Could you please post the full stack trace of the exception?
Most probably there is another hive client using the same metastore and you are
using the default configuration where there can be only one hive client against
one metastore. Look at this wiki page for more d
system.
Prasad
From: Haritha Javvadi
Date: Tue, 7 Jul 2009 17:34:28 -0700
To: Prasad Chakka
Subject: Re: Regarding Hive
Hey,
OK.got it.Can you suggest me whats the method involved in killing
the process. What should be done to kill the process. I think, the mistake I
have done
Previously, this property was defined in jpox.properties but HIVE-610 moved all
such properties to hive-default.xml. It is possible that you may still be using
old hive-default.xml thus hive is not able to find this property.
From: Prasad Chakka
Reply-To
Make sure that you have hive-default.xml and hive-site.xml in the classpath
while invoking hive.
Prasad
From: Min Zhou
Reply-To:
Date: Sun, 12 Jul 2009 18:35:49 -0700
To: hive-user
Subject: how to define my metastore since jpox be removed
I found all things
One way is to create external table pointing to the original table location.
From: Min Zhou
Reply-To:
Date: Mon, 13 Jul 2009 21:44:38 -0700
To: hive-user
Subject: It there any way sharing directories between different users in hive
Hi all,
We've set up a envir
It's also my way. but actually, our table wanna sharing is one partitioned by
date , it's metadata changed everyday. it's boring loading.
On Tue, Jul 14, 2009 at 12:51 PM, Prasad Chakka wrote:
One way is to create external table pointing to the original
aring directories between different users in
hive
The fact is that only a few tables need to share, users usually create their
own tables for their jobs.
On Tue, Jul 14, 2009 at 12:58 PM, Prasad Chakka wrote:
Then the question is why not everyone share the metadata and let hadoop enforce
the
ute each other solely
on HDFS permissions?
On Jul 13, 2009 10:18 PM, "Min Zhou" wrote:
Hmm, HIVE-493 is okay since authentication subsystem isn't existing. I was
considering using something like crontab to automatically add paritions for our
tables.
Thanks, Prasad!
On Tue,
Peter, can you open a JIRA for the output delimiter issue?
Also the method described by Peter is the best (you could even split the output
into multiple files and use Oracle's parallel loader if you have a multi-core
client and server). JDBC/ODBC/OCI will not be able to handle large data volumes
You can put the jar in hive/lib
From: Abhishek Tiwari
Reply-To:
Date: Thu, 16 Jul 2009 08:09:38 -0700
To:
Subject: Using external Jars in UDF
Hi,
I am writing an UDF which can 'select' required rows/records from a hive-table
and dump it into my mysql databas
Yeah, we know of this optimization and and will add it as we start optimizing
filter queries using indexes.
From: Namit Jain
Reply-To:
Date: Thu, 16 Jul 2009 22:42:25 -0700
To:
Subject: Re: Selecting data based on the clustered columns
I am not sure if they ar
It may be because you may be using the default config which creates metadata in
a directory relative to current directory. Check whether you have multiple
'metastore_db' directories. If so then try to set up a common metastore by
changing config (conf/hive-default.xml --> conf/hive-site.xml) or
The below information is not enough to figure out what is going on.
Send the correct stack trace from
/tmp//hive.log
You can change the log level to DEBUG in log4j.properties and rerun the query.
Also if the mappers/reducers are failing then check the log of the failed tasks
from hadoop.
Prasa
Sequence Block compression happens on smaller chunks (around 1MB I think) so
the compression ration would be smaller than compressing complete file.
From: Saurabh Nanda
Reply-To:
Date: Mon, 27 Jul 2009 08:38:08 -0700
To:
Subject: Re: Re: bz2 Splits.
#2 Compr
Hive does not produce any results because the external table does not have any
partitions defined. Partitions are not inferred from file system but are
created when 'load data local...' or 'alter table ... add partition ..'
commands are executed. So you need to execute the later cmd on the exter
Are you sure you are getting the same error even with the schema below (i.e.
trying to set a string to an int column?). Can you give the full stack trace
that you might see in /tmp/$USER/hive.log?
From: Bill Graham
Reply-To: ,
Date: Thu, 30 Jul 2009 10:02:54 -
The hive logs go into /tmp/$USER/hive.log not hive_job_log*.txt.
From: Bill Graham
Reply-To:
Date: Thu, 30 Jul 2009 10:52:06 -0700
To: Prasad Chakka
Cc: , Zheng Shao
Subject: Re: partitions not being created
I'm trying to set a string to a string an
rasad
From: Bill Graham
Reply-To:
Date: Thu, 30 Jul 2009 11:47:41 -0700
To: Prasad Chakka
Cc:
Subject: Re: partitions not being created
That file contains a similar error as the Hive Server logs:
2009-07-30 11:44:21,095 WARN mapred.JobClient
(JobClient.java:configureCommandLineOption
use totally different
table names (do not rename with older version) if you don't want want to drop
tables.
From: Bill Graham
Reply-To:
Date: Thu, 30 Jul 2009 11:47:41 -0700
To: Prasad Chakka
Cc:
Subject: Re: partitions not being created
That file c
blems. HIVE-592 fixes
this to rename directories correctly. So if you have created all tables after
HIVE-592 patch went in, you should be fine.
From: Bill Graham
Reply-To:
Date: Thu, 30 Jul 2009 13:09:03 -0700
To: Prasad Chakka
Cc:
Subject: Re: partitions not
From: Bill Graham
Reply-To:
Date: Fri, 31 Jul 2009 14:00:20 -0700
To: Prasad Chakka
Cc:
Subject: Re: partitions not being created
I just completely removed my all of my Hive tables and folders in HDFS, as well
as metadata_db. I then re-built Hive from the latest from the trunk
Quotes should work.
Prasad
From: Saurabh Nanda
Reply-To:
Date: Tue, 4 Aug 2009 05:43:34 -0700
To:
Subject: Passing parameters to custom map/reduce scripts
Can I pass parameters to custom map/reduce scripts like this:
from try_hits
select transform ip
: Saurabh Nanda
Reply-To:
Date: Tue, 4 Aug 2009 10:28:34 -0700
To:
Subject: Re: Passing parameters to custom map/reduce scripts
Prasad, my query was more about the concept. Not the syntax. I hope
this will still work when hive 0.4 is released. I'm basing a lot on
it.
On 8/4/09, Prasad C
Are you using the latest hive-default.xml? It should contain more datanucleus
properties than below. It is looking for a table called 'NUCLEUS_TABLES' which
contains list of tables that got created when the original schema was created.
Prasad
From: Bill Graham
Ok then add this (after schema has been created) and set all autoCreate* to
false.
datanucleus.storeManagerType
rdbms
metadata store type
From: Bill Graham
Reply-To:
Date: Wed, 5 Aug 2009 11:13:41 -0700
To: Prasad Chakka
Cc:
Subject: Re: Errors
We could make this feature per table property which doesn't have the extended
feature set supported...
From: Frederick Oko
Reply-To:
Date: Thu, 13 Aug 2009 02:12:54 -0700
To:
Subject: Re: Dynamic Partitioning?
Actually this is what Hive originally did -- it u
I don't think I understand it exactly but check the metastore db for a table
'SDS' and the the table's location is specified in 'LOCATION' column. You could
write a SQL query to change this to what you want.
I suggest you backup the metastore db first before doing this.
Prasad
much work is involved for such a feature?
-Chris
On Aug 17, 2009, at 6:19 PM, Prasad Chakka wrote:
We could make this feature per table property which doesn't have the extended
feature set supported...
From: Frederick Oko >
Reply-To:
>
Date: Thu,
Vijay, there is no solution for it yet. There may be a jira open but AFAIK, no
one is working on it. You are welcome to contribute this feature.
Prasad
From: Vijay
Reply-To:
Date: Mon, 24 Aug 2009 16:59:28 -0700
To:
Subject: Re: Adding jar files when running
Yeah, you are correct. It is a bug. It will be great if you can submit a patch.
Prasad
From: David Lerman
Reply-To:
Date: Fri, 4 Sep 2009 13:19:28 -0700
To:
Subject: MetaStoreClient not getting cached
Planning a query is taking a very long time, and during th
Thanks David. This will be in trunk & 0.4 soon.
Prasad
From: David Lerman
Reply-To:
Date: Fri, 4 Sep 2009 13:41:43 -0700
To:
Subject: Re: MetaStoreClient not getting cached
Thanks Prasad, patch is available in HIVE-816.
On 9/4/09 4:24 PM, "Pras
If a certain input file/dir does not exist then the job can’t be submitted.
Since only a few reducers are failing, the problem could be something else.
Eva, Does the same job succeed on a second try? Ie. Is the file/dir available
eventually? What is the replication factor?
Prasad
_
The latest version does create all the parents if they do not exist. Can you
try with the latest version (or 0.4)?
Prasad
From: james warren
Reply-To:
Date: Tue, 8 Sep 2009 22:44:35 -0700
To:
Subject: possible INSERT OVERWRITE TABLE bug
I encountered this be
couldn’t find those files that it thought it has seen
before and failed. So, it looks like a concurrency issue.
Yongqiang,
Could you elaborate a bit on why you say this is not a bug?
Thanks,
Eva.
On 9/9/09 9:55 AM, "Prasad Chakka" wrote:
If a certain input file/dir does not exis
.
On 9/9/09 10:29 AM, "Prasad Chakka" wrote:
The first query will not return unless it copied the files to the dest
directory and this operation is atomic (FileSystem.rename() guarantees that).
Since second query is not executed until the first query returns, this problem
may be due t
. This is
actually one big advantage of Hive. The r/w lock or access control can be
achieved by a DB row with lock count for each partition, etc. This might be
over-simplfying it, but the metastore DB seems to be the ideal candidate.
Thoughts?
On 9/9/09 12:52 PM, "Prasad Chakka"
You should create a daily partition table. So you just need to create a new
partition which is automatic if you use 'LOAD DATA... INTO TABLE ... PARTITION
(ds='2009-09-01')'
Prasad
From: Mayuran Yogarajah
Reply-To:
Date: Fri, 11 Sep 2009 12:20:25 -0700
To:
S
arajah
Reply-To:
Date: Fri, 11 Sep 2009 13:16:27 -0700
To:
Subject: Re: General design/schema question
Prasad Chakka wrote:
> You should create a daily partition table. So you just need to create
> a new partition which is automatic if you use 'LOAD DATA... INTO TABLE
> ... PARTITION
SerDe may be needed to deserialize the result from Hive server. But I also
thought the results are in delimited form (LazySimpleSerDe or
MetadataTypedColumnSetSerDe or something like that) so it is possible to
decouple hadoop jar but will take some work.
From:
Yes, it should.
From: Abhijit Pol
Reply-To:
Date: Wed, 16 Sep 2009 11:04:24 -0700
To:
Subject: hive queries over tables in different formats
does hive support queries over multiple tables stored in different formats? for
example tableA is stroed as TEXTFILE a
I think this can be a good feature though I would like the filename to be a
partition column (one of such) instead of a separate type of column. Would that
work?
create external table FOO ( )
row format delimited fields terminated by ','
partitioned by (file_name FILENAME)
stored as textfile l
ubstr(filename, 4, 7),
substr(filename, 8, 10) ;
thanks,
-A
From: Prasad Chakka [mailto:pcha...@facebook.com]
Sent: Wednesday, September 16, 2009 11:10 AM
To: hive-user@hadoop.apache.org
Subject: Re: adding filenames as new columns via Hive
I think this can be a good feature though I would lik
1 - 100 of 122 matches
Mail list logo
