Unsubscribe
- Original Message -
From: "Kevin Risden"
To: user@hive.apache.org
Sent: Tuesday, October 16, 2018 9:44:17 AM
Subject: Re: HiveServer2 performance references?
Thanks for the responses. I'll wait to test with HIVE-20621 to see how much
that helps. For HIVE-17194, doesn't loo
: Re: Unable to read hive external table data which is linked to s3
after upgradation from 2.1.1 to 2.3.3
Hi Richard ,
After looking at hive 2.3.3 logs i found that when we are loading all
configuration parameters then below message is getting printed :-
Values omitted for securi
s.key
* fs.s3a.secret.key
* fs.s3a.connection.maximum
* fs.s3a.impl
Regards,
Anup Tiwari
On Thu, Apr 12, 2018 at 7:19 PM, Richard A. Bross < r...@oaktreepeak.com >
wrote:
Based on the exception, it looks more like an AWS credentials issue than a Hive
issue. Are you runni
Based on the exception, it looks more like an AWS credentials issue than a Hive
issue. Are you running in AWS EMR, on-prem?
In AWS the resource accessing the S3 bucket would have to have an IAM that gave
permission. If you are running somewhere else whatever AWS login you use would
have to ha
make sure that
the partitions are correctly loaded and then try again dropping that particular
partition?
Or look at your s3 folder if you see any such "partition folder file" and check
if it is missing for this particular partition?
On 5 April 2018 at 15:40, Richard A. Bros
Storage Desc Params:
serialization.format1
Time taken: 0.467 seconds, Fetched: 98 row(s)
- Original Message -
From: "Furcy Pin"
To: user@hive.apache.org
Sent: Thursday, April 5, 2018 9:21:06 AM
Subject: Re: ALTER TABLE DROP PARTITION not working on
No definitely internal. It's loaded from an external table.
- Original Message -
From: "Furcy Pin"
To: user@hive.apache.org
Sent: Thursday, April 5, 2018 9:21:06 AM
Subject: Re: ALTER TABLE DROP PARTITION not working on S3
Hi Richard,
could you please check i
I think that someone put a file in there manually. Would that prevent Hive
from dropping the partition. I also did a "drop table" and the s3 object keys
persisted.
- Original Message -----
From: "Richard A. Bross"
To: user@hive.apache.org
Sent: Thursday, April 5, 201
Hi,
I have a Hive managed table on S3, "api_measurements". I've tried dropping a
partition like so:
hive> alter table api_measurements drop if exists
partition(daydate='2018-04-04', epoch=1522876500);
Dropped the partition daydate=2018-04-04/epoch=1522876500
OK
Time taken: 2.109 seconds
Yet t
Partitioning_and_Bucketing_in_Hive-Based_Big_Data_Warehouses
Cheers,
Gopal
On 4/3/18, 1:42 PM, "Richard A. Bross" wrote:
Gopal,
The Presto devs say they are willing to make the changes to adhere to the
Hive bucket spec. I quoted
"Presto could fix their fai
Mahender,
To really address your question I think that you'd have to supply a bit more
information, such as the kind of data that you want to save; RBDMS type look
ups, key/value/index type look ups, insert velocity, etc. These wide choices
of technologies are suited to different use cases, al
Gopal,
The Presto devs say they are willing to make the changes to adhere to the Hive
bucket spec. I quoted
"Presto could fix their fail-safe for bucketing implementation to actually
trust the Hive bucketing spec & get you out of this mess - the bucketing
contract for Hive is actual file nam
Gopal,
Thanks for this. Great information and something to look at more closely to
better understand the internals.
Rick
- Original Message -
From: "Gopal Vijayaraghavan"
To: user@hive.apache.org
Sent: Tuesday, April 3, 2018 3:15:46 AM
Subject: Re: Hive, Tez, clustering, buckets, and
I'm really confused and could use help understanding. The Hive documentation
here
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL+BucketedTables
says:
"Bucketed tables are fantastic in that they allow much more efficient sampling
than do non-bucketed tables, and they may l
Gopal,
Thanks for taking the time to try and help. A few things in relation to your
response:
* Yes, the 'epoch' column is an hourly timestamp. Clustering by a column with
high cardinality would make little sense.
* I'm interested in your statement that CLUSTERED BY does not CLUSTER BY. My
Hello there,
I've done a ton of reading and testing between Hive, Presto, Tez, etc. In our
test environment I have 8 tables being loaded with Hive that works fine under
certain circumstances.
Most of our data is taken from JSON files in from S3, creates a temporary
external table, and then p
Use JDK8 and try G1 first, we saw lower GC pause under G1 especially in our
large memory environment
On Wednesday, November 29, 2017, 8:14:03 AM PST, eric wong
wrote:
-- 已转发邮件 --
From: eric wong
To: user@hive.apache.org
Cc:
Bcc:
Date: Thu, 30 Nov 2017 00:07:47 +
Hello!
New message, please read <http://xn--80acveirclejkb.xn--p1ai/opinion.php?ib>
Rosenfeld Richard E
Hey friend!
Check this out http://rhaquellasupplierkosmetik.com/different.php?oq
Rosenfeld Richard E
Hi Stephen,
Try increasing the max connections limit in MySQL. I ran into a similar
problem and this resolved things after a Hive 0.12 upgrade. You can
increase the connection limit without a restart, but pay attention to other
settings as well so that you don't over allocate resources on your db
You can use: ALTER TABLE {table_name} ENABLE NO_DROP;
And it will keep a user from dropping the table - but it can be over ridden.
Rick
On Nov 22, 2013 12:36 PM, "Echo Li" wrote:
> Good Friday!
>
> I was trying to apply certain level of security in our hive data
> warehouse, by modifying access
good idea, I will try. thanks
At 2013-10-16 19:12:30,"Ed Soniat" wrote:
You could use a modular math to transform the data in to single value
representations of each range you intend to represent with your boundary using
a sub select.
On Wed, Oct 16, 2013 at 7:09 AM, Richard
with descritized values.
thanks.
Richard
Hi
Is it possible to create a Hive external table pointing to an HBase column
family and for all versions of the data for the keys to be returned rather
than just I presume the latest version of key/column to be returned.
Regards
Hi John,
Do you have a copy of the MySQL JDBC driver in your Hive library path?
Rick
On Apr 3, 2013 3:57 PM, "John Omernik" wrote:
> Not sure what the issues is, conf is good, validated I can log in to mysql
> with username in the hive-site, and I ran the metastore update scripts.
>
> show tabl
I got some hive generated files with .defate extension. I know this is a
compressed file.
It is not my data so i canot change the option to uncompressed. I just want to
view
the file content. But when I used hadoop fs -text, i cannot get plaintext
output. The
output is still binary. How can I fi
What do you have set in core-site.XML for io.sort.mb, io.sort.factor, and
io.file.buffer.size? You should be able to adjust these and get past the
heap issue. Be careful about how much ram you ave though, and don't st them
too high.
Rick
On Jan 30, 2013 8:55 AM, "John Omernik" wrote:
> So it's f
format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
At 2013-01-23 11:45:13,Richard wrote:
I am wondering how to determine the number of map-reduce for a hive query.
for example, the following query
select
sum(c1),
sum(c2),
k1
from
{
select transform(*)
I am wondering how to determine the number of map-reduce for a hive query.
for example, the following query
select
sum(c1),
sum(c2),
k1
from
{
select transform(*) using 'mymapper' as c1, c2, k1
from t1
} a group by k1;
when i run this query, it takes two map-reduce, but I expect it to take
Hi,
I usually write hive script. I found that sometimes if an error occurs, the
hive script will continue
to execute. I do not want this behavor becuase it may hide some problems. I
want it to quit when
any error occurs. Is there any option I should set?
thanks.
Richard
quencefile format
http://my.safaribooksonline.com/book/databases/hadoop/9780596521974/file-based-data-structures/id3555432
that tab is to separate key from values in the record (I may be wrong but this
is how I interpreted it)
On Wed, Jan 9, 2013 at 12:49 AM, Richard wrote:
more informati
I am trying to create a table and insert overwrite it, so the data is supposed
to be generated.
At 2013-01-09 17:17:06,"Anurag Tangri" wrote:
Hi Richard,
You should set the format in create external table command based on the format
of your data on HDFS.
Is your data text f
more information:
if I set the format as textfile, there is no tab space.
if I set the format as sequencefile and view the content via hadoop fs -text, I
saw a tab space in the head of each line.
At 2013-01-09 15:44:00,Richard wrote:
hi there
I have a problem with creating a hive table
delimited fields terminated by '\001' lines terminated by '\n'
49 stored as sequencefile
50 location '${HADOOP_PATH_4_MY_HIVE}/${HIVETBL_my_table}';
thanks for help.
Richard
wrong. otherwise the error message is hard to be nocited.
thanks.
Richard
Hi, I want to use Hive on some data in the following format:
\0x01
field1=val1\0x01
field2=val2\0x01
...
\0x01
the lines between and are a record. How should I define the table?
thanks.
Richard
erwrite a partition
(not even a record/file) if you frame an optimized query.
Hope it helps!...
Regards
Bejoy.K.S
From: Bejoy Ks
To: "user@hive.apache.org"
Sent: Friday, March 16, 2012 5:56 PM
Subject: Re: update a hive table
Hey Richard
First, don't treat h
if I wang to update a table, e.g,
insert overwrite table mytable
select lower(col1), col2, col3 from mytable;
if mytable has many columns but I only need to update one of them, how can I
write the
statement short?
this approach works, I did not realize "var=NULL" is different from "var is
NULL" before.
Richard
At 2012-03-12 12:03:00,"Evan Pollan" wrote:
I used a left outer join with a constraint on a joined row column value being
null -- logically equivalent to a &quo
Hi Sachin
I have tried LEFT SEMI JOIN, but the results are the same to a JOIN operation.
Richard
At 2012-03-12 12:03:39,sac.khur...@gmail.com wrote:
Hi richard,
You can use left semi join.
SELECT a.key.
FROM a LEFT SEMI JOIN b on (a.key = b.key)
Thanks,
Sachin
Sent from BlackBerry® on
proper to use histogram_numeric(cast(col to BIGINT), n).
thanks.
Richard
hi,
how can I join two tables A and B so that the result is "In A but not in B"?
let's take an example, say, the column to identify record is id.
e.g.
select A.* from A join B on (A.id = B.id)
thanks.
Richard
A custom SerDe would be your best bet. We're using one to do exactly that.
Regards,
Rick
On Apr 28, 2011 11:29 AM, "Shantian Purkad"
wrote:
> Any suggestions?
>
>
>
>
> From: Shantian Purkad
> To: user@hive.apache.org
> Sent: Tue, April 26, 2011 11:05:46 PM
> Su
43 matches
Mail list logo
