Re: Cannot INSERT OVERWRITE on clustered table with > 8 buckets

2018-07-13 Thread Gopal Vijayaraghavan
> I'm using Hive 1.2.1 with LLAP on HDP 2.6.5. Tez AM is 3GB, there are 3 > daemons for a total of 34816 MB. Assuming you're using Hive2 here (with LLAP) and LLAP kinda sucks for ETL workloads, but this is a different problem. > PARTITIONED BY (DATAPASSAGGIO string, ORAPASSAGGIO string) > CLU

Re: Cannot INSERT OVERWRITE on clustered table with > 8 buckets

2018-07-14 Thread Nicolas Paris
​Hi Gopal​ Can you try running with (& see what your query read-perf looks like) > https://gist.github.com/t3rmin4t0r/087b61f79514673c307bb9a88327a4db > > CREATE TABLE IF NOT EXISTS passaggi1718 > ( > ... > ) > PARTITIONED BY (DATAPASSAGGIO string) > CLUSTERED BY (ORAPASSAGGIO) INTO

Re: Cannot INSERT OVERWRITE on clustered table with > 8 buckets

2018-07-14 Thread Gopal Vijayaraghavan
​​> Or a simple insert will be automatically sorted as the table DDL mention ? Simple insert should do the sorting, older versions of Hive had ability to disable that (which is a bad thing & therefore these settings are now just hard-configed to =true in Hive3.x) -- set hive.enforce.bucketin

Re: Cannot INSERT OVERWRITE on clustered table with > 8 buckets

2018-07-16 Thread Federico D'Ambrosio
Hi Gopal, Thank you very much for your response and your very helpful insight on my problem. As you correctly guessed, I'm quite new to Hive. I tried the query you suggested, in your gist, for the creation of the table without using LLAP and it worked fine, as in it didn't hang like the one I tri