Hi experts
I'm very new in Hive and Hadoop and I want to create a very simple demo to
analyse sample twitts like this:
T 2009-06-08 21:49:37
U http://twitter.com/evion
W I think data mining is awesome!
T 2009-06-08 21:49:37
U http://twitter.com/hyungjin
W I don’t think so. I don’t like data
Hi Shan,
If you happen to have a lot of repeated data (in the most general
grouping), you might get some speedup by little pre-aggregation. The
following code should produce the same results as the example in your
first post:
From (
SELECT a, b , c, count(*) AS cnt
FROM X
group by a,b,c
)
If you get output onto a single line it will be much easier for hive to process.
On Tue, Jun 5, 2012 at 5:20 AM, Babak Bastan babak...@gmail.com wrote:
Hi experts
I'm very new in Hive and Hadoop and I want to create a very simple demo to
analyse sample twitts like this:
T 2009-06-08
ok, no difference for me records in a line or not
2009-06-08 21:49:37 - http://twitter.com/
http://twitter.com/evionblablabla- I
think data mining is awesome!
2009-06-08 21:49:37 - http://twitter.com/
http://twitter.com/hyungjinbliblibli
- I don’t think so. I don’t like data mining
How can
Hi Babak
There isn't anything called hdfs files. Hdfs is just a file system that can
store any type of file. You just need to transfer your file from lfs to hdfs
and the following command helps you out for that
hadoop fs -copyFromLocal location of file in lfs destination location in
hdfs
Thank you for your answer
location of file in lfs
That means the location of my *txt file on my computer ? and I have no
destination address in hdfs,where can I get this location?
could you please write an example?
On Tue, Jun 5, 2012 at 4:29 PM, Bejoy Ks bejoy...@yahoo.com wrote:
Hi Babak
If you type hadoop fs -ls / it will show you the folders that currently exist
on your hadoop cluster.
Regards,
[02AXP_4C_grad]
Anurag Gulati | Lead Programmer Analyst | Disruptive Innovation -
Socializing Acquisition
' (602) 537-7265 | * anurag.gul...@aexp.com
Lfs means local file system.
Hadoop fs -copyFromLocal will help to copy data from your local file system to
the Hadoop distributed file system. Not sure what kind of cluster setup you
have, are you running in local or pseudo distributed mode?
Here is a link to get you started on hive
Thank you Bejoy for your complete answer :)
if I run this command:
hadoop fs -ls /
I get this results:
drwxr-xr-x - root root 4096 2011-04-26 01:06 /var
drwxrwxrwx - root root 4096 2012-06-05 18:38 /tmp
drwxr-xr-x - root root 12288 2012-06-05 17:44 /etc
-rw-r--r-- 1 root
Hi Babak
Looks like your hadoop is not configured correctly. The list gives me a pulse
that it is showing lfs rather than hdfs. Have you configured your
'fs.default.name' in core-site.xl to point to hdfs:// instead of file:/// .
You may need to revisit your hadoop setup. Try out the book I
HI Babak
It looks like a hadoop configuration problem for me. Have you configured your
'fs.default.name' in core-site.xl to point to hdfs:// instead of file:/// ? If
not that is likely to be the issue. Also if you are usinh hdfs for first time
there are other steps like formatting namenode
set
hive.metastore.warehouse.dir in hive-site.xml
property
namehive.metastore.local/name
valuetrue/value
/property
namehive.metastore.warehouse.dir/name
value/home/your username/hivefolder/value
descriptionlocation of default database for the
Thanks sShashwat, and where is this hive-site.xml
On Tue, Jun 5, 2012 at 8:02 PM, shashwat shriparv dwivedishash...@gmail.com
wrote:
set
hive.metastore.warehouse.dir in hive-site.xml
property
namehive.metastore.local/name
valuetrue/value
/property
It will be inside hive/conf
On Tue, Jun 5, 2012 at 11:43 PM, Babak Bastan babak...@gmail.com wrote:
Thanks sShashwat, and where is this hive-site.xml
On Tue, Jun 5, 2012 at 8:02 PM, shashwat shriparv
dwivedishash...@gmail.com wrote:
set
hive.metastore.warehouse.dir in hive-site.xml
It must be inside the configuration/configuration or outside this?
On Tue, Jun 5, 2012 at 8:15 PM, shashwat shriparv dwivedishash...@gmail.com
wrote:
It will be inside hive/conf
On Tue, Jun 5, 2012 at 11:43 PM, Babak Bastan babak...@gmail.com wrote:
Thanks sShashwat, and where is this
Thank you so much my friend your idee works fine(no error) you are the best
:)
On Tue, Jun 5, 2012 at 8:20 PM, Babak Bastan babak...@gmail.com wrote:
It must be inside the configuration/configuration or outside this?
On Tue, Jun 5, 2012 at 8:15 PM, shashwat shriparv
inside configuration. all properties will be inside the configuration tags
On Tue, Jun 5, 2012 at 11:53 PM, Babak Bastan babak...@gmail.com wrote:
Thank you so much my friend your idee works fine(no error) you are the
best :)
On Tue, Jun 5, 2012 at 8:20 PM, Babak Bastan babak...@gmail.com
Ok sorry but that was my Mistake .I thought it works but no.
I wrote the command without ; and then I think It works but with ; at the
end of command
CREATE TABLE pokes (foo INT, bar STRING);
does'nt work
On Tue, Jun 5, 2012 at 8:34 PM, shashwat shriparv dwivedishash...@gmail.com
wrote:
@Bejoy: I set the fs.default.name in the core-site.xml and I did all of
thing that was mentioned in the reference but no effect
On Tue, Jun 5, 2012 at 8:43 PM, Babak Bastan babak...@gmail.com wrote:
Ok sorry but that was my Mistake .I thought it works but no.
I wrote the command without ; and
Hi Babak
You gotta follow those instructions in the apace site to set up hadoop from
scratch and ensure that hdfs is working first. You should be able to read and
write files to hdfs before you do your next steps.
Are you on CDH or apache distribution of hadoop? If it is CDH there are
no I'm not working on CDH.Is there a way to test if my Hadoop works fine or
not?
On Tue, Jun 5, 2012 at 9:55 PM, Bejoy KS bejoy...@yahoo.com wrote:
**
Hi Babak
You gotta follow those instructions in the apace site to set up hadoop
from scratch and ensure that hdfs is working first. You
Hello everyone, I develop a scientific research for my college, where
I conduct experiments involving hive and I wonder what is the easiest
way to install the hive.
I've tried installing using claudera, but I had trouble picking up the mirrors.
I'd rather install it using ubuntu or other debian
Hi Rafael,
Just download Hive from apache and set all the variables properly,
and you are good to go. For detailed help you can visit -
https://cwiki.apache.org/confluence/display/Hive/GettingStarted;...
And this is the official wikipage -
Thanks, Mark.
I was hoping the expressions could be evaluated at query time, but
generating the query works just as well.
On Mon, Jun 4, 2012 at 7:42 PM, Mark Grover mgro...@oanda.com wrote:
Hi Matthew,
unix_timestamp() is being executed on the Hive server in your case.
Therefore, as your
Hi Guys, While I am trying to understand the archive functionality/code in
HIVE. I noticed that the default value of “hive.archive.har.parentdir.settable”
setting is false. But shim is using this setting to pass one of the mandatory
argument “–P” to HadoopArchives class.
Is there any place
Hi Jan,
The quick answer is I don't know but may be someone else on the mailing
list does:-)
Looking at the wiki page for Lateral view(
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView),
there was a problem related to predicate pushdown on UDTF's (
Hello!
I can't understand what's wrong.
drop table if exists t0;
create table t0 (short_descr string) STORED AS SEQUENCEFILE;
from (
select cast(2 as int) identifier2, short_descr area_name from t0) asdf
select *--area_name
order by identifier2 desc, area_name asc
;
The query works fine
27 matches
Mail list logo
