I am interested in ML but I want a Hadoop base because I am learning
hadoop. Mahout seems to be for ML at this time. Not Hadoop.
Thanks,
Mohan
On Tue, Apr 29, 2014 at 7:38 AM, Shahab Yunus wrote:
> For Machine Learning based applications of Hadoop you can check-out Mahout
> framework.
>
> Regar
Hi Folks,
I am running a map only (no reduce) job on Hadoop 2.3.0 and the
/tmp/hadoop-df/nm-local-dir (df being my user name) directory is growing
exponentially causing me run out of my 80GB disk space.
I would like to know if there is any mechanism(automatic or manual) by
which I can free up th
Hi Raj,
> Should 'john1' be included in the 'sudoers' file ?
Hadoop don't use root privileges. But it has some built-in users and
groups like hdfs, mapred, etc. I think you should add your admin user at
least to groups hdfs and mapred.
--
Thanks,
Sergey
On 28/04/14 23:40, Raj Hadoop wrote:
> Hi,
For Machine Learning based applications of Hadoop you can check-out Mahout
framework.
Regards,
Shahab
On Mon, Apr 28, 2014 at 10:02 PM, Mohan Radhakrishnan <
radhakrishnan.mo...@gmail.com> wrote:
> Hi,
>I have been reading the definitive guide and taking online courses.
> Now I would li
Hi Harsh,
Hadoop write once a packet, and GZIP compressed file should write
completely, so I think if the packet size bigger than the compressed file, I
can make sure the compressed file is not written at all or completed
written.
Is it right ?
Thanks a lot.
Regards,
Ken Huang
-邮件原件-
发件人
Hi,
I have been reading the definitive guide and taking online courses.
Now I would like to understand how Hadoop is used for more real-time
scenarios. Are machine learning, language processing and fraud detection
examples available ? What are the other practical usecases ?
I am familiar wi
You do not need to alter the packet size to write files - why do you
think you need larger packets than the default one?
On Mon, Apr 28, 2014 at 4:04 PM, wrote:
> Hi Harsh,
>
>
>
> I’m using HDFS client to write GZIP compressed files, I want to write once a
> file, in order to not uncompressing
Regards, Motty
This kind of questions, I think that should be asked in the CDH Users mailing
list. There, you will obtain a better and a faster answer.
Best wishes
On Monday, April 28, 2014 01:00:13 PM motty cruz wrote:
> Hello, I'm upgrading to CDH5. I download latest parcel from
> http://archiv
Hello, I'm upgrading to CDH5. I download latest parcel from
http://archive.cloudera.com/cdh5/parcels/latest/
to /oprt/cloudera/parcel-repo next to cluster on cludera under parcels -->
I hit the distribution button, started to distribute got to 50% but it does
not go any further. any ideas how to p
Hi All,
I just joined this group, and not sure whether this question was discussed
before.
Is it possible to run Application Master within the same process as Node
Manager? If not, any plan to support it future?
I am asking this because we might want to use YARN as a job dispatching
system. We h
Hi,
I have a cluster CDH4. How can one perform hadoop admin without root access.
Basically an account like 'john1' on the cluster want to have access to hdfs,
mapred etc.,
Should 'john1' be included in the 'sudoers' file ?
What instructions should I ask System Admin team to have 'john1' acc
Hi,
Following seq is done
hdfs dfs -mkdir /a
take snapshot s_0
hdfs dfs -mkdir -p /a/b/c
hdfs dfs -put foo /a/b/c
take snapshot s_1
Now the command line snapshotdiff between s_0 and s_1 shows just the
addition of directory "b". It should show addition of directory "b/c" as
well as addition of "b
Not 755, but yeah. See DefaultContainerExecutor.createAppLogDirs(). You may
have to debug more though.
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Fri, Apr 25, 2014 at 8:42 AM, sam liu wrote:
> My version is 2.1.0 and the cluster uses DefaultContainerExecutor. Is it
> possible that Defa
Currently, we are working on setting up a production hadoop cluster. We have a
smaller cluster of four nodes we are working on for development (nn, snn, and
two datanodes). From a high level, we flume our data into HDFS, and then due to
the way that flume may resend data on error (due to the wa
Hi Silvina,
Thanks for your reply.
cmake is installed, I try the following:
apt-get install cmake
Reading package lists... Done
Building dependency tree
Reading state information... Done
cmake is already the newest version.
0 upgraded, 0 newly insta
Sudhakar,
will this set for the new files being written to those directories?
On Mon, Apr 28, 2014 at 5:52 PM, sudhakara st wrote:
> Changes the replication factor of a file. -R option is for recursively
> increasing the replication factor of files within a directory.
>
> Example:
>
>- had
Changes the replication factor of a file. -R option is for recursively
increasing the replication factor of files within a directory.
Example:
- hadoop fs -setrep -w 3 -R /user/hadoop/dir1
hadoop dfs -setrep -R -w 1 /dir/
On Mon, Apr 28, 2014 at 2:29 PM, Nitin Pawar wrote:
> DFS replicati
Hello,
I wish to disable checksum calculation & verification in Hadoop's framework,
I run the TestDfsIO application with the parameter dfs.checksum.type=NULL,
To my best understanding, this was supposed to turnoff check summing in the
framework,
But I received the following error -
...
...
...
1
Are you sure that CMake is installed?
Best,
Silvina
On 28 April 2014 13:05, ascot.m...@gmail.com wrote:
> Hi,
>
> I am trying to install Hadoop 2.4.0 from source, I got the following
> error, please help!!
>
> Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
>
> Regards
>
>
Hi,
I am trying to install Hadoop 2.4.0 from source, I got the following error,
please help!!
Can anyone share the "apache-maven-3.1.1/conf/settings.xml” setting?
Regards
O/S Ubuntu: 12.04 (64-bit)
Java: java version "1.6.0_45"
protoc —version: libprotoc 2.5.0
Command: mvn package -Pdist,na
Hi Harsh,
I’m using HDFS client to write GZIP compressed files, I want to write once
a file, in order to not uncompressing it. So I should make every write
completely, otherwise file will corrupted.
I’m raising up the client’s write packet size to avoid partially write.
But it doesn’t work, s
Colons in path names may not currently work well:
https://issues.apache.org/jira/browse/HADOOP-7945
On Mon, Apr 28, 2014 at 3:26 PM, Patcharee Thongtra
wrote:
> Hi,
>
> I tried to put escape chars around it, but it does not work.
>
> Patcharee
>
>
> On 04/28/2014 11:45 AM, Nitin Pawar wrote:
>
>
Hi,
I tried to put escape chars around it, but it does not work.
Patcharee
On 04/28/2014 11:45 AM, Nitin Pawar wrote:
try putting escape chars around it
On Mon, Apr 28, 2014 at 2:52 PM, Patcharee Thongtra
mailto:patcharee.thong...@uni.no>> wrote:
Hi,
My file name contains ":" and
try putting escape chars around it
On Mon, Apr 28, 2014 at 2:52 PM, Patcharee Thongtra <
patcharee.thong...@uni.no> wrote:
> Hi,
>
> My file name contains ":" and I got error "copyFromLocal: unexpected
> URISyntaxException" when I try to copy this file to Hadoop. See below.
>
> [patcharee@comput
Hi,
My file name contains ":" and I got error "copyFromLocal: unexpected
URISyntaxException" when I try to copy this file to Hadoop. See below.
[patcharee@compute-1-0 ~]$ hadoop fs -copyFromLocal
wrfout_d01_2001-01-01_00:00:00 netcdf_data/
copyFromLocal: unexpected URISyntaxException
I am u
DFS replication is set on the file level (block level) or at cluster level
(if you do not specify the replication factor while writing the file then
this one is picked).
As per my understanding, there is nothing for directories.
On Mon, Apr 28, 2014 at 2:12 PM, Meng QingPing wrote:
> Hi,
>
>
Hi,
I want set dfs.replication as 1 for dfs /tmp, set dfs.replication as 3 for
dfs /user. How to configure? Both /tmp and /user are generated by
mapreduce job or hive or sqoop.
Thanks,
Jack
27 matches
Mail list logo
