Are you sure the RDD that you were saving isn't empty!?
Are you seeing a _SUCCESS file in this location? hdfs://
masteripaddress:9000/root/test-app/test1/
(Do hadoop fs -ls hdfs://masteripaddress:9000/root/test-app/test1/)
Thanks
Best Regards
On Thu, Jul 24, 2014 at 4:24 PM, lmk
Hi Akhil,
I am sure that the RDD that I saved is not empty. I have tested it using
take.
But is there no way that I can see this saved physically like we do in the
normal context? Can't I view this folder as I am already logged into the
cluster?
And, should I run hadoop fs -ls
This piece of code
saveAsHadoopFile[TextOutputFormat[NullWritable,Text]](hdfs://
masteripaddress:9000/root/test-app/test1/)
Saves the RDD into HDFS, and yes you can physically see the files using the
hadoop command (hadoop fs -ls /root/test-app/test1 - yes you need to login
to the cluster). In
Thanks Akhil.
I was able to view the files. Actually I was trying to list the same using
regular ls and since it did not show anything I was concerned.
Thanks for showing me the right direction.
Regards,
lmk
--
View this message in context:
