Hello Everyone,
I am trying to read(open) and write files in hdfs inside a python script. But
having error. Can someone tell me what is wrong here.
Code (full): sample.py
#!/usr/bin/python
from subprocess import Popen, PIPE
print "Before Loop"
cat = Popen(["hadoop", "fs", "-cat", "./sample.txt"],
stdout=PIPE)
put = Popen(["hadoop", "fs", "-put", "-", "./modifiedfile.txt"],
stdin=PIPE)
for line in cat.stdout:
line += "Blah"
print line
put.stdin.write(line)
cat.stdout.close()
cat.wait()
put.stdin.close()
put.wait()
When I execute :
hadoop jar
/usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.5.1.jar -file
./sample.py -mapper './sample.py' -input sample.txt -output fileRead
It executes properly I couldn't find the file which supposed to create in hdfs
modifiedfile
And When I execute :
hadoop fs -getmerge ./fileRead/ file.txt
Inside the file.txt, I got :
Before Loop
Before Loop
Can someone please tell me what I am doing wrong here ?? I dont think it reads
from the sample.txt
I would really appreciate the help.
--
Thanks & Regards,
Shalini Ravishankar.
--
https://mail.python.org/mailman/listinfo/python-list
