Re: accessing hdfs cluster through ssh tunnel

You will have to use a socks proxy (-D option in ssh tunnel). In addition, when invoking hadoop fs command, you will have to add -Dsocks.proxyHost and - Dsocks.proxyPort. Thanks, Hariharan On Thu, 12 Sep 2019, 23:26 saurabh pratap singh, wrote: > Thank you so much for your reply . > I have furt

Re: Python Hadoop Example

Hi Wei-Chiu, You can look at Dask [1]. It can work with HDFS [2] and integrates well with YARN as well [3]. 1 - https://dask.org 2 - http://docs.dask.org/en/latest/remote-data-services.html 3 - http://yarn.dask.org/en/latest/ Thanks, Hari On Sun, 16 Jun 2019, 23:31 Wei-Chiu Chuang, wrote: >

Re: aggregated logs on HDFS garbled

Application logs are not saved in text format. They are saved in a binary format called TFile. You can use the "yarn logs"[1] command to view them. Alternatively you can use a tool like this tfile reader[2]. [1] - https://hadoop.apache.org/docs/r2.7.4/hadoop-yarn/hadoop-yarn-site/YarnCommands.htm