Re: WordCount w/ YARN and EMR local filesystem and/or HDFS

2016-08-23 Thread Robert Metzger
o on the > cluster. > > > > > > > > *From: *Stephan Ewen > *Reply-To: *"user@flink.apache.org" > *Date: *Tuesday, August 23, 2016 at 11:47 AM > *To: *"user@flink.apache.org" , Robert Metzger < > rmetz...@data-artisans.com> > > *Subje

Re: WordCount w/ YARN and EMR local filesystem and/or HDFS

2016-08-23 Thread Stephan Ewen
ponseMessageWrapper. > readFields(ProtobufRpcEngine.java:417) > > at org.apache.hadoop.security.SaslRpcClient.saslConnect( > SaslRpcClient.java:370) > > at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client. > java:560) > > at org.apache.hadoop.ipc.Client$

Re: WordCount w/ YARN and EMR local filesystem and/or HDFS

2016-08-23 Thread Stephan Ewen
Hi! The file "/home/hadoop/LICENSE.txt" probably exists only on the machine that starts the job (your workstation, laptop), not in the cluster. The Flink processes in the cluster cannot find the file under that address. The input data must be in a filesystem that all cluster nodes can access, lik

WordCount w/ YARN and EMR local filesystem and/or HDFS

2016-08-23 Thread Foster, Craig
I'm trying to use the wordcount example with the local file system, but it's giving me permissions error or it's not finding it. It works just fine for input and output on S3. What is the correct URI usage for the local file system and HDFS? I have installed Flink on EMR and am just using the f