Hi, This is the log with setting “localhost” flink-Vidura-jobmanager-localhost.log <https://gist.github.com/viduranga/e9d43521587697de3eb5#file-flink-vidura-jobmanager-localhost-log>
And this is the log with setting “127.0.0.1” flink-Vidura-jobmanager-localhost.log <https://gist.github.com/viduranga/5af6b05f204e1f4b344f#file-flink-vidura-jobmanager-localhost-log> > On Mar 5, 2015, at 2:23 PM, Till Rohrmann <trohrm...@apache.org> wrote: > > What does the jobmanager log says? I think Stephan added some more logging > output which helps us to debug this problem. > > On Thu, Mar 5, 2015 at 9:36 AM, Dulaj Viduranga <vidura...@icloud.com> > wrote: > >> Using start-locat.sh. >> I’m using the original config yaml. I also tried changing jobmanager >> address in config to “127.0.0.1 but no luck. With my changes it works ok. >> The conf file follows. >> >> >> ################################################################################ >> # Licensed to the Apache Software Foundation (ASF) under one >> # or more contributor license agreements. See the NOTICE file >> # distributed with this work for additional information >> # regarding copyright ownership. The ASF licenses this file >> # to you under the Apache License, Version 2.0 (the >> # "License"); you may not use this file except in compliance >> # with the License. You may obtain a copy of the License at >> # >> # http://www.apache.org/licenses/LICENSE-2.0 >> # >> # Unless required by applicable law or agreed to in writing, software >> # distributed under the License is distributed on an "AS IS" BASIS, >> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. >> # See the License for the specific language governing permissions and >> # limitations under the License. >> >> ################################################################################ >> >> >> >> #============================================================================== >> # Common >> >> #============================================================================== >> >> jobmanager.rpc.address: 127.0.0.1 >> >> jobmanager.rpc.port: 6123 >> >> jobmanager.heap.mb: 256 >> >> taskmanager.heap.mb: 512 >> >> taskmanager.numberOfTaskSlots: 1 >> >> parallelization.degree.default: 1 >> >> >> #============================================================================== >> # Web Frontend >> >> #============================================================================== >> >> # The port under which the web-based runtime monitor listens. >> # A value of -1 deactivates the web server. >> >> jobmanager.web.port: 8081 >> >> # The port uder which the standalone web client >> # (for job upload and submit) listens. >> >> webclient.port: 8080 >> >> >> #============================================================================== >> # Advanced >> >> #============================================================================== >> >> # The number of buffers for the network stack. >> # >> # taskmanager.network.numberOfBuffers: 2048 >> >> # Directories for temporary files. >> # >> # Add a delimited list for multiple directories, using the system directory >> # delimiter (colon ':' on unix) or a comma, e.g.: >> # /data1/tmp:/data2/tmp:/data3/tmp >> # >> # Note: Each directory entry is read from and written to by a different I/O >> # thread. You can include the same directory multiple times in order to >> create >> # multiple I/O threads against that directory. This is for example >> relevant for >> # high-throughput RAIDs. >> # >> # If not specified, the system-specific Java temporary directory >> (java.io.tmpdir >> # property) is taken. >> # >> # taskmanager.tmp.dirs: /tmp >> >> # Path to the Hadoop configuration directory. >> # >> # This configuration is used when writing into HDFS. Unless specified >> otherwise, >> # HDFS file creation will use HDFS default settings with respect to >> block-size, >> # replication factor, etc. >> # >> # You can also directly specify the paths to hdfs-default.xml and >> hdfs-site.xml >> # via keys 'fs.hdfs.hdfsdefault' and 'fs.hdfs.hdfssite'. >> # >> # fs.hdfs.hadoopconf: /path/to/hadoop/conf/ >> >> >>> On Mar 5, 2015, at 2:03 PM, Till Rohrmann <trohrm...@apache.org> wrote: >>> >>> How did you start the flink cluster? Using the start-local.sh, the >>> start-cluster.sh or starting the job manager and task managers >> individually >>> using taskmanager.sh/jobmanager.sh. Could you maybe post the >>> flink-conf.yaml file, you're using? >>> >>> With your changes, everything works, right? >>> >>> On Thu, Mar 5, 2015 at 8:55 AM, Dulaj Viduranga <vidura...@icloud.com> >>> wrote: >>> >>>> Hi Till, >>>> I’m sorry. It doesn’t seem to solve the problem. The taskmanager still >>>> tries a 10.0.0.0/8 IP. >>>> >>>> Best regards. >>>> >>>>> On Mar 5, 2015, at 1:00 PM, Till Rohrmann <till.rohrm...@gmail.com> >>>> wrote: >>>>> >>>>> Hi Dulaj, >>>>> >>>>> I looked through your commit and noticed that the JobClient might not >> be >>>>> listening on the right network interface. Your commit seems to fix it. >> I >>>>> just want to understand the problem properly and therefore I opened a >>>>> branch with a small change. Could you try out whether this change would >>>>> also fix your problem? You can find the code here [1]. Would be awesome >>>> if >>>>> you checked it out and let it run on your cluster setting. Thanks a lot >>>>> Dulaj! >>>>> >>>>> [1] >>>>> >>>> >> https://github.com/tillrohrmann/flink/tree/fixLocalFlinkMiniClusterJobClient >>>>> >>>>> On Thu, Mar 5, 2015 at 4:21 AM, Dulaj Viduranga <vidura...@icloud.com> >>>>> wrote: >>>>> >>>>>> The every change in the commit b7da22a is not required but I thought >>>> they >>>>>> are appropriate. >>>>>> >>>>>>> On Mar 5, 2015, at 8:11 AM, Dulaj Viduranga <vidura...@icloud.com> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> I found many other places “localhost” is hard coded. I changed them >> in >>>> a >>>>>> better way I think. I made a pull request. Please review. b7da22a < >>>>>> >>>> >> https://github.com/viduranga/flink/commit/b7da22a562d3da5a9be2657308c0f82e4e2f80cd >>>>>>> >>>>>>> >>>>>>>> On Mar 4, 2015, at 8:17 PM, Stephan Ewen <se...@apache.org> wrote: >>>>>>>> >>>>>>>> If I recall correctly, we only hardcode "localhost" in the local >> mini >>>>>>>> cluster - do you think it is problematic there as well? >>>>>>>> >>>>>>>> Have you found any other places? >>>>>>>> >>>>>>>> On Mon, Mar 2, 2015 at 10:26 AM, Dulaj Viduranga < >>>> vidura...@icloud.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> In some places of the code, "localhost" is hard coded. When it is >>>>>> resolved >>>>>>>>> by the DNS, it is posible to be directed to a different IP other >>>> than >>>>>>>>> 127.0.0.1 (like private range 10.0.0.0/8). I changed those places >> to >>>>>>>>> 127.0.0.1 and it works like a charm. >>>>>>>>> But hard coding 127.0.0.1 is not a good option because when the >>>>>> jobmanager >>>>>>>>> ip is changed, this becomes an issue again. I'm thinking of setting >>>>>>>>> jobmanager ip from the config.yaml to these places. >>>>>>>>> If you have a better idea on doing this with your experience, >> please >>>>>> let >>>>>>>>> me know. >>>>>>>>> >>>>>>>>> Best. >>>>>>>>> >>>>>>> >>>>>> >>>>>> >>>> >>>> >> >>
