Re: Netty issues while deploying Flink with Yarn on MapR
Just FYI: There is now a documentation page on how to use Flink on MapR (Thanks to Gordon): https://ci.apache.org/projects/flink/flink-docs- release-1.3/setup/mapr_setup.html On Tue, Feb 7, 2017 at 6:34 PM, Robert Metzger wrote: > Hi, > cool! > > Yes, creating a JIRA for the problem is a good idea. > Once you've found a way to fix the issue, you can open a pull request > referencing the issue. > > Regards, > Robert > > > > On Tue, Feb 7, 2017 at 6:20 PM, ani.desh1512 > wrote: > >> Thanks Robert. >> I would love to try to solve this problem so that future MapR and Flink >> users do not face these issues. Should I create a JIRA for it? Let me know >> how I can be of help. >> >> >> >> -- >> View this message in context: http://apache-flink-user-maili >> ng-list-archive.2336050.n4.nabble.com/Netty-issues-while-dep >> loying-Flink-with-Yarn-on-MapR-tp11411p11497.html >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> > >
Re: Netty issues while deploying Flink with Yarn on MapR
Hi, cool! Yes, creating a JIRA for the problem is a good idea. Once you've found a way to fix the issue, you can open a pull request referencing the issue. Regards, Robert On Tue, Feb 7, 2017 at 6:20 PM, ani.desh1512 wrote: > Thanks Robert. > I would love to try to solve this problem so that future MapR and Flink > users do not face these issues. Should I create a JIRA for it? Let me know > how I can be of help. > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Netty-issues- > while-deploying-Flink-with-Yarn-on-MapR-tp11411p11497.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. >
Re: Netty issues while deploying Flink with Yarn on MapR
Thanks Robert. I would love to try to solve this problem so that future MapR and Flink users do not face these issues. Should I create a JIRA for it? Let me know how I can be of help. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Netty-issues-while-deploying-Flink-with-Yarn-on-MapR-tp11411p11497.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Netty issues while deploying Flink with Yarn on MapR
Hi Aniket, great analysis of the problem! Thank you for looking so well into it! Would you be interested in trying to solve the problem for Flink? We could try to provide a maven build profile that sets the correct versions and excludes. We could maybe also provide a MapR specific release of Flink in the future. I've seen many problems with Flink on MapR recently, and it would be good to fix them "forever". On Tue, Feb 7, 2017 at 4:41 PM, Ufuk Celebi wrote: > Thanks for reporting the solution. > > @Robert: Is that a general issue we have with Flink YARN on MapR? > > On 7 February 2017 at 15:50:17, ani.desh1512 (ani.desh1...@gmail.com) > wrote: > > In case anyone is having similar issues with Flink on Yarn on MapR, I > > managed to solve > > > > this issue with help from the MapR community. > > > > > > > > -- > > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Netty-issues- > while-deploying-Flink-with-Yarn-on-MapR-tp11411p11486.html > > Sent from the Apache Flink User Mailing List archive. mailing list > archive at Nabble.com. > > > >
Re: Netty issues while deploying Flink with Yarn on MapR
Thanks for reporting the solution. @Robert: Is that a general issue we have with Flink YARN on MapR? On 7 February 2017 at 15:50:17, ani.desh1512 (ani.desh1...@gmail.com) wrote: > In case anyone is having similar issues with Flink on Yarn on MapR, I > managed to solve > > this issue with help from the MapR community. > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Netty-issues-while-deploying-Flink-with-Yarn-on-MapR-tp11411p11486.html > > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com. >
Re: Netty issues while deploying Flink with Yarn on MapR
In case anyone is having similar issues with Flink on Yarn on MapR, I managed to solve <https://community.mapr.com/message/55664-re-netty-issues-while-deploying-flink-with-yarn-on-mapr> this issue with help from the MapR community. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Netty-issues-while-deploying-Flink-with-Yarn-on-MapR-tp11411p11486.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Netty issues while deploying Flink with Yarn on MapR
I am trying to run Flink using Yarn on MapR. My previous issue
<http://http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Flink-with-Yarn-on-MapR-tt15448.html>
got resolved and I have updated the original post accordingly so.
Accordingly, I modified pom.xml to change the zookeeper version to *mapr
zookeeper jar* version which in my case was: *3.4.5-mapr-1604*
I then built flink (*flink-1.3-SNAPSHOT*) as follows:
*mvn clean install -DskipTests -Pvendor-repos
-Dhadoop.version=2.7.0-mapr-1607*
The build is successfull. Then I try to run *./bin/yarn-session.sh -n 3* and
get the following error:
/2017-02-02 16:11:10,717 INFO org.apache.flink.yarn.YarnClusterDescriptor
- Using values:
2017-02-02 16:11:10,718 INFO org.apache.flink.yarn.YarnClusterDescriptor
- TaskManager count = 3
2017-02-02 16:11:10,718 INFO org.apache.flink.yarn.YarnClusterDescriptor
- JobManager memory = 1024
2017-02-02 16:11:10,718 INFO org.apache.flink.yarn.YarnClusterDescriptor
- TaskManager memory = 1024
2017-02-02 16:11:10,928 INFO com.mapr.util.zookeeper.ZKDataRetrieval
- Process path: null. Event state: SyncConnected. Event type: None
2017-02-02 16:11:10,928 INFO com.mapr.util.zookeeper.ZKDataRetrieval
- Connected to ZK:
ip-10-101-2-111.ec2.internal:5181,ip-10-101-2-112.ec2.internal:5181,ip-10-101-2-113.ec2.internal:5181
2017-02-02 16:11:10,929 INFO com.mapr.util.zookeeper.ZKDataRetrieval
- Getting serviceData for master node of resourcemanager
2017-02-02 16:11:10,935 INFO com.mapr.util.zookeeper.ZKDataRetrieval
- Process path: null. Event state: SaslAuthenticated. Event type: None
2017-02-02 16:11:10,948 INFO
org.apache.hadoop.yarn.client.MapRZKBasedRMFailoverProxyProvider - Updated
RM address to ip-10-101-2-111.ec2.internal/10.101.2.111:8032
2017-02-02 16:11:11,216 WARN org.apache.flink.yarn.YarnClusterDescriptor
- The configuration directory
('/home/ubuntu/flink/flink-dist/target/flink-1.3-SNAPSHOT-bin/flink-1.3-SNAPSHOT/conf')
contains both LOG4J and Logback configuration files. Please delete or rename
one of them.
2017-02-02 16:11:11,225 INFO org.apache.flink.yarn.Utils
- Copying from
file:///home/ubuntu/flink/flink-dist/target/flink-1.3-SNAPSHOT-bin/flink-1.3-SNAPSHOT/conf/log4j.properties
to
maprfs:/user/ubuntu/.flink/application_1485984594262_0007/log4j.properties
2017-02-02 16:11:11,249 INFO org.apache.flink.yarn.Utils
- Copying from
file:///home/ubuntu/flink/flink-dist/target/flink-1.3-SNAPSHOT-bin/flink-1.3-SNAPSHOT/lib
to maprfs:/user/ubuntu/.flink/application_1485984594262_0007/lib
2017-02-02 16:11:11,680 INFO org.apache.flink.yarn.Utils
- Copying from
file:///home/ubuntu/flink/flink-dist/target/flink-1.3-SNAPSHOT-bin/flink-1.3-SNAPSHOT/conf/logback.xml
to maprfs:/user/ubuntu/.flink/application_1485984594262_0007/logback.xml
2017-02-02 16:11:11,685 INFO org.apache.flink.yarn.Utils
- Copying from
file:///home/ubuntu/flink/flink-dist/target/flink-1.3-SNAPSHOT-bin/flink-1.3-SNAPSHOT/lib/flink-dist_2.10-1.3-SNAPSHOT.jar
to
maprfs:/user/ubuntu/.flink/application_1485984594262_0007/flink-dist_2.10-1.3-SNAPSHOT.jar
2017-02-02 16:11:12,932 INFO org.apache.flink.yarn.Utils
- Copying from
/home/ubuntu/flink/flink-dist/target/flink-1.3-SNAPSHOT-bin/flink-1.3-SNAPSHOT/conf/flink-conf.yaml
to maprfs:/user/ubuntu/.flink/application_1485984594262_0007/flink-conf.yaml
2017-02-02 16:11:12,949 INFO org.apache.flink.yarn.YarnClusterDescriptor
- Submitting application master application_1485984594262_0007
2017-02-02 16:11:12,977 INFO
org.apache.hadoop.yarn.security.ExternalTokenManagerFactory - Initialized
external token manager class -
com.mapr.hadoop.yarn.security.MapRTicketManager
2017-02-02 16:11:13,195 INFO
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted
application application_1485984594262_0007
2017-02-02 16:11:13,195 INFO org.apache.flink.yarn.YarnClusterDescriptor
- Waiting for the cluster to be allocated
2017-02-02 16:11:13,196 INFO org.apache.flink.yarn.YarnClusterDescriptor
- Deploying cluster, current state ACCEPTED
Error while deploying YARN cluster: Couldn't deploy Yarn cluster
java.lang.RuntimeException: Couldn't deploy Yarn cluster
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:425)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:620)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:476)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSe
Re: Flink with Yarn on MapR
Hi,
I think this is a re-post of a question I've already "answered":
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Flink-with-Yarn-on-MapR-td15448.html
On Wed, Jan 25, 2017 at 12:12 AM, ani.desh1512
wrote:
> Hi,
> I am trying to setup flink with Yarn on Mapr cluster. I built flink
> (flink-1.3-SNAPSHOT) as follows:
>
> mvn clean install -DskipTests -Pvendor-repos
> -Dhadoop.version=2.7.0-mapr-1607
>
> The build is successful. Then I try to run ./bin/yarn-session.sh -n 4
> (without changing any config or whatsoever) and get the following error:
>
> Error while deploying YARN cluster: Couldn't deploy Yarn cluster
> java.lang.RuntimeException: Couldn't deploy Yarn cluster
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(
> AbstractYarnClusterDescriptor.java:425)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(
> FlinkYarnSessionCli.java:620)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(
> FlinkYarnSessionCli.java:476)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(
> FlinkYarnSessionCli.java:473)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext$1.run(
> HadoopSecurityContext.java:43)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1595)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(
> HadoopSecurityContext.java:40)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(
> FlinkYarnSessionCli.java:473)
> Caused by: java.lang.NumberFormatException: For input string:
> "${nodemanager.resource.cpu-vcores}"
> at
> java.lang.NumberFormatException.forInputString(
> NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:569)
> at java.lang.Integer.parseInt(Integer.java:615)
> at
> org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1271)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(
> AbstractYarnClusterDescriptor.java:315)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(
> AbstractYarnClusterDescriptor.java:434)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(
> AbstractYarnClusterDescriptor.java:423)
> ... 9 more
>
> Now, the property that is causing this error nodemanager.resource.cpu-
> vcores
> is appropriately set in yarn-site.xml. The cluster is 3 ResourceManager (2
> on standby) and 5 NodeManager. To be extra safe, I changed the value for
> this property at ALL the Nodemanager’s yarn-site.xml.
> I believe that this property is default set to 4. So I am trying to
> understand as to why is this error cropping up.
> The required environment variable is set as follows:
> YARN_CONF_DIR=/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/
>
> I also tried setting the fs.hdfs.hadoopconf property (to point to the
> Hadoop
> conf directory) in flink-config.yaml. But I still get the same error.
>
>
> Any help with this error will be greatly appreciated
>
>
> Thanks in advance,
>
> Aniket D
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Flink-with-Yarn-
> on-MapR-tp11275.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
Flink with Yarn on MapR
Hi,
I am trying to setup flink with Yarn on Mapr cluster. I built flink
(flink-1.3-SNAPSHOT) as follows:
mvn clean install -DskipTests -Pvendor-repos
-Dhadoop.version=2.7.0-mapr-1607
The build is successful. Then I try to run ./bin/yarn-session.sh -n 4
(without changing any config or whatsoever) and get the following error:
Error while deploying YARN cluster: Couldn't deploy Yarn cluster
java.lang.RuntimeException: Couldn't deploy Yarn cluster
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:425)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:620)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:476)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:473)
at
org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
at java.security.AccessController.doPrivileged(Native
Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:473)
Caused by: java.lang.NumberFormatException: For input string:
"${nodemanager.resource.cpu-vcores}"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:569)
at java.lang.Integer.parseInt(Integer.java:615)
at
org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1271)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:315)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:434)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:423)
... 9 more
Now, the property that is causing this error nodemanager.resource.cpu-vcores
is appropriately set in yarn-site.xml. The cluster is 3 ResourceManager (2
on standby) and 5 NodeManager. To be extra safe, I changed the value for
this property at ALL the Nodemanager’s yarn-site.xml.
I believe that this property is default set to 4. So I am trying to
understand as to why is this error cropping up.
The required environment variable is set as follows:
YARN_CONF_DIR=/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/
I also tried setting the fs.hdfs.hadoopconf property (to point to the Hadoop
conf directory) in flink-config.yaml. But I still get the same error.
Any help with this error will be greatly appreciated
Thanks in advance,
Aniket D
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-with-Yarn-on-MapR-tp11275.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
