Re: No yarn option in self-built flink version
@Arnaud: Turns out those examples are on purpose. As Chesnay pointed out in the ticket, there are also cases where you don't necessarily want to bundle the Hadoop dependency, but still want to set a version like that. On Wed, Jun 12, 2019 at 9:32 AM Ufuk Celebi wrote: > I created https://issues.apache.org/jira/browse/FLINK-12813 for > this. @Arnaud: Would you be interested in opening a PR with a fix? > > – Ufuk > > > On Tue, Jun 11, 2019 at 11:10 AM LINZ, Arnaud > wrote: > >> Hello, >> >> >> >> Thanks a lot, it works. However, may I suggest that you update the >> documentation page : >> >> >> >> mvn clean install -DskipTests -Pvendor-repos >> -Dhadoop.version*=*2.6.0-cdh5.16.1 >> >> has absolutely no interest if you don’t include hadoop, that’s why I >> thought that -Pvendor-repos was including the -Pinclude-hadoop stated in >> the above paragraph… >> >> >> >> Arnaud >> >> >> >> *De :* Ufuk Celebi >> *Envoyé :* vendredi 7 juin 2019 12:00 >> *À :* LINZ, Arnaud >> *Cc :* user ; ches...@apache.org >> *Objet :* Re: No yarn option in self-built flink version >> >> >> >> Hey Arnaud, >> >> >> >> I think you need to active the Hadoop profile via -Pinclude-hadoop (the >> default was changed to not include Hadoop as far as I know). >> >> >> >> For more details, check out: >> >> >> https://ci.apache.org/projects/flink/flink-docs-release-1.8/flinkDev/building.html#packaging-hadoop-into-the-flink-distribution >> >> >> >> >> >> If this does not work for you, I would wait for Chesnay's input (cc'd). >> >> >> >> – Ufuk >> >> >> >> >> >> On Fri, Jun 7, 2019 at 11:04 AM LINZ, Arnaud >> wrote: >> >> Hello, >> >> I am trying to build my own flink distribution with proper Cloudera >> dependencies. >> Reading >> https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html >> I've done : >> git clone https://github.com/apache/flink >> cd flink >> git checkout tags/release-1.8.0 >> $MAVEN_HOME/bin/mvn clean install -DskipTests -Pvendor-repos >> -Dhadoop.version=2.6.0-cdh5.16.1 -Dfast >> cd flink-dist >> $MAVEN_HOME/bin/mvn install -DskipTests -Pvendor-repos >> -Dhadoop.version=2.6.0-cdh5.16.1 >> >> Everything was successful. >> >> However when running using flink-dist/target/flink-1.8.0-bin/flink-1.8.0 >> Running /bin/flink -h prints no yarn/Hadoop options. >> And running >> bin/flink run -m yarn-cluster -yn 4 -yjm 1024m -ytm 4096m >> ./examples/batch/WordCount.jar >> Prints : >> Could not build the program from JAR file. >> >> Am I missing something? >> >> Best regards, >> Arnaud >> >> >> >> >> L'intégrité de ce message n'étant pas assurée sur internet, la société >> expéditrice ne peut être tenue responsable de son contenu ni de ses pièces >> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si >> vous n'êtes pas destinataire de ce message, merci de le détruire et >> d'avertir l'expéditeur. >> >> The integrity of this message cannot be guaranteed on the Internet. The >> company that sent this message cannot therefore be held liable for its >> content nor attachments. Any unauthorized use or dissemination is >> prohibited. If you are not the intended recipient of this message, then >> please delete it and notify the sender. >> >>
Re: No yarn option in self-built flink version
I created https://issues.apache.org/jira/browse/FLINK-12813 for this. @Arnaud: Would you be interested in opening a PR with a fix? – Ufuk On Tue, Jun 11, 2019 at 11:10 AM LINZ, Arnaud wrote: > Hello, > > > > Thanks a lot, it works. However, may I suggest that you update the > documentation page : > > > > mvn clean install -DskipTests -Pvendor-repos > -Dhadoop.version*=*2.6.0-cdh5.16.1 > > has absolutely no interest if you don’t include hadoop, that’s why I > thought that -Pvendor-repos was including the -Pinclude-hadoop stated in > the above paragraph… > > > > Arnaud > > > > *De :* Ufuk Celebi > *Envoyé :* vendredi 7 juin 2019 12:00 > *À :* LINZ, Arnaud > *Cc :* user ; ches...@apache.org > *Objet :* Re: No yarn option in self-built flink version > > > > Hey Arnaud, > > > > I think you need to active the Hadoop profile via -Pinclude-hadoop (the > default was changed to not include Hadoop as far as I know). > > > > For more details, check out: > > > https://ci.apache.org/projects/flink/flink-docs-release-1.8/flinkDev/building.html#packaging-hadoop-into-the-flink-distribution > > > > > > If this does not work for you, I would wait for Chesnay's input (cc'd). > > > > – Ufuk > > > > > > On Fri, Jun 7, 2019 at 11:04 AM LINZ, Arnaud > wrote: > > Hello, > > I am trying to build my own flink distribution with proper Cloudera > dependencies. > Reading > https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html > I've done : > git clone https://github.com/apache/flink > cd flink > git checkout tags/release-1.8.0 > $MAVEN_HOME/bin/mvn clean install -DskipTests -Pvendor-repos > -Dhadoop.version=2.6.0-cdh5.16.1 -Dfast > cd flink-dist > $MAVEN_HOME/bin/mvn install -DskipTests -Pvendor-repos > -Dhadoop.version=2.6.0-cdh5.16.1 > > Everything was successful. > > However when running using flink-dist/target/flink-1.8.0-bin/flink-1.8.0 > Running /bin/flink -h prints no yarn/Hadoop options. > And running > bin/flink run -m yarn-cluster -yn 4 -yjm 1024m -ytm 4096m > ./examples/batch/WordCount.jar > Prints : > Could not build the program from JAR file. > > Am I missing something? > > Best regards, > Arnaud > > > > > L'intégrité de ce message n'étant pas assurée sur internet, la société > expéditrice ne peut être tenue responsable de son contenu ni de ses pièces > jointes. Toute utilisation ou diffusion non autorisée est interdite. Si > vous n'êtes pas destinataire de ce message, merci de le détruire et > d'avertir l'expéditeur. > > The integrity of this message cannot be guaranteed on the Internet. The > company that sent this message cannot therefore be held liable for its > content nor attachments. Any unauthorized use or dissemination is > prohibited. If you are not the intended recipient of this message, then > please delete it and notify the sender. > >
RE: No yarn option in self-built flink version
Hello, Thanks a lot, it works. However, may I suggest that you update the documentation page : mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.16.1 has absolutely no interest if you don’t include hadoop, that’s why I thought that -Pvendor-repos was including the -Pinclude-hadoop stated in the above paragraph… Arnaud De : Ufuk Celebi Envoyé : vendredi 7 juin 2019 12:00 À : LINZ, Arnaud Cc : user ; ches...@apache.org Objet : Re: No yarn option in self-built flink version Hey Arnaud, I think you need to active the Hadoop profile via -Pinclude-hadoop (the default was changed to not include Hadoop as far as I know). For more details, check out: https://ci.apache.org/projects/flink/flink-docs-release-1.8/flinkDev/building.html#packaging-hadoop-into-the-flink-distribution If this does not work for you, I would wait for Chesnay's input (cc'd). – Ufuk On Fri, Jun 7, 2019 at 11:04 AM LINZ, Arnaud mailto:al...@bouyguestelecom.fr>> wrote: Hello, I am trying to build my own flink distribution with proper Cloudera dependencies. Reading https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html I've done : git clone https://github.com/apache/flink cd flink git checkout tags/release-1.8.0 $MAVEN_HOME/bin/mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.16.1 -Dfast cd flink-dist $MAVEN_HOME/bin/mvn install -DskipTests -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.16.1 Everything was successful. However when running using flink-dist/target/flink-1.8.0-bin/flink-1.8.0 Running /bin/flink -h prints no yarn/Hadoop options. And running bin/flink run -m yarn-cluster -yn 4 -yjm 1024m -ytm 4096m ./examples/batch/WordCount.jar Prints : Could not build the program from JAR file. Am I missing something? Best regards, Arnaud L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: No yarn option in self-built flink version
Hey Arnaud, I think you need to active the Hadoop profile via -Pinclude-hadoop (the default was changed to not include Hadoop as far as I know). For more details, check out: https://ci.apache.org/projects/flink/flink-docs-release-1.8/flinkDev/building.html#packaging-hadoop-into-the-flink-distribution If this does not work for you, I would wait for Chesnay's input (cc'd). – Ufuk On Fri, Jun 7, 2019 at 11:04 AM LINZ, Arnaud wrote: > Hello, > > I am trying to build my own flink distribution with proper Cloudera > dependencies. > Reading > https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html > I've done : > git clone https://github.com/apache/flink > cd flink > git checkout tags/release-1.8.0 > $MAVEN_HOME/bin/mvn clean install -DskipTests -Pvendor-repos > -Dhadoop.version=2.6.0-cdh5.16.1 -Dfast > cd flink-dist > $MAVEN_HOME/bin/mvn install -DskipTests -Pvendor-repos > -Dhadoop.version=2.6.0-cdh5.16.1 > > Everything was successful. > > However when running using flink-dist/target/flink-1.8.0-bin/flink-1.8.0 > Running /bin/flink -h prints no yarn/Hadoop options. > And running > bin/flink run -m yarn-cluster -yn 4 -yjm 1024m -ytm 4096m > ./examples/batch/WordCount.jar > Prints : > Could not build the program from JAR file. > > Am I missing something? > > Best regards, > Arnaud > > > > > L'intégrité de ce message n'étant pas assurée sur internet, la société > expéditrice ne peut être tenue responsable de son contenu ni de ses pièces > jointes. Toute utilisation ou diffusion non autorisée est interdite. Si > vous n'êtes pas destinataire de ce message, merci de le détruire et > d'avertir l'expéditeur. > > The integrity of this message cannot be guaranteed on the Internet. The > company that sent this message cannot therefore be held liable for its > content nor attachments. Any unauthorized use or dissemination is > prohibited. If you are not the intended recipient of this message, then > please delete it and notify the sender. >