I'm glad you figured it out. It sure looks like a mess. On Wednesday, November 18, 2020 at 5:02:41 PM UTC-5 st.mail...@gmail.com wrote:
> Thanks Jonathan and Gianluca for your explanations. I now got most of my > confusion cleared up, most importantly the fact that String interpolation > is done either by Groovy or by the bash but never by Jenkins. > > I think the first sentence here got me on the wrong path: "Jenkins > Pipeline uses rules identical to Groovy <http://groovy-lang.org/> for > string interpolation" > https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#string-interpolation > To me that suggests that Jenkins also does some kind of string > interpolation. > > @Gianluca: Thanks for your attempts to solve my problem. I've tried again, > and I am pretty sure now that mixing simple strings (single quotes) and > GStrings (double quotes) does not help because I suspect that there is some > automatic type conversion between simple strings and GStrings involved here > that destroys the simple strings - see below. > > Also keep in mind that I need to quote all of jdbcUrl/username/password in > the bash command because they can contain special characters (& ! etc). > > So trying your approach like this: > sh "${mvnCommand(m2RepoIdentifier)}" + > > ' -f proj-reactor/pom.xml ' + > ' -P GenerateStorables -DdeployAtEnd=true' + > " '-Dproj.build.generatefromdb.url=${jdbcUrlNoData()}'" + > ' -Dproj.build.generatefromdb.user=${GEN_USR}' + > ' "-Dproj.build.generatefromdb.password=${GEN_PASSWD}"' + > " clean ${mvnBuildGoal} pmd:pmd pmd:cpd" > -> Jenkins *will still print the warning* about insecure interpolation of > sensitive variables > -> The build works fine though, e.g. all variables get passed to the maven > build as expected > > *BUT what actually worked* in the end is the following: > sh "${mvnCommand(m2RepoIdentifier)}".toString() + > > ' -f proj-reactor/pom.xml ' + > ' -P GenerateStorables -DdeployAtEnd=true' + > " '-Dproj.build.generatefromdb.url=${jdbcUrlNoData()}'".toString() + > > ' -Dproj.build.generatefromdb.user=${GEN_USR}' + > ' "-Dproj.build.generatefromdb.password=${GEN_PASSWD}"' + > " clean ${mvnBuildGoal} pmd:pmd pmd:cpd".toString() > -> No warning about insecure interpolation of sensitive variables > -> The build works fine, e.g. all variables get passed to the maven build > as expected > -> Also nice for readability to not need to escape any quotes/characters > in any part of the command above > > Hope this helps other folks to clear up any string interpolation confusion > :-) > stefan. > > > On Wed, Nov 18, 2020 at 1:14 AM JonathanRRogers <jonatha...@gmail.com> > wrote: > >> Unless "mvnBuildGoal" is the name of a Jenkins environment variable, >> which would be added to the shell process environment, it won't work inside >> a single-quoted string. >> >> On Tuesday, November 17, 2020 at 4:11:32 PM UTC-5 Gianluca wrote: >> >>> Somethings said are correctly … somethings said are wrong. >>> I’ll try to summarise pointing to the lines in your pipeline and explain >>> what’s wrong and how to fix: >>> >>> Single quoted string are literals. They never get interpolated and >>> passed as they are. >>> So, in your case: >>> >>> sh '${JAVA_HOME}/bin/java -version' // single quotes - this works fine! >>>> >>> >>> Work … not because ${JAVA_HOME} is interpolated by Jenkins … but because >>> for an unfortunate coincidence … ${JAVA_HOME} is a correct bash syntax for >>> reference an environment variable in bash and get expanded correctly. >>> DO NOT confuse Jenkins pipeline environment variable that exists only in >>> the pipeline context with the BASH environment variable that exists in the >>> shell when “sh” is executed. >>> Now, also take into account that Jenkins also creates a corresponding >>> BASH environment variable for each pipeline environment variable. >>> So, what in Groovy double-quoted string like “${env.GEN_USR}” get >>> expanded using the value of the Jenkins environment variable GEN_USR, but >>> if you want to let bash expand the environment variable then you have to >>> pass '${GEN_USR}’ … in single-quoted string … or … you need to escape “$” >>> in double-quoted string to avoid that Groovy does the interpolation instead >>> of bash. >>> >>> Your multi-line command suffer from the fact somethings need to be >>> expanded by Groovy and something else need to be expanded by BASH, so you >>> need to split in different string and concatenate or escape $: >>> >>> sh script: ''' >>>> ${mvnCommand(m2RepoIdentifier)}\ >>>> >>> This is a Jenkins function … so need to be expanded by Groovy and it’s >>> in the double-quoted string… so far so good >>> >>> -f proj-reactor/pom.xml\ >>>> -P GenerateStorables -DdeployAtEnd=true\ >>>> -Dproj.build.generatefromdb.url=${jdbcUrlNoData()}\ >>>> >>> That is the same … so it’s good >>> >>> -Dproj.build.generatefromdb.user=${GEN_USR}\ >>>> >>> That’s the problem. You are using “withCredentials” and that command >>> only creates BASH environment variable … and you are letting Groovy do the >>> expansion but GEN_USR doesn’t exist into Jenkins pipeline … and neither >>> env.GEN_USR … the only one that exist is the BASH environment variable and >>> you have to let bash expand … so you need to escape $ to pass as it is to >>> BASH: >>> >>> -Dproj.build.generatefromdb.user=\${GEN_USR}\ >>> >>> -Dproj.build.generatefromdb.password=${GEN_PASSWD}\ >>>> >>> Same as above… it’s a BASH environment variable >>> >>> clean ${mvnBuildGoal} pmd:pmd pmd:cpd\ >>>> >>> That is probably a Groovy variable but I’m not entire sure. >>> >>> ''' >>>> >>> >>> I hope that helps. >>> >>> As suggestion, try to split and use a mix of double-quoted and >>> single-quoted so you avoid escaping: >>> >>> sh script: >>> "${mvnCommand(m2RepoIdentifier)} “ + >>> '-f proj-reactor/pom.xml ‘ + >>> '-P GenerateStorables -DdeployAtEnd=true ‘ + >>> "-Dproj.build.generatefromdb.url=${jdbcUrlNoData()}” + >>> '-Dproj.build.generatefromdb.user=${GEN_USR} ‘ + >>> '-Dproj.build.generatefromdb.password=${GEN_PASSWD} ‘ + >>> 'clean ${mvnBuildGoal} pmd:pmd pmd:cpd' >>> >>> >>> Cheers, >>> Gianluca. >>> >>> >>> On 17 Nov 2020, at 20:45, JonathanRRogers <jonatha...@gmail.com> wrote: >>> >>> I don't think you've read the Jenkins pipeline documentation correctly. >>> String interpolation is performed by Groovy. Jenkins doesn't add any >>> interpolation behavior. Jenkins will always treat single-quoted strings as >>> simple literals. The "sh" step calls a shell which can do its own variable >>> expansion, which is the approach recommended for passing credentials to >>> external commands. >>> >>> Your multi-line sh step includes calls to "mvnCommand()" and >>> "jdbcUrlNoData()". Unless those are shell functions, you're unlikely to get >>> the results you want. >>> >>> On Tuesday, November 17, 2020 at 10:41:51 AM UTC-5 st.mail...@gmail.com >>> wrote: >>> >>>> Thanks Dirk, but that does not solve my problem, I'm still stuck. >>>> I think that my question boils down to why the multi-line sh command >>>> below does not work? >>>> According to >>>> https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#string-interpolation >>>> single-quoted strings should be interpolated by Jenkins (instead of >>>> using Groovy string interpolation). I would assume this applies >>>> to both variables (e.g. GEN_USR below) and method calls returning a >>>> string (e.g. jdbcUrlNoData() below)? >>>> >>>> withCredentials([usernamePassword(credentialsId: >>>> 'credentialsGenerateFromDb', >>>> usernameVariable: 'GEN_USR', >>>> passwordVariable: 'GEN_PASSWD')]) { >>>> String m2RepoIdentifier = "${env.BRANCH_NAME}_${env.BUILD_NUMBER}" >>>> >>>> withEnv(javaAndMavenEnvArray()) { >>>> sh '${JAVA_HOME}/bin/java -version' // single quotes - this works >>>> fine! >>>> >>>> sh script: ''' >>>> ${mvnCommand(m2RepoIdentifier)}\ >>>> -f proj-reactor/pom.xml\ >>>> -P GenerateStorables -DdeployAtEnd=true\ >>>> -Dproj.build.generatefromdb.url=${jdbcUrlNoData()}\ >>>> -Dproj.build.generatefromdb.user=${GEN_USR}\ >>>> -Dproj.build.generatefromdb.password=${GEN_PASSWD}\ >>>> clean ${mvnBuildGoal} pmd:pmd pmd:cpd\ >>>> ''' >>>> } >>>> } >>>> >>>> On Tue, Nov 17, 2020 at 2:06 PM 'Dirk Heinrichs' via Jenkins Users < >>>> jenkins...@googlegroups.com> wrote: >>>> >>>>> Am Dienstag, den 17.11.2020, 13:45 +0100 schrieb ST: >>>>> >>>>> " '-Dproj.build.generatefromdb.user=${env.GEN_USR}'" >>>>> >>>>> >>>>> Try with ${GEN_USR} inside shell scripts, not ${env.GEN_USR}. >>>>> >>>>> HTH... >>>>> >>>>> Dirk >>>>> >>>>> -- >>>>> >>>>> *Dirk Heinrichs* >>>>> Senior Systems Engineer, Delivery Pipeline >>>>> OpenText ™ Discovery | Recommind >>>>> *Phone*: +49 2226 15966 18 <+49%202226%201596618> >>>>> *Email*: dhei...@opentext.com >>>>> *Website*: www.recommind.de >>>>> Recommind GmbH, Von-Liebig-Straße 1, 53359 Rheinbach >>>>> Vertretungsberechtigte Geschäftsführer Gordon Davies, Madhu >>>>> Ranganathan, Christian Waida, Registergericht Amtsgericht Bonn, >>>>> Registernummer HRB 10646 >>>>> This e-mail may contain confidential and/or privileged information. If >>>>> you are not the intended recipient (or have received this e-mail in >>>>> error) >>>>> please notify the sender immediately and destroy this e-mail. Any >>>>> unauthorized copying, disclosure or distribution of the material in this >>>>> e-mail is strictly forbidden >>>>> Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte >>>>> Informationen. Wenn Sie nicht der richtige Adressat sind oder diese >>>>> E-Mail >>>>> irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und >>>>> vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte >>>>> Weitergabe dieser Mail sind nicht gestattet. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Jenkins Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to jenkinsci-use...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/jenkinsci-users/4dd32ea372d43c9dde0800bed9a16253d45cb2e5.camel%40opentext.com >>>>> >>>>> <https://groups.google.com/d/msgid/jenkinsci-users/4dd32ea372d43c9dde0800bed9a16253d45cb2e5.camel%40opentext.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to jenkinsci-use...@googlegroups.com. >>> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/bed31997-28f8-4e9b-b205-43f02d4348a5n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/jenkinsci-users/bed31997-28f8-4e9b-b205-43f02d4348a5n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to jenkinsci-use...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/jenkinsci-users/b5086225-d400-4500-a936-0413c025edf6n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/jenkinsci-users/b5086225-d400-4500-a936-0413c025edf6n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/69dc4fce-d19a-4cc4-b38a-b3029dca9d44n%40googlegroups.com.
