[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020838#comment-16020838 ] Hudson commented on HBASE-15199: FAILURE: Integrated in Jenkins build HBase-HBASE-14614 #244 (See [https://builds.apache.org/job/HBase-HBASE-14614/244/]) HBASE-15199 (addendum) - When JRUBY_HOME is specified, update CLASSPATH (busbey: rev b67f6fecc173ff1272284f3e47f95d493fab331d) * (edit) bin/hbase * (edit) bin/hbase.cmd > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003858#comment-16003858 ] Hudson commented on HBASE-15199: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2981 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2981/]) HBASE-15199 (addendum) - When JRUBY_HOME is specified, update CLASSPATH (busbey: rev b67f6fecc173ff1272284f3e47f95d493fab331d) * (edit) bin/hbase * (edit) bin/hbase.cmd > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003805#comment-16003805 ] Xiang Li commented on HBASE-15199: -- Thanks everyone for the review! > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002770#comment-16002770 ] Sean Busbey commented on HBASE-15199: - [~water] nothing else needed, thanks for the fast turn around. I got a bit bogged down in other things, but I'll push this addendum later today. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001957#comment-16001957 ] Jerry He commented on HBASE-15199: -- +1 on the addendum. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001935#comment-16001935 ] Xiang Li commented on HBASE-15199: -- [~busbey] Would you please let me know if I need to something else for this JIRA? > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999465#comment-15999465 ] Ted Yu commented on HBASE-15199: Addendum looks good to me. You don't need to wait for my exercising the addendum before committing. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999285#comment-15999285
]
Hadoop QA commented on HBASE-15199:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 10s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
7s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
58m 35s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 53s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12866716/HBASE-15199-addendum.master.000.patch
|
| JIRA Issue | HBASE-15199 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 73e1b0026cda 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e99ed99 |
| shellcheck | v0.4.6 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6706/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch,
> HBASE-15199.master.001.patch, HBASE-15199.master.002.patch,
> HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999271#comment-15999271 ] Hudson commented on HBASE-15199: FAILURE: Integrated in Jenkins build HBase-HBASE-14614 #223 (See [https://builds.apache.org/job/HBase-HBASE-14614/223/]) HBASE-15199 Move jruby jar so only on runtime classpath for hbase-shell (busbey: rev 083796d2e67cbfedf30053e826dbd6e0c0b2baec) * (edit) bin/hbase.cmd * (edit) hbase-assembly/src/main/assembly/hadoop-two-compat.xml * (edit) bin/hbase > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999257#comment-15999257 ] Sean Busbey commented on HBASE-15199: - Looks reasonable. [~yuzhih...@gmail.com] does this addendum remove the warnings you saw before? > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999256#comment-15999256
]
Sean Busbey commented on HBASE-15199:
-
{quote}
I uploaded a new patch based on the current master branch. I never worked for a
patch as "addendum" before, but it means a patch to contain the diff based on
the current status, and does not contain the already committed changes for
HBASE-15199, then it is as uploaded.
{quote}
yep, exactly. reviewing now. thanks for this!
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch,
> HBASE-15199.master.001.patch, HBASE-15199.master.002.patch,
> HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999255#comment-15999255 ] Xiang Li commented on HBASE-15199: -- I tested it on both Linux and Windows. it took some time ^_^ > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999251#comment-15999251 ] Xiang Li commented on HBASE-15199: -- Hi [~busbey] I uploaded a new patch based on the current master branch. I never worked for a patch as "addendum" before, but it means a patch to contain the diff based on the current status, and does not contain the already committed changes for HBASE-15199, then it is as uploaded. Would you please review? The logic is updated as : (1) for the commands which need jruby (currently shell and org.jruby.Main) A. when JRUBY_HOME is specified explicitly, eg. export JRUBY_HOME=/usr/local/share/jruby CLASSPATH and HBASE_OPTS are updated according to JRUBY_HOME specified B. when JRUBY_HOME is not specified explicitly add jruby packaged with HBase to CLASSPATH (2) for other commands, do nothing > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199-addendum.master.000.patch, > HBASE-15199.master.001.patch, HBASE-15199.master.002.patch, > HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998488#comment-15998488 ] Sean Busbey commented on HBASE-15199: - I think more experience operators can already manually change things such that JRuby stuff ends up in the server environment. We should stay focused on the path everyone ends up walking down by default. [~water], would you mind putting together the change? An addendum is fine, but if it looks like things will take until sometime next week, then let's do a new JIRA instead. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998483#comment-15998483 ] Xiang Li commented on HBASE-15199: -- Hi Sean, the proposed also change makes sense to me. The only reason for me to keep the logic as it is now is we might need to leave a way for (senior) operators to control JRUBY_HOME if they really want and be aware of the consequence, even for server. But I am just wondering if it is really needed. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998428#comment-15998428
]
Sean Busbey commented on HBASE-15199:
-
{quote}
The patch of HBASE-15199 does change the behavior of
{code}HBASE_OPTS="$HBASE_OPTS -Djruby.home=$JRUBY_HOME
-Djruby.lib=$JRUBY_HOME/lib"{code}
Without the patch, it only takes effect when hbase shell.
With the patch, it takes effect whenever JRUBY_HOME is specified explicitly, so
jruby jar appears in hbase server's CLASSPATH.
{quote}
Hurm. Any reason we should keep it in server CLASSPATH? that sounds like a
source of operator headache. I think I grok [~jerryhe]'s comment now; putting
hte JRUBY_HOME check within the "do we need jruby now?" block makes more sense
to me.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997933#comment-15997933
]
Xiang Li commented on HBASE-15199:
--
Hi Ted,
1. The error you met (LoadError: no such file to load -- irb/completion) when
specifying JRUBY_HOME explicitly is due to the following code in hbase shell
script
{code}
HBASE_OPTS="$HBASE_OPTS -Djruby.home=$JRUBY_HOME -Djruby.lib=$JRUBY_HOME/lib"
{code}
So two conditions:
(1) When you specify JRUBY_HOME explicitly (by e.g. export), that statement is
needed. But you need a full jruby install, not a renamed jruby-complete jar, as
mentioned by Sean. (My tests show that only -Djruby.home=$JRUBY_HOME is needed,
-Djruby.lib=$JRUBY_HOME/lib could be removed. Correct me if I am wrong)
(2) When you do not specify JRUBY_HOME explicitly, it works by adding
jruby-complete jar to CLASSPATH when jruby needed.
2. The patch of HBASE-15199 does change the behavior of
{code}
HBASE_OPTS="$HBASE_OPTS -Djruby.home=$JRUBY_HOME -Djruby.lib=$JRUBY_HOME/lib"
{code}
Without the patch, it only takes effect when hbase shell.
With the patch, it takes effect whenever JRUBY_HOME is specified explicitly, so
jruby jar appears in hbase server's CLASSPATH.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997752#comment-15997752
]
Sean Busbey commented on HBASE-15199:
-
{quote}
However, there is a catch - if JRUBY_HOME is set as environment variable and
you use bin/start-hbase.sh, the jruby jar would appear in the classpath:
java.class.path=/a/jruby-1.6.8/lib/jruby.jar
{quote}
this part might not be HBASE-17878. One shouldn't have JRUBY_HOME exported in
the environment when launching the server processes normally. You should only
have it exported when you actually need your specific JRuby in the classpath.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997684#comment-15997684 ] Jerry He commented on HBASE-15199: -- Addendum to put this case in the 'if jruby_needed' of the original patch too? > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997606#comment-15997606
]
Ted Yu commented on HBASE-15199:
The above procedure allows shell to work.
However, there is a catch - if JRUBY_HOME is set as environment variable and
you use bin/start-hbase.sh, the jruby jar would appear in the classpath:
java.class.path=/a/jruby-1.6.8/lib/jruby.jar
leading to the following exception:
{code}
2017-05-04 23:53:48,192 FATAL [cn012:46854.activeMasterManager] master.HMaster:
Failed to become active master
java.nio.file.AccessDeniedException: : getFileStatus on :
com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires
a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error
Code: AccessDenied; Request ID: B1725B25836D2604), S3 Extended Request ID:
f8+vQ1XvtF3iCC9RBCtOnsQrMdQTvWSJb930LTkS4BJxRqZHuUNoI/fFELMV8ndMBnYgRp4x91g=
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:158)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1635)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:117)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1447)
at org.apache.hadoop.fs.s3a.S3AFileSystem.exists(S3AFileSystem.java:2040)
at
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:462)
at
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:162)
at
org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:142)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:724)
{code}
Should we document this ?
If so, I can open a JIRA.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997598#comment-15997598
]
Sean Busbey commented on HBASE-15199:
-
JRUBY_HOME needs to be jruby install, not just a location that includes a
renamed jruby-complete jar. jruby.jar that it looks for isn't meant to be the
whole shebang (note how the shell sets the jruby support files to look in
JRUBY_HOME/lib, that's where it expects to find e.g. the ruby support files
that your invocation is complaining about).
{code}
Downloads busbey$ curl -O
https://s3.amazonaws.com/jruby.org/downloads/1.6.8/jruby-bin-1.6.8.tar.gz
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 14.9M 100 14.9M0 0 3960k 0 0:00:03 0:00:03 --:--:-- 3960k
Downloads busbey$ tar xzf jruby-bin-1.6.8.tar.gz
Downloads busbey$ cd hbase-2.0.0-SNAPSHOT
hbase-2.0.0-SNAPSHOT busbey$ JRUBY_HOME=~/Downloads/jruby-1.6.8 ./bin/hbase
shell
2017-05-04 18:37:26,740 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Users/busbey/tmp_projects/hbase/hbase-assembly/target/hbase-2.0.0-SNAPSHOT/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/Cellar/hadoop/2.7.3/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 2.0.0-SNAPSHOT, r01af27061e97dd82de79f6900189c868f8919a2c, Thu May 4
18:22:39 CDT 2017
Took 0.0440 seconds
1.8.7-p357 :001 > exit
{code}
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997472#comment-15997472
]
Ted Yu commented on HBASE-15199:
Here is the dir listing:
{code}
dir/a/jruby:
total 4
drwxr-xr-x 2 hbase hadoop 4096 May 4 17:57 lib
dir/a/jruby/lib:
total 13512
-rw-r--r-- 1 hbase hadoop 13832273 May 4 17:57 jruby.jar
{code}
jruby.jar is actually jruby-complete-1.6.8.jar
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997467#comment-15997467 ] Sean Busbey commented on HBASE-15199: - Does your install work with the shipped jruby? Does it work if you export JRUBY_HOME prior to invocation? Can you paste bin a recursive ls of wherever JRUBY_HOME is pointing? > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997439#comment-15997439
]
Ted Yu commented on HBASE-15199:
This was what I did:
. produced hbase-2.0.0-SNAPSHOT-bin.tar.gz
. expanded the tar ball under /tmp
. made sure JRUBY_HOME points to a directory where lib/jruby.jar exists (copied
from jruby-complete jar)
. used the following command to launch shell:
{code}
JRUBY_HOME=/xx/jruby bin/hbase shell
{code}
I got:
{code}
LoadError: no such file to load -- irb/completion
require at org/jruby/RubyKernel.java:1062
(root) at /tmp/hbase-2.0.0-SNAPSHOT/bin/../bin/hirb.rb:41
{code}
What did I miss ?
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994103#comment-15994103 ] Hudson commented on HBASE-15199: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2939 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2939/]) HBASE-15199 Move jruby jar so only on runtime classpath for hbase-shell (busbey: rev 083796d2e67cbfedf30053e826dbd6e0c0b2baec) * (edit) bin/hbase.cmd * (edit) bin/hbase * (edit) hbase-assembly/src/main/assembly/hadoop-two-compat.xml > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993985#comment-15993985 ] Xiang Li commented on HBASE-15199: -- Thanks [~busbey] and [~jerryhe] for the review! Really appreciate it! > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993698#comment-15993698 ] Jerry He commented on HBASE-15199: -- +1 > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993144#comment-15993144 ] Sean Busbey commented on HBASE-15199: - +1. will commit later today if no one else has feedback. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch, HBASE-15199.master.003.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992674#comment-15992674
]
Hadoop QA commented on HBASE-15199:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 10s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 39s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
6s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 7s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 13s
{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 152m 21s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
59s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 202m 8s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.client.TestMultiRespectsLimits |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865886/HBASE-15199.master.003.patch
|
| JIRA Issue | HBASE-15199 |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
xml |
| uname | Linux 5efff7a71178 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / c8a7e80 |
| Default Java | 1.8.0_121 |
| shellcheck | v0.4.6 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6658/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6658/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6658/testReport/ |
| modules | C: hbase-assembly . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6658/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL:
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992440#comment-15992440
]
Xiang Li commented on HBASE-15199:
--
Hi [~busbey] thanks for the comments! The errors/warnings reported by
shellcheck and your comments are addressed. Patch 003 for master branch is
uploaded.
By the way, regarding
bq. Shouldn't this be just the jruby jar, rather than e.g. also the ruby
sources.
If I understand it correctly, for CLASSPATH, /* contains jar files under
only. The following paragraph could be found in
http://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html
{quote}
Class path entries can contain the base name wildcard character ( * ), which is
considered equivalent to specifying a list of all of the files in the directory
with the extension .jar or .JAR. For example, the class path entry mydir/*
specifies all JAR files in the directory named mydir. A class path entry
consisting of * expands to a list of all the jar files in the current directory
{quote}
But I still made the change in patch 003 to use *.jar and a loop to add all
jars one by one, which I believe is more easy to read, explicit, and more
controllable (does not rely on Java implementation as mentioned above).
Thanks for your comment!
I re-submit the patch and will watch the Hadoop QA output to see if there are
new errors/warnings reported by shellcheck.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch, HBASE-15199.master.003.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991340#comment-15991340
]
Sean Busbey commented on HBASE-15199:
-
General approach looks good.
{quote}
./bin/hbase:313:14: error: Double quote array expansions to avoid re-splitting
elements. [SC2068]
./bin/hbase:314:19: warning: Quote the rhs of == in [[ ]] to prevent glob
matching. [SC2053]
./bin/hbase:321:29: warning: Brace expansions and globs are literal in
assignments. Quote it or use an array. [SC2125]
{quote}
Could you correct these shellcheck errors please?
{code}
+ # add JRUBY_PACKAGED_WITH_HBASE to CLASSPATH when jruby is needed
+ JRUBY_PACKAGED_WITH_HBASE=$HBASE_HOME/lib/ruby/*
{code}
Shouldn't this be just the jruby jar, rather than e.g. also the ruby sources.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch,
> HBASE-15199.master.002.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990906#comment-15990906
]
Hadoop QA commented on HBASE-15199:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 10s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 12s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 3m
5s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m 8s
{color} | {color:red} The patch generated 3 new + 498 unchanged - 0 fixed = 501
total (was 498) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
55m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 124m 26s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
38s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 211m 18s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12865755/HBASE-15199.master.002.patch
|
| JIRA Issue | HBASE-15199 |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
xml |
| uname | Linux dce17c242ef5 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1848353 |
| Default Java | 1.8.0_121 |
| shellcheck | v0.4.6 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/artifact/patchprocess/diff-patch-shellcheck.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/testReport/ |
| modules | C: hbase-assembly . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6643/console |
| Powered by |
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990735#comment-15990735 ] Xiang Li commented on HBASE-15199: -- Updated patch 002 for master branch to address [~busbey]'s comments! The logic with the patch is (1) if JRUBY_HOME is defined, CLASSPATH is updated according to JRUBY_HOME defined (2) if JRUBY_HOME is not defined, check if the command issued belongs to a pre-defined command set (currently it contains shell and org.jruby.Main). A. if yes, add jruby jar packaged with HBase into CLASSPATH B. if no, do nothing There is a behavior change when comparing with original logic(without the patch) (1) without the patch, JRUBY_HOME and JRUBY_OPTS only takes effect when "hbase shell", that is, it does not take effect when "hbase org.jruby.Main xxx.rb“. It is a bug I believe (2) with the patch, JRUBY_HOME takes effect, for all commands, as long as it is specified. When JRUBY_HOME is specified, the jruby-complete jar packaged with HBase will be ignored. Make any sense to you? [~busbey], [~stack], [~jinghe] > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch, > HBASE-15199.master.002.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985120#comment-15985120 ] Jerry He commented on HBASE-15199: -- bq. we can solve this with a check for `org.jruby.Main` Make sense. `org.jruby.Main` is fixed. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985078#comment-15985078
]
Sean Busbey commented on HBASE-15199:
-
in addition to the shell, I think we also need to add this when someone is
expressly calling one of our jruby based tools.
e.g. `bin/shutdown_regionserver.rb` says that it should be called like:
{code}
# This script is used to issue a stop command to a regionserver via RPC.
# Intended for use in environments where sshing around is inappropriate
# Run it like this by passing it to a jruby interpreter:
#
# ./bin/hbase org.jruby.Main bin/shutdown_regionserver.rb c2021:16020
{code}
I haven't looked at all of our `bin/*.rb` examples, but I suspect we can solve
this with a check for `org.jruby.Main`
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Assignee: Xiang Li
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt, HBASE-15199.master.001.patch
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985044#comment-15985044 ] Xiang Li commented on HBASE-15199: -- Updated patch 001 (considering Stack's patch as 000) for master, in accordance with Stack's idea, by moving jruby-complete from lib to lib/ruby. 3 files are updated: (1) hadoop-two-compat.xml in assembly is updated to move jruby-complete from lib to lib/ruby when doing assembly. (2) hbase(for Linux) and hbase.cmd(for Windows) are updated to add jruby-complete to classpath only when hbase shell. I tested the patch on Linux (not on Windows), by using HBase 1.2.4. HBase shell worked well, while master and region server could be launched correctly without jruby-complete in the classpath. [~stack] and [~busbey], make sense to you? > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Assignee: Xiang Li >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt, HBASE-15199.master.001.patch > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976175#comment-15976175 ] stack commented on HBASE-15199: --- I just tried assembly and yeah, nothing special done... the jruby jar ends up in same place as all other jars in ./lib/jruby-complete-1.6.8.jar ... So, this patch doesn't help (as per Sean). Need to change assembly so this jruby jar is elsewhere and then the hbase-shell goes and includes it from whereever it lives. In ./lib we currently have a ruby subdir which we make as part of assembly (long time since I dug in here... ) Could put jruby dir under there... or probably better, formalize where modules go to get extras... a lib/hbase-shell subdir... I'm sure there are better ideas and even conventions for how to do this but have spent no time digging. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976160#comment-15976160 ] stack commented on HBASE-15199: --- Looks like I was lazy and didn't do work to ensure my patch didn't bundle jruby -- thanks [~busbey] (I thought it fine removing all reference from top-level pom, even bit in dependency management, down into submodule... did similar for other encapsulations such as pb version in hbase-protocol-shaded ).. I should at least see what our assembly does w/ the patch directive; i.e. confirm it doesn't do as i suspected but rather does as [~busbey] suggests > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976036#comment-15976036 ] Xiang Li commented on HBASE-15199: -- Hi, to centralize our discussion on jruby for future reference, if you agree, could we get back to HBASE-17878 to continue our discussion? Or it is ok for me that I can close HBASE-17878 a dup of this JIRA and we continue to have the discussion here. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976032#comment-15976032 ] Xiang Li commented on HBASE-15199: -- Hi [~stack] Agree with Sean. Your patch removes jruby-complete from of hbase parent pom, which can not move it down to hbase-shell. Currently, it is already scoped to hbase-shell. If an artifact is only declared in , it will not be actually included. Only when it is added to (the same level as , not the one under ), it is added as a dependency. can be used to control the version, scope or exclusion.., so that you do not have to specify those in every pom of child modules. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974914#comment-15974914
]
Hadoop QA commented on HBASE-15199:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 9s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 12s {color}
| {color:red} HBASE-15199 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12838695/15199.txt |
| JIRA Issue | HBASE-15199 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6505/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974905#comment-15974905
]
Sean Busbey commented on HBASE-15199:
-
this looks like the wrong approach. the bit removed from the top level pom is
just a {{dependencyManagement}} entry; it says what version of jruby-complete
ought to be used if anyone uses it. Only the hbase-shell module actually uses
it both before and after.
Likely what we need to fix this is a packaging change in when we include
jruby-complete on the classpath at runtime. We need it for the shell, but that
should be it. AFAICT, the current patch will still place it in the runtime
classpath for all of our server processes.
> Move jruby jar so only on hbase-shell module classpath; currently globally
> available
>
>
> Key: HBASE-15199
> URL: https://issues.apache.org/jira/browse/HBASE-15199
> Project: HBase
> Issue Type: Task
> Components: dependencies, jruby, shell
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15199.txt
>
>
> A suggestion that came up out of internal issue (filed by Mr Jan Van Besien)
> was to move the scope of the jruby include down so it is only a dependency
> for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time
> for example) which can clash with the includes of others. Our Sean suggests
> that could be good to shut down exploit possibilities if jruby was not
> globally available. Only downside I can think is that it may no longer be
> available to our bin/*rb scripts if we move the jar but perhaps these can be
> changed so they can find the ruby jar in new location.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[
https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662698#comment-15662698
]
Hadoop QA commented on HBASE-15199:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 56s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 13s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 49s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 3s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 150m 59s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestMetaWithReplicas |
| | org.apache.hadoop.hbase.master.TestTableLockManager |
| | org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | org.apache.hadoop.hbase.client.TestHCM |
| | org.apache.hadoop.hbase.client.TestTableSnapshotScanner |
| | org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12838695/15199.txt |
| JIRA Issue | HBASE-15199 |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux 744f378a9983 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 9250bf8 |
| Default Java | 1.8.0_111 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4451/artifact/patchprocess/patch-unit-root.txt
|
|
[jira] [Commented] (HBASE-15199) Move jruby jar so only on hbase-shell module classpath; currently globally available
[ https://issues.apache.org/jira/browse/HBASE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662480#comment-15662480 ] stack commented on HBASE-15199: --- Do it for 2.0.0 only. > Move jruby jar so only on hbase-shell module classpath; currently globally > available > > > Key: HBASE-15199 > URL: https://issues.apache.org/jira/browse/HBASE-15199 > Project: HBase > Issue Type: Task > Components: dependencies, jruby, shell >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 15199.txt > > > A suggestion that came up out of internal issue (filed by Mr Jan Van Besien) > was to move the scope of the jruby include down so it is only a dependency > for the hbase-shell. jruby jar brings in a bunch of dependencies (joda time > for example) which can clash with the includes of others. Our Sean suggests > that could be good to shut down exploit possibilities if jruby was not > globally available. Only downside I can think is that it may no longer be > available to our bin/*rb scripts if we move the jar but perhaps these can be > changed so they can find the ruby jar in new location. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
