[
https://issues.apache.org/jira/browse/GOBBLIN-1671?focusedWorklogId=795158&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795158
]
ASF GitHub Bot logged work on GOBBLIN-1671:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 26/Jul/22 07:32
Start Date: 26/Jul/22 07:32
Worklog Time Spent: 10m
Work Description: bharos opened a new pull request, #3531:
URL: https://github.com/apache/gobblin/pull/3531
When using `mapreduce` mode in gobblin.sh, the additional jars passed to
gobblin.sh through --jars are comma separated. They are incorrectly
added to HADOOP_CLASSPATH that takes colon (:) separated jars.
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1671
### Description
- [ ] Here are some details about my PR, including screenshots (if
applicable):
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Tested locally. The error was reproducible locally, which caused
ClassNotFoundException.
The PR fixes the issue and jobs are able to get the dependent jars correctly.
### Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
Issue Time Tracking
-------------------
Worklog Id: (was: 795158)
Remaining Estimate: 0h
Time Spent: 10m
> ClassNotFoundException for class dependency in external jar while running
> Gobblin with mapreduce mode
> -----------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1671
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1671
> Project: Apache Gobblin
> Issue Type: Bug
> Affects Versions: 0.16.0
> Reporter: Bharath Krishna
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Faced issue while trying to upgrade from Gobblin {{0.14}} to {{0.16.}}
> We have a 'Gobblin consumer' jar, which contains some consumer/job
> implementations.
> We pass the consumer jar along with some other jars through the {{--jars}}
> field in gobblin.sh MapReduce mode
> ([https://github.com/apache/gobblin/blob/b400089035fe7ada1a523f9b7e5321e11d46d651/bin/gobblin.sh#L129])
>
> It is noticed that the Gobblin runs fail with ClassNotFoundException, not
> able to find the classes from the consumer jar, although it is present in the
> {{libjars}} as well as the {{HADOOP_CLASSPATH}}
> One workaround for this is to copy the custom jar onto
> [GOBBLIN_LIB|https://github.com/apache/gobblin/blob/b400089035fe7ada1a523f9b7e5321e11d46d651/bin/gobblin.sh#L30],
> although this does not seem correct as the consumer jar should have been
> already included through {{--jars.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
