[jira] [Commented] (DRILL-8321) Change kafka_2.13 dependency scope to test
[
https://issues.apache.org/jira/browse/DRILL-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612019#comment-17612019
]
ASF GitHub Bot commented on DRILL-8321:
---
cgivre merged PR #2662:
URL: https://github.com/apache/drill/pull/2662
> Change kafka_2.13 dependency scope to test
> ---
>
> Key: DRILL-8321
> URL: https://issues.apache.org/jira/browse/DRILL-8321
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.20.2
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.20.3
>
>
> Drill has 2 scala dependencies:
> * {{org.apache.kafka.kafka_2.13}}
> * {{com.madhukaraphatak.java-sizeof_2.11}}
> which are targets on different scala versions 2.13 and 2.11. But Scala has no
> backward compatibility for major releases, so we can’t have 2 libraries
> compiled on various versions of scala.
> To solve the issue there are only 2 ways:
> # Compile both libraries on the same major Scala version.
> # Remove one of the libraries from Drill
> {{kafka_2.13}} is server side (kafka’s server side) dependency and is
> unnecessary on the client side (Drill). Probably, it was added carelessly to
> Drill to a compile scope, while it is necessary only in a test scope.
> So {{kafka_2.13}} can be removed from compile scope. It will reduce the Drill
> package size and the main – it will solve scala version conflict.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8321) Change kafka_2.13 dependency scope to test
[
https://issues.apache.org/jira/browse/DRILL-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611539#comment-17611539
]
PJ Fanning commented on DRILL-8321:
---
I opened an issue and PR at https://issues.apache.org/jira/browse/DRILL-8324
> Change kafka_2.13 dependency scope to test
> ---
>
> Key: DRILL-8321
> URL: https://issues.apache.org/jira/browse/DRILL-8321
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.20.2
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.20.3
>
>
> Drill has 2 scala dependencies:
> * {{org.apache.kafka.kafka_2.13}}
> * {{com.madhukaraphatak.java-sizeof_2.11}}
> which are targets on different scala versions 2.13 and 2.11. But Scala has no
> backward compatibility for major releases, so we can’t have 2 libraries
> compiled on various versions of scala.
> To solve the issue there are only 2 ways:
> # Compile both libraries on the same major Scala version.
> # Remove one of the libraries from Drill
> {{kafka_2.13}} is server side (kafka’s server side) dependency and is
> unnecessary on the client side (Drill). Probably, it was added carelessly to
> Drill to a compile scope, while it is necessary only in a test scope.
> So {{kafka_2.13}} can be removed from compile scope. It will reduce the Drill
> package size and the main – it will solve scala version conflict.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8321) Change kafka_2.13 dependency scope to test
[
https://issues.apache.org/jira/browse/DRILL-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611526#comment-17611526
]
Maksym Rymar commented on DRILL-8321:
-
[~pj.fanning] Yes, I was thinking about this too. One way is to replace
[https://github.com/phatak-dev/java-sizeof] with
[https://github.com/DimitrisAndreou/memory-measurer] or even with a fork from
the last written on a pure java 8 [https://github.com/shihyuho/memory-measurer]
. Another way is to try to implement something own, as this library is used
only for one special place.
> Change kafka_2.13 dependency scope to test
> ---
>
> Key: DRILL-8321
> URL: https://issues.apache.org/jira/browse/DRILL-8321
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.20.2
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.20.3
>
>
> Drill has 2 scala dependencies:
> * {{org.apache.kafka.kafka_2.13}}
> * {{com.madhukaraphatak.java-sizeof_2.11}}
> which are targets on different scala versions 2.13 and 2.11. But Scala has no
> backward compatibility for major releases, so we can’t have 2 libraries
> compiled on various versions of scala.
> To solve the issue there are only 2 ways:
> # Compile both libraries on the same major Scala version.
> # Remove one of the libraries from Drill
> {{kafka_2.13}} is server side (kafka’s server side) dependency and is
> unnecessary on the client side (Drill). Probably, it was added carelessly to
> Drill to a compile scope, while it is necessary only in a test scope.
> So {{kafka_2.13}} can be removed from compile scope. It will reduce the Drill
> package size and the main – it will solve scala version conflict.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8321) Change kafka_2.13 dependency scope to test
[
https://issues.apache.org/jira/browse/DRILL-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611063#comment-17611063
]
ASF GitHub Bot commented on DRILL-8321:
---
rymarm opened a new pull request, #2662:
URL: https://github.com/apache/drill/pull/2662
# [DRILL-8321](https://issues.apache.org/jira/browse/DRILL-8321): Change
kafka_2.13 dependency scope to test
## Description
Move `org.apache.kafka.kafka_2.13` to test dependency scope. It prevents
conflict between Scala versions of `org.apache.kafka.kafka_2.13` (Scala 2.13)
and `com.madhukaraphatak.java-sizeof_2.11` (Scala 2.11). Solves such exceptions
that could appear during Drill-On-Yarn startup:
```
Caused by: java.util.ServiceConfigurationError:
com.fasterxml.jackson.databind.Module: Provider
com.fasterxml.jackson.module.scala.DefaultScalaModule could not be instantiated
...
Caused by: java.lang.NoSuchMethodError: 'scala.collection.immutable.Seq$
scala.package$.Seq()'
```
Also, it cleans Drill's classpath from unnecessary dependencies.
Probably `org.apache.kafka.kafka_2.13` was added to compile scope by
mistake.
## Documentation
No changes.
## Testing
Unit tests pass successfully. Tried to connect to Kafka and read a topic -
everything works fine.
> Change kafka_2.13 dependency scope to test
> ---
>
> Key: DRILL-8321
> URL: https://issues.apache.org/jira/browse/DRILL-8321
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.20.2
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 2.0.0
>
>
> Drill has 2 scala dependencies:
> * {{org.apache.kafka.kafka_2.13}}
> * {{com.madhukaraphatak.java-sizeof_2.11}}
> which are targets on different scala versions 2.13 and 2.11. But Scala has no
> backward compatibility for major releases, so we can’t have 2 libraries
> compiled on various versions of scala.
> To solve the issue there are only 2 ways:
> # Compile both libraries on the same major Scala version.
> # Remove one of the libraries from Drill
> {{kafka_2.13}} is server side (kafka’s server side) dependency and is
> unnecessary on the client side (Drill). Probably, it was added carelessly to
> Drill to a compile scope, while it is necessary only in a test scope.
> So {{kafka_2.13}} can be removed from compile scope. It will reduce the Drill
> package size and the main – it will solve scala version conflict.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8321) Change kafka_2.13 dependency scope to test
[
https://issues.apache.org/jira/browse/DRILL-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611053#comment-17611053
]
PJ Fanning commented on DRILL-8321:
---
Any chance we can drop [https://github.com/phatak-dev/java-sizeof] ? If it
isn't published for recent scala versions, it is a real millstone around our
necks.
> Change kafka_2.13 dependency scope to test
> ---
>
> Key: DRILL-8321
> URL: https://issues.apache.org/jira/browse/DRILL-8321
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.20.2
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 2.0.0
>
>
> Drill has 2 scala dependencies:
> * {{org.apache.kafka.kafka_2.13}}
> * {{com.madhukaraphatak.java-sizeof_2.11}}
> which are targets on different scala versions 2.13 and 2.11. But Scala has no
> backward compatibility for major releases, so we can’t have 2 libraries
> compiled on various versions of scala.
> To solve the issue there are only 2 ways:
> # Compile both libraries on the same major Scala version.
> # Remove one of the libraries from Drill
> {{kafka_2.13}} is server side (kafka’s server side) dependency and is
> unnecessary on the client side (Drill). Probably, it was added carelessly to
> Drill to a compile scope, while it is necessary only in a test scope.
> So {{kafka_2.13}} can be removed from compile scope. It will reduce the Drill
> package size and the main – it will solve scala version conflict.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
