[jira] [Commented] (RATIS-64) Do not pre-shade artifacts
[ https://issues.apache.org/jira/browse/RATIS-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688490#comment-16688490 ] Josh Elser commented on RATIS-64: - Closing: I think I inadvertently resolved this via the ratis-thirdparty module. Please reopen if I got something wrong. > Do not pre-shade artifacts > -- > > Key: RATIS-64 > URL: https://issues.apache.org/jira/browse/RATIS-64 > Project: Ratis > Issue Type: Bug > Components: build >Reporter: Enis Soztutar >Priority: Major > > Hugo and I were discussing a case were the shading should happen at the last > step, rather than what we do today. > I think there are 3 possible strategies of shading that one can do: > (1) pre-shade some of your dependencies, so that your other dependencies can > work. This what we do today, we shade PB+ GRPC, etc so that Hadoop can work. > (2) pre-shade some of your dependencies' transitive dependencies so that you > depend on already-shaded artifacts. This will be like having maven artifacts > of shaded-hadoop so that hadoop itself does not bring in any more dependency. > If hadoop has shaded artifacts, or we do shading of hadoop's dependencies in > another repository, we won't need to pre-shade PB, etc. > (3) post-shade. This means that in the code itself we do not depend on > shaded packages anymore, but do the shading as a different module so that we > publish shaded artifacts. This allows the downstreamers to be able to consume > ratis, while allowing ratis source code to be saner. > Obviously for doing (3), we need to kick out ratis-hadoop module to the > hadoop project. Thinking about it, I think ratis-hadoop does not belong in > Ratis itself anyway. What do you guys think about moving the code over to > Hadoop, and getting rid of the hadoop dependency to end this shading madness? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-64) Do not pre-shade artifacts
[ https://issues.apache.org/jira/browse/RATIS-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954027#comment-15954027 ] Jing Zhao commented on RATIS-64: Yes maybe we can use this chance to remove hadoop related code from the ratis library. I agree with Enis that the ratis-hadoop code belongs more to hadoop and we can reuse it when we use ratis in Hadoop (e.g. NameNode HA). > Do not pre-shade artifacts > -- > > Key: RATIS-64 > URL: https://issues.apache.org/jira/browse/RATIS-64 > Project: Ratis > Issue Type: Bug >Reporter: Enis Soztutar > > Hugo and I were discussing a case were the shading should happen at the last > step, rather than what we do today. > I think there are 3 possible strategies of shading that one can do: > (1) pre-shade some of your dependencies, so that your other dependencies can > work. This what we do today, we shade PB+ GRPC, etc so that Hadoop can work. > (2) pre-shade some of your dependencies' transitive dependencies so that you > depend on already-shaded artifacts. This will be like having maven artifacts > of shaded-hadoop so that hadoop itself does not bring in any more dependency. > If hadoop has shaded artifacts, or we do shading of hadoop's dependencies in > another repository, we won't need to pre-shade PB, etc. > (3) post-shade. This means that in the code itself we do not depend on > shaded packages anymore, but do the shading as a different module so that we > publish shaded artifacts. This allows the downstreamers to be able to consume > ratis, while allowing ratis source code to be saner. > Obviously for doing (3), we need to kick out ratis-hadoop module to the > hadoop project. Thinking about it, I think ratis-hadoop does not belong in > Ratis itself anyway. What do you guys think about moving the code over to > Hadoop, and getting rid of the hadoop dependency to end this shading madness? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RATIS-64) Do not pre-shade artifacts
[ https://issues.apache.org/jira/browse/RATIS-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15952391#comment-15952391 ] Enis Soztutar commented on RATIS-64: bq. First of all, what is the problem we try to solve? Is it to avoid the imports from org.apache.ratis.shaded packages in our source code? Indeed. It is for keeping the source code shading-free inside Ratis so that we do not fight with the IDE every day. bq. #3 sounds like a good idea. But why we need to kick out ratis-hadoop? I think so, in HBase master code we have both (1) and (3) for example. (3) is implemented in empty-modules hbase-shaded-client and hbase-shaded-server where they each create uber-jars with shaded deps. However, we needed (1), because Hadoop would bring in PB, Guava, etc dependencies which would otherwise cause compilation problems. Again, without removing ratis-hadoop module, we can instead depend on fully shaded Hadoop artifacts so that we can have our versions of PB and Guava in the classpath. However regardless of shading, I feel that ratis-hadoop really does not belong inside Ratis, but it should be maintained inside Hadoop. Maybe once the RPC API is more stable we can consider that refactoring. > Do not pre-shade artifacts > -- > > Key: RATIS-64 > URL: https://issues.apache.org/jira/browse/RATIS-64 > Project: Ratis > Issue Type: Bug >Reporter: Enis Soztutar > > Hugo and I were discussing a case were the shading should happen at the last > step, rather than what we do today. > I think there are 3 possible strategies of shading that one can do: > (1) pre-shade some of your dependencies, so that your other dependencies can > work. This what we do today, we shade PB+ GRPC, etc so that Hadoop can work. > (2) pre-shade some of your dependencies' transitive dependencies so that you > depend on already-shaded artifacts. This will be like having maven artifacts > of shaded-hadoop so that hadoop itself does not bring in any more dependency. > If hadoop has shaded artifacts, or we do shading of hadoop's dependencies in > another repository, we won't need to pre-shade PB, etc. > (3) post-shade. This means that in the code itself we do not depend on > shaded packages anymore, but do the shading as a different module so that we > publish shaded artifacts. This allows the downstreamers to be able to consume > ratis, while allowing ratis source code to be saner. > Obviously for doing (3), we need to kick out ratis-hadoop module to the > hadoop project. Thinking about it, I think ratis-hadoop does not belong in > Ratis itself anyway. What do you guys think about moving the code over to > Hadoop, and getting rid of the hadoop dependency to end this shading madness? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (RATIS-64) Do not pre-shade artifacts
[
https://issues.apache.org/jira/browse/RATIS-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15952385#comment-15952385
]
Enis Soztutar commented on RATIS-64:
This is from [~szetszwo], from the mail thread:
{quote}
I am open to change the shading procedure but I like to understand the
motivation. First of all, what is the problem we try to solve? Is it
to avoid the imports from org.apache.ratis.shaded packages in our
source code?
#3 sounds like a good idea. But why we need to kick out ratis-hadoop?
I mean why it can't be post-shaded?
{quote}
> Do not pre-shade artifacts
> --
>
> Key: RATIS-64
> URL: https://issues.apache.org/jira/browse/RATIS-64
> Project: Ratis
> Issue Type: Bug
>Reporter: Enis Soztutar
>
> Hugo and I were discussing a case were the shading should happen at the last
> step, rather than what we do today.
> I think there are 3 possible strategies of shading that one can do:
> (1) pre-shade some of your dependencies, so that your other dependencies can
> work. This what we do today, we shade PB+ GRPC, etc so that Hadoop can work.
> (2) pre-shade some of your dependencies' transitive dependencies so that you
> depend on already-shaded artifacts. This will be like having maven artifacts
> of shaded-hadoop so that hadoop itself does not bring in any more dependency.
> If hadoop has shaded artifacts, or we do shading of hadoop's dependencies in
> another repository, we won't need to pre-shade PB, etc.
> (3) post-shade. This means that in the code itself we do not depend on
> shaded packages anymore, but do the shading as a different module so that we
> publish shaded artifacts. This allows the downstreamers to be able to consume
> ratis, while allowing ratis source code to be saner.
> Obviously for doing (3), we need to kick out ratis-hadoop module to the
> hadoop project. Thinking about it, I think ratis-hadoop does not belong in
> Ratis itself anyway. What do you guys think about moving the code over to
> Hadoop, and getting rid of the hadoop dependency to end this shading madness?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
