Stack wrote:
On Wed, Apr 12, 2017 at 8:22 AM, Josh Elser<[email protected]> wrote:
....
This makes me wonder if we could construct source jars just the same as
we're creating shaded jars. Google has lead me to [2][3], but I've never
tried either. The latter option seems to be acknowledging that the source
might not actually compile, but the package names would at least be correct.
I think this would be a good early experiment which, if it does work out,
removes the only acknowledged "hole" in the current plan.
Mighty Josh.
As it happens, the 'createSources' you cite is currently in use by our
hbase-protocol-shaded module for the specific purpose of keeping our IDE'rs
happy; we relocate protobuf itself so we can have pb3.2 and pb2.5 on our
CLASSPATH but IDEs don't complain because the src for the relocated pb is
checked-in to our codebase.
Aha! Wizards are ahead of me on the path already :)
Here's more if interested:
All modules that make use of protos by convention require a 'pre-build'
step. Generally, the pre-build is required if you add or mod .proto files.
At pre-build we generate class files from the amended .protos and then
check-in the product. See the README in each of our proto-carrying modules.
In the hbase-protocol-shaded case, we abuse the pre-build notion so we can
house an internal pb version, one that does not agree w/ hadoops nor w/
that used by our CPs describing Endpoint services. We do as follows (the
order may not be exact in below):
* Generate class files from protos
* Shade the built artifact (with the createSource flag set).
* Unjar the artifact which has generated proto and protobuf src classes in
it
* Apply a few patches to protobuf to support offheap work (stuff we need
to push back up to pb)
* Overlay our current protobuf src w/ the new version in the src tree -- a
big no-no (smile).
You then check it all in....
It has been suggested we undo these hokey pre-build steps and just generate
classes from protobuf inline w/ the main build. A plugin we just figured
makes this possible since it provides the platform-appropriate protoc
(org.xolstice.maven.plugins). We could do the simplification currently for
all but the hbase-protocol-shaded module because of the above jujitsu. One
thought is that if we had a pre-build artifact as is being suggested in
this thread, I could move the patched pb there and purge the pre-build step
everywhere.
The xolstice plugin has been working well for me elsewhere so far. +1
I think we're speaking the same language. My realization was that,
hopefully, we could automatically create that pre-built artifact via
Maven magic instead of having a human copy source files around.
I think pushing this part forward with some code is the next logical
step. Seems to be consensus about taking our known internal dependencies
and performing this shade magic.
Don't want to stomp on your worries, Nick. I think your worries are more
about the presentation to downstream and we're in agreement about
isolating our internal deps with the described approach?
Thanks,
St.Ack
We (The HBase PMC) will have to make releases of this new artifact and vote
on them. I think it will be a relatively rare event.
I'd be up for doing the first cut if folks are game.
St.Ack
1. URL via Sean but for committers to view only:
https://reporeq.apache.org/
[2] https://maven.apache.org/plugins/maven-shade-plugin/shade-
mojo.html#createSourcesJar
[3] https://maven.apache.org/plugins/maven-shade-plugin/shade-
mojo.html#shadeSourcesContent
