Re: mvn, Edgent samples, and the broader app development and packaging topic

[John D. Ament]

I personally like seeing samples/examples in their own repo.  They avoid
issues with releases that may not be perfect and may bring in other
dependencies.

On Wed, Jun 21, 2017 at 9:22 AM Dale LaBossiere 
wrote:

> Today (pre-mvn), Edgent has a number of sample programs under /samples.
> They are pre-built and included in the binary release bundle as a
> collection of jars edgent.samples.{apps,…}.jar and are run via scripts
> under /scripts.
>
> Your thoughts on samples in the new world? As things are now, the scripts
> don’t work. They presume a (binary release) directory tree where all of the
> edgent jars reside in a particular layout.  They also use the
> non-version-labeled-name form of the Edgent jars.
>
> At a high level there's “how do I build and package/assemble my app for
> execution on a device, and run my app”.  Edgent, mostly, isn’t in the
> business of how to package the bits or get them to / install them on the
> device.
>
> Today’s binary release bundle only addresses the “run” part of that.  The
> binary bundle includes the sample sources but doesn’t include
> tooling/scripts to build them. How lame is that?  :-)
>
> In my mind it’s NOT a hard requirement that we (a) provide pre-built
> samples, nor (b) provide an Edgent binary release bundle (containing all
> the Edgent jars and deps).  I think it’s just the opposite :-)
>
> I’ll stop there just to kickoff this thread.
>
> — Dale
>
>

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

The examples are something I’m currently working on … 

They will work differently than right now (definitely no classpath in the 
Manifest).

In Maven there are several ways to do this … I’ll have to dig into how the 
samples are run to find out how to do it in an ideal way.

I have also seen the current way things are bundled in the binaries … I also 
think changing this would be a good idea as the way it is handled now forces 
the directory structure on the users. I wouldn’t like to do that. I would much 
more prefer a directory with edgent artifacts and an “ext” directory containing 
external dependencies and passing in an individual classpath to the 
applications. … but as I said … I’ll have to dig a little more into the 
examples. First I’ll finish the android modules.

Chris

Am 21.06.17, 15:21 schrieb "Dale LaBossiere" :

Today (pre-mvn), Edgent has a number of sample programs under /samples.  
They are pre-built and included in the binary release bundle as a collection of 
jars edgent.samples.{apps,…}.jar and are run via scripts under /scripts.

Your thoughts on samples in the new world? As things are now, the scripts 
don’t work. They presume a (binary release) directory tree where all of the 
edgent jars reside in a particular layout.  They also use the 
non-version-labeled-name form of the Edgent jars.

At a high level there's “how do I build and package/assemble my app for 
execution on a device, and run my app”.  Edgent, mostly, isn’t in the business 
of how to package the bits or get them to / install them on the device.

Today’s binary release bundle only addresses the “run” part of that.  The 
binary bundle includes the sample sources but doesn’t include tooling/scripts 
to build them. How lame is that?  :-)

In my mind it’s NOT a hard requirement that we (a) provide pre-built 
samples, nor (b) provide an Edgent binary release bundle (containing all the 
Edgent jars and deps).  I think it’s just the opposite :-)

I’ll stop there just to kickoff this thread.

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

Got a couple of pointers to such projects for examples - particularly mvn based 
ones? I’ve been through 10 pages of https://github.com/apache without any luck 
:-)  Maybe only 1 proj that even had a separate repo 
(https://github.com/apache/geode-examples).

> On Jun 21, 2017, at 10:00 AM, John D. Ament  wrote:
> 
> I personally like seeing samples/examples in their own repo.  They avoid
> issues with releases that may not be perfect and may bring in other
> dependencies.

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Hi,

Just a little on what I’m currently working on. 
The maven dependency plugin is able to create an xml representation of a 
projects dependencies. I’m currently trying to create a build that pipes that 
through an XSLT and hereby creates a start-script for the example they are 
built in. This will refer to a directory structure in which all Edgent 
artifacts are output into one directory and all non-edgent dependencies to an 
“ext” subdirectory within that lib directory. The scripts should be able to 
correctly build an applications classpath based on that.

But this weekend a lot more is going on than the last ones, so I will probably 
work on this next week.

Chris




Am 22.06.17, 15:00 schrieb "Dale LaBossiere" :

Got a couple of pointers to such projects for examples - particularly mvn 
based ones? I’ve been through 10 pages of https://github.com/apache without any 
luck :-)  Maybe only 1 proj that even had a separate repo 
(https://github.com/apache/geode-examples).

> On Jun 21, 2017, at 10:00 AM, John D. Ament  wrote:
> 
> I personally like seeing samples/examples in their own repo.  They avoid
> issues with releases that may not be perfect and may bring in other
> dependencies.

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

Chris, thanks for the update.

First, and most importantly, I hope you have a great weekend! :-)

At a high level, I was thinking there’s value in a couple of build/packaging 
approaches.
- create a totally self contained executable-jar
- something that supports a model where a device may have a separately 
installed and managed Edgent that any/all apps on the device use.
- a twist on the former where just the Edgent jars needed by the app are copied 
to a local dir and bundled up

For the samples:
- we should provide tooling that supports all of the above
- the tooling should be able to generate j8, j7, or android
- it might be good if each sample had its own pom.xml / generated .jar rather 
than one bit one
- the user should be able to easily run a CLI-built sample locally (no need to 
actually copy things to a “device”)
- the user should be able to easily import a sample into a maven-integrated IDE 
and build and run it
- no need to include/demo building without mvn tooling

Expanding on that...

- create a totally self contained executable-jar
  This is what you get from Eclipse when selecting a maven-based Edgent app 
project
  and doing Export… > Runnable JAR > “package required libraries into jar”.
  The user just gets that jar onto their device and a “java -jar ” and 
starts their app.  
  Very simple.  No additional work to get/install Edgent.  
  Only the app’s dependent Edgent jars (and deps) are included in the jar - 
i.e., it’s ~minimal/
  This is perhaps an actual production mode of use for simple "single Edgent 
app" devices.

- something that supports a model where a device may have a separately 
installed and managed Edgent that any/all apps on the device use.
  This has some characteristics that you mentioned. 
  I’m not sure how important it is to separate the Edgent vs “ext” 
jars/directories.
  Unless there’s a compelling reason to do so, and it’s really not that much 
effort/complication, I think I’d just use a single dir for all :-)

  Provide some cmd that populates a local dir with *all* the Edgent jars and 
their deps (the analog of a full Edgent binary bundle).
  Provide an edgent-classpath.sh cmd that returns a CLASSPATH value that simply 
includes everything in that dir
  Bundle up that that local dir and the edgent-classpath script.
  The user needs to get bundle onto the device and unpack it.
  Note, building the app wouldn’t use this dir/bundle, it’s just for deployment.
 (well, a user that had non-maven-intergrated app built tooling could use the 
generated dir)

  Create a jar containing just the Edgent app; make its manifest have and entry 
for main class.
  Provide a run-app.sh script that essentially just does:
export CLASSPATH=`edgent-classpath.sh` # user can always augment this 
stmt if they have other things to add
java -jar# start the app
  Bundle up that jar and the run script.
  The user needs to get bundle onto the device and unpack it.
  The app is started via “./run-app.sh”

- a twist on the former where just the Edgent jars needed by the app are copied 
to a local dir and bundled up
  This leverages mvn to create the ~minimal set of Edgent jars and their deps 
needed by the app.
  The dir contains what “mvn dependency:copy-dependency” creates.
  This seems to be the same thing as created by Eclipse> Export … 
> Runnable JAR > “copy required jars into sub-folder”.
  (Note, unlike that Eclipse export, the app’s jar would NOT contain a manifest 
classpath.  The edgent-classpath.sh scheme would be used.)
  The dir can be bundled up and brought to the device and unpacked.

Hope that’s all mostly making sense / sensible :-)

— Dale

> On Jun 23, 2017, at 3:39 AM, Christofer Dutz  
> wrote:
> ...Just a little on what I’m currently working on. 
> The maven dependency plugin is able to create an xml representation of a 
> projects dependencies. I’m currently trying to create a build that pipes that 
> through an XSLT and hereby creates a start-script for the example they are 
> built in. This will refer to a directory structure in which all Edgent 
> artifacts are output into one directory and all non-edgent dependencies to an 
> “ext” subdirectory within that lib directory. The scripts should be able to 
> correctly build an applications classpath based on that.

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Hi Dale,

I just paused my work on the classpath generation and simply added the 
maven-shade-plugin which creates uber-jars, which produces jars that contain 
not only the modules content, but also includes all dependencies into one big 
fat jar … I also added a sample start script to the topology project. 

Would this do as a startging point?

Regarding your points:
First section:
- Well it now produces an additional self-contained jar additionally
- When using Maven you usually don’t use this model of having something 
installed, it is possible using the “system” scope in dependencies and 
providing the systemPath element pointing to some 
“${edgent.home}/my-edgent-lib-1.2.0.jar” but that wouldn’t be good style. 
Usually the single-source for the artifacts would be the maven local repo on 
that device.
Second section:
- The shade plugin would work on all platforms. All I would have to do, is to 
add the plugin configuration to the other platforms.
- Having jars for each example would require splitting them up into separate 
maven modules. I don’t think that’s a good idea. I’d prefer a start script that 
lets the user choose the sample to run. I know this would require adjusting the 
start script whenever a sample is added/deleted/changed, but it would reduce 
the number of these super-fat uber jars.
- If the user ran the maven build (mvn clean package) any user would be able to 
run the samples.
- IntelliJ was able to immediately run every example I tried, even before the 
shade plugin was added and I think Eclipse this should also work.

Regarding the binary distribution. I think we should put non-edgent libs in a 
separate directory. Mainly because this way we clearly draw a line to what’s 
our stuff and which is from others (even if these “others” are also ASF 
projects)

Chris

Am 23.06.17, 18:40 schrieb "Dale LaBossiere" :

Chris, thanks for the update.

First, and most importantly, I hope you have a great weekend! :-)

At a high level, I was thinking there’s value in a couple of 
build/packaging approaches.
- create a totally self contained executable-jar
- something that supports a model where a device may have a separately 
installed and managed Edgent that any/all apps on the device use.
- a twist on the former where just the Edgent jars needed by the app are 
copied to a local dir and bundled up

For the samples:
- we should provide tooling that supports all of the above
- the tooling should be able to generate j8, j7, or android
- it might be good if each sample had its own pom.xml / generated .jar 
rather than one bit one
- the user should be able to easily run a CLI-built sample locally (no need 
to actually copy things to a “device”)
- the user should be able to easily import a sample into a maven-integrated 
IDE and build and run it
- no need to include/demo building without mvn tooling

Expanding on that...

- create a totally self contained executable-jar
  This is what you get from Eclipse when selecting a maven-based Edgent app 
project
  and doing Export… > Runnable JAR > “package required libraries into jar”.
  The user just gets that jar onto their device and a “java -jar ” 
and starts their app.  
  Very simple.  No additional work to get/install Edgent.  
  Only the app’s dependent Edgent jars (and deps) are included in the jar - 
i.e., it’s ~minimal/
  This is perhaps an actual production mode of use for simple "single 
Edgent app" devices.

- something that supports a model where a device may have a separately 
installed and managed Edgent that any/all apps on the device use.
  This has some characteristics that you mentioned. 
  I’m not sure how important it is to separate the Edgent vs “ext” 
jars/directories.
  Unless there’s a compelling reason to do so, and it’s really not that 
much effort/complication, I think I’d just use a single dir for all :-)

  Provide some cmd that populates a local dir with *all* the Edgent jars 
and their deps (the analog of a full Edgent binary bundle).
  Provide an edgent-classpath.sh cmd that returns a CLASSPATH value that 
simply includes everything in that dir
  Bundle up that that local dir and the edgent-classpath script.
  The user needs to get bundle onto the device and unpack it.
  Note, building the app wouldn’t use this dir/bundle, it’s just for 
deployment.
 (well, a user that had non-maven-intergrated app built tooling could use 
the generated dir)

  Create a jar containing just the Edgent app; make its manifest have and 
entry for main class.
  Provide a run-app.sh script that essentially just does:
export CLASSPATH=`edgent-classpath.sh` # user can always augment this 
stmt if they have other things to add
java -jar# start the app
  Bundle up that jar and the run script.
  The user needs to get bundle onto the device and unpack it.
 

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

Chris, thanks for helping to work through this.

> On Jun 25, 2017, at 9:44 AM, Christofer Dutz  
> wrote:
> 
> Hi Dale,
> 
> I just paused my work on the classpath generation and simply added the 
> maven-shade-plugin which creates uber-jars, which produces jars that contain 
> not only the modules content, but also includes all dependencies into one big 
> fat jar … I also added a sample start script to the topology project. 
> 
> Would this do as a startging point?
I pulled down the latest PR changes and built for all platforms.
I can’t seem to eliminate all errors in a Eclipse dev context (CLI builds are 
fine).  I’ll follow up with that separately.

> 
> Regarding your points:
> First section:
> - Well it now produces an additional self-contained jar additionally
Observation: the uber-jar contains the extracted dependent component/jar 
contents as opposed to containing the actual dependent jars.  Any benefits of 
one form over there other?
There’s some “xsl” stuff in the samples pom - part of your paused classpath 
generation work that should be removed?

> - When using Maven you usually don’t use this model of having something 
> installed, it is possible using the “system” scope in dependencies and 
> providing the systemPath element pointing to some 
> “${edgent.home}/my-edgent-lib-1.2.0.jar” but that wouldn’t be good style. 
> Usually the single-source for the artifacts would be the maven local repo on 
> that device.
I’m confident I don’t know the various ways in which organizations will want to 
produce edge devices that contain Edgent-based apps :-)  So I’m hesitant to 
constrain things / require that maven technology be resident on the device.  
That’s what’s motivates me to think we need to ~easily support “install” based 
cases… and I’m not confident that supplying “uber-jar” as the only initial 
alternative will be OK.

I guess if we also provide a way to construct a dir(s) containing all of the 
edgent jars dependencies (not just a single sample’s dependencies yet) then 
maybe that and the uber-jar alternative will be good enough to start.  At least 
the user won’t have to scratch their head on how to get the jars out of the 
local repo if they really need them.

> Second section:
> - The shade plugin would work on all platforms. All I would have to do, is to 
> add the plugin configuration to the other platforms.
> - Having jars for each example would require splitting them up into separate 
> maven modules. I don’t think that’s a good idea. I’d prefer a start script 
> that lets the user choose the sample to run. I know this would require 
> adjusting the start script whenever a sample is added/deleted/changed, but it 
> would reduce the number of these super-fat uber jars.
At one level I appreciate what you’re saying.  Maybe theres a need for another 
type of sample that differs from this collection?  Let me try to clarify…

At a high level it feels like we need at least one sample project that we can 
say “copy this sample project (source, pom, scripts if any) to jump-start the 
development of your Edgent-based app”.  Do you think that what you’re 
considering will serve that purpose well?

Let’s fully treat samples like app code that a user would write.
Don't build them as part of edgent-parent, don’t depend at all on 
edgent-parent/pom.xml.  Depending on a common sample-parent pom is OK.
Don’t include sample jars in a binary release.
Get things into a shape where samples could live in a separate repo.

I don’t think “create uber jar” has to be included in a sample default build 
config.  It might be fine / preferable? for the user to have to specially 
request it.

The connector-sample is an example of why a multi-sample uber-jar isn’t a 
great.  If someone wants to try the mqtt connector sample on a device where 
they didn’t build the sample, the easiest thing would be to get the sample’s 
uber-jar to the device.  But that uber-jar is large ~22MB.  I’m pretty sure 
that a lot of that is associated with other connector samples (e.g., the 
kafka-connector’s dependent jars).  As a user of a sample I think I’d 
appreciate being able to bring over a much smaller jar.  And being able to show 
a ~minimal uber-jar for an app might start to help with questions about “size”.

> - If the user ran the maven build (mvn clean package) any user would be able 
> to run the samples.
> - IntelliJ was able to immediately run every example I tried, even before the 
> shade plugin was added and I think Eclipse this should also work.
Yup, I knew that Eclipse already supported that (and assumed similarly for 
INtelliJ).  Was just trying to be complete :-)

> 
> Regarding the binary distribution. I think we should put non-edgent libs in a 
> separate directory. Mainly because this way we clearly draw a line to what’s 
> our stuff and which is from others (even if these “others” are also ASF 
> projects)
OK, seems reasonable.

> 
> Chris
> 
> Am 23.06.17, 18:40 schrieb "Dale LaBossiere" :
> 
>Chris, tha

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Hi Dale,

Well it would be possible to reduce the classes in the uber-jars to only the 
used classes, but this usually causes issues as soon as there’s reflection in 
the software somewhere.

Right now, I only concentrated on building the samples not the distribution. 
That was next on my list. For people using an (let’s call it an Edgent 
installation) would simply add a reference to the lib and lib/ext directories 
to their applications classpath and be done.

Yes … the XSL stuff was for my start-script generation approach … I deactivated 
that for now, giving the uber-jar a try. If we stick to that, I’ll remove that 
completely.

To help people getting started, I would rather suggest using Maven archetypes. 
These are special Maven modules which act as templates which you can use to 
automatically setup a new project. In the Flex project, I created several 
archetypes to setup libraries, applications, even servers running on Spring 
boot. All a user must do is run the archetype maven plugin and pass in the 
archetype module he wants to use, and the plugin creates a new project for him. 

I could move the uber-jar generation config into a separate profile which the 
user must manually activate …

So, on my to-do list now I have noted:
- Move the uber-jar configuration into a profile
- No longer make the main Edgent build also build the examples
- Create an example archetype based on the Hello World example
- Create an assembly for a binary distribution

Correct?

Chris


Am 26.06.17, 23:26 schrieb "Dale LaBossiere" :

Chris, thanks for helping to work through this.

> On Jun 25, 2017, at 9:44 AM, Christofer Dutz  
wrote:
> 
> Hi Dale,
> 
> I just paused my work on the classpath generation and simply added the 
maven-shade-plugin which creates uber-jars, which produces jars that contain 
not only the modules content, but also includes all dependencies into one big 
fat jar … I also added a sample start script to the topology project. 
> 
> Would this do as a startging point?
I pulled down the latest PR changes and built for all platforms.
I can’t seem to eliminate all errors in a Eclipse dev context (CLI builds 
are fine).  I’ll follow up with that separately.

> 
> Regarding your points:
> First section:
> - Well it now produces an additional self-contained jar additionally
Observation: the uber-jar contains the extracted dependent component/jar 
contents as opposed to containing the actual dependent jars.  Any benefits of 
one form over there other?
There’s some “xsl” stuff in the samples pom - part of your paused classpath 
generation work that should be removed?

> - When using Maven you usually don’t use this model of having something 
installed, it is possible using the “system” scope in dependencies and 
providing the systemPath element pointing to some 
“${edgent.home}/my-edgent-lib-1.2.0.jar” but that wouldn’t be good style. 
Usually the single-source for the artifacts would be the maven local repo on 
that device.
I’m confident I don’t know the various ways in which organizations will 
want to produce edge devices that contain Edgent-based apps :-)  So I’m 
hesitant to constrain things / require that maven technology be resident on the 
device.  That’s what’s motivates me to think we need to ~easily support 
“install” based cases… and I’m not confident that supplying “uber-jar” as the 
only initial alternative will be OK.

I guess if we also provide a way to construct a dir(s) containing all of 
the edgent jars dependencies (not just a single sample’s dependencies yet) then 
maybe that and the uber-jar alternative will be good enough to start.  At least 
the user won’t have to scratch their head on how to get the jars out of the 
local repo if they really need them.

> Second section:
> - The shade plugin would work on all platforms. All I would have to do, 
is to add the plugin configuration to the other platforms.
> - Having jars for each example would require splitting them up into 
separate maven modules. I don’t think that’s a good idea. I’d prefer a start 
script that lets the user choose the sample to run. I know this would require 
adjusting the start script whenever a sample is added/deleted/changed, but it 
would reduce the number of these super-fat uber jars.
At one level I appreciate what you’re saying.  Maybe theres a need for 
another type of sample that differs from this collection?  Let me try to 
clarify…

At a high level it feels like we need at least one sample project that we 
can say “copy this sample project (source, pom, scripts if any) to jump-start 
the development of your Edgent-based app”.  Do you think that what you’re 
considering will serve that purpose well?

Let’s fully treat samples like app code that a user would write.
Don't build them as part of edgent-parent, don’t depend at all on 
edgent-parent/pom.xml.  Depending on a common s

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

> On Jun 27, 2017, at 3:24 AM, Christofer Dutz  
> wrote:
> 
> Hi Dale,
> 
> Well it would be possible to reduce the classes in the uber-jars to only the 
> used classes, but this usually causes issues as soon as there’s reflection in 
> the software somewhere.
Sorry for the confusion.  I wasn’t asking for that sort of additional 
uber-included classes filtering (an interesting idea; I seem to recall some 
tool that can do that).  Rather, in Eclipse, " > Export … > Java > 
Runnable JAR > package required libraries into generated JAR" creates a jar 
that includes the actual dependent jars, not their extracted classes.  The jar 
also includes a tiny “Jar in Jar” loader which is the “main-class”.  Seems very 
clean/nice.  Was interested in your opinion on it.

> ...
> To help people getting started, I would rather suggest using Maven 
> archetypes. ...
I had a feeling you might suggest that :-)  Works for me.

> I could move the uber-jar generation config into a separate profile which the 
> user must manually activate …
Right now I don’t have a strong opinion on that.  I had mentioned it in case it 
made it easier to consider/accept alternative ideas like 
per-sample-projects/poms.  If you still think per-sample-poms are best avoided 
for other reasons then OK, stick with the current structure and uber generation 
scheme.

> So, on my to-do list now I have noted:
> - Move the uber-jar configuration into a profile
see above.  your call.
> - No longer make the main Edgent build also build the examples
sounds good (I guess you agreed with the rationale)
> - Create an example archetype based on the Hello World example
sounds good
> - Create an assembly for a binary distribution
I’m unclear on the full implications of that. I’m now of the, perhaps 
incorrect?, opinion that we should NOT distribute binary distribution tgz 
bundles.  Are we on the same page?  
If this assembly/tgz is just something that someone can run/create if they need 
a way to access the Edgent jars and deps outside of the repo then I get that.

I’m imagining a tool that you can point at component in a repo and it will do 
the jar copying/bundling.  No need for any other sources, etc.
e.g., if I identify edgent-parent:1.2.0-SNAPSHOT it creates a bundle with all 
of the jars/deps of that ss/release.  extra credit? - If I identify 
edgent-connectors-mqtt:1.2.0-SNAPSHOT the generated bundle only includes that 
subset of jars/deps.  Does that make sense to you?  There’s nothing edgent 
specific about such a tool; maybe one already exists?

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Regarding the assembly,

The way the build is currently setup, we would be distributing jars via Apaches 
Maven Repo which is synced with Maven Central. So, everyone using maven would 
easily be able to use these. But if someone isn’t using Maven or Ivy with a 
Maven resolver or Gradle with a Maven resolver, they would have to either build 
Edgent locally or manually download all the jars from Maven Central. I don’t 
think people would like that. I know of quite a few that would be used to 
creating an Eclipse project with a libs directory, which they simply copy a set 
of jars into. For that type of user, a binary distribution would be beneficial.

Regarding the Jar with jars inside … seems I could use the assembly plugin for 
that, it’s jar-with-dependencies goal seems to be for creating exactly such a 
thing.
I’ll check that out right away.

Chris


Am 27.06.17, 15:15 schrieb "Dale LaBossiere" :


> On Jun 27, 2017, at 3:24 AM, Christofer Dutz  
wrote:
> 
> Hi Dale,
> 
> Well it would be possible to reduce the classes in the uber-jars to only 
the used classes, but this usually causes issues as soon as there’s reflection 
in the software somewhere.
Sorry for the confusion.  I wasn’t asking for that sort of additional 
uber-included classes filtering (an interesting idea; I seem to recall some 
tool that can do that).  Rather, in Eclipse, " > Export … > Java > 
Runnable JAR > package required libraries into generated JAR" creates a jar 
that includes the actual dependent jars, not their extracted classes.  The jar 
also includes a tiny “Jar in Jar” loader which is the “main-class”.  Seems very 
clean/nice.  Was interested in your opinion on it.

> ...
> To help people getting started, I would rather suggest using Maven 
archetypes. ...
I had a feeling you might suggest that :-)  Works for me.

> I could move the uber-jar generation config into a separate profile which 
the user must manually activate …
Right now I don’t have a strong opinion on that.  I had mentioned it in 
case it made it easier to consider/accept alternative ideas like 
per-sample-projects/poms.  If you still think per-sample-poms are best avoided 
for other reasons then OK, stick with the current structure and uber generation 
scheme.

> So, on my to-do list now I have noted:
> - Move the uber-jar configuration into a profile
see above.  your call.
> - No longer make the main Edgent build also build the examples
sounds good (I guess you agreed with the rationale)
> - Create an example archetype based on the Hello World example
sounds good
> - Create an assembly for a binary distribution
I’m unclear on the full implications of that. I’m now of the, perhaps 
incorrect?, opinion that we should NOT distribute binary distribution tgz 
bundles.  Are we on the same page?  
If this assembly/tgz is just something that someone can run/create if they 
need a way to access the Edgent jars and deps outside of the repo then I get 
that.

I’m imagining a tool that you can point at component in a repo and it will 
do the jar copying/bundling.  No need for any other sources, etc.
e.g., if I identify edgent-parent:1.2.0-SNAPSHOT it creates a bundle with 
all of the jars/deps of that ss/release.  extra credit? - If I identify 
edgent-connectors-mqtt:1.2.0-SNAPSHOT the generated bundle only includes that 
subset of jars/deps.  Does that make sense to you?  There’s nothing edgent 
specific about such a tool; maybe one already exists?

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Forget the assembly jar-with-dependencies goal … it produces the same output as 
the shade plugin :-(

I think the packaging type you are looking for is like that of Spring Boot 
applications. I’ll try to find out if this is possible without spring boot.

Chris


Am 27.06.17, 15:28 schrieb "Christofer Dutz" :

Regarding the assembly,

The way the build is currently setup, we would be distributing jars via 
Apaches Maven Repo which is synced with Maven Central. So, everyone using maven 
would easily be able to use these. But if someone isn’t using Maven or Ivy with 
a Maven resolver or Gradle with a Maven resolver, they would have to either 
build Edgent locally or manually download all the jars from Maven Central. I 
don’t think people would like that. I know of quite a few that would be used to 
creating an Eclipse project with a libs directory, which they simply copy a set 
of jars into. For that type of user, a binary distribution would be beneficial.

Regarding the Jar with jars inside … seems I could use the assembly plugin 
for that, it’s jar-with-dependencies goal seems to be for creating exactly such 
a thing.
I’ll check that out right away.

Chris


Am 27.06.17, 15:15 schrieb "Dale LaBossiere" :


> On Jun 27, 2017, at 3:24 AM, Christofer Dutz 
 wrote:
> 
> Hi Dale,
> 
> Well it would be possible to reduce the classes in the uber-jars to 
only the used classes, but this usually causes issues as soon as there’s 
reflection in the software somewhere.
Sorry for the confusion.  I wasn’t asking for that sort of additional 
uber-included classes filtering (an interesting idea; I seem to recall some 
tool that can do that).  Rather, in Eclipse, " > Export … > Java > 
Runnable JAR > package required libraries into generated JAR" creates a jar 
that includes the actual dependent jars, not their extracted classes.  The jar 
also includes a tiny “Jar in Jar” loader which is the “main-class”.  Seems very 
clean/nice.  Was interested in your opinion on it.

> ...
> To help people getting started, I would rather suggest using Maven 
archetypes. ...
I had a feeling you might suggest that :-)  Works for me.

> I could move the uber-jar generation config into a separate profile 
which the user must manually activate …
Right now I don’t have a strong opinion on that.  I had mentioned it in 
case it made it easier to consider/accept alternative ideas like 
per-sample-projects/poms.  If you still think per-sample-poms are best avoided 
for other reasons then OK, stick with the current structure and uber generation 
scheme.

> So, on my to-do list now I have noted:
> - Move the uber-jar configuration into a profile
see above.  your call.
> - No longer make the main Edgent build also build the examples
sounds good (I guess you agreed with the rationale)
> - Create an example archetype based on the Hello World example
sounds good
> - Create an assembly for a binary distribution
I’m unclear on the full implications of that. I’m now of the, perhaps 
incorrect?, opinion that we should NOT distribute binary distribution tgz 
bundles.  Are we on the same page?  
If this assembly/tgz is just something that someone can run/create if 
they need a way to access the Edgent jars and deps outside of the repo then I 
get that.

I’m imagining a tool that you can point at component in a repo and it 
will do the jar copying/bundling.  No need for any other sources, etc.
e.g., if I identify edgent-parent:1.2.0-SNAPSHOT it creates a bundle 
with all of the jars/deps of that ss/release.  extra credit? - If I identify 
edgent-connectors-mqtt:1.2.0-SNAPSHOT the generated bundle only includes that 
subset of jars/deps.  Does that make sense to you?  There’s nothing edgent 
specific about such a tool; maybe one already exists?

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

> On Jun 27, 2017, at 9:28 AM, Christofer Dutz  
> wrote:
> 
> Regarding the assembly,
> 
> The way the build is currently setup, we would be distributing jars via 
> Apaches Maven Repo which is synced with Maven Central. So, everyone using 
> maven would easily be able to use these. But if someone isn’t using Maven or 
> Ivy with a Maven resolver or Gradle with a Maven resolver, they would have to 
> either build Edgent locally or manually download all the jars from Maven 
> Central. I don’t think people would like that. I know of quite a few that 
> would be used to creating an Eclipse project with a libs directory, which 
> they simply copy a set of jars into. For that type of user, a binary 
> distribution would be beneficial.

Yes, I completely understand that overall use case / need.  I’d just like to 
address it without actually distributing a composite binary bundle :-)
Instead I’d like to just give / tell them a way to be able to just easily “get” 
the jars/deps that we will already be distributing in repos.
I definitely don’t want such a user to have to resort to building Edgent just 
to get their hands on the jars.

I'm surprised there isn’t already such a “get-from-repo [—recursive] 
component-coordinate” tool.

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

A brief search turned up 
https://github.com/simpligility/maven-repository-tools/tree/master/maven-repository-provisioner
 

I cursory look at its README makes it sound almost exactly like what I was 
hoping for… except it looks like it may only copy out to another repo and not 
offer an “copy out to arbitrary dir” sort of mode :-)  I’ll have to look more 
closely.

> On Jun 27, 2017, at 10:51 AM, Dale LaBossiere  wrote:
> 
> 
>> On Jun 27, 2017, at 9:28 AM, Christofer Dutz  
>> wrote:
>> 
>> Regarding the assembly,
>> 
>> The way the build is currently setup, we would be distributing jars via 
>> Apaches Maven Repo which is synced with Maven Central. So, everyone using 
>> maven would easily be able to use these. But if someone isn’t using Maven or 
>> Ivy with a Maven resolver or Gradle with a Maven resolver, they would have 
>> to either build Edgent locally or manually download all the jars from Maven 
>> Central. I don’t think people would like that. I know of quite a few that 
>> would be used to creating an Eclipse project with a libs directory, which 
>> they simply copy a set of jars into. For that type of user, a binary 
>> distribution would be beneficial.
> 
> Yes, I completely understand that overall use case / need.  I’d just like to 
> address it without actually distributing a composite binary bundle :-)
> Instead I’d like to just give / tell them a way to be able to just easily 
> “get” the jars/deps that we will already be distributing in repos.
> I definitely don’t want such a user to have to resort to building Edgent just 
> to get their hands on the jars.
> 
> I'm surprised there isn’t already such a “get-from-repo [—recursive] 
> component-coordinate” tool.
> 
> — Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

That only seems to work for deploying things and not for downloading them :-(

Chris

Am 27.06.17, 17:01 schrieb "Dale LaBossiere" :

A brief search turned up 
https://github.com/simpligility/maven-repository-tools/tree/master/maven-repository-provisioner
 

I cursory look at its README makes it sound almost exactly like what I was 
hoping for… except it looks like it may only copy out to another repo and not 
offer an “copy out to arbitrary dir” sort of mode :-)  I’ll have to look more 
closely.

> On Jun 27, 2017, at 10:51 AM, Dale LaBossiere  
wrote:
> 
> 
>> On Jun 27, 2017, at 9:28 AM, Christofer Dutz  
wrote:
>> 
>> Regarding the assembly,
>> 
>> The way the build is currently setup, we would be distributing jars via 
Apaches Maven Repo which is synced with Maven Central. So, everyone using maven 
would easily be able to use these. But if someone isn’t using Maven or Ivy with 
a Maven resolver or Gradle with a Maven resolver, they would have to either 
build Edgent locally or manually download all the jars from Maven Central. I 
don’t think people would like that. I know of quite a few that would be used to 
creating an Eclipse project with a libs directory, which they simply copy a set 
of jars into. For that type of user, a binary distribution would be beneficial.
> 
> Yes, I completely understand that overall use case / need.  I’d just like 
to address it without actually distributing a composite binary bundle :-)
> Instead I’d like to just give / tell them a way to be able to just easily 
“get” the jars/deps that we will already be distributing in repos.
> I definitely don’t want such a user to have to resort to building Edgent 
just to get their hands on the jars.
> 
> I'm surprised there isn’t already such a “get-from-repo [—recursive] 
component-coordinate” tool.
> 
> — Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Dale LaBossiere]

maven-repository-provisioner seemed oh so close… :-)  You can give it an 
arbitrary destination dir to copy into.
But it didn’t work for an artifact like 
org.apache.edgent:edgent-parent:1.2.0-SNAPSHOT.
There were complaints and it didn’t go include the sub-projects. Enhancing the 
code didn’t look promising. 
So hard to believe that mvn doesn’t have simple tools for this sort of thing.

So I’ve create a shell script that does the trick.  All one will have to do is 
get this single script and run it (so I claim :-).

The script creates a new mvn project and then tweaks its pom to add 
dependencies to the various Edgent components.
It then does a “mvn compile" (to get the jars into the local maven repo)
and “mvn dependency:copy-dependencies” (to copy them out of the local repo).
It separates them into ext-jars and edgent-jars dirs and creates a classpath.sh 
that
can be used by apps in setting CLASSPATH.

Still tweaking, etc.  I’ll share in a bit.

— Dale


> On Jun 29, 2017, at 5:22 AM, Christofer Dutz  
> wrote:
> 
> That only seems to work for deploying things and not for downloading them :-(
> 
> Chris
> 
> Am 27.06.17, 17:01 schrieb "Dale LaBossiere" :
> 
>A brief search turned up 
> https://github.com/simpligility/maven-repository-tools/tree/master/maven-repository-provisioner
>  
> 
>I cursory look at its README makes it sound almost exactly like what I was 
> hoping for… except it looks like it may only copy out to another repo and not 
> offer an “copy out to arbitrary dir” sort of mode :-)  I’ll have to look more 
> closely.
> 
>> On Jun 27, 2017, at 10:51 AM, Dale LaBossiere  wrote:
>> 
>> 
>>> On Jun 27, 2017, at 9:28 AM, Christofer Dutz  
>>> wrote:
>>> 
>>> Regarding the assembly,
>>> 
>>> The way the build is currently setup, we would be distributing jars via 
>>> Apaches Maven Repo which is synced with Maven Central. So, everyone using 
>>> maven would easily be able to use these. But if someone isn’t using Maven 
>>> or Ivy with a Maven resolver or Gradle with a Maven resolver, they would 
>>> have to either build Edgent locally or manually download all the jars from 
>>> Maven Central. I don’t think people would like that. I know of quite a few 
>>> that would be used to creating an Eclipse project with a libs directory, 
>>> which they simply copy a set of jars into. For that type of user, a binary 
>>> distribution would be beneficial.
>> 
>> Yes, I completely understand that overall use case / need.  I’d just like to 
>> address it without actually distributing a composite binary bundle :-)
>> Instead I’d like to just give / tell them a way to be able to just easily 
>> “get” the jars/deps that we will already be distributing in repos.
>> I definitely don’t want such a user to have to resort to building Edgent 
>> just to get their hands on the jars.
>> 
>> I'm surprised there isn’t already such a “get-from-repo [—recursive] 
>> component-coordinate” tool.
>> 
>> — Dale
> 
> 
>