Re: [DISCUSS] Let's switch for GitHub for our primary repo!

[Christofer Dutz]

Hi guys,

This is the first mail I got after signing up ;-)
Fascinating project you have here … guess I will hang around quite some time in 
the (hopefully) near future.

Guess I might be able to help you with this GIT topic. Are you thinking of 
switching to GIT in general from SVN? Or do you want to commit to GitHub and 
hope for this to get synced back to the ASF SVN?

Chris



Am 23.05.17, 15:50 schrieb "Samantha Chan" :

Hi Dale,

+1!  I would like to use Github as a primary repo.
I think it will make it easier for people to discover, adopt and contribute
code back. Using Github issues vs Jira also seems like a good idea.  We can
have everything at a single point of entry.

Samantha



On Wed, May 17, 2017 at 2:02 PM, John D. Ament 
wrote:

> Assuming we have a CSV type of dump, it seems like a client could be
> written to do the import.
>
> http://stackoverflow.com/questions/31125655/is-there-a-
> way-to-import-jira-issues-to-github
>
> John
>
> On Wed, May 17, 2017 at 1:58 PM Dale LaBossiere 
> wrote:
>
> > Hey all,
> >
> > Today @ApacheCon I learned that infra now supports GitHub as a primary
> > repo (in contrast to our GitHub mirror of svn).  What would switching 
buy
> > us?   The Merge button! :-)  and use of GitHub issues.
> >
> > One roadbump, apparently the migration of JIRA issues to GitHub isn’t
> > automated.  At least one group with a something like 100 JIRAs just did
> it
> > all by hand.  (don’t know if that was a total of 100 open/closed or just
> > open).  In any case that’s something to better understand.
> >
> > What do you think?
> >
> > — Dale
>

Re: [DISCUSS] Let's switch for GitHub for our primary repo!

[Christofer Dutz]

Hi,

I’m just asking, because the Flex project started with SVN and we migrated to 
ASF GIT. Now we have the GitHub mirrors setup and can process Pull requests 
etc. But it’s not our main repo. Our main repo is the ASF one. 

Are you sure you can really use the Merge button on pull requests? Wouldn’t 
this require deeper access to Github. We currently do things by sending 
commands to close pull requests etc. in the commit messages. So, if we say: 
“closes #238” the ASF bots close the pull requests for us. If things have 
changed, this might be interesting for the Flex Project too.

In general, these types of migrations are really easy … just create one of the 
Infra tickets and things happen automatically. I think they even have issue 
templates for things like this.

And regarding help … The sort of thing I do quite a lot in my paid job and here 
at Apache, is build systems cleanup … my speciality are Maven migrations. 
Reading other emails it looked a little complicated with your current Gradle 
setup … want any help in that … just ask ;-)

Looking forward do digging deeper into this (

Chris

Am 23.05.17, 17:13 schrieb "Dale LaBossiere" :

Hi Christofer,  

Sorry I didn’t manage to catch up with you @ ApacheCon following your SCADA 
talk and mention of interest in Edgent.  We’re greatly interested in your 
interest/feedback!

Re git, Edgent(Quarks) was a GitHub based project prior to 
donation/incubating @ ASF.  The Apache Edgent ASF project was created with SVN 
as the main repo with an ASF git repo mirror and a read-only mirror of that to 
GitHub — apparently standard ASF fare at the time for those wanting to use 
git/github.  We promote using the gitflow model of forking the GitHub mirror 
for committing/creating and reviewing PRs.  Then committers use git to merge 
the PR to the ASF git repo - ugh.

The proposal is to convert to the newly supported “GitHub as the main repo” 
approach.  Then we’ll be able to fully use the normal gitflow model including 
the Merge button on the PR’s page!  And use of GitHub Issues instead of JIRA.

As I understand it, the repo switch issue is (easily?) handled by 
generating an infra request once the project decides to head in that direction.

— Dale


> On May 23, 2017, at 10:29 AM, Christofer Dutz  
wrote:
> 
> Hi guys,
> 
> This is the first mail I got after signing up ;-)
> Fascinating project you have here … guess I will hang around quite some 
time in the (hopefully) near future.
> 
> Guess I might be able to help you with this GIT topic. Are you thinking 
of switching to GIT in general from SVN? Or do you want to commit to GitHub and 
hope for this to get synced back to the ASF SVN?
> 
> Chris
> 
> 
> 
> Am 23.05.17, 15:50 schrieb "Samantha Chan" :
> 
>Hi Dale,
> 
>+1!  I would like to use Github as a primary repo.
>I think it will make it easier for people to discover, adopt and 
contribute
>code back. Using Github issues vs Jira also seems like a good idea.  
We can
>have everything at a single point of entry.
> 
>Samantha
> 
> 
> 
>On Wed, May 17, 2017 at 2:02 PM, John D. Ament 
>wrote:
> 
>> Assuming we have a CSV type of dump, it seems like a client could be
>> written to do the import.
>> 
>> http://stackoverflow.com/questions/31125655/is-there-a-
>> way-to-import-jira-issues-to-github
>> 
>> John
>> 
>> On Wed, May 17, 2017 at 1:58 PM Dale LaBossiere 
>> wrote:
>> 
>>> Hey all,
>>> 
>>> Today @ApacheCon I learned that infra now supports GitHub as a primary
>>> repo (in contrast to our GitHub mirror of svn).  What would switching 
buy
>>> us?   The Merge button! :-)  and use of GitHub issues.
>>> 
>>> One roadbump, apparently the migration of JIRA issues to GitHub isn’t
>>> automated.  At least one group with a something like 100 JIRAs just did
>> it
>>> all by hand.  (don’t know if that was a total of 100 open/closed or just
>>> open).  In any case that’s something to better understand.
>>> 
>>> What do you think?
>>> 
>>> — Dale
>> 
> 
> 

Interest in a Edgent talk at a german conference?

[Christofer Dutz]

Hi guys,

On Apache community mailinglist 
(d...@community.apache.org) we are currently 
discussing an interesting option of having an Apache Day at this years 
Solutions.Hamburg conference. If you think that’s interesting, please join the 
discussion there.

Chris

Re: [DISCUSS] Let's switch for GitHub for our primary repo!

[Christofer Dutz]

Hi Dale,

Cool … will have to investigate … think this would be good for the Flex project 
too ;-)

Regarding the build system … what is the projects general opinion on a Maven 
build? I know that Ant and Gradle are a lot more flexible as Maven, but I think 
it is a good thing that Maven is this strict, as it forces the developers to 
address structural issues instead of using workarounds. Especially in 
Open-Source projects with a lot of different levels of build system expertise I 
usually always had problems with non Maven builds. But this is just a general 
question without having seen the current state … just managing to reach the 
summit of that pile of work you find when coming back from a 1,5 week leave ;-)

Chris 


Am 24.05.17, 15:33 schrieb "Dale LaBossiere" :

Yeah, the new "GitHub as the main repo” config came up in John’s incubator 
talk @ ApacheCon.  Pretty sure OpenWisk was one of the projects using it.  
Don’t recall the other incubator project that was gushing over it.  As you 
might expect there was a LOT of interest in it.

Consider this a personal invitation to help cleanup the build system :-)

— Dale

> On May 23, 2017, at 4:26 PM, Christofer Dutz  
wrote:
> 
> Hi,
> 
> I’m just asking, because the Flex project started with SVN and we 
migrated to ASF GIT. Now we have the GitHub mirrors setup and can process Pull 
requests etc. But it’s not our main repo. Our main repo is the ASF one. 
> 
> Are you sure you can really use the Merge button on pull requests? 
Wouldn’t this require deeper access to Github. We currently do things by 
sending commands to close pull requests etc. in the commit messages. So, if we 
say: “closes #238” the ASF bots close the pull requests for us. If things have 
changed, this might be interesting for the Flex Project too.
> 
> In general, these types of migrations are really easy … just create one 
of the Infra tickets and things happen automatically. I think they even have 
issue templates for things like this.
> 
> And regarding help … The sort of thing I do quite a lot in my paid job 
and here at Apache, is build systems cleanup … my speciality are Maven 
migrations. Reading other emails it looked a little complicated with your 
current Gradle setup … want any help in that … just ask ;-)
> 
> Looking forward do digging deeper into this (
> 
> Chris
> 
> Am 23.05.17, 17:13 schrieb "Dale LaBossiere" :
> 
>Hi Christofer,  
> 
>Sorry I didn’t manage to catch up with you @ ApacheCon following your 
SCADA talk and mention of interest in Edgent.  We’re greatly interested in your 
interest/feedback!
> 
>Re git, Edgent(Quarks) was a GitHub based project prior to 
donation/incubating @ ASF.  The Apache Edgent ASF project was created with SVN 
as the main repo with an ASF git repo mirror and a read-only mirror of that to 
GitHub — apparently standard ASF fare at the time for those wanting to use 
git/github.  We promote using the gitflow model of forking the GitHub mirror 
for committing/creating and reviewing PRs.  Then committers use git to merge 
the PR to the ASF git repo - ugh.
> 
>The proposal is to convert to the newly supported “GitHub as the main 
repo” approach.  Then we’ll be able to fully use the normal gitflow model 
including the Merge button on the PR’s page!  And use of GitHub Issues instead 
of JIRA.
> 
>As I understand it, the repo switch issue is (easily?) handled by 
generating an infra request once the project decides to head in that direction.
> 
>— Dale
> 
> 
>> On May 23, 2017, at 10:29 AM, Christofer Dutz 
 wrote:
>> 
>> Hi guys,
>> 
>> This is the first mail I got after signing up ;-)
>> Fascinating project you have here … guess I will hang around quite some 
time in the (hopefully) near future.
>> 
>> Guess I might be able to help you with this GIT topic. Are you thinking 
of switching to GIT in general from SVN? Or do you want to commit to GitHub and 
hope for this to get synced back to the ASF SVN?
>> 
>> Chris
>> 
>> 
>> 
>> Am 23.05.17, 15:50 schrieb "Samantha Chan" :
>> 
>>   Hi Dale,
>> 
>>   +1!  I would like to use Github as a primary repo.
>>   I think it will make it easier for people to discover, adopt and 
contribute
>>   code back. Using Github issues vs Jira also seems like a good idea.  
We can
>>   have everything at a single point of entry.
>> 
>>   Samantha
>> 
>> 
>> 
>>   On Wed, May 17, 2017 at 2:02 PM, John D. Ament 
>>   wrote:
>> 
 

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi Dale,

I just checked out everything and managed to get things imported in IntelliJ 
after a little struggle … this is a good job you did :-)

Right now, it seems as if there were Ant+Gradle+Eclipse build files in there. 
While it might seem convenient to be able to build with the build system of 
choice, the Flex project currently has big trouble with this as it’s a 
nightmare to keep all the different builds in sync. So, if someone is reporting 
problems we now usually ask “what build system are you using?” before asking 
any other questions or being able to help. So, I would suggest deciding which 
one it’s going to be and to stick to that. 

From a first glance at the build, it does seem structurally quite simple. But 
probably there will be the one or other “rock” with a little “monster” 
underneath, if you look closer (there always is …)

The things you mentioned, that you seem to be missing with Gradle (even if I 
can’t quite understand why):
- Javadoc
- release bundles
- manifests

etc. are handled quite nicely by default Maven setups.

JavaDoc is generated automatically when running a release build together with 
the usual Maven project reports (is even configured in the apache parent POM 
together with rat, deployment etc.)

I didn’t quite understand the “Manifest” thing, but the jar plugin does 
generate this with reasonable defaults, and can be extended to also export the 
dependencies into that (even if I don’t recommend that). 

Finally, the assembly-plugin is perfect for generating the binary (and source) 
release packages.

In the Flex project, I also setup the build to run SonarQube analysis and 
automatically generate the documentation from markdown and/or asciidoctor 
(which I think is very convenient) even automatically update and deploy the 
project website. 

And as I said before, with Gradle you will be able to create an equally good 
build … but you will have to do a lot of the configuration manually and it does 
always offer the cheap way out. Unfortunately, every now and then someone will 
try and use this cheap way out, may the reason be just not knowing better or 
needing to get “this little feature out the door before the next release”. 

It took me about every free minute for about 2 years to untangle and refactor 
the FlexJS Ant build converting it to a clean Maven build. If we had started 
working cleanly from the beginning, this wouldn’t have been a problem. Now the 
build times went down from 8 hours to 20 minutes, we have Jenkins automatically 
build and test all feature branches using the multi-branch pipeline plugin. I’m 
really satisfied with the results … well except the fact that I must re-sync 
the Ant and Maven build all the time ;-)

I could offer to create a fork on GitHub, create a feature branch there and try 
to whip up a set of poms that add Maven as fourth build system to the list … 
you could check it out and play around with it. But I’d only do this, if there 
is any interest in it.

Chris




Am 24.05.17, 17:08 schrieb "Dale LaBossiere" :


On May 24, 2017, at 9:38 AM, Christofer Dutz  
wrote:
...
Regarding the build system … what is the projects general opinion on a 
Maven build? I know that Ant and Gradle are a lot more flexible as Maven, but I 
think it is a good thing that Maven is this strict, as it forces the developers 
to address structural issues instead of using workarounds. Especially in 
Open-Source projects with a lot of different levels of build system expertise I 
usually always had problems with non Maven builds. But this is just a general 
question without having seen the current state


I think we headed down the gradle path because someone stepped up and
contributed an initial set of gradle build files.  And it was simple.  But 
it omitted 
building javadoc, release bundles, the jar manifest class-path that I’ve
mentioned, and maybe some additional stuff (oh, like building java7 based
jars using retrolambda, as well as building for an android tgt).  

I don’t know that the project has an overall position on gradle vs maven.  
Folks?

I don’t have a strong opinion (though I was happy to see the elimination of 
an 
xml based build spec :-)

Once “forced to address structural issues instead of using gradle 
workarounds",
might one then end up with an equivalently simple/clean gradle based system?
Though the path to get to that point may be less clear :-)

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

I did have some free time this weekend, so I thought I’d give the Maven thing a 
try. And right now I haven finished, but I think I’m about 80% done. So far 
there have been one or two little monsters under some rocks … but they weren’t 
too nasty ;-)

Regarding the classpath stuff in the manifest: I really dislike this way of 
setting the classpath as it usually causes a lot of problems when using it. It 
requires libs to be in a predefined directory structure and follow a given 
naming convention. Try to use this in a project built with Maven (just picking 
this as I saw this would be a problem for a Maven built project)

Regarding tests: I did see that there are several Unit tests I would probably 
more call integration-tests. Especially the ones that require loading of a war 
(console.war). So I guess you have both some classical unit-tests as well as 
integration-tests … no need to treat them the same way. Maven ususally has the 
unit-test and the integration-test phase for exactly this.

Regarding jacoco: So I’ll activate that in the maven build too 

Regarding automatic build: For Flex I setup the build to do a multibranch 
pipeline build. So whenever someone checks something in “develop”, “releae/*” 
or “feature/*” it automatically builds that particular branch. However develop 
and “release/*” are the only ones uploading SNAPSHOT versions to the ASF Nexus.

Regarding my commitment: I think Edgent is the tool I was looking for in order 
to make my SCADA project feature-perfect. So I have a strong interest in the 
project maturing. I do have plans for a longer involvement in the project. 
Probably I would be trying to create an open-source “plc” connector (which 
would probably have to be developed outside of Apache due to libraries still 
being GPL). But I do plan on staying around some time ;-)

Regarding the usage: Maven or Gradle based application developers could 
directly utilize SNAPSHOT and release artifacts as they are released. 
Additionally, I would create a so-called assembly that produces the same zip or 
tar-gz archives the users have been using. So there shouldn’t be too much of a 
change.

So now I’ll try to finish the rest of the connector modules maven conversion (

Chris

Am 25.05.17, 19:37 schrieb "Dale LaBossiere" :

Awaiting others to chime in on this “switch to mvn” subject.
A “don’t care” (+/- 0) response is preferred over silence.

In the mean time...

> On May 24, 2017, at 11:41 AM, Christofer Dutz  
wrote:
> ...
> I just checked out everything and managed to get things imported in 
IntelliJ after a little struggle … this is a good job you did :-)
Can’t take all the credit (or blame :-)
> 
> Right now, it seems as if there were Ant+Gradle+Eclipse build files in 
there. While it might seem ... 
gradle is the only CLI way to build edgent.  Any ant build.xml files are 
either (a) residual cruft that should be removed (my bad) or (b) needed to 
leverage the ant machinery for invoking retrolambda in building java7/android 
compatible versions of the jars (machinery that wasn’t converted to pure gradle 
due to time/effort/value at the time).  Attempting to use ant at the top level 
tells you it doesn’t work :-)  As for Eclipse .project/.classpath, yup those 
are live and haven’t been a maintenance issue.

fwiw, I just removed all build.xml except the top level one and those under 
platform and “gradle release” worked fine so I’ll create a PR to clean them up 
as a first step.

> ...JavaDoc is generated automatically when running a release build 
together with the usual Maven project reports (is even configured in the apache 
parent POM together with rat, deployment etc.)
Javadoc was complicated by creating groupings as well as excluding it for 
non-API classes.

> ...I didn’t quite understand the “Manifest” thing, but the jar plugin 
does generate this with reasonable defaults, and can be extended to also export 
the dependencies into that (even if I don’t recommend that). 
An edgent jar’s manifest class-path includes references to its immediate 
dependent edgent jars (not transitive) as well as references to its external 
jar dependencies (transitively).  I agree that "compiled in" references to 
specific versions of external dependencies may not be a great idea / is perhaps 
best eliminated.

My recollection is that by default gradle did not generate any manifest 
class-path.

e.g., just to make this a bit more concrete, this is from 
edgent.connectors.kafka.jar/MANIFEST.MF (see connectors/kafka/build.gradle for 
more info)

Class-Path: ../../../lib/edgent.api.topology.jar ../../../ext/gson-2.2
 .4.jar ../../../ext/slf4j-api-1.7.12.jar ../../../ext/metrics-core-3.
 1.2.jar ../ext/kafka_2.10-0.8.2.2.jar ../ext/kafka-clients-0.8.2.2.ja
 r ../ext/log4j-1.2.16.jar ../ext/metrics-core-2.2.0.jar ../ext/scala-
 library-

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi guys,

Strange people these Germans … if it rains, they complain about it and how much 
they would like to go swimming in the sun … now that it’s 32° and the sun is 
shining, no one wants to go swimming, cause it’s too fu***g hot ;-) … well this 
way I had some time to finish the maven migration … I just pushed the changes 
to the “feature/maven” branch of my fork on github: 
https://github.com/chrisdutz/incubator-edgent/tree/feature/maven

Here a short list on what little monsters I found siting under some rocks:

1) Usage of test-jars --> Bad practice → Move code to separate test-util jar 
module
2) AppServiceTest in provides/direct relies on system-properties to load jar 
produced by another module → Ideally this should be refactored to find the jar 
on the classpath
3) HttpServerTest in console/server relies on console.war
4) console/servlets module compiled to something with a name completely 
unrelated to the project
5) test/svt requires samples project → Some modules rely on samples. While I 
think, this is ok for Samples themselves, nothing outside the samples should 
rely on samples (my opinion)
6) Almost all tests relying on successful SSL handshakes in 
wsclient-javax.websocket are failing (I’ll investigate this as soon as I have 
time, I could imagine this to be related to non US JVM encryption issues we are 
also seeing in the Flex project)
7) Some unit-tests should more be integration-tests

I didn’t migrate the stuff in “platform” as I didn’t quite understand what they 
are used for and what they should do.

Hope you like the changes. With a build like this it should be easy to setup 
the CI to auto distribute SNAPSHOTs and execute releases on ASF machines with a 
simple one or two-liner.

Chris


Am 27.05.17, 19:16 schrieb "Christofer Dutz" :

I did have some free time this weekend, so I thought I’d give the Maven 
thing a try. And right now I haven finished, but I think I’m about 80% done. So 
far there have been one or two little monsters under some rocks … but they 
weren’t too nasty ;-)

Regarding the classpath stuff in the manifest: I really dislike this way of 
setting the classpath as it usually causes a lot of problems when using it. It 
requires libs to be in a predefined directory structure and follow a given 
naming convention. Try to use this in a project built with Maven (just picking 
this as I saw this would be a problem for a Maven built project)

Regarding tests: I did see that there are several Unit tests I would 
probably more call integration-tests. Especially the ones that require loading 
of a war (console.war). So I guess you have both some classical unit-tests as 
well as integration-tests … no need to treat them the same way. Maven ususally 
has the unit-test and the integration-test phase for exactly this.

Regarding jacoco: So I’ll activate that in the maven build too 

Regarding automatic build: For Flex I setup the build to do a multibranch 
pipeline build. So whenever someone checks something in “develop”, “releae/*” 
or “feature/*” it automatically builds that particular branch. However develop 
and “release/*” are the only ones uploading SNAPSHOT versions to the ASF Nexus.

Regarding my commitment: I think Edgent is the tool I was looking for in 
order to make my SCADA project feature-perfect. So I have a strong interest in 
the project maturing. I do have plans for a longer involvement in the project. 
Probably I would be trying to create an open-source “plc” connector (which 
would probably have to be developed outside of Apache due to libraries still 
being GPL). But I do plan on staying around some time ;-)

Regarding the usage: Maven or Gradle based application developers could 
directly utilize SNAPSHOT and release artifacts as they are released. 
Additionally, I would create a so-called assembly that produces the same zip or 
tar-gz archives the users have been using. So there shouldn’t be too much of a 
change.

So now I’ll try to finish the rest of the connector modules maven 
conversion (

Chris

Am 25.05.17, 19:37 schrieb "Dale LaBossiere" :

Awaiting others to chime in on this “switch to mvn” subject.
A “don’t care” (+/- 0) response is preferred over silence.

In the mean time...

> On May 24, 2017, at 11:41 AM, Christofer Dutz 
 wrote:
> ...
> I just checked out everything and managed to get things imported in 
IntelliJ after a little struggle … this is a good job you did :-)
Can’t take all the credit (or blame :-)
> 
> Right now, it seems as if there were Ant+Gradle+Eclipse build files 
in there. While it might seem ... 
gradle is the only CLI way to build edgent.  Any ant build.xml files 
are either (a) residual cruft that should be removed (my bad) or (b) needed to 
leverage the ant machinery for invoking retrolambda in building

Re: Understanding the snapshot and release process

[Christofer Dutz]

The Maven build should work on the ASF Jenkins and hereby publish the 
snapshots. I think that in the flex project I added this to ensure its added to 
the generated "site".

Which one is the hosted Jenkins you are referring to? The asf Jenkins?

And what should be the problem with Travis? Why are you using two ci servers?

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: "John D. Ament" 
Datum: 28.05.17 16:20 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Understanding the snapshot and release process

The maven based build seems to be much more maintainable.  I'm glad you
guys came together to solve this problem :-)

RE distribution of snapshots.  We only presently support publishing via our
hosted jenkins, however Edgent currently builds in Travis.  So that would
need to be changed, or some other solution.

You also can drop the repositories/pluginRepositories sections of the new
pom.  Its not needed.


John

On Sun, May 28, 2017 at 10:03 AM Christofer Dutz 
wrote:

> Hi guys,
>
> Strange people these Germans … if it rains, they complain about it and how
> much they would like to go swimming in the sun … now that it’s 32° and the
> sun is shining, no one wants to go swimming, cause it’s too fu***g hot ;-)
> … well this way I had some time to finish the maven migration … I just
> pushed the changes to the “feature/maven” branch of my fork on github:
> https://github.com/chrisdutz/incubator-edgent/tree/feature/maven
>
> Here a short list on what little monsters I found siting under some rocks:
>
> 1) Usage of test-jars --> Bad practice → Move code to separate test-util
> jar module
> 2) AppServiceTest in provides/direct relies on system-properties to load
> jar produced by another module → Ideally this should be refactored to find
> the jar on the classpath
> 3) HttpServerTest in console/server relies on console.war
> 4) console/servlets module compiled to something with a name completely
> unrelated to the project
> 5) test/svt requires samples project → Some modules rely on samples. While
> I think, this is ok for Samples themselves, nothing outside the samples
> should rely on samples (my opinion)
> 6) Almost all tests relying on successful SSL handshakes in
> wsclient-javax.websocket are failing (I’ll investigate this as soon as I
> have time, I could imagine this to be related to non US JVM encryption
> issues we are also seeing in the Flex project)
> 7) Some unit-tests should more be integration-tests
>
> I didn’t migrate the stuff in “platform” as I didn’t quite understand what
> they are used for and what they should do.
>
> Hope you like the changes. With a build like this it should be easy to
> setup the CI to auto distribute SNAPSHOTs and execute releases on ASF
> machines with a simple one or two-liner.
>
> Chris
>
>
> Am 27.05.17, 19:16 schrieb "Christofer Dutz" :
>
> I did have some free time this weekend, so I thought I’d give the
> Maven thing a try. And right now I haven finished, but I think I’m about
> 80% done. So far there have been one or two little monsters under some
> rocks … but they weren’t too nasty ;-)
>
> Regarding the classpath stuff in the manifest: I really dislike this
> way of setting the classpath as it usually causes a lot of problems when
> using it. It requires libs to be in a predefined directory structure and
> follow a given naming convention. Try to use this in a project built with
> Maven (just picking this as I saw this would be a problem for a Maven built
> project)
>
> Regarding tests: I did see that there are several Unit tests I would
> probably more call integration-tests. Especially the ones that require
> loading of a war (console.war). So I guess you have both some classical
> unit-tests as well as integration-tests … no need to treat them the same
> way. Maven ususally has the unit-test and the integration-test phase for
> exactly this.
>
> Regarding jacoco: So I’ll activate that in the maven build too
>
> Regarding automatic build: For Flex I setup the build to do a
> multibranch pipeline build. So whenever someone checks something in
> “develop”, “releae/*” or “feature/*” it automatically builds that
> particular branch. However develop and “release/*” are the only ones
> uploading SNAPSHOT versions to the ASF Nexus.
>
> Regarding my commitment: I think Edgent is the tool I was looking for
> in order to make my SCADA project feature-perfect. So I have a strong
> interest in the project maturing. I do have plans for a longer involvement
> in the project. Probably I would be trying to create an open-source “plc”
> connector (which would probably have to be developed outside of Apache due
> to libraries still bein

Re: Understanding the snapshot and release process

[Christofer Dutz]

Well the big advantage of running the jobs on ASF Jenkins is that you can 
auto-deploy to Nexus (ASF Maven Repo) as well as commit to Git without having 
to store some ones credentials on an external machine. Infra is very helpful in 
getting even complicated setups running (I manage the Windows based flex 
builds, which need Flashplayer, Browsers and Selenium). I would be more than 
happy to help setup everything.

Chris

Am 28.05.17, 17:26 schrieb "John D. Ament" :

On Sun, May 28, 2017 at 11:12 AM Christofer Dutz 
wrote:

> The Maven build should work on the ASF Jenkins and hereby publish the
> snapshots. I think that in the flex project I added this to ensure its
> added to the generated "site".
>
> Which one is the hosted Jenkins you are referring to? The asf Jenkins?
>
>
Yes, the ASF jenkins.


> And what should be the problem with Travis? Why are you using two ci
> servers?
>

AFAIK, Edgent only uses travis, and doesn't currently run jobs on ASF
Jenkins.  At least if they do, I was unable to find a job with a name that
was easy to identify.


> Chris
>
>
>
> Von meinem Samsung Galaxy Smartphone gesendet.
>
>
>  Ursprüngliche Nachricht 
> Von: "John D. Ament" 
> Datum: 28.05.17 16:20 (GMT+01:00)
> An: dev@edgent.apache.org
> Betreff: Re: Understanding the snapshot and release process
>
> The maven based build seems to be much more maintainable.  I'm glad you
> guys came together to solve this problem :-)
>
> RE distribution of snapshots.  We only presently support publishing via 
our
> hosted jenkins, however Edgent currently builds in Travis.  So that would
> need to be changed, or some other solution.
>
> You also can drop the repositories/pluginRepositories sections of the new
> pom.  Its not needed.
>
>
> John
>
> On Sun, May 28, 2017 at 10:03 AM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Hi guys,
> >
> > Strange people these Germans … if it rains, they complain about it and
> how
> > much they would like to go swimming in the sun … now that it’s 32° and
> the
> > sun is shining, no one wants to go swimming, cause it’s too fu***g hot
> ;-)
> > … well this way I had some time to finish the maven migration … I just
> > pushed the changes to the “feature/maven” branch of my fork on github:
> > https://github.com/chrisdutz/incubator-edgent/tree/feature/maven
> >
> > Here a short list on what little monsters I found siting under some
> rocks:
> >
> > 1) Usage of test-jars --> Bad practice → Move code to separate test-util
> > jar module
> > 2) AppServiceTest in provides/direct relies on system-properties to load
> > jar produced by another module → Ideally this should be refactored to
> find
> > the jar on the classpath
> > 3) HttpServerTest in console/server relies on console.war
> > 4) console/servlets module compiled to something with a name completely
> > unrelated to the project
> > 5) test/svt requires samples project → Some modules rely on samples.
> While
> > I think, this is ok for Samples themselves, nothing outside the samples
> > should rely on samples (my opinion)
> > 6) Almost all tests relying on successful SSL handshakes in
> > wsclient-javax.websocket are failing (I’ll investigate this as soon as I
> > have time, I could imagine this to be related to non US JVM encryption
> > issues we are also seeing in the Flex project)
> > 7) Some unit-tests should more be integration-tests
> >
> > I didn’t migrate the stuff in “platform” as I didn’t quite understand
> what
> > they are used for and what they should do.
> >
> > Hope you like the changes. With a build like this it should be easy to
> > setup the CI to auto distribute SNAPSHOTs and execute releases on ASF
> > machines with a simple one or two-liner.
> >
> > Chris
> >
> >
> > Am 27.05.17, 19:16 schrieb "Christofer Dutz"  >:
> >
> > I did have some free time this weekend, so I thought I’d give the
> > Maven thing a try. And right now I haven finished, but I think I’m about
> > 80% done. So far there have been one or two little monsters under some
> > rocks … but they weren’t too nasty ;-)
> >
> > Regarding t

Changing the way the the servlets are loaded.

[Christofer Dutz]

Hi guys,

While finishing the Maven migration of the last missing tests and finding a way 
to get rid of the one or the other monster while at it, I noticed the two 
modules console/server and console/servlets.
It seems as if “servlets” contain the real servlets for the server module and 
the reason the modules are split up, are to be able to create a war containing 
the servlets and to load that in an embedded jetty in the “server” module. I 
think we could simplify things greatly here.

In the Flex project the submodule “BlazeDS” is a server component based on a 
Servlet. Here initially the test-setup was done similarly and we were having a 
similarly complex test setup. I then refactored the tests to directly 
instantiate a Jetty servlet container and to programmatically register the 
servlets. The cool thing with this, is the “server” module could contain the 
“servlets” content and there would be no need to find the Servlets by loading 
some external jars.

Have a look at my “TestServer” implementation to get a feeling what I’m talking 
about:
https://github.com/apache/flex-blazeds/blob/42ecd811bec2c7825da250c73b9d3c463f990320/remoting/src/test/java/flex/messaging/util/TestServer.java

Chris

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi,

So, I just create a PR as I think I am currently finished with everything I 
wanted to fix. 

Edgent should now be buildable with Maven command. At least I managed to get 
the travis build working with maven ☺

Here some structural changes I did. I refactored the project and extracted the 
TestApplications.java together with its service descriptor into a dedicated 
module located in the test submodule. Wherever a jar was referenced for 
dynamically loading, now I don’t reference the jar where it’s created during 
the build, I use the maven-dependency plugin to copy it to a location local to 
the current module and simply load it from there. This way I didn’t have to 
deal with any too complicated classpath lookups. I could not only run the tests 
as part of the Maven build, but also directly inside my IDE (IntellIJ).

Now some helpful maven commands:

Simply build the project running all unit tests:

mvn clean install

Deploy SNAPSHOTS in Apaches Maven repository:

mvn clean deploy

Additionally, build and deploy source and javadoc artifacts (all signed and 
require correct pgp setup on the client side):

mvn clean deploy -Papache-release

Build without running tests:

mvn clean install -DskipTests

The site generation still needs some love as it would currently run all test 
twice.

Please have a look at this. 

Chris





Am 31.05.17, 18:41 schrieb "Dale LaBossiere" :

Christofer, can you create a [WIP] PR for this work?  Then I’ll have a 
context from which I can review / ask stupid nubie questions :-)

Also, what’s the executive summary for running maven to build a release (or 
at least all of the jars) using this work?  And to run the tests?  I’d like to 
pull the changes into my workspace and really see what’s generated, etc

Thanks!
— Dale

> On May 28, 2017, at 10:03 AM, Christofer Dutz  
wrote:
> 
> had some time to finish the maven migration … I just pushed the changes 
to the “feature/maven” branch of my fork on github: 
https://github.com/chrisdutz/incubator-edgent/tree/feature/maven 
<https://github.com/chrisdutz/incubator-edgent/tree/feature/maven>

Re: Changing the way the the servlets are loaded.

[Christofer Dutz]

Hi,

Ok … what you write regarding the platform independence makes absolute sense. 
The original code was actualy using the same Jetty Server classes as I was in 
BlazeDS, so an initial attempt was to build servlets as normal jar archive, to 
depend on that from the server and to manually register the servlets. That 
worked nicely. Unfortunately, I see that servlets should be a normal “war” web 
application to be usable for others. So I made it a war again and simply 
changed the way the jars are discovered. Instead of relying on a relative path 
to another module, I made sure servlets is built before server and simply have 
the maven-dependency-plugin copy that into the target directory under a name I 
have control over. This simplified the code for finding and accessing the war 
archive greatly.

Another thing I did was to extract one TestApplications class from the testcode 
of one module and made a new module in “tests” containing that file. Now I 
could access the jar file of that in other applications testsuites. 

From my point of view the build should now be structurally untangled. I created 
a pull request, so please feel free to investigate my changes: 
https://github.com/apache/incubator-edgent/pull/309

Travis is no longer complaining and seems to be happy with it ;-)

Chris



Am 31.05.17, 20:53 schrieb "Susan Cline" :

Hi, Chris,

I thought I would answer based on what I recall to be the original purpose 
of splitting them into a ‘server’ and ‘servlet’ directory.

It was written in a way to allow other servlet containers to run the 
‘servlet’ code without making them dependent on each other.
With that said, looking at your TestServer although it uses a jetty server, 
could it be rewritten to use other servlet containers?

If so, then I think what you propose seems okay to me.  I’m not sure if 
some of the examples would have to be rewritten though?

Cheers,

Susan

> On May 31, 2017, at 12:51 AM, Christofer Dutz  
wrote:
> 
> Hi guys,
> 
> While finishing the Maven migration of the last missing tests and finding 
a way to get rid of the one or the other monster while at it, I noticed the two 
modules console/server and console/servlets.
> It seems as if “servlets” contain the real servlets for the server module 
and the reason the modules are split up, are to be able to create a war 
containing the servlets and to load that in an embedded jetty in the “server” 
module. I think we could simplify things greatly here.
> 
> In the Flex project the submodule “BlazeDS” is a server component based 
on a Servlet. Here initially the test-setup was done similarly and we were 
having a similarly complex test setup. I then refactored the tests to directly 
instantiate a Jetty servlet container and to programmatically register the 
servlets. The cool thing with this, is the “server” module could contain the 
“servlets” content and there would be no need to find the Servlets by loading 
some external jars.
> 
> Have a look at my “TestServer” implementation to get a feeling what I’m 
talking about:
> 
https://github.com/apache/flex-blazeds/blob/42ecd811bec2c7825da250c73b9d3c463f990320/remoting/src/test/java/flex/messaging/util/TestServer.java
> 
> Chris

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi,

So, I just added some changes to get the site generation working. To have a 
look, just run

mvn clean install site:site site:stage

After the build just go to the target/staging directory and open the index.html 
in any browser. There you can see the generated site for the project.

Chris


Am 02.06.17, 20:59 schrieb "Dale LaBossiere" :

Thanks, I’ve pulled it into my workspace and started to explore.
— Dale

> On Jun 2, 2017, at 11:26 AM, Christofer Dutz  
wrote:
> 
> Hi,
> 
> So, I just create a PR as I think I am currently finished with everything 
I wanted to fix. 
> 
> Edgent should now be buildable with Maven command. At least I managed to 
get the travis build working with maven ☺
> 
> Here some structural changes I did. I refactored the project and 
extracted the TestApplications.java together with its service descriptor into a 
dedicated module located in the test submodule. Wherever a jar was referenced 
for dynamically loading, now I don’t reference the jar where it’s created 
during the build, I use the maven-dependency plugin to copy it to a location 
local to the current module and simply load it from there. This way I didn’t 
have to deal with any too complicated classpath lookups. I could not only run 
the tests as part of the Maven build, but also directly inside my IDE 
(IntellIJ).
> 
> Now some helpful maven commands:
> 
> Simply build the project running all unit tests:
> 
>mvn clean install
> 
> Deploy SNAPSHOTS in Apaches Maven repository:
> 
>mvn clean deploy
> 
> Additionally, build and deploy source and javadoc artifacts (all signed 
and require correct pgp setup on the client side):
> 
>mvn clean deploy -Papache-release
> 
> Build without running tests:
> 
>mvn clean install -DskipTests
> 
> The site generation still needs some love as it would currently run all 
test twice.
> 
> Please have a look at this. 
> 
> Chris
> 
> 
> 
> 
> 
> Am 31.05.17, 18:41 schrieb "Dale LaBossiere" :
> 
>Christofer, can you create a [WIP] PR for this work?  Then I’ll have a 
context from which I can review / ask stupid nubie questions :-)
> 
>Also, what’s the executive summary for running maven to build a 
release (or at least all of the jars) using this work?  And to run the tests?  
I’d like to pull the changes into my workspace and really see what’s generated, 
etc
> 
>Thanks!
>— Dale
> 
>> On May 28, 2017, at 10:03 AM, Christofer Dutz 
 wrote:
>> 
>> had some time to finish the maven migration … I just pushed the changes 
to the “feature/maven” branch of my fork on github: 
https://github.com/chrisdutz/incubator-edgent/tree/feature/maven 
<https://github.com/chrisdutz/incubator-edgent/tree/feature/maven>
> 
> 
> 

Re: Maven wiki page [was Re: Understanding the snapshot and release process]

[Christofer Dutz]

I’d love to, but you would need to give me the charma to do so (

Chris

Am 05.06.17, 20:14 schrieb "Dale LaBossiere" :

I created a wiki page [1] to help us work through / collaborate on this.
IMO there are too many questions, etc to effectively deal with it just on 
the dev@ list

Chris, maybe you could fill in some of the (red) blanks on that page while 
continue my maven education.

[1] https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 


— Dale

Re: Maven wiki page [was Re: Understanding the snapshot and release process]

[Christofer Dutz]

Hi Queeniw,

thanks … I added most of the things in the upper section. 

For the questions in the lower one, I would like to ask you to watch the little 
maven introduction video I did. It should answer a lot of questions. Questions 
that my answers would probably sound confusing without this minimal Maven 
background.

Chris

Am 05.06.17, 20:39 schrieb "Queenie Ma" :

Hi Chris, I just granted you permissions to make changes to the wiki.

—
Queenie

> On Monday, Jun 05, 2017 at 11:37 AM, Christofer Dutz 
mailto:christofer.d...@c-ware.de)> wrote:
> I’d love to, but you would need to give me the charma to do so (
>
> Chris
>
> Am 05.06.17, 20:14 schrieb "Dale LaBossiere" :
>
> I created a wiki page [1] to help us work through / collaborate on this.
> IMO there are too many questions, etc to effectively deal with it just on 
the dev@ list
>
> Chris, maybe you could fill in some of the (red) blanks on that page 
while continue my maven education.
>
> [1] https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>
>
> — Dale
>
>
>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi,

I’ll answer this one quickly as I can do that without digging in too deep.

I wouldn’t add the apache-edgent to the artifact names as the groupId already 
qualifies them as apache edgent modules.
However when creating a release zip/tar.gz with the sources or binary 
distribution I would add the “apache-edgent-” prefix.

Chris


Am 06.06.17, 23:02 schrieb "Dale LaBossiere" :

Currently the generated JAR/WAR lack an “edgent” or “apache-edgent” prefix, 
e.g., "api-topology-1.2.0-SNAPSHOT.jar".  How important/required is it to add 
that before we start publishing these to mvn repos?

What’s the current ASF best-practice in this regard?  e.g., I some 
(limited) # of “apache-XYZZY-…” in maven central.  Presumably there are many 
many others that omit the “apache-" - e.g., “beam-sdks-…”

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Well looking through the Apache repo there is no 100% standard, but most 
artifacts are indeed prefixed with the project name but not apache … I’ll add 
the edgent to each of the artifacts.

Chris

Am 07.06.17, 13:47 schrieb "John D. Ament" :

Usually the maven norm is something like
com.mycompany.myproduct:myproduct-some-module.

So I would expect to see groupIds of org.apache.edgent, and artifact id's
of edgent-some-module.  However, I've noticed that both Eclipse and IBM
(obviously not related in any way) seem to use completely different
artifact formats.  I would recommend being an Apache project you adopt the
Apache Maven standard format.

John

On Wed, Jun 7, 2017 at 7:44 AM Christofer Dutz 
wrote:

> Hi,
>
> I’ll answer this one quickly as I can do that without digging in too deep.
>
> I wouldn’t add the apache-edgent to the artifact names as the groupId
> already qualifies them as apache edgent modules.
> However when creating a release zip/tar.gz with the sources or binary
> distribution I would add the “apache-edgent-” prefix.
>
> Chris
>
>
> Am 06.06.17, 23:02 schrieb "Dale LaBossiere" :
>
> Currently the generated JAR/WAR lack an “edgent” or “apache-edgent”
> prefix, e.g., "api-topology-1.2.0-SNAPSHOT.jar".  How important/required 
is
> it to add that before we start publishing these to mvn repos?
>
> What’s the current ASF best-practice in this regard?  e.g., I some
> (limited) # of “apache-XYZZY-…” in maven central.  Presumably there are
> many many others that omit the “apache-" - e.g., “beam-sdks-…”
>
> — Dale
>
>
>
>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Checking our repo:
https://repository.apache.org/#view-repositories;releases~browsestorage

I can see that fewer than about 10% of all apache projects do that … But I 
wouldn’t mind adding that, if the others agree.

Chris



Am 07.06.17, 14:54 schrieb "Justin Mclean" :

Hi,

While not a requirement prefixing with apache may add some legal protection 
and is good from a branding point of of view.

Thanks,
Justin

Re: Understanding the snapshot and release process

[Christofer Dutz]

No worries,

As long as you didn't do a "deploy" nothing should be uploaded :-)

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 07.06.17 19:35 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Understanding the snapshot and release process

install -Papache-release is generating the (arc) signature file :-)

Chris, what needs to be done to get it to also generate the .md5 and .sha files?

Also, did I now just populate the ASF snapshot repo? :-)

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi,

Right now I didn't configure anything regarding the assembly of the source 
artifact. It's apaches default config. I'll look into a more fine-tuned version.

And regarding the artifact name ... Justin suggested that change, but 90% of 
the other projects don't do it that way. I was waiting for your opinions before 
doing that. It does make the dependencies a lot more verbose. After all you 
can't use the artifacts without the groupId "org.apache.effect" which includes 
"apache" and "edgent".

But sure, I'll change that if you all want me to.

Chris

Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 07.06.17 18:45 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Understanding the snapshot and release process

OK, so mvn install -Papache-release generates the source release bundle zip.

I poked around for a while but couldn’t come up with the incantation
needed to start excluding things from the zip.  e.g., the zip includes
all of the residual gradle generated **/build/

Chris, can you enhance the configs so as to exclude the various
things that the gradle build was configured to exclude?
See top level build.gradle/srcReleaseTarGz task.
Also exclude src/site (that present in the zip too), right?

The generated zip’s name is:
  target/edgent-parent-1.2.0-SNAPSHOT-source-release.zip

“edgent-parent” ?  today we, I thought correctly, have “apache-edgent” there.
Also, the required “incubating” is missing in the release bundle name.

Thanks,
— Dale

> On Jun 7, 2017, at 10:35 AM, Christofer Dutz  
> wrote:
>
> Checking our repo:
> https://repository.apache.org/#view-repositories;releases~browsestorage
>
> I can see that fewer than about 10% of all apache projects do that … But I 
> wouldn’t mind adding that, if the others agree.
>
> Chris
>
>
>
> Am 07.06.17, 14:54 schrieb "Justin Mclean" :
>
>Hi,
>
>While not a requirement prefixing with apache may add some legal 
> protection and is good from a branding point of of view.
>
>Thanks,
>Justin
>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Regarding java8 and java7 artifacts …

This could be problematic, if we wanted to publish both.

Having a binary tar.gz with a java8 and one with java7 doesn’t seem to be a 
problem, but having two artifacts deployed to Maven-Central is no trivial task.
Exactly this type of jars for different java versions is one of the things the 
Maven guys are currently working on as part of the Java 9 – Jigsaw support. But 
it’s a nightmare. We could start creating java8 jars as default and add a 
second “java7” jar using the “java7” classifier, but then a lot of the 
mechanisms in Maven no longer work. 

I could offer to generally build java7 versions and to test them with java7 and 
java8 … I think this topic should be discussed a little more before I start 
implementing this.

Right now, I would opt for distributing java8 binaries to Maven Central and to 
offer downloadable java7 binary distributions.

Chris



Am 07.06.17, 21:06 schrieb "Christofer Dutz" :

Hi,

Right now I didn't configure anything regarding the assembly of the source 
artifact. It's apaches default config. I'll look into a more fine-tuned version.

And regarding the artifact name ... Justin suggested that change, but 90% 
of the other projects don't do it that way. I was waiting for your opinions 
before doing that. It does make the dependencies a lot more verbose. After all 
you can't use the artifacts without the groupId "org.apache.effect" which 
includes "apache" and "edgent".

But sure, I'll change that if you all want me to.

Chris

Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 07.06.17 18:45 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Understanding the snapshot and release process

OK, so mvn install -Papache-release generates the source release bundle zip.

I poked around for a while but couldn’t come up with the incantation
needed to start excluding things from the zip.  e.g., the zip includes
all of the residual gradle generated **/build/

Chris, can you enhance the configs so as to exclude the various
things that the gradle build was configured to exclude?
See top level build.gradle/srcReleaseTarGz task.
Also exclude src/site (that present in the zip too), right?

The generated zip’s name is:
  target/edgent-parent-1.2.0-SNAPSHOT-source-release.zip

“edgent-parent” ?  today we, I thought correctly, have “apache-edgent” 
there.
Also, the required “incubating” is missing in the release bundle name.

    Thanks,
— Dale

> On Jun 7, 2017, at 10:35 AM, Christofer Dutz  
wrote:
>
> Checking our repo:
> https://repository.apache.org/#view-repositories;releases~browsestorage
>
> I can see that fewer than about 10% of all apache projects do that … But 
I wouldn’t mind adding that, if the others agree.
>
> Chris
>
>
>
> Am 07.06.17, 14:54 schrieb "Justin Mclean" :
>
>Hi,
>
>While not a requirement prefixing with apache may add some legal 
protection and is good from a branding point of of view.
>
>Thanks,
>Justin
>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi Dale,

Today I had some progress on the java7 side.
I successfully added the retrolambda-maven-plugin in a dedicated java7 profile 
to the main pom. Now the plugin automatically kicks in and converts the classes 
after compiling them. I also added the animal-sniffer-maven-plugin to 
explicitly check against the signatures of 1.7 or 1.8 JDKs. This all seems to 
be working. Unfortunately, the backporting of default methods in interfaces 
seems to be a problem. The default methods in DirectTestSetup can’t seem to be 
backported if the interface and the implementing class are not in the same 
retrolambda execution. Eventually refactoring the tests to work without default 
methods would be a good idea.

The problem with classifiers is that maven automatically resolves artifacts. 
Depending on the scope different artifacts are pulled in. In general, we would 
be using the “compile” scope. So, if I add a dependency with classifier “java7” 
then it would pull in that jar in the java7 version, but the transitive 
dependencies would probably be the java8 ones, so you would have to explicitly 
define all the transitive dependencies with the classifier. We don’t want that 
;-)

I also contacted the maven guys on their mailing list. They did have the one or 
the other solution. In general, they would all come down to adding the type to 
the version. So, we would start producing “1.2.0.java7” and “1.2.0.java8” or 
“1.2.0.7” and “1.2.0.8” versions (I would prefer the first as it doesn’t 
suggest the java 8 version being slightly newer than the 7 version). Another 
alternative would be to prefix the version with the target version. “7.1.2.0” 
and “8.1.2.0” … but I don’t really like that option. In this case, I would have 
to refactor the build to use variables in all modules except the parent poms 
(you can’t use variables for parent poms). Then travis/Jenkins would run two 
build:

mvn clean deploy
mvn clean deploy –Pjava7

In the end, it would result in both versions being built, tested and deployed, 
but in the end the java7 version would remain in the workspace as the second 
executions “clean” removed any reference to the java8 version.

Hope that explains the dilemma a little more. At least with these 
version-suffixes we would have a fully working solution. I’ll get into changing 
this tomorrow.

By the way … anyone got an idea how I can make Travis not die on out of memory? 
Tried several things, but it didn’t seem to do the job.

Chris




Am 08.06.17, 16:13 schrieb "Dale LaBossiere" :


> On Jun 8, 2017, at 4:19 AM, Christofer Dutz  
wrote:
> 
> Regarding java8 and java7 artifacts …

Coincidentally, I started investigating this space yesterday and 
retrolambda-maven-plugin
and was just composing mail on the question of publishing java8 and java7 
to mvn repos :-)

[ fwiw, so far I had defined a Profile (e.g., mvn install -Ptgt=java7) and
changed the “target” build directory to “target/${edgent.target.kind}”
(e.g., target/{java8,java7}).  Things were working fine up to that point:
generated class files and module jars. ]

> This could be problematic, if we wanted to publish both.
> ...
> I could offer to generally build java7 versions and to test them with 
java7 and java8 … I think this topic should be discussed a little more before I 
start implementing this.
> 
> Right now, I would opt for distributing java8 binaries to Maven Central 
and to offer downloadable java7 binary distributions.

Ugh.  Publishing java8 in a mvn repo but java7/android as a binary tgz 
seems really bad.
It creates two different ways (and needed doc) for users building an Edgent 
app
and installing Edgent jars onto a “device”.

Could you elaborate on the problems using the maven coordinate “classifier”?
The maven doc [1] has this exactly as use case for it (they use “jdk15” vs 
“jdk14”).
At a high (ignorant) level I don’t have a problem with the classifier being 
nil for java8 :-)

If using a coordinate classifier is a no-go, what about (grimace) 
incorporating it into the
groupId:  org.apache.edgent, org.apache.edgent.java7  ?

A bit related, what coordinate component is “incubating” supposed to be 
incorporated into?

— Dale

[1] https://maven.apache.org/pom.html <https://maven.apache.org/pom.html>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Eventually your approach with a sub-directory inside the target would be a 
valid option … would probably have to adjust the maven-clean-plugin too, but 
that should work in getting all results inside the target directory in parallel.

Chris

Am 08.06.17, 16:13 schrieb "Dale LaBossiere" :


> On Jun 8, 2017, at 4:19 AM, Christofer Dutz  
wrote:
> 
> Regarding java8 and java7 artifacts …

Coincidentally, I started investigating this space yesterday and 
retrolambda-maven-plugin
and was just composing mail on the question of publishing java8 and java7 
to mvn repos :-)

[ fwiw, so far I had defined a Profile (e.g., mvn install -Ptgt=java7) and
changed the “target” build directory to “target/${edgent.target.kind}”
(e.g., target/{java8,java7}).  Things were working fine up to that point:
generated class files and module jars. ]

> This could be problematic, if we wanted to publish both.
> ...
> I could offer to generally build java7 versions and to test them with 
java7 and java8 … I think this topic should be discussed a little more before I 
start implementing this.
> 
> Right now, I would opt for distributing java8 binaries to Maven Central 
and to offer downloadable java7 binary distributions.

Ugh.  Publishing java8 in a mvn repo but java7/android as a binary tgz 
seems really bad.
It creates two different ways (and needed doc) for users building an Edgent 
app
and installing Edgent jars onto a “device”.

Could you elaborate on the problems using the maven coordinate “classifier”?
The maven doc [1] has this exactly as use case for it (they use “jdk15” vs 
“jdk14”).
At a high (ignorant) level I don’t have a problem with the classifier being 
nil for java8 :-)

If using a coordinate classifier is a no-go, what about (grimace) 
incorporating it into the
groupId:  org.apache.edgent, org.apache.edgent.java7  ?

A bit related, what coordinate component is “incubating” supposed to be 
incorporated into?

— Dale

[1] https://maven.apache.org/pom.html <https://maven.apache.org/pom.html>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi Dale,

Thanks for that … I had noticed the merged pull request … today I pulled in the 
upstream changes and tweaked the build a little. 
Now both the java8 and java7 builds seem to be working. I also made the build 
compile to different target sub-directories so now you can inspect the output 
from both runs without having them interfering. 

Chris


Am 09.06.17, 22:29 schrieb "Dale LaBossiere" :

Hi Chris,  in case you hadn’t noticed the following merged PR (on master) 
eliminates the
use of “default”.  So you should be able to rebase your branch your branch 
and be all set.

https://github.com/apache/incubator-edgent/pull/310 
<https://github.com/apache/incubator-edgent/pull/310>

> On Jun 8, 2017, at 10:40 AM, Christofer Dutz  
wrote:
> ...problem. The default methods in DirectTestSetup can’t seem to be 
backported if the interface and the implementing class are not in the same 
retrolambda execution. Eventually refactoring the tests to work without default 
methods would be a good idea.

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Ok … so I’m working on the open issues … I do have one question though … 
Compiling with one java version and then running a different one with the tests 
will be challenging, if not impossible. 
What I did a while ago was to introduce the animal-sniffer plugin. This very 
strictly checks the resulting class files against the signatures for a given 
JDK to ensure the classes would work in that particular JDK. Would this be 
enough for you? It’s what we settled for in the Flex project. In the past we 
did have to re-release stuff because even if we compiled to output 1.7 code, 
the code wasn’t runnable on 1.7. Since introducing the animal-sniffer plugin we 
never had problems like this again. 

Chris


Am 10.06.17, 22:45 schrieb "Christofer Dutz" :

Hi Dale,

Thanks for that … I had noticed the merged pull request … today I pulled in 
the upstream changes and tweaked the build a little. 
Now both the java8 and java7 builds seem to be working. I also made the 
build compile to different target sub-directories so now you can inspect the 
output from both runs without having them interfering. 

Chris


Am 09.06.17, 22:29 schrieb "Dale LaBossiere" :

Hi Chris,  in case you hadn’t noticed the following merged PR (on 
master) eliminates the
use of “default”.  So you should be able to rebase your branch your 
branch and be all set.

https://github.com/apache/incubator-edgent/pull/310 
<https://github.com/apache/incubator-edgent/pull/310>

> On Jun 8, 2017, at 10:40 AM, Christofer Dutz 
 wrote:
> ...problem. The default methods in DirectTestSetup can’t seem to be 
backported if the interface and the implementing class are not in the same 
retrolambda execution. Eventually refactoring the tests to work without default 
methods would be a good idea.

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Well “mvn test” would run the maven lifecycle up to the test phase, this 
includes compiling etc … and compiling doesn’t work with Java7.
We could probably create some solution to manually load and run the tests 
inside the test-jars but integrating this into one nice Maven build could be a 
challenging task, so say the least … but we wouldn’t have to, if you would be 
ok with the signature-verification option. 

Seems not, so I’ll continue digging … but this one will probably take a little 
longer. 

Chris

Am 12.06.17, 15:37 schrieb "Dale LaBossiere" :

Thanks for all the progress!

It’s a little unsettling to not be able to run the regression 
unit/functional tests w/1.7 :-)

If the local mvn repo has the up-to-date built 1.7 compat Edgent jars and 
the local workspace has the up-to-date built 1.7 compat test class files,  why 
doesn’t running “mvn test -Pjava7” w/1.7 just reuse them?

— Dale

> On Jun 12, 2017, at 7:35 AM, Christofer Dutz  
wrote:
> 
> Ok … so I’m working on the open issues … I do have one question though … 
> Compiling with one java version and then running a different one with the 
tests will be challenging, if not impossible. 
> What I did a while ago was to introduce the animal-sniffer plugin. This 
very strictly checks the resulting class files against the signatures for a 
given JDK to ensure the classes would work in that particular JDK. Would this 
be enough for you? It’s what we settled for in the Flex project. In the past we 
did have to re-release stuff because even if we compiled to output 1.7 code, 
the code wasn’t runnable on 1.7. Since introducing the animal-sniffer plugin we 
never had problems like this again. 
> 

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi Dale,

Glad you like that … I was going to address the “checked in IDE settings” thing 
soon ;-)

I am working with IntelliJ without any issues in my Maven setup. Had to rename 
the Gradle build to make IntelliJ chose Maven though.

In my opinion IDE settings shouldn’t be checked in and should rather be added 
to the gitignore list. Don’t know how often we had issues in Flex with checked 
in IDE settings :-(

The warnings are related to the fact that all Apache Maven configurations have 
the “Apache root pom” as top-most parent. A lot Some things are configured 
there. However I’m using different versions of some plugins and re-defining 
them in the edgent-parent. That’s what the first block of warnings are about.

The rest of the blocks are because in order to work correctly the e2m plugin 
has built-in configurations on what to do for which default maven plugin. This 
is how Eclipse knows when to copy resources, compile classes and eventually 
even generate code. However per default not all plugins are handled by the m2e 
plugin. That’s what the second block is about. There is a way to get rid of 
them and eventually even tell Eclipse what to do, but this would need some fine 
tuning. I would like to address that as soon as we’re finished. Otherwise I 
would probably have to re-configure that over and over again. Mabe it would be 
good to tell Eclipse to ignore the plugins it doesn’t know so the errors go 
away and then to fine tune them afterwards.

Chris



Von: Dale LaBossiere 
Antworten an: "dev@edgent.apache.org" 
Datum: Montag, 12. Juni 2017 um 22:43
An: "dev@edgent.apache.org" 
Betreff: Re: Understanding the snapshot and release process

Chris, I started to try to get Eclipse working with the maven PR-309 changes.  
Making some progress.

At a high level, I removed all existing .project and .classpath Eclipse files 
from my local git clone.
Then I created a new Eclipse workspace and did an “Import existing Maven 
Projects”.
Eclipse then created a .project and .classpath from the poms.
Seemingly all good stuff.  And presumably analogous to importing a maven 
project into IntelliJ.

But I have 3 build problems and 66 (mostly pom.xml related) warnings.

What do you make of these warnings in the top-level pom.xml?   Googling seems 
to tell me that the “Overriding managed version” warnings are because the 
top-level pom.xml is overriding the versions for the plugins inherited from the 
parent pom.xml (org.apache:apache:18 ???) ???

[cid:image001.png@01D2E3CF.2BA7FA50]


There are a lot (every project/module I assume) of these:
[cid:image002.png@01D2E3CF.2BA7FA50]

Any thoughts on the 3 errors?
[cid:image003.png@01D2E3CF.2BA7FA50]
the full error msgs are:
[cid:image004.png@01D2E3CF.2BA7FA50]

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Well Maven can do quite a lot of things. But there’s one base principal which I 
lay great emphasis on in my Maven trainings. 
“If it’s hard to do with Maven, you’re probably doing it wrong” :-)

I guess the main problem is that we need Java8 to compile the tests and Java7 
to run them. As a maven build fires up a VM to do the build the version of java 
used throughout the build is sort of defined by this call to run maven. 
Anything else would require to fork a new process with the JavaVM VM … this is 
not handled. If it was possible to build and test with java7, this wouldn’t be 
a problem.

I guess also executing the pre-built test with java7 should also be possible, 
but I’m struggling with one other of your requirements:
“Only deploy everything if the java8, java7 and android builds succeed”. I hope 
you can see the dilemma.

Here it might be a better option to add a Jenkinsfile to the repository and 
setup a Jenkins job on the ASF Jenkins. With this I am a lot more flexible than 
with Travis. We could continue to use Travis for the auto-testing of pull 
requests, but would live with running the java7 testsuite with a java8 VM, but 
have the Jenkins job deploy the SNAPSHOTS to the ASF repo.

What do you think about that? Building on Apache Machines also gives a lot of 
other benefits.

Chris




Am 12.06.17, 18:55 schrieb "Dale LaBossiere" :


> On Jun 12, 2017, at 10:23 AM, Christofer Dutz  
wrote:
> 
> Well “mvn test” would run the maven lifecycle up to the test phase, this 
includes compiling etc … and compiling doesn’t work with Java7.
Maven doesn’t have the notion of build-avoidance / artifact reuse?
I can imagine that even if it did, it might record the JDK version used to 
generate the artifacts and then not reuse them when the current JDK version is 
different.

> We could probably create some solution to manually load and run the tests 
inside the test-jars but integrating this into one nice Maven build could be a 
challenging task, so say the least … but we wouldn’t have to, if you would be 
ok with the signature-verification option. 
I’m holding out till I better understand maven’s behavior/limitations and 
what other strategies might exist :-)

> 
> Seems not, so I’ll continue digging … but this one will probably take a 
little longer. 

What do other projects do if for example they build/release to a 
least-common-demonitor, e.g., 1.7, and still want to do actual test 
verification against the higher compatible/supported version, e.g., 1.8?  They 
don’t/can’t use standard maven machinery for that?  They must build some 
parallel test execution / reporting machinery?

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Aaahh … well that’s a different thing … that’s already possible with maven :-)

Right now, all Apache releases go to a so-called release-repository. That’s a 
Maven repository containing only the artifacts of the current release. The 
project can vote on this. If the vote doesn’t pass, the repo is simply dropped 
and it was as if nothing had happened. If the release passes, the artifacts are 
released to the Apache Maven Release Repo and this is automatically mirrored to 
Maven Central. 

So, it seems this part is already taken care of … one more thing to check off 
the list ;-)

I guess Travis was chosen, because Github was selected as primary repo for the 
project and the integration with auto-built pull requests is great, I have to 
admit that. But it is problematic to setup auto deployment of SNAPSHOT versions 
as we would have to check in credentials for deploying to the Apache Maven repo 
and that’s not going to happen. As the build setup is quite trivial I think 
using both is the best option. Use Travis for the auto-testing of 
pull-requests. Use Apache Jenkins for deploying SNAPSHOTS and doing 
code-analysis on Sonarqube and stuff like that. 

Right now, I am still having issues running all Tests in the java7 version … 
math3 is sort of causing issues, I haven’t quite figured out why.

Chris



Am 13.06.17, 15:59 schrieb "Dale LaBossiere" :


> On Jun 12, 2017, at 5:04 PM, Christofer Dutz  
wrote:
> ...
> I guess also executing the pre-built test with java7 should also be 
possible, but I’m struggling with one other of your requirements:
> “Only deploy everything if the java8, java7 and android builds succeed”. 
I hope you can see the dilemma.
It’s possible my terminology is confusing/inaccurate.  Mostly I think I 
meant that for releases (not talking about snapshots nor release-candidate 
staging for voting) we should only make the final release artifacts, including 
convenience binaries, public only if/once all configurations have passed 
testing&voting.

I think it’s time for me to go back to all the ASF release doc info.  I 
know a lot has changed and earlier I didn’t pay any attention to snapshots nor 
maven related things because we weren’t doing them.

> Here it might be a better option to add a Jenkinsfile to the repository 
and setup a Jenkins job on the ASF Jenkins. With this I am a lot more flexible 
than with Travis. We could continue to use Travis for the auto-testing of pull 
requests, but would live with running the java7 testsuite with a java8 VM, but 
have the Jenkins job deploy the SNAPSHOTS to the ASF repo.
> 
> What do you think about that? Building on Apache Machines also gives a 
lot of other benefits.
I don’t know why Travis was chosen.  Is the ASF Jenkins service relatively 
new or not good/viable for PR-auto-test use?  Regardless, partially or fully 
switching to Jenkins tooling as appropriate is OK with me - unless there are 
some things you forgot to mention :-)

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Ok, so I just pushed a major update.

On my machine with these changes both the java7 and java8 builds seem to be 
working. In the next few days I’ll probably play around with some of the 
excluded modules and see if we can include them in the java7 build. Right now, 
it sort of looks as if they were excluded simply because they didn’t 
immediately work. So I’ll go through them and try to find out which ones we can 
add and write down why some cant (This information seems to be missing at the 
moment)

Another thing I did, was to add the RAT plugin, go through the exclusions and 
clean up the exclusions. Some were excluding stuff that shouldn’t be excluded 
and it reported quite a hand full of files without ASF headers. I added them so 
please have a look if this was ok.

The Eclipse support is also on my list (even if I have to admit that I always 
hate having to open that IDE ;-) )

Chris

Am 13.06.17, 17:45 schrieb "Dale LaBossiere" :


> On Jun 12, 2017, at 4:56 PM, Christofer Dutz  
wrote:
> ...
> In my opinion IDE settings shouldn’t be checked in and should rather be 
added to the gitignore list. Don’t know how often we had issues in Flex with 
checked in IDE settings :-(
Agreed, with maven projects/pom’s in place, the Eclipse IDE 
settings/.project/.classpath would be removed from the repo.

>  
> The warnings are related to the fact that all Apache Maven configurations 
have the “Apache root pom” as top-most parent. A lot Some things are configured 
there. However I’m using different versions of some plugins and re-defining 
them in the edgent-parent. That’s what the first block of warnings are about.
Since the overrides are intentional, can you add the “annotations” to 
eliminate the warnings?  (and maybe a comment as to why a different version is 
being selected?) 
https://stackoverflow.com/questions/30782453/maven-suppress-overriding-managed-version-warning-in-eclipse
 
<https://stackoverflow.com/questions/30782453/maven-suppress-overriding-managed-version-warning-in-eclipse>

>  
> The rest of the blocks are because in order to work correctly the e2m 
plugin has built-in configurations on what to do for which default maven 
plugin. This is how Eclipse knows when to copy resources, compile classes and 
eventually even generate code. However per default not all plugins are handled 
by the m2e plugin. That’s what the second block is about. There is a way to get 
rid of them and eventually even tell Eclipse what to do, but this would need 
some fine tuning. I would like to address that as soon as we’re finished. 
Otherwise I would probably have to re-configure that over and over again. Mabe 
it would be good to tell Eclipse to ignore the plugins it doesn’t know so the 
errors go away and then to fine tune them afterwards.
Dealing with the warnings later seems OK.  Can you make the change to 
eliminate the errors now?  You probably know what needs to be done but fwiw 
here’s what Eclipse quick-fix added:
@dales-mbp:1049> diff test/fvtiot/pom.xml{,.sav}
62,96d61
< 
<   
<   
<   
<   org.eclipse.m2e
<   lifecycle-mapping
<   1.0.0
<   
<   
<   
<   
<   

<   
<   
org.apache.maven.plugins
<   
<   
<   
maven-dependency-plugin
<   

<   

<   
[3.0.1,)
<   

<   
<   
copy
<   
<   

<   
<   

<   
<   
<   
<   
<   
<   
<   
< 
@dales-mbp:1050> 

Thanks,
— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Your wish is my command ;-)

I just installed the latest Eclipse “Neon.3” in the Developer edition (which 
comes with Maven support).

Then I manually deleted all existing eclipse settings in the Edgent project as 
otherwise Eclipse would have insisted on using them instead. Also, I added them 
to the main gitignore file to prevent them from being tracked when re-created.

After that I imported my existing Maven project into my Eclipse workspace. It 
correctly detected all plugins I am using in the build and correctly setup the 
M2E plugins for these (glad I’m just using the usual suspects of maven 
plugins). In the end, I still got 3 errors complaining about the copy plugin 
not being able to copy because the jars not having been assembled yet. So, I 
moved all tests that require the loading of WAR or JAR files to the 
integration-test phase (where they belong anyway) and re-tried. Eclipse still 
had the same issues. In the end, I forced Eclipse to ignore the plugin 
execution. So, If you try to execute any of the “IT” suffixed tests in Eclipse, 
these will probably fail if you haven’t done a Maven build prior to test 
execution.

Right now, I have the project imported in Eclipse and am getting 0 errors. I 
tried running some Junit tests from Eclipse and all worked nicely :-)

So, I guess were coming close to the finishing line.

Chris



Am 14.06.17, 18:09 schrieb "Dale LaBossiere" :

Yup, just noticed it. Looking good!

> On Jun 14, 2017, at 11:36 AM, Christofer Dutz  
wrote:
> 
> Ok, so I just pushed a major update.
> 
> On my machine with these changes both the java7 and java8 builds seem to 
be working. In the next few days I’ll probably play around with some of the 
excluded modules and see if we can include them in the java7 build. Right now, 
it sort of looks as if they were excluded simply because they didn’t 
immediately work. So I’ll go through them and try to find out which ones we can 
add and write down why some cant (This information seems to be missing at the 
moment)
I suggest deferring the “add more to java7” effort until everything else 
needed to reach the state of being able to develop and make a release with the 
new machinery is done.  Maybe deferring until after actually making a release 
with the new machinery - IMO we should do one asap.

> Another thing I did, was to add the RAT plugin, go through the exclusions 
and clean up the exclusions. Some were excluding stuff that shouldn’t be 
excluded and it reported quite a hand full of files without ASF headers. I 
added them so please have a look if this was ok.
LGTM

> 
> The Eclipse support is also on my list (even if I have to admit that I 
always hate having to open that IDE ;-) )
:-) the sooner the better so that more folks can pull in the PR to 
experiment.

Thanks!
— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Just another question …

It seems as if some unit-tests tend to fail in random intervals. Right now, I 
had 3 Travis runs without any functional changes and I got failures on 3 
different places. Is this “normal”? If yes … I think I should start 
identifiying the “random failers” and investigate the reason.

Chris 

Am 15.06.17, 12:49 schrieb "Christofer Dutz" :

Your wish is my command ;-)

I just installed the latest Eclipse “Neon.3” in the Developer edition 
(which comes with Maven support).

Then I manually deleted all existing eclipse settings in the Edgent project 
as otherwise Eclipse would have insisted on using them instead. Also, I added 
them to the main gitignore file to prevent them from being tracked when 
re-created.

After that I imported my existing Maven project into my Eclipse workspace. 
It correctly detected all plugins I am using in the build and correctly setup 
the M2E plugins for these (glad I’m just using the usual suspects of maven 
plugins). In the end, I still got 3 errors complaining about the copy plugin 
not being able to copy because the jars not having been assembled yet. So, I 
moved all tests that require the loading of WAR or JAR files to the 
integration-test phase (where they belong anyway) and re-tried. Eclipse still 
had the same issues. In the end, I forced Eclipse to ignore the plugin 
execution. So, If you try to execute any of the “IT” suffixed tests in Eclipse, 
these will probably fail if you haven’t done a Maven build prior to test 
execution.

Right now, I have the project imported in Eclipse and am getting 0 errors. 
I tried running some Junit tests from Eclipse and all worked nicely :-)

So, I guess were coming close to the finishing line.

Chris



Am 14.06.17, 18:09 schrieb "Dale LaBossiere" :

Yup, just noticed it. Looking good!

> On Jun 14, 2017, at 11:36 AM, Christofer Dutz 
 wrote:
> 
> Ok, so I just pushed a major update.
> 
> On my machine with these changes both the java7 and java8 builds seem 
to be working. In the next few days I’ll probably play around with some of the 
excluded modules and see if we can include them in the java7 build. Right now, 
it sort of looks as if they were excluded simply because they didn’t 
immediately work. So I’ll go through them and try to find out which ones we can 
add and write down why some cant (This information seems to be missing at the 
moment)
I suggest deferring the “add more to java7” effort until everything 
else needed to reach the state of being able to develop and make a release with 
the new machinery is done.  Maybe deferring until after actually making a 
release with the new machinery - IMO we should do one asap.

> Another thing I did, was to add the RAT plugin, go through the 
exclusions and clean up the exclusions. Some were excluding stuff that 
shouldn’t be excluded and it reported quite a hand full of files without ASF 
headers. I added them so please have a look if this was ok.
LGTM

> 
> The Eclipse support is also on my list (even if I have to admit that 
I always hate having to open that IDE ;-) )
:-) the sooner the better so that more folks can pull in the PR to 
experiment.

Thanks!
— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Unfortunately we have to live with that.
Using variables for the versions can cause problems. In our case I made sure 
they don't. But we need that in order to create the java 8 and java 7 packages.

But Maven tends to be over careful.

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 15.06.17 17:19 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Understanding the snapshot and release process

I just pulled down the latest PR contents into a new clone.  I get a bunch of 
warnings like the one below (but things work).

mvn clean install
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for 
org.apache.edgent.analytics:edgent-analytics-sensors:jar:1.2.0-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ 
org.apache.edgent.analytics:edgent-analytics-sensors:${edgent.version}, 
/Users/dlaboss/git/incubator-edgen\
t-4/analytics/sensors/pom.xml, line 31, column 12

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

In case of a variable for the groupId the warning is the same except that it 
mentions groupId instead of version :-(

Chris



Am 15.06.17, 23:18 schrieb "Dale LaBossiere" :

ugh.  There are ~160 lines of this every time you run a mvn cmd…  and I 
hadn’t noticed the:

[WARNING]
[WARNING] It is highly recommended to fix these problems because they 
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support 
building such malformed projects.
[WARNING]

Sounds pretty ominous.

Seems to me like people will quickly tire of / get annoyed by this.
Might it be a lesser of evils to put the java17 in the groupId instead?

— Dale 

> On Jun 15, 2017, at 11:55 AM, Christofer Dutz  
wrote:
> 
> Unfortunately we have to live with that.
> Using variables for the versions can cause problems. In our case I made 
sure they don't. But we need that in order to create the java 8 and java 7 
packages.
> 
> But Maven tends to be over careful.
> 
> Chris
> 
> 
> 
> Von meinem Samsung Galaxy Smartphone gesendet.
> 
> 
>  Ursprüngliche Nachricht 
> Von: Dale LaBossiere 
> Datum: 15.06.17 17:19 (GMT+01:00)
> An: dev@edgent.apache.org
> Betreff: Re: Understanding the snapshot and release process
> 
> I just pulled down the latest PR contents into a new clone.  I get a 
bunch of warnings like the one below (but things work).
> 
> mvn clean install
> [INFO] Scanning for projects...
> [WARNING]
> [WARNING] Some problems were encountered while building the effective 
model for 
org.apache.edgent.analytics:edgent-analytics-sensors:jar:1.2.0-SNAPSHOT
> [WARNING] 'version' contains an expression but should be a constant. @ 
org.apache.edgent.analytics:edgent-analytics-sensors:${edgent.version}, 
/Users/dlaboss/git/incubator-edgen\
> t-4/analytics/sensors/pom.xml, line 31, column 12
> 
> — Dale

Setting up an ASF Jenkins?

[Christofer Dutz]

Hi guys,

I was thinking, if you would be interested in me setting up ASF Jenkins for 
Edgent?
The way Travis is currently used to automatically test pull requests is really 
good. I would like to additionally setup Jenkins Jobs on ASF Infra to publish 
SNAPSHOTS in the near future. Also, it would allow automating the process of 
updating the Edgent website.

I don’t know if I would need additional permissions to do this, but I wanted to 
make sure you guys would like me to take care of this.

Chris

Re: Understanding the snapshot and release process

[Christofer Dutz]

Just have a look at the last few builds on Travis that my pull request 
initiated. You will see it failing with different tests every time I ran the 
build even if nothing changed (I think in the last commit I simply removed a 
space in one of the poms).

Was successful …

After moving some tests from Unit-Test to Integration-Test phase:
Failure in module “Apache Edgent: Providers: Development”
- DevelopmentPlumbingTest>PlumbingTest.testParallelBalanced:628

After adding the explice lifecycle mapping plugin config (has no effect on the 
build)
Failure in module “Apache Edgent: Connectors: WSClient Websocket”
- [ERROR]   
WebSocketClientGlobalTest>WebSocketClientTest.testReconnect:359->TopologyAbstractTest.completeAndValidate:82->TopologyAbstractTest.completeAndValidate:98
  contents:[一, 二, 四]
- [ERROR]   
WebSocketClientGlobalTest>WebSocketClientTest.testSslReconnect:571->TopologyAbstractTest.completeAndValidate:82->TopologyAbstractTest.completeAndValidate:98
  contents:[一, 二, 四]

After removing a simple space character in a pom
Failure in module “Apache Edgent: Providers: Direct”
- [ERROR]   DirectPlumbingTest>PlumbingTest.testPressureReliever:194 
delay:TAI:332@380

So it seems that there is a hand full of tests that could need some hardening …

Chris

Am 15.06.17, 14:42 schrieb "Dale LaBossiere" :

Thx for the cleanup for Eclipse.

    > On Jun 15, 2017, at 7:09 AM, Christofer Dutz  
wrote:
> ...It seems as if some unit-tests tend to fail in random intervals. Right 
now, I had 3 Travis runs without any functional changes and I got failures on 3 
different places. Is this “normal”? If yes … I think I should start 
identifiying the “random failers” and investigate the reason.

What were the failing tests?

MqttOpenTest and Http[Global]Test use external public servers and can be 
impacted by them.  An attempt has been made to desensitize them (e.g., trying 
to connect and then skipping instead of failing tests) but sometimes things 
seem to leak through.

All of the tests have time limits imposed and will “fail" if they don’t 
complete in time (some have had more narrow limits than others).  I’ve seen 
these sort of failures on occasion such as when the test system is heavily 
loaded.

— Dale

Re: Setting up an ASF Jenkins?

[Christofer Dutz]

That depends on your current setup. If you could describe that, I could go a 
little more into details.

Chris

Am 16.06.17, 14:45 schrieb "Dale LaBossiere" :

+1 from me.  Could you elaborate on the website updating?  What is updated 
under what conditions?

Thanks,
— Dale

> On Jun 16, 2017, at 4:58 AM, Christofer Dutz  
wrote:
> 
> Hi guys,
> 
> I was thinking, if you would be interested in me setting up ASF Jenkins 
for Edgent?
> The way Travis is currently used to automatically test pull requests is 
really good. I would like to additionally setup Jenkins Jobs on ASF Infra to 
publish SNAPSHOTS in the near future. Also, it would allow automating the 
process of updating the Edgent website.
> 
> I don’t know if I would need additional permissions to do this, but I 
wanted to make sure you guys would like me to take care of this.
> 
> Chris

Re: Setting up an ASF Jenkins?

[Christofer Dutz]

No, generating snapshots from every pull request would be dangerous. Snapshots 
would only be generated from the develop branch.

It would be possible to auto generate the site and have that pushed to the 
asf-site branch, which is picked up by the infra bots. I already finished such 
a setup for the flex project.

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: William Marshall 
Datum: 16.06.17 20:19 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Setting up an ASF Jenkins?

Hey Chris,

Would this create a SNAPSHOT for every pull request? Where would snapshots
be hosted?

I like the idea of changing the website workflow. Currently we manually
generate the html using jekyll, commit the changed html to the asf-site
branch, and then push that to origin.

-Will

On Fri, Jun 16, 2017 at 9:47 AM, Dale LaBossiere 
wrote:

> Does this tell you what you need to know?
>
> https://github.com/apache/incubator-edgent-website/blob/master/README.md <
> https://github.com/apache/incubator-edgent-website/blob/master/README.md>
>
> — Dale
>
>
> > On Jun 16, 2017, at 9:03 AM, Christofer Dutz 
> wrote:
> >
> > That depends on your current setup. If you could describe that, I could
> go a little more into details.
> >
> > Chris
> >
> > Am 16.06.17, 14:45 schrieb "Dale LaBossiere" :
> >
> >+1 from me.  Could you elaborate on the website updating?  What is
> updated under what conditions?
> >
> >Thanks,
> >— Dale
> >
> >> On Jun 16, 2017, at 4:58 AM, Christofer Dutz 
> wrote:
> >>
> >> Hi guys,
> >>
> >> I was thinking, if you would be interested in me setting up ASF Jenkins
> for Edgent?
> >> The way Travis is currently used to automatically test pull requests is
> really good. I would like to additionally setup Jenkins Jobs on ASF Infra
> to publish SNAPSHOTS in the near future. Also, it would allow automating
> the process of updating the Edgent website.
> >>
> >> I don’t know if I would need additional permissions to do this, but I
> wanted to make sure you guys would like me to take care of this.
> >>
> >> Chris
> >
> >
> >
>
>

Re: Understanding the snapshot and release process

[Christofer Dutz]

Ok … so I think I might have a possible solution, thanks to Karl Heinz Marbaise 
on the Maven list.

We don’t use profiles at all and build all artifacts in one run. The difference 
would be that we create mini modules in the “java7” directory which build the 
java7 versions of the java8 artifacs by copying and unpacking the java8 
versions to the mini projects target directory, have retrolambda do it’s magic 
there and package them normally. In this case, I would like to use a different 
groupId for the java7 artifacts instead of using different versions as this way 
we will have far less trouble releasing stuff.

Using maven toolchains I might even be able to run the transformed tests in the 
mini projects with java7 … haven’t used toolchains till now, but it looks as if 
they would deliver exactly the functionality we need.

So I’m going to give this approach a try.

Chris


Am 16.06.17, 18:43 schrieb "Dale LaBossiere" :


> On Jun 16, 2017, at 4:46 AM, Christofer Dutz  
wrote:
> 
> In case of a variable for the groupId the warning is the same except that 
it mentions groupId instead of version :-(

Hmm…  any thoughts on plausible alternatives? ...and implications to 
dev, release, and consumption of java7&android Edgent?

At a high level, something just seems wrong that it’s this hard to
create multiple variants of a product w/mvn, where the variants’
jars need different coordinates along some dimension.
The retrolambda plugin may be complicating things but it doesn't
feel like it’s the cause of the main issue: variable coordinate decls.

Related, without the use of variable-version decls, even ignoring
this j8/j7 thing, there would be 1.2.0-SNAPSHOT
throughout every one of our (~65) poms.  Even using
variable-version decls, that static decl is present in every pom’s
 decl!  Is it the really case that when we want to release
1.2.0 (not SNAPSHOT) or when it’s time for 1.3.0, every pom
has to be edited?

Just brainstorming…   my head hurts :-\   I apologize in advance…  :-)

E means Edgent-java ...

Is there any ~convenient way to synthesize/use alternate pom’s for E7?
Might there be a way to leverage “mvn —file ” and also 
pom.xml:?

e.g., have the pom.xml implicitly be j8/E8 oriented with non-variable 
coordinate decl for the project.

Imagine building / staging a RC for E7 would be something like:

mk-j7-poms  # create pom-j7.xml files adding “-java7” in versions and a 
“relativePath” of ../pom-j7.xml
 # **/pom-7.xml added to .gitignore; target dir is 
still variable

mvn -f pom-j7.xml -Pjava7 clean install -DskipTests
# TBD do j7 testing
# …
mvn -f pom-j7.xml -Pjava7 deploy -Papache-release -DskipTests

Or if we’re going that far, then maybe needing/using a separate 
clone/workspace to build & deploy E7 might be tolerable?

Though… while the gradle/ant tooling created j7 jars using *the* j8 jars, 
this mvn tooling is already regenerating the j8 classes 
and then running them through retrolambda, right? (if so, ugh)

Kinda feels like E7 artifacts should instead just declare a
dependency on their corresponding E8 jar artifact (not src)
and just do some (retrolambda) transform to generate the E7 jar artifacts.
Which seems to imply separate (~parallel) simpler poms for E7
which can then also have non-variable-version decl?
e.g., then something like
cd java7
mvn clean install -DskipTests
# TBD test
mvn deploy -Papache-release -DskipTests

Also, the current "mvn install -Papache-release -Pjava7"
generates -sources.jar, -javadoc.jar, -source-release.zip
Do we really want to distribute those (duplicate) set of those artifacts?
Don’t know if that might just mean more config tweaking or
if the retrolambda plugin / “deploy” flow for java isn’t really what
we want.

Your thoughts on all of this?

— Dale

Re: Understanding the snapshot and release process

[Christofer Dutz]

Ok … reporting some progress here.

I managed to get the build to create the java8 and the java7 artifacts in one 
run and to run the tests in both versions.
The only thing I am currently struggling with is the Maven toolchain support.

Maven provides a mechanism called “toolchains” in which you can for example 
define multiple JDKs and use this to do exactly what we want: run all tests in 
the normal modules with Java8 and the ones in the Java7 modules with Java7. 
Unfortunately, we would need the paths to the JDKs on Travis to use this. It 
would be easy on Apache Jenkins, but right now I think it would probably be 
better to let the tests on Travis all run with Java8 and to run the Java7 tests 
with Java7 on the ASF Jenkins.

What do you think?

Will continue porting all the individual modules … currently only ported the 
api modules . A lot more to go ;-)

Chris



Am 17.06.17, 16:29 schrieb "Christofer Dutz" :

Ok … so I think I might have a possible solution, thanks to Karl Heinz 
Marbaise on the Maven list.

We don’t use profiles at all and build all artifacts in one run. The 
difference would be that we create mini modules in the “java7” directory which 
build the java7 versions of the java8 artifacs by copying and unpacking the 
java8 versions to the mini projects target directory, have retrolambda do it’s 
magic there and package them normally. In this case, I would like to use a 
different groupId for the java7 artifacts instead of using different versions 
as this way we will have far less trouble releasing stuff.

Using maven toolchains I might even be able to run the transformed tests in 
the mini projects with java7 … haven’t used toolchains till now, but it looks 
as if they would deliver exactly the functionality we need.

So I’m going to give this approach a try.

Chris


Am 16.06.17, 18:43 schrieb "Dale LaBossiere" :


> On Jun 16, 2017, at 4:46 AM, Christofer Dutz 
 wrote:
> 
> In case of a variable for the groupId the warning is the same except 
that it mentions groupId instead of version :-(

Hmm…  any thoughts on plausible alternatives? ...and implications to 
dev, release, and consumption of java7&android Edgent?

At a high level, something just seems wrong that it’s this hard to
create multiple variants of a product w/mvn, where the variants’
jars need different coordinates along some dimension.
The retrolambda plugin may be complicating things but it doesn't
feel like it’s the cause of the main issue: variable coordinate decls.

Related, without the use of variable-version decls, even ignoring
this j8/j7 thing, there would be 1.2.0-SNAPSHOT
throughout every one of our (~65) poms.  Even using
variable-version decls, that static decl is present in every pom’s
 decl!  Is it the really case that when we want to release
1.2.0 (not SNAPSHOT) or when it’s time for 1.3.0, every pom
has to be edited?

Just brainstorming…   my head hurts :-\   I apologize in advance…  :-)

E means Edgent-java ...

Is there any ~convenient way to synthesize/use alternate pom’s for E7?
Might there be a way to leverage “mvn —file ” and also 
pom.xml:?

e.g., have the pom.xml implicitly be j8/E8 oriented with non-variable 
coordinate decl for the project.

Imagine building / staging a RC for E7 would be something like:

mk-j7-poms  # create pom-j7.xml files adding “-java7” in 
versions and a “relativePath” of ../pom-j7.xml
 # **/pom-7.xml added to .gitignore; target 
dir is still variable

mvn -f pom-j7.xml -Pjava7 clean install -DskipTests
# TBD do j7 testing
# …
mvn -f pom-j7.xml -Pjava7 deploy -Papache-release -DskipTests

Or if we’re going that far, then maybe needing/using a separate 
clone/workspace to build & deploy E7 might be tolerable?

Though… while the gradle/ant tooling created j7 jars using *the* j8 
jars, 
this mvn tooling is already regenerating the j8 classes 
and then running them through retrolambda, right? (if so, ugh)

Kinda feels like E7 artifacts should instead just declare a
dependency on their corresponding E8 jar artifact (not src)
and just do some (retrolambda) transform to generate the E7 jar 
artifacts.
Which seems to imply separate (~parallel) simpler poms for E7
which can then also have non-variable-version decl?
e.g., then something like
cd java7
mvn clean install -DskipTests
# TBD test
mvn deploy -Papache-rele

Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Hi,

While fine tuning my maven migration, I stumbled over a problem in the 
analytics/sensors modules.
As part of the build I am validating the classes for Java7 against the 
signatures of the Java 7 SDK. Here my plugin found an issue I thought was worth 
reporting.
In the class org.apache.edgent.analytics.sensors.Range almost at the end in the 
method toUnsignedString we are using Byte.toUnsignedInt, which is only 
available in Java 8 … there is no back-port for java 7 for this code. 
Eventually we should replace this with code that works on Java 7 too.

Chris

Re: Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Ok guys … now some MAJOR update :-)

I managed to setup a build that build all modules with Java8, if the profile 
“platform-java7” is enabled, also Java7 versions are produced by unpacking the 
Java8 versions in some pom-only modules, applying retrolambda there and then 
running all unit- and integration-tests with Java7. The cool thing is, I didn’t 
port only the ones build for java7 in the past, but all edgent modules and 
fixed the ones that were having problems (I added a helper that does the same 
as the Java8 extensions did). Now all modules except some examples are also 
available on Java7 (I had do disable some Tests though, as they were using 
Java8 APIs, but this should be fixable quite easily). The cool thing … all 
should also work on Travis too (I found a place where the Java paths were 
documented …  see the .travis.yml).

The build of Java8 should work out of the box on any machine with Maven and 
Java8, if you want to build the Java7 versions, you need to activate the 
profile “platform-java7” and provide a so-called toolchain (either you create a 
toolchain.xml in your .m2 directory or you create a file elsewhere and pass 
that files path in with the “-t” command-line option). As retrolambda needs 
Java8 to run, you also need to provide a property: “java8.home” that points to 
your Java8 home directory.

Regarding the Android package, I would start working on this next. If the 
“android” modules are only used in the “android” platform, I would suggest 
building a similar structure for android as for Java7 and to move the real 
android modules into that directory. Then they wouldn’t be part of the Java8 
and Java7 package and be included only in the android package.

You can use the toolchains-travis.xml file as a template to create your own. I 
added an git exclusion for toolchains-local.xml. So, if you just copy that to 
toolchains-local.xml and adjust your java-home paths inside, you should be able 
to build everything with the following command:

mvn –Pplatform-java7,platform-android –Djava8.home={JAVA8_HOME} –t 
toolchains-local.xml clean install

Hope it works on your machines too … if not I’ll handle the problems as soon as 
possible.

Chris




Am 19.06.17, 15:20 schrieb "Dale LaBossiere" :

Nice catch!
I created https://issues.apache.org/jira/browse/EDGENT-423
— Dale

> On Jun 18, 2017, at 8:04 AM, Christofer Dutz  
wrote:
> 
> While fine tuning my maven migration, I stumbled over a problem in the 
analytics/sensors modules.
> As part of the build I am validating the classes for Java7 against the 
signatures of the Java 7 SDK. Here my plugin found an issue I thought was worth 
reporting.
> In the class org.apache.edgent.analytics.sensors.Range almost at the end 
in the method toUnsignedString we are using Byte.toUnsignedInt, which is only 
available in Java 8 … there is no back-port for java 7 for this code. 
Eventually we should replace this with code that works on Java 7 too.

Re: Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Hi Dale,

Glad it seems to be working :-)

If you could write down exactly what the difference is between java8/7 and 
android, then I could probably finish that tomorrow.

Chris


Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 19.06.17 21:01 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Java7 compatability issue in analytics/sensors

Success!  Now I need to spend some time understanding the results :-)   
questions to follow

$ mvn -t toolchains-local.xml -Djava8.home=$JAVA_HOME clean install  site:site 
-Pplatform-java7,platform-android
…
[INFO] Apache Edgent (Java 7): Test: SVT .. SUCCESS [ 57.896 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 59:17 min
[INFO] Finished at: 2017-06-19T14:23:59-04:00
[INFO] Final Memory: 136M/969M
[INFO] 

> On Jun 19, 2017, at 12:10 PM, Dale LaBossiere  wrote:
>
> Hmm… changed the CLI options order to match that in .travis.yml and it’s 
> working now.
> I’ll report once everything has run.
>
>> On Jun 19, 2017, at 12:05 PM, Dale LaBossiere > <mailto:dml.apa...@gmail.com>> wrote:
>>
>> Did you commit&push all of your changes?
>> I pulled the latest PR-309 stuff.  Created my toolchains-local.xml 
>> (copied/edited toolchains-travis.xml)
>> and then...
>>
>>> On Jun 19, 2017, at 10:13 AM, Christofer Dutz >> <mailto:christofer.d...@c-ware.de>> wrote:
>>>
>>> mvn –Pplatform-java7,platform-android –Djava8.home={JAVA8_HOME} –t 
>>> toolchains-local.xml clean install
>>
>> $ mvn –Pplatform-java7 –Djava8.home=$JAVA_HOME –t toolchains-local.xml clean 
>> install
>> [ERROR] Unknown lifecycle phase "–Pplatform-java7". You must specify a valid 
>> lifecycle phase or a goal in the format : or 
>> :[:]:
>> ...
>

Re: Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Hi Dale,

Ok … so to me it sounds like I should simply move the android directory to the 
android platform part of the build - So it’s only built, when building the 
android distribution. Then to remove it from the java7 platform. That should be 
all from a build point of view. I could duplicate the pom-only modules of java7 
in android to give them a matching groupId, but the artifacts should be 
identical from the ones in java7 and they would add a lot of additional time 
for running the tests twice for java7. I think we shouldn’t do that.

What’s still missing is the (binary-) distributions themselves. While with 
Maven we would be ready to go to start building Edgent applications, when using 
Ant or other - more manual tools – a regular distribution package, bundling 
jars and scripts is probably preferable. This is handled by so-called 
“assemblies” … I would handle building android, java7 and java8 distributions 
with dedicated assembly modules.

Chris

PS: By the way … are there any other Edgent developers? It seems quite quiet 
here on the list, if you subtract my email-spam ;-)




Am 19.06.17, 22:58 schrieb "Dale LaBossiere" :


> On Jun 19, 2017, at 4:20 PM, Christofer Dutz  
wrote:
> ...
> If you could write down exactly what the difference is between java8/7 
and android, then I could probably finish that tomorrow.


Android is our j7 platform release with these changes:
  - will not support due to no JMX support (ideally exclude artifacts for 
these)
edgent.api.developent
edgent.runtime.jmxcontrol
edgent.console.servlets
edgent.console.server
  - adds
android.topology.jar
android.hardware.jar

— Dale

Re: Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Hi all,

So, I just pushed the finished andoid platform support.

What I did, was to move the android module from the root into the android 
platform. After that I had to refactor the code to be directly compilable with 
Java7.
Also, I ensured that as soon as someone wants to build the android platform he 
is informed that he also has to enable the java7 platform for it to work.

Next (and probably the final big step) would be to create assemblies for the 3 
versions (java8, java7, android) which create binary distributions for all 
platforms.

Chris




Am 20.06.17, 08:22 schrieb "Christofer Dutz" :

Hi Dale,

Ok … so to me it sounds like I should simply move the android directory to 
the android platform part of the build - So it’s only built, when building the 
android distribution. Then to remove it from the java7 platform. That should be 
all from a build point of view. I could duplicate the pom-only modules of java7 
in android to give them a matching groupId, but the artifacts should be 
identical from the ones in java7 and they would add a lot of additional time 
for running the tests twice for java7. I think we shouldn’t do that.

What’s still missing is the (binary-) distributions themselves. While with 
Maven we would be ready to go to start building Edgent applications, when using 
Ant or other - more manual tools – a regular distribution package, bundling 
jars and scripts is probably preferable. This is handled by so-called 
“assemblies” … I would handle building android, java7 and java8 distributions 
with dedicated assembly modules.

Chris

PS: By the way … are there any other Edgent developers? It seems quite 
quiet here on the list, if you subtract my email-spam ;-)




Am 19.06.17, 22:58 schrieb "Dale LaBossiere" :


> On Jun 19, 2017, at 4:20 PM, Christofer Dutz 
 wrote:
> ...
> If you could write down exactly what the difference is between 
java8/7 and android, then I could probably finish that tomorrow.


Android is our j7 platform release with these changes:
  - will not support due to no JMX support (ideally exclude artifacts 
for these)
edgent.api.developent
edgent.runtime.jmxcontrol
edgent.console.servlets
edgent.console.server
  - adds
android.topology.jar
android.hardware.jar

— Dale

Re: Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Hi Dale,

Well I could move the module back outside of the android module. The problem 
with this is, that as far as I understood it, it won’t work in the Java8 
version and it is only used in the android distribution – which must be java7. 
The current version will definitely mislead people to try to use it with java8. 
I agree the workflow would be easier leaving it the way it was, but it’s more 
ambiguous.

Same with the “fake artifacts” which simply reference the java7 version … in 
this case you would have to use the “type” of “pom” for every artifact or we 
would be creating a load of empty jars that have a dependency to the java7 
version. I don’t really like both cases … I’ll leave things as they are in the 
PR for now and keep that in mind. Who knows which solution will pop my mind 
when I’m doing something completely different ;-)

I was calling the module “android.android” because it’s the android 
distribution and the android module of that. It does sound strange – I agree – 
eventually we could rename the module to make it sort of “android.sensors” or 
whatever would make sense.

We could create convenience poms that contain all the dependencies to the 
platform they belong to … then all you would need to do, would be to create one 
pom-dependency to that and you would immediately get all jars that belong to 
that pulled in. But, as I said, I’ll let it hang for a little while … mabe I’ll 
come up with a different solution.

Chris



Am 20.06.17, 16:34 schrieb "Dale LaBossiere" :


> On Jun 20, 2017, at 2:22 AM, Christofer Dutz  
wrote:
> ...
> Ok … so to me it sounds like I should simply move the android directory 
to the android platform part of the build - So it’s only built, when building 
the android distribution. Then to remove it from the java7 platform. That 
should be all from a build point of view. I could duplicate the pom-only 
modules of java7 in android to give them a matching groupId, but the artifacts 
should be identical from the ones in java7 and they would add a lot of 
additional time for running the tests twice for java7. I think we shouldn’t do 
that.

There seem to be two parts to this.

Not keen on the handling of the android specific modules. i.e., the 
project's development model is to freely use constrained-java8 (j7 + lambdas) 
everywhere and then have those modules processed w/retrolambda for j7&android 
targets.  Makes sense, doesn’t it?  Can’t we retain that dev model for the 
android specific modules?

Agree it’s nice to avoid duplicating the j7 jars.
I’m trying to imagine how an android app developer would consume Edgent for 
building their app. (not an environment I’m familiar with)
With the current PR state, I think they’d have to declare dependencies on 
the j7 components.
Imagine they’re using the IotProvider,  the WIoTP connector and 
android-topology.  
I think they’d have to declare the dependencies:

c.a.e.java7.providers:edgent-providers-iot:1.2.0-SNAPSHOT
c.a.e.java7.connectors:edgent-connectors-iotp:1.2.0-SNAPSHOT
c.a.e.android.android:edgent-android-topology:1.2.0-SNAPSHOT  # must we 
have “android.android”?

Kinda feels like it would be better to have them strictly think in terms of 
only the Edgent android platform.  No?

c.a.e.android.providers:edgent-providers-iot:1.2.0-SNAPSHOT
c.a.e.android.connectors:edgent-connectors-iotp:1.2.0-SNAPSHOT
c.a.e.android.android:edgent-android-topology:1.2.0-SNAPSHOT  # must we 
have “android.android”?

there wouldn’t be a 
c.a.e.android.providers:edgent-providers-development since it’s not supported 
for android

Presumably, the android poms for the non-android-specific components would 
just declare a dependency on their corresponding j7 artifact.

As for android tests/testing, we don’t have any automated mechanism for 
doing that and we wouldn’t want a mvn build to run all the tests again :-)
I guess we mostly take comfort in knowing android's 99% the same (tested) 
j7 jars and currently only two small android-specific modules that were 
manually tested.


> What’s still missing is the (binary-) distributions themselves. While 
with Maven we would be ready to go to start building Edgent applications, when 
using
Looking fwd to that installment!
Having (presumably edgent-only) binary bundles will be nice / simplify our 
project.  I’ll follow-on with separate questions for that.

> PS: By the way … are there any other Edgent developers? It seems quite 
quiet here on the list, if you subtract my email-spam ;-)
Yeah, 5 or so of my colleague contributors have been heads-down on some 
rather demanding / high priority projects.
More incentive for folks to join and grow the community :-)  So glad you’re 
here!!!
Making Edgent easily consumable via maven-central will lower the bar for 
using it.

— Dale

Re: Java7 compatability issue in analytics/sensors

[Christofer Dutz]

Ok I changed the android stuff to build with Java8 and to use retrolambda to 
produce a java7 version and ensure it’s valid java7 using the animalsniffer 
plugin. So that should be ok.

The thing with empty jars is that if for example I would be building some 
application with maven targeting android, the application will contain one and 
the same artifact name twice … it would be 
edgent-connectors-common-1.2.0-SNAPSHOT.jar (the one from android) and 
edgent-connectors-common-1.2.0-SNAPSHOT.jar (the one from java7) … this will 
definitely cause problems. 

I think I’ll go down the path of repacking the java7 jars as android jars and 
to deploy them with the android groupId. I know that we will be providing 
technically the same content twice, but I guess the benefits of this approach 
overweigh the disadvantages of the other options.

Chris




Am 20.06.17, 19:29 schrieb "Dale LaBossiere" :

I ask things without understanding some of the implications/consequences :-)

> On Jun 20, 2017, at 11:03 AM, Christofer Dutz  
wrote:
> ...
> Well I could move the module back outside of the android module. The 
problem with this is, that as far as I understood it, it won’t work in the 
Java8 version and it is only used in the android distribution – which must be 
java7. The current version will definitely mislead people to try to use it with 
java8. I agree the workflow would be easier leaving it the way it was, but it’s 
more ambiguous.

Your understanding is correct and I also like it in its new location under 
platforms/android.  I was hoping that under platforms/android, the build for 
.java files could be like what used to happen for j7 before you reworked things 
- i.e., in platforms/android for -Pplatform-android, inject retrolambda into 
the compile flow and then just compile as usual with j8. (generating things 
under target/android)

> Same with the “fake artifacts” which simply reference the java7 version … 
in this case you would have to use the “type” of “pom” for every artifact or we 
would be creating a load of empty jars that have a dependency to the java7 
version. I don’t really like both cases … I’ll leave things as they are in the 
PR for now and keep that in mind. Who knows which solution will pop my mind 
when I’m doing something completely different ;-)

:-)   I think I now understand the need for a fake-artifact to support use 
of dependency declaration like o.a.e.android:edgent-provider-iot
Hmm… my initial reaction is that “empty” o.a.e.android jars maybe don’t 
seem all that terrible in order to present that usage model to the user.  It 
feels a bit better than having a copy of the j7 jar in the repo (with a 
different name/coordinate).


> I was calling the module “android.android” because it’s the android 
distribution and the android module of that. It does sound strange – I agree – 
eventually we could rename the module to make it sort of “android.sensors” or 
whatever would make sense.
> 
> We could create convenience poms that contain all the dependencies to the 
platform they belong to … then all you would need to do, would be to create one 
pom-dependency to that and you would immediately get all jars that belong to 
that pulled in. But, as I said, I’ll let it hang for a little while … mabe I’ll 
come up with a different solution.


I can imagine such a total-platform-pom might be useful for a different use 
case, but I think for building apps, users will be better off declaring fine 
grained dependencies. Then when it comes time for them to create some minimal 
“bundle” needed to get their app&dependencies to a “device” where it can be 
run, mvn can help pull together only the needed pieces.

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

The examples are something I’m currently working on … 

They will work differently than right now (definitely no classpath in the 
Manifest).

In Maven there are several ways to do this … I’ll have to dig into how the 
samples are run to find out how to do it in an ideal way.

I have also seen the current way things are bundled in the binaries … I also 
think changing this would be a good idea as the way it is handled now forces 
the directory structure on the users. I wouldn’t like to do that. I would much 
more prefer a directory with edgent artifacts and an “ext” directory containing 
external dependencies and passing in an individual classpath to the 
applications. … but as I said … I’ll have to dig a little more into the 
examples. First I’ll finish the android modules.

Chris

Am 21.06.17, 15:21 schrieb "Dale LaBossiere" :

Today (pre-mvn), Edgent has a number of sample programs under /samples.  
They are pre-built and included in the binary release bundle as a collection of 
jars edgent.samples.{apps,…}.jar and are run via scripts under /scripts.

Your thoughts on samples in the new world? As things are now, the scripts 
don’t work. They presume a (binary release) directory tree where all of the 
edgent jars reside in a particular layout.  They also use the 
non-version-labeled-name form of the Edgent jars.

At a high level there's “how do I build and package/assemble my app for 
execution on a device, and run my app”.  Edgent, mostly, isn’t in the business 
of how to package the bits or get them to / install them on the device.

Today’s binary release bundle only addresses the “run” part of that.  The 
binary bundle includes the sample sources but doesn’t include tooling/scripts 
to build them. How lame is that?  :-)

In my mind it’s NOT a hard requirement that we (a) provide pre-built 
samples, nor (b) provide an Edgent binary release bundle (containing all the 
Edgent jars and deps).  I think it’s just the opposite :-)

I’ll stop there just to kickoff this thread.

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Hi,

Just a little on what I’m currently working on. 
The maven dependency plugin is able to create an xml representation of a 
projects dependencies. I’m currently trying to create a build that pipes that 
through an XSLT and hereby creates a start-script for the example they are 
built in. This will refer to a directory structure in which all Edgent 
artifacts are output into one directory and all non-edgent dependencies to an 
“ext” subdirectory within that lib directory. The scripts should be able to 
correctly build an applications classpath based on that.

But this weekend a lot more is going on than the last ones, so I will probably 
work on this next week.

Chris




Am 22.06.17, 15:00 schrieb "Dale LaBossiere" :

Got a couple of pointers to such projects for examples - particularly mvn 
based ones? I’ve been through 10 pages of https://github.com/apache without any 
luck :-)  Maybe only 1 proj that even had a separate repo 
(https://github.com/apache/geode-examples).

> On Jun 21, 2017, at 10:00 AM, John D. Ament  wrote:
> 
> I personally like seeing samples/examples in their own repo.  They avoid
> issues with releases that may not be perfect and may bring in other
> dependencies.

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

o the device and unpack it.
  The app is started via “./run-app.sh”

- a twist on the former where just the Edgent jars needed by the app are 
copied to a local dir and bundled up
  This leverages mvn to create the ~minimal set of Edgent jars and their 
deps needed by the app.
  The dir contains what “mvn dependency:copy-dependency” creates.
  This seems to be the same thing as created by Eclipse> 
Export … > Runnable JAR > “copy required jars into sub-folder”.
  (Note, unlike that Eclipse export, the app’s jar would NOT contain a 
manifest classpath.  The edgent-classpath.sh scheme would be used.)
  The dir can be bundled up and brought to the device and unpacked.

Hope that’s all mostly making sense / sensible :-)

— Dale

> On Jun 23, 2017, at 3:39 AM, Christofer Dutz  
wrote:
> ...Just a little on what I’m currently working on. 
> The maven dependency plugin is able to create an xml representation of a 
projects dependencies. I’m currently trying to create a build that pipes that 
through an XSLT and hereby creates a start-script for the example they are 
built in. This will refer to a directory structure in which all Edgent 
artifacts are output into one directory and all non-edgent dependencies to an 
“ext” subdirectory within that lib directory. The scripts should be able to 
correctly build an applications classpath based on that.

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Hi Dale,

Well it would be possible to reduce the classes in the uber-jars to only the 
used classes, but this usually causes issues as soon as there’s reflection in 
the software somewhere.

Right now, I only concentrated on building the samples not the distribution. 
That was next on my list. For people using an (let’s call it an Edgent 
installation) would simply add a reference to the lib and lib/ext directories 
to their applications classpath and be done.

Yes … the XSL stuff was for my start-script generation approach … I deactivated 
that for now, giving the uber-jar a try. If we stick to that, I’ll remove that 
completely.

To help people getting started, I would rather suggest using Maven archetypes. 
These are special Maven modules which act as templates which you can use to 
automatically setup a new project. In the Flex project, I created several 
archetypes to setup libraries, applications, even servers running on Spring 
boot. All a user must do is run the archetype maven plugin and pass in the 
archetype module he wants to use, and the plugin creates a new project for him. 

I could move the uber-jar generation config into a separate profile which the 
user must manually activate …

So, on my to-do list now I have noted:
- Move the uber-jar configuration into a profile
- No longer make the main Edgent build also build the examples
- Create an example archetype based on the Hello World example
- Create an assembly for a binary distribution

Correct?

Chris


Am 26.06.17, 23:26 schrieb "Dale LaBossiere" :

Chris, thanks for helping to work through this.

> On Jun 25, 2017, at 9:44 AM, Christofer Dutz  
wrote:
> 
> Hi Dale,
> 
> I just paused my work on the classpath generation and simply added the 
maven-shade-plugin which creates uber-jars, which produces jars that contain 
not only the modules content, but also includes all dependencies into one big 
fat jar … I also added a sample start script to the topology project. 
> 
> Would this do as a startging point?
I pulled down the latest PR changes and built for all platforms.
I can’t seem to eliminate all errors in a Eclipse dev context (CLI builds 
are fine).  I’ll follow up with that separately.

> 
> Regarding your points:
> First section:
> - Well it now produces an additional self-contained jar additionally
Observation: the uber-jar contains the extracted dependent component/jar 
contents as opposed to containing the actual dependent jars.  Any benefits of 
one form over there other?
There’s some “xsl” stuff in the samples pom - part of your paused classpath 
generation work that should be removed?

> - When using Maven you usually don’t use this model of having something 
installed, it is possible using the “system” scope in dependencies and 
providing the systemPath element pointing to some 
“${edgent.home}/my-edgent-lib-1.2.0.jar” but that wouldn’t be good style. 
Usually the single-source for the artifacts would be the maven local repo on 
that device.
I’m confident I don’t know the various ways in which organizations will 
want to produce edge devices that contain Edgent-based apps :-)  So I’m 
hesitant to constrain things / require that maven technology be resident on the 
device.  That’s what’s motivates me to think we need to ~easily support 
“install” based cases… and I’m not confident that supplying “uber-jar” as the 
only initial alternative will be OK.

I guess if we also provide a way to construct a dir(s) containing all of 
the edgent jars dependencies (not just a single sample’s dependencies yet) then 
maybe that and the uber-jar alternative will be good enough to start.  At least 
the user won’t have to scratch their head on how to get the jars out of the 
local repo if they really need them.

> Second section:
> - The shade plugin would work on all platforms. All I would have to do, 
is to add the plugin configuration to the other platforms.
> - Having jars for each example would require splitting them up into 
separate maven modules. I don’t think that’s a good idea. I’d prefer a start 
script that lets the user choose the sample to run. I know this would require 
adjusting the start script whenever a sample is added/deleted/changed, but it 
would reduce the number of these super-fat uber jars.
At one level I appreciate what you’re saying.  Maybe theres a need for 
another type of sample that differs from this collection?  Let me try to 
clarify…

At a high level it feels like we need at least one sample project that we 
can say “copy this sample project (source, pom, scripts if any) to jump-start 
the development of your Edgent-based app”.  Do you think that what you’re 
considering will serve that purpose well?

Let’s fully treat samples like app code that a user would write.
Don't build them as part of edgent-parent, don’t dep

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Regarding the assembly,

The way the build is currently setup, we would be distributing jars via Apaches 
Maven Repo which is synced with Maven Central. So, everyone using maven would 
easily be able to use these. But if someone isn’t using Maven or Ivy with a 
Maven resolver or Gradle with a Maven resolver, they would have to either build 
Edgent locally or manually download all the jars from Maven Central. I don’t 
think people would like that. I know of quite a few that would be used to 
creating an Eclipse project with a libs directory, which they simply copy a set 
of jars into. For that type of user, a binary distribution would be beneficial.

Regarding the Jar with jars inside … seems I could use the assembly plugin for 
that, it’s jar-with-dependencies goal seems to be for creating exactly such a 
thing.
I’ll check that out right away.

Chris


Am 27.06.17, 15:15 schrieb "Dale LaBossiere" :


> On Jun 27, 2017, at 3:24 AM, Christofer Dutz  
wrote:
> 
> Hi Dale,
> 
> Well it would be possible to reduce the classes in the uber-jars to only 
the used classes, but this usually causes issues as soon as there’s reflection 
in the software somewhere.
Sorry for the confusion.  I wasn’t asking for that sort of additional 
uber-included classes filtering (an interesting idea; I seem to recall some 
tool that can do that).  Rather, in Eclipse, " > Export … > Java > 
Runnable JAR > package required libraries into generated JAR" creates a jar 
that includes the actual dependent jars, not their extracted classes.  The jar 
also includes a tiny “Jar in Jar” loader which is the “main-class”.  Seems very 
clean/nice.  Was interested in your opinion on it.

> ...
> To help people getting started, I would rather suggest using Maven 
archetypes. ...
I had a feeling you might suggest that :-)  Works for me.

> I could move the uber-jar generation config into a separate profile which 
the user must manually activate …
Right now I don’t have a strong opinion on that.  I had mentioned it in 
case it made it easier to consider/accept alternative ideas like 
per-sample-projects/poms.  If you still think per-sample-poms are best avoided 
for other reasons then OK, stick with the current structure and uber generation 
scheme.

> So, on my to-do list now I have noted:
> - Move the uber-jar configuration into a profile
see above.  your call.
> - No longer make the main Edgent build also build the examples
sounds good (I guess you agreed with the rationale)
> - Create an example archetype based on the Hello World example
sounds good
> - Create an assembly for a binary distribution
I’m unclear on the full implications of that. I’m now of the, perhaps 
incorrect?, opinion that we should NOT distribute binary distribution tgz 
bundles.  Are we on the same page?  
If this assembly/tgz is just something that someone can run/create if they 
need a way to access the Edgent jars and deps outside of the repo then I get 
that.

I’m imagining a tool that you can point at component in a repo and it will 
do the jar copying/bundling.  No need for any other sources, etc.
e.g., if I identify edgent-parent:1.2.0-SNAPSHOT it creates a bundle with 
all of the jars/deps of that ss/release.  extra credit? - If I identify 
edgent-connectors-mqtt:1.2.0-SNAPSHOT the generated bundle only includes that 
subset of jars/deps.  Does that make sense to you?  There’s nothing edgent 
specific about such a tool; maybe one already exists?

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

Forget the assembly jar-with-dependencies goal … it produces the same output as 
the shade plugin :-(

I think the packaging type you are looking for is like that of Spring Boot 
applications. I’ll try to find out if this is possible without spring boot.

Chris


Am 27.06.17, 15:28 schrieb "Christofer Dutz" :

Regarding the assembly,

The way the build is currently setup, we would be distributing jars via 
Apaches Maven Repo which is synced with Maven Central. So, everyone using maven 
would easily be able to use these. But if someone isn’t using Maven or Ivy with 
a Maven resolver or Gradle with a Maven resolver, they would have to either 
build Edgent locally or manually download all the jars from Maven Central. I 
don’t think people would like that. I know of quite a few that would be used to 
creating an Eclipse project with a libs directory, which they simply copy a set 
of jars into. For that type of user, a binary distribution would be beneficial.

Regarding the Jar with jars inside … seems I could use the assembly plugin 
for that, it’s jar-with-dependencies goal seems to be for creating exactly such 
a thing.
I’ll check that out right away.

Chris


Am 27.06.17, 15:15 schrieb "Dale LaBossiere" :


> On Jun 27, 2017, at 3:24 AM, Christofer Dutz 
 wrote:
> 
> Hi Dale,
> 
> Well it would be possible to reduce the classes in the uber-jars to 
only the used classes, but this usually causes issues as soon as there’s 
reflection in the software somewhere.
Sorry for the confusion.  I wasn’t asking for that sort of additional 
uber-included classes filtering (an interesting idea; I seem to recall some 
tool that can do that).  Rather, in Eclipse, " > Export … > Java > 
Runnable JAR > package required libraries into generated JAR" creates a jar 
that includes the actual dependent jars, not their extracted classes.  The jar 
also includes a tiny “Jar in Jar” loader which is the “main-class”.  Seems very 
clean/nice.  Was interested in your opinion on it.

> ...
> To help people getting started, I would rather suggest using Maven 
archetypes. ...
I had a feeling you might suggest that :-)  Works for me.

> I could move the uber-jar generation config into a separate profile 
which the user must manually activate …
Right now I don’t have a strong opinion on that.  I had mentioned it in 
case it made it easier to consider/accept alternative ideas like 
per-sample-projects/poms.  If you still think per-sample-poms are best avoided 
for other reasons then OK, stick with the current structure and uber generation 
scheme.

> So, on my to-do list now I have noted:
> - Move the uber-jar configuration into a profile
see above.  your call.
> - No longer make the main Edgent build also build the examples
sounds good (I guess you agreed with the rationale)
> - Create an example archetype based on the Hello World example
sounds good
> - Create an assembly for a binary distribution
I’m unclear on the full implications of that. I’m now of the, perhaps 
incorrect?, opinion that we should NOT distribute binary distribution tgz 
bundles.  Are we on the same page?  
If this assembly/tgz is just something that someone can run/create if 
they need a way to access the Edgent jars and deps outside of the repo then I 
get that.

I’m imagining a tool that you can point at component in a repo and it 
will do the jar copying/bundling.  No need for any other sources, etc.
e.g., if I identify edgent-parent:1.2.0-SNAPSHOT it creates a bundle 
with all of the jars/deps of that ss/release.  extra credit? - If I identify 
edgent-connectors-mqtt:1.2.0-SNAPSHOT the generated bundle only includes that 
subset of jars/deps.  Does that make sense to you?  There’s nothing edgent 
specific about such a tool; maybe one already exists?

— Dale

Re: mvn, Edgent samples, and the broader app development and packaging topic

[Christofer Dutz]

That only seems to work for deploying things and not for downloading them :-(

Chris

Am 27.06.17, 17:01 schrieb "Dale LaBossiere" :

A brief search turned up 
https://github.com/simpligility/maven-repository-tools/tree/master/maven-repository-provisioner
 
<https://github.com/simpligility/maven-repository-tools/tree/master/maven-repository-provisioner>
I cursory look at its README makes it sound almost exactly like what I was 
hoping for… except it looks like it may only copy out to another repo and not 
offer an “copy out to arbitrary dir” sort of mode :-)  I’ll have to look more 
closely.

> On Jun 27, 2017, at 10:51 AM, Dale LaBossiere  
wrote:
> 
> 
>> On Jun 27, 2017, at 9:28 AM, Christofer Dutz  
wrote:
>> 
>> Regarding the assembly,
>> 
>> The way the build is currently setup, we would be distributing jars via 
Apaches Maven Repo which is synced with Maven Central. So, everyone using maven 
would easily be able to use these. But if someone isn’t using Maven or Ivy with 
a Maven resolver or Gradle with a Maven resolver, they would have to either 
build Edgent locally or manually download all the jars from Maven Central. I 
don’t think people would like that. I know of quite a few that would be used to 
creating an Eclipse project with a libs directory, which they simply copy a set 
of jars into. For that type of user, a binary distribution would be beneficial.
> 
> Yes, I completely understand that overall use case / need.  I’d just like 
to address it without actually distributing a composite binary bundle :-)
> Instead I’d like to just give / tell them a way to be able to just easily 
“get” the jars/deps that we will already be distributing in repos.
> I definitely don’t want such a user to have to resort to building Edgent 
just to get their hands on the jars.
> 
> I'm surprised there isn’t already such a “get-from-repo [—recursive] 
component-coordinate” tool.
> 
> — Dale

Progress on Maven?

[Christofer Dutz]

Hi,

Reporting back from my recreational 2-week break. Just in the process on 
picking up the ball again … I remember some email about progress on the maven 
distribution. What’s the status of this? Otherwise I would start work on the 
distribution assembly.

Chris

Re: Progress on Maven?

[Christofer Dutz]

Hi Guys,

As I had a little thinking-time here, I implemented a first version of a 
distribution module. 
You need to enable the “distribution” profile to build it with the rest (I 
thought probably not everyone wants the distribution built on every build).
Right now, it produces a zip, tar.gz and tar.bz2 archive.

I haven’t included the console, examples etc. as I didn’t know if we want to 
keep the examples in the repo. 

Chris


Am 10.07.17, 10:17 schrieb "Christofer Dutz" :

Hi,

Reporting back from my recreational 2-week break. Just in the process on 
picking up the ball again … I remember some email about progress on the maven 
distribution. What’s the status of this? Otherwise I would start work on the 
distribution assembly.

Chris

Re: Progress on Maven?

[Christofer Dutz]

Hi Dale,

Can I treat the last part with the table as a ToDo list?
TODO Lists sort of are the thing I work with best ;-)

Chris


Am 10.07.17, 23:52 schrieb "Dale LaBossiere" :

BTW, I’ve been working to cleanup / record items in

https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 


— Dale

Re: Progress on Maven?

[Christofer Dutz]

Hi Dale,

Seems the script was stripped from the email … eventually use 
https://paste.apache.org/ for that.

Chris

Von: Dale LaBossiere 
Antworten an: "dev@edgent.apache.org" 
Datum: Montag, 10. Juli 2017 um 17:20
An: "dev@edgent.apache.org" 
Betreff: Re: Progress on Maven?

needed

Re: Progress on Maven?

[Christofer Dutz]

Hi,

Ok … so I updated the wiki page and just pushed another update to the pull 
request, in which I cleaned up the artifacts and directories as wished in the 
bottom table of the wiki page.

Also, I had to fix one or two problems. Now the tests and the apache-release 
profile seem to be running fine in the java7 and android platforms.

I also added distributions to the two additional platforms.

Chris




Am 11.07.17, 08:55 schrieb "Christofer Dutz" :

Hi Dale,

Can I treat the last part with the table as a ToDo list?
TODO Lists sort of are the thing I work with best ;-)

Chris


Am 10.07.17, 23:52 schrieb "Dale LaBossiere" :

BTW, I’ve been working to cleanup / record items in

https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>

— Dale

Re: Progress on Maven?

[Christofer Dutz]

Ups … that’s already done and finished … at least in the PR :-)

I didn’t like the old names as they only confused me…

Chris

Am 11.07.17, 16:41 schrieb "Dale LaBossiere" :

Regarding the table and web socket connector, unless there’s discussion 
needed, always welcome, please rename the generated artifacts as indicated.  
Thanks!

— Dale

> On Jul 11, 2017, at 2:55 AM, Christofer Dutz  
wrote:
> 
> Hi Dale,
> 
> Can I treat the last part with the table as a ToDo list?
> TODO Lists sort of are the thing I work with best ;-)
> 
> Chris
> 
> 
> Am 10.07.17, 23:52 schrieb "Dale LaBossiere" :
> 
>BTW, I’ve been working to cleanup / record items in
> 
>https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>
> 
>— Dale
> 

Re: Progress on Maven?

[Christofer Dutz]

Hmpf … 

seems I can’t build all on Travis as the job is killed after 4MB, which seems 
to be something about half way through the build … any Idea how to extend that 
limit?

Chris

Am 11.07.17, 16:44 schrieb "Dale LaBossiere" :

I’ll check the updates, pull down the latest, etc.  Thanks!
— Dale

> On Jul 11, 2017, at 9:51 AM, Christofer Dutz  
wrote:
> 
> Hi,
> 
> Ok … so I updated the wiki page and just pushed another update to the 
pull request, in which I cleaned up the artifacts and directories as wished in 
the bottom table of the wiki page.
> 
> Also, I had to fix one or two problems. Now the tests and the 
apache-release profile seem to be running fine in the java7 and android 
platforms.
> 
> I also added distributions to the two additional platforms.
> 
> Chris
> 
> 
> 
> 
> Am 11.07.17, 08:55 schrieb "Christofer Dutz" :
> 
>Hi Dale,
> 
>Can I treat the last part with the table as a ToDo list?
>TODO Lists sort of are the thing I work with best ;-)
> 
>Chris
> 
> 
>Am 10.07.17, 23:52 schrieb "Dale LaBossiere" :
> 
>BTW, I’ve been working to cleanup / record items in
> 
>https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>
> 
>— Dale
> 
> 
> 

Re: Progress on Maven?

[Christofer Dutz]

Sounds good for me :-)

I just added "the works" for the sake of completeness. If we had an ASF Jenkins 
build doing the full thing, I guess a Travis build just doing the j8 build + 
unit-tests + integration-tests that would be enough and should be below the 4mb 
limit.

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 11.07.17 21:43 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Progress on Maven?

No idea from me.  Any way to reduce the mvn output verbosity?

Assuming that we setup Jenkins for nightly/snapshots and that does full 
j8/j7/android building/testing,
I’m content to reduce what’s being done in travis for PR validation.
e.g., toss android build&test, toss j7 test, toss j7 build if also necessary.
How’s that sound to you?

> On Jul 11, 2017, at 3:01 PM, Christofer Dutz  
> wrote:
>
> Hmpf …
>
> seems I can’t build all on Travis as the job is killed after 4MB, which seems 
> to be something about half way through the build … any Idea how to extend 
> that limit?

dev@edgent.apache.org

[Christofer Dutz]

I'm not demanding the assembly ... It was just the last missing part in the 
migration to Maven. Most ASF projects provide such convenience binary archives, 
but an ASF release is the source only anyway.

I admit that your script would do the job.

Another thing worth looking into might be the Maven wrapper, which would 
eliminate the need to download Maven, as someone not wanting to use Maven would 
have to install Maven in order to NOT use it afterwards ;-) ... Users using 
Maven wouldn't use the script.

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 11.07.17 21:29 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: maven and distribution bundles with transitive deps: licensing¬ice

Hi,

Can’t we eliminate releasing binary bundles containing transitive deps and in 
doing so
eliminate a huge amount of licensing/notice pain for us?  But still supply what 
users need.

As I understand it / ASF policy, the newly generated distribution bundles lack 
the necessary
license/notice info for transitive deps contained in the bundle.

That’s what the binary-release/LICENSE and licenses/binary-release/* was all 
about
(included in the gradle generated binary bundles).
Justin indicated we needed to have full copies of license/notice text not just 
URLs to it
(since the URL may have different text at a later date that doesn’t match that 
text
applicable for the contained deps).

I see the new distribution stuff also generates edgent-distribution-.jar.
Its META-INF/DEPENDENCIES has a nice listing of all the transitive deps
license info but it’s (a) just URLs to licenses, (b) lacks any notice info,
and (c) isn’t included in the generated binary release bundle.
Maybe I just don’t understand how this is supposed to work.

Not releasing a binary bundle also eliminates validation/testing of it.

My hope was that viable / better alternative is to provide an easy way for users
themselves to get the Edgent jars and transitive external deps themselves from
maven-central (or any maven repo).

Presumably we'd just need to tell the user something like:
“This command retrieves the Edgent jars and their transitive external 
dependencies.
The external dependencies come with their own licensing terms that you should 
review.
A summary of the transitive dependencies and their licenses can be found here
.
Continue? ”

That’s what get-edgent-jars.sh (https://paste.apache.org/p/GI0n 
) was working towards.

Is this making sense / compelling / a valid and (sufficiently) user friendly 
approach?

To clarify for all, this tool (equivalently a binary bundle) is only needed by
Edgent users that don’t use maven/maven-repo enabled app development tools
or that don’t use those tools to generate a standalone “über jar” for their 
Edgent app
to deploy to their edge device.

— Dale

Jquery, JqueryUI and D3

[Christofer Dutz]

Hi,

I noticed that the “servlets” module also contains HTML, JavaScript and CSS 
(Eventually better to call it webapp instead of servlets). The JS and CSS seems 
to be mainly copied from Jquery, JqueryUI and D3 distribution … how about not 
checking this code in with our code, but have it downloaded and added as part 
of the build?

One thing I did notice, was a comment about a modified version of these libs - 
without mentioning what the changes were. Does anyone remember what these 
changes were and why they were needed? I personally don’t like copying code of 
other projects and checking that in with another project.

Chris

Re: Jquery, JqueryUI and D3

[Christofer Dutz]

Well as this is the sort of thing I have come across quite often … I just did 
that change locally. 

It does require changing the directory structure in the output a little, but 
those changes make things even cleaner.

Which version of d3 are we using? Jquery was 1.11.2, JqueryUI was 1.11.4, but 
what version is d3?

And regarding the sankey_edgent thing … it seems like this isn’t part of d3, 
but it uses d3 and might contain code copied from somewhere else.
It also seems as if jquery or jquery-ui might contain some legend extension 
copied from somewhere else … but can’t really tell :-(

Chris


Am 12.07.17, 14:13 schrieb "Dale LaBossiere" :

Chris, I agree with your general sentiment.  Don’t know how difficult it 
will be to get there, downloading/bundling only what’s really desired - 
generally we've tried to avoid bloat. If non-trivial, maybe this is something 
we can/should address after the maven conversion (track with a JIRA)?

If console/servlets/README is to be believed, only one of the files 
(sankey_edgent.js) is a modified file from d3.  Presumably one can do a diff 
against the noted original to see what was changed.

Susan Cline, can you clarify?

— Dale

> On Jul 12, 2017, at 7:05 AM, Christofer Dutz  
wrote:
> 
> Hi,
> 
> I noticed that the “servlets” module also contains HTML, JavaScript and 
CSS (Eventually better to call it webapp instead of servlets). The JS and CSS 
seems to be mainly copied from Jquery, JqueryUI and D3 distribution … how about 
not checking this code in with our code, but have it downloaded and added as 
part of the build?
> 
> One thing I did notice, was a comment about a modified version of these 
libs - without mentioning what the changes were. Does anyone remember what 
these changes were and why they were needed? I personally don’t like copying 
code of other projects and checking that in with another project.
> 
> Chris

dev@edgent.apache.org

[Christofer Dutz]

Well the problem is, the wrapper would require us to check in a jar and that’s 
a no-go for an Apache release.

There seems to be long-going discussions on the maven-wrapper page about such a 
no-deps update. Right now, I think the most promising solution would be to 
extend the mvnw scripts to check if the jar is present and if it’s not, to 
compile a single java class and run it locally. This would then download the 
jar and place it where the wrapper expects it to be. In that case only sources 
would be checked in and the only dependency needed would be to have a JDKs bin 
directory on the systems path (Which is needed for almost everything else 
anyway).

Unfortunately, I am totally at war with platform (independent) shell-scripts. 
Sort of have the same love for them as for regular expressions ;-)

Volunteers to look into this?

Chris


Am 12.07.17, 14:34 schrieb "Dale LaBossiere" :

The wrapper sounds like a win (we currently leverage the gradle wrapper to 
ease the pain).

I’m continuing to tweak the script for j7/android.

> On Jul 11, 2017, at 5:07 PM, Christofer Dutz  
wrote:
> 
> I'm not demanding the assembly ... It was just the last missing part in 
the migration to Maven. Most ASF projects provide such convenience binary 
archives, but an ASF release is the source only anyway.
> 
> I admit that your script would do the job.
> 
> Another thing worth looking into might be the Maven wrapper, which would 
eliminate the need to download Maven, as someone not wanting to use Maven would 
have to install Maven in order to NOT use it afterwards ;-) ... Users using 
Maven wouldn't use the script.
> 
> Chris

dev@edgent.apache.org

[Christofer Dutz]

Hi John,

Is this considered a best-practice at Apache? I remember us having a similar 
discussion at the Flex project and we settled with not checking in the jar and 
not using maven wrapper.

Chris

Am 12.07.17, 15:11 schrieb "John D. Ament" :

On Wed, Jul 12, 2017 at 9:09 AM Christofer Dutz 
wrote:

> Well the problem is, the wrapper would require us to check in a jar and
> that’s a no-go for an Apache release.
>

you can check in JARs to the repo.  Just exclude it from the source release.


>
> There seems to be long-going discussions on the maven-wrapper page about
> such a no-deps update. Right now, I think the most promising solution 
would
> be to extend the mvnw scripts to check if the jar is present and if it’s
> not, to compile a single java class and run it locally. This would then
> download the jar and place it where the wrapper expects it to be. In that
> case only sources would be checked in and the only dependency needed would
> be to have a JDKs bin directory on the systems path (Which is needed for
> almost everything else anyway).
>
> Unfortunately, I am totally at war with platform (independent)
> shell-scripts. Sort of have the same love for them as for regular
> expressions ;-)
>
> Volunteers to look into this?
>
> Chris
>
>
> Am 12.07.17, 14:34 schrieb "Dale LaBossiere" :
>
> The wrapper sounds like a win (we currently leverage the gradle
> wrapper to ease the pain).
>
> I’m continuing to tweak the script for j7/android.
>
> > On Jul 11, 2017, at 5:07 PM, Christofer Dutz <
> christofer.d...@c-ware.de> wrote:
> >
> > I'm not demanding the assembly ... It was just the last missing part
> in the migration to Maven. Most ASF projects provide such convenience
> binary archives, but an ASF release is the source only anyway.
> >
> > I admit that your script would do the job.
> >
> > Another thing worth looking into might be the Maven wrapper, which
> would eliminate the need to download Maven, as someone not wanting to use
> Maven would have to install Maven in order to NOT use it afterwards ;-) 
...
> Users using Maven wouldn't use the script.
> >
> > Chris
>
>
>
>

dev@edgent.apache.org

[Christofer Dutz]

Hi Dale,

I just tested your script and it worked perfectly :-)

Regarding your observations:
- You mentioned in another post, that I should exclude any transitive 
dependency of Kafka … The only valid reason I could see to need this, if using 
Edgent in an environment in which Kafka is provided by other means. The Maven 
way to handle such a situation is to mark that dependency “provided” in which 
case it is used during compilation, but is excluded from being pulled in 
automatically … like the servlet-api jars which usually are provided by every 
servlet container.
- I’ll investigate why the animal-sniffer-annotations and alike are included … 
I guess they shouldn’t. The reason for a different Jetty version in Java7, is 
that the version used in java8 is not compatible with java7, so I had to go 
back to the newest Jetty version that supports Java7.

Just forked the Maven-Wrapper project and will start working on a version of it 
that works without a checked in jar. Any other solution (excluding jar from the 
release) sounds like a hack and it would break the one requirement of Apache 
releases, that you should be able to download and unpack the release and simply 
run it to reproduce the same results. I guess the changes to the wrapper needed 
here are minimal and if we had them all Apache projects could easily use that 
versions without any problems.

Chris


Am 12.07.17, 19:37 schrieb "Dale LaBossiere" :

Chris,

I’ve uploaded a new version of get-edgent-jars.sh to 
https://paste.apache.org/VuU3 

After you’ve built the j8,j7,android platforms you can do the following to 
get the jars:

sh get-edgent-jars.sh --platform java8 --version 1.2.0-SNAPSHOT
sh get-edgent-jars.sh --platform java7 --version 1.2.0-SNAPSHOT
sh get-edgent-jars.sh --platform android --version 1.2.0-SNAPSHOT

I’m seeing some odd results that I believe are purely a result of what the 
Edgent poms have in them (I think I have your most recent changes).
Comparing what’s collected for j8 vs j7 vs android…

- with the “provided” that was recently added, the kafka_2.10-0.8.2.2.jar 
(and deps) are “missing” from j8,j7,android
- j8 vs j7 ext-jars - there are extra jars in j7 like 
animal-sniffer-annotations-1.14 and others; there are different versions of 
jetty jars
- j8 vs android ext-jars - similar extra jars in android

— Dale

dev@edgent.apache.org

[Christofer Dutz]

First version of a auto-jar-downloading version: 

Am 13.07.17, 08:48 schrieb "Christofer Dutz" :

Hi Dale,

I just tested your script and it worked perfectly :-)

Regarding your observations:
- You mentioned in another post, that I should exclude any transitive 
dependency of Kafka … The only valid reason I could see to need this, if using 
Edgent in an environment in which Kafka is provided by other means. The Maven 
way to handle such a situation is to mark that dependency “provided” in which 
case it is used during compilation, but is excluded from being pulled in 
automatically … like the servlet-api jars which usually are provided by every 
servlet container.
- I’ll investigate why the animal-sniffer-annotations and alike are 
included … I guess they shouldn’t. The reason for a different Jetty version in 
Java7, is that the version used in java8 is not compatible with java7, so I had 
to go back to the newest Jetty version that supports Java7.

Just forked the Maven-Wrapper project and will start working on a version 
of it that works without a checked in jar. Any other solution (excluding jar 
from the release) sounds like a hack and it would break the one requirement of 
Apache releases, that you should be able to download and unpack the release and 
simply run it to reproduce the same results. I guess the changes to the wrapper 
needed here are minimal and if we had them all Apache projects could easily use 
that versions without any problems.

Chris


Am 12.07.17, 19:37 schrieb "Dale LaBossiere" :

Chris,

I’ve uploaded a new version of get-edgent-jars.sh to 
https://paste.apache.org/VuU3 <https://paste.apache.org/VuU3>

After you’ve built the j8,j7,android platforms you can do the following 
to get the jars:

sh get-edgent-jars.sh --platform java8 --version 1.2.0-SNAPSHOT
sh get-edgent-jars.sh --platform java7 --version 1.2.0-SNAPSHOT
sh get-edgent-jars.sh --platform android --version 1.2.0-SNAPSHOT

I’m seeing some odd results that I believe are purely a result of what 
the Edgent poms have in them (I think I have your most recent changes).
Comparing what’s collected for j8 vs j7 vs android…

- with the “provided” that was recently added, the 
kafka_2.10-0.8.2.2.jar (and deps) are “missing” from j8,j7,android
- j8 vs j7 ext-jars - there are extra jars in j7 like 
animal-sniffer-annotations-1.14 and others; there are different versions of 
jetty jars
- j8 vs android ext-jars - similar extra jars in android

— Dale

dev@edgent.apache.org

[Christofer Dutz]

Ok ... that wen’t away too fast …

Here’s the link:
https://github.com/chrisdutz/maven-wrapper/commit/b0487654b032ee59446919f22c01c2235d8c4e93

Hopefully my colleagues will help me make it solid and perfect :-)

Chris

Am 13.07.17, 08:48 schrieb "Christofer Dutz" :

Hi Dale,

I just tested your script and it worked perfectly :-)

Regarding your observations:
- You mentioned in another post, that I should exclude any transitive 
dependency of Kafka … The only valid reason I could see to need this, if using 
Edgent in an environment in which Kafka is provided by other means. The Maven 
way to handle such a situation is to mark that dependency “provided” in which 
case it is used during compilation, but is excluded from being pulled in 
automatically … like the servlet-api jars which usually are provided by every 
servlet container.
- I’ll investigate why the animal-sniffer-annotations and alike are 
included … I guess they shouldn’t. The reason for a different Jetty version in 
Java7, is that the version used in java8 is not compatible with java7, so I had 
to go back to the newest Jetty version that supports Java7.

Just forked the Maven-Wrapper project and will start working on a version 
of it that works without a checked in jar. Any other solution (excluding jar 
from the release) sounds like a hack and it would break the one requirement of 
Apache releases, that you should be able to download and unpack the release and 
simply run it to reproduce the same results. I guess the changes to the wrapper 
needed here are minimal and if we had them all Apache projects could easily use 
that versions without any problems.

Chris


Am 12.07.17, 19:37 schrieb "Dale LaBossiere" :

Chris,

I’ve uploaded a new version of get-edgent-jars.sh to 
https://paste.apache.org/VuU3 <https://paste.apache.org/VuU3>

After you’ve built the j8,j7,android platforms you can do the following 
to get the jars:

sh get-edgent-jars.sh --platform java8 --version 1.2.0-SNAPSHOT
sh get-edgent-jars.sh --platform java7 --version 1.2.0-SNAPSHOT
sh get-edgent-jars.sh --platform android --version 1.2.0-SNAPSHOT

I’m seeing some odd results that I believe are purely a result of what 
the Edgent poms have in them (I think I have your most recent changes).
Comparing what’s collected for j8 vs j7 vs android…

- with the “provided” that was recently added, the 
kafka_2.10-0.8.2.2.jar (and deps) are “missing” from j8,j7,android
- j8 vs j7 ext-jars - there are extra jars in j7 like 
animal-sniffer-annotations-1.14 and others; there are different versions of 
jetty jars
- j8 vs android ext-jars - similar extra jars in android

— Dale

Re: Jquery, JqueryUI and D3

[Christofer Dutz]

Hi Susan,

I already had done that locally, I just pushed the changes to my pull-request. 
Would be super-awesome, if you could verify I didn’t break anything.
I did run the application and after tweaking a few things in the js and html, I 
am no longer getting any errors or warnings in the browser.

You can easily start the edgent server by building edgent with maven and then 
running

mvn tomcat:run-war 

in the servlets directory. This would automatically fire up a tomcat and run 
the application in there.
I downgraded the Sankey version to 1.1.0 as this was very similar to the 
version we use in edgent.

Chris



Am 13.07.17, 23:02 schrieb "Susan Cline" :

Hi,

The README under the console/servlets can be believed!

I just did a diff between the current version of sankey.js here: 
https://github.com/d3/d3-sankey/tree/master/src 
<https://github.com/d3/d3-sankey/tree/master/src>  and the sankey_edgent.js 
file under console/servlets.

There are many changes, but I believe most of them are due to the continued 
development of sankey.js.  It was updated as recently as about a month ago.
I don’t recall if I made any substantial changes when I originally worked 
on this - I may have just removed functions that I was not using.

With that said if we modify the build to use sankey.js I volunteer to test 
it with the console.

Susan

> On Jul 12, 2017, at 5:13 AM, Dale LaBossiere  wrote:
> 
> Chris, I agree with your general sentiment.  Don’t know how difficult it 
will be to get there, downloading/bundling only what’s really desired - 
generally we've tried to avoid bloat. If non-trivial, maybe this is something 
we can/should address after the maven conversion (track with a JIRA)?
> 
> If console/servlets/README is to be believed, only one of the files 
(sankey_edgent.js) is a modified file from d3.  Presumably one can do a diff 
against the noted original to see what was changed.
> 
> Susan Cline, can you clarify?
> 
> — Dale
> 
>> On Jul 12, 2017, at 7:05 AM, Christofer Dutz  
wrote:
>> 
>> Hi,
>> 
>> I noticed that the “servlets” module also contains HTML, JavaScript and 
CSS (Eventually better to call it webapp instead of servlets). The JS and CSS 
seems to be mainly copied from Jquery, JqueryUI and D3 distribution … how about 
not checking this code in with our code, but have it downloaded and added as 
part of the build?
>> 
>> One thing I did notice, was a comment about a modified version of these 
libs - without mentioning what the changes were. Does anyone remember what 
these changes were and why they were needed? I personally don’t like copying 
code of other projects and checking that in with another project.
>> 
>> Chris
> 

What's left to do for Maven migration?

[Christofer Dutz]

Hi guys,

So right now, I sort of lost track of what’s still left to do on your wish list 
for a successful maven migration.
If someone could compile a list of things to do, I would gladly work on those 
issues. Must admit that I lost track a little on the confluence page.

Chris

Re: What's left to do for Maven migration?

[Christofer Dutz]

By the way … my pull request for the maven-wrapper is currently being finalized 
… hopefully this will be finished soon and then it will make things even easier 
;-)
https://github.com/takari/maven-wrapper/pull/60

Chris

Am 17.07.17, 16:03 schrieb "Dale LaBossiere" :

Sorry for that confusion.  There are so many details to track / deal with.

The Issues / TODOs in [1] all need to be reviewed and need resolutions.  
Can we just work from that? (marking done items as such, including the 
resolution, and then just doing a strikethrough it the resolved item)

Right now, I think dealing with the binary release bundle and samples are 
the highest priority / largest unknowns.

Thanks for all your continued diligence!

— Dale

> On Jul 17, 2017, at 2:43 AM, Christofer Dutz  
wrote:
> 
> Hi guys,
> 
> So right now, I sort of lost track of what’s still left to do on your 
wish list for a successful maven migration.
> If someone could compile a list of things to do, I would gladly work on 
those issues. Must admit that I lost track a little on the confluence page.
> 
> Chris

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi,

I just pushed a change that includes my improved jar-free version of the 
maven-wrapper that should be 100% compliant with Apache Release rules.
It’s currently the exact same version I submitted as pull-request for the 
maven-wrapper project, but as the scripts are duplicated and checked in anyway, 
I thought I’d just go ahead and add them to Edgent.
My first tests were perfect :-)

So now, if you checked out Edgent and have JAVA_HOME set all you need to do, is 
run: 

./mvnw clean install

and it will download the maven version, install it and use it. So you can now 
reduce the requirements to having Java 8 Installed.

One thing I noticed today – as I’m currently setting up my new laptop – is that 
it’s no longer trivial to get a Java 7 JDK. 
I will try to figure out how to setup the toolchain to support building Java 7 
with only Java 8 in the next few days … hopefully it will be as easy as 
defining a java 7 JDK which points to the Java 8 version.

Chris



Am 19.07.17, 11:13 schrieb "Christofer Dutz" :

By the way … my pull request for the maven-wrapper is currently being 
finalized … hopefully this will be finished soon and then it will make things 
even easier ;-)
https://github.com/takari/maven-wrapper/pull/60

Chris

Am 17.07.17, 16:03 schrieb "Dale LaBossiere" :

Sorry for that confusion.  There are so many details to track / deal 
with.

The Issues / TODOs in [1] all need to be reviewed and need resolutions. 
 Can we just work from that? (marking done items as such, including the 
resolution, and then just doing a strikethrough it the resolved item)

Right now, I think dealing with the binary release bundle and samples 
are the highest priority / largest unknowns.

Thanks for all your continued diligence!

— Dale

> On Jul 17, 2017, at 2:43 AM, Christofer Dutz 
 wrote:
> 
> Hi guys,
> 
> So right now, I sort of lost track of what’s still left to do on your 
wish list for a successful maven migration.
> If someone could compile a list of things to do, I would gladly work 
on those issues. Must admit that I lost track a little on the confluence page.
> 
> Chris

Re: maven: DevelopmentProvider console not working

[Christofer Dutz]

I’ll look into this next week.

Right now I just added some really small changes:
1. Moved the toolchain support (Testing Java7 with Java7) into a separate 
Profile as I think It’s a bad idea to require Java 7 for building. Having just 
setup my new Mac I had to notice, that it is no longer as easy to obtain a Java 
7 SDK as it was in the past. This way everything can be built with Java 8 and 
the results should be the same, but to be on the safe side we can enable native 
Java 7 testing at will by enabling the “toolchain” profile
2. I started re-writing the DEVELOPMENT.md and some of the other text files to 
reflect latest changes.

Haven’t committed those changes however … just wanted to let you know I’m on it 
;-)

Chris



Am 31.07.17, 22:23 schrieb "Dale LaBossiere" :

Hi Chris,

With the latest PR-309 content (31July) the Edgent console isn’t working.

In case it wasn’t previously understood, the Edgent app developer/user must 
not be required to do anything special for the console to work.
i.e., the app should only need to declare a dependency on the 
DevelopmentProvider and it’s embedded jetty server / app should “just work”.

Add an slf4j runtime implementation dependency to the sample/pom.xml in 
order to see problems that are encountered on startup.


  
  org.slf4j
  slf4j-jdk14
  1.7.12
  runtime


Then run the DevelopmentSample app from say Eclipse (same thing happens if 
an uber-jar is generated and used from the cmd line):

Jul 31, 2017 4:07:27 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @500ms
Jul 31, 2017 4:07:27 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.3.6.v20151106
Jul 31, 2017 4:07:28 PM org.eclipse.jetty.server.handler.ContextHandler 
doStart
INFO: Started o.e.j.s.ServletContextHandler@5e5792a0{/jobs,null,AVAILABLE}
Jul 31, 2017 4:07:28 PM org.eclipse.jetty.server.handler.ContextHandler 
doStart
INFO: Started 
o.e.j.s.ServletContextHandler@26653222{/metrics,null,AVAILABLE}
Jul 31, 2017 4:07:28 PM org.eclipse.jetty.webapp.WebInfConfiguration 
getCanonicalNameForWebAppTmpDir
WARNING: Can't generate resourceBase as part of webapp tmp dir name: 
java.lang.IllegalStateException: No resourceBase or war set for context
Jul 31, 2017 4:07:28 PM org.eclipse.jetty.webapp.WebAppContext doStart
WARNING: Failed startup of context 
o.e.j.w.WebAppContext@3532ec19{/console,null,null}
java.lang.IllegalStateException: No resourceBase or war set for context
at 
org.eclipse.jetty.webapp.WebInfConfiguration.unpack(WebInfConfiguration.java:406)
at 
org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure(WebInfConfiguration.java:72)
at 
org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:480)
at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:516)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.server.Server.start(Server.java:405)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:372)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.apache.edgent.console.server.HttpServer.startServer(HttpServer.java:108)
at 
org.apache.edgent.providers.development.DevelopmentProvider.(DevelopmentProvider.java:95)
at 
org.apache.edgent.samples.topology.DevelopmentSample.main(DevelopmentSample.java:33)

Jul 31, 2017 4:07:28 PM org.eclipse.jetty.server.AbstractConnector doStart
INFO: Started ServerConnector@4f970963{HTTP/1.1,[http/1.1]}{0.0.0.0:59881}
Jul 31, 2017 4:07:28 PM org.eclipse.jetty.server.Server doStart
INFO: Started @820ms
http://localhost:59881/console
.



With that I guess it’s no surprise that opening that URL yields:

HTTP ERROR: 503

Problem accessing /console/. Reason:

Service Unavailable

Powered by Jetty:// 9.3.6.v20151106 

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi all,

I just pulled in Dales changes to my forks branch. I like excluding the 
examples from the core build. However there is one problem as the test/svt 
project has a test dependency on the samples/apps project. If this is excluded, 
the build will probably fail.
I would suggest to adjust the test to not rely on a sample. Hereby I could 
remove the top most issue in the “problems” document.

Should we leave everything the way it currently is, or should I create a 
feature/maven branch in the Edgent repo? I’m fine with both options. If anyone 
else needs write access to my fork, just send me an email. 

Chris


Am 23.07.17, 20:05 schrieb "Christofer Dutz" :

Hi,

I just pushed a change that includes my improved jar-free version of the 
maven-wrapper that should be 100% compliant with Apache Release rules.
It’s currently the exact same version I submitted as pull-request for the 
maven-wrapper project, but as the scripts are duplicated and checked in anyway, 
I thought I’d just go ahead and add them to Edgent.
My first tests were perfect :-)

So now, if you checked out Edgent and have JAVA_HOME set all you need to 
do, is run: 

./mvnw clean install

and it will download the maven version, install it and use it. So you can 
now reduce the requirements to having Java 8 Installed.

One thing I noticed today – as I’m currently setting up my new laptop – is 
that it’s no longer trivial to get a Java 7 JDK. 
I will try to figure out how to setup the toolchain to support building 
Java 7 with only Java 8 in the next few days … hopefully it will be as easy as 
defining a java 7 JDK which points to the Java 8 version.

Chris



Am 19.07.17, 11:13 schrieb "Christofer Dutz" :

By the way … my pull request for the maven-wrapper is currently being 
finalized … hopefully this will be finished soon and then it will make things 
even easier ;-)
https://github.com/takari/maven-wrapper/pull/60

Chris

Am 17.07.17, 16:03 schrieb "Dale LaBossiere" :

Sorry for that confusion.  There are so many details to track / 
deal with.

The Issues / TODOs in [1] all need to be reviewed and need 
resolutions.  Can we just work from that? (marking done items as such, 
including the resolution, and then just doing a strikethrough it the resolved 
item)

Right now, I think dealing with the binary release bundle and 
samples are the highest priority / largest unknowns.

Thanks for all your continued diligence!

— Dale

> On Jul 17, 2017, at 2:43 AM, Christofer Dutz 
 wrote:
> 
> Hi guys,
> 
> So right now, I sort of lost track of what’s still left to do on 
your wish list for a successful maven migration.
> If someone could compile a list of things to do, I would gladly 
work on those issues. Must admit that I lost track a little on the confluence 
page.
> 
> Chris

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi Dale,

great you’re looking into this issue … I would have to work myself into the 
topic a little more in order to address that.

Regarding the samples issues: I would strongly suggest to request a separate 
GIT repo for the samples. While it is possible to keep them in there, there are 
a lot of issues that have to be dealt with this way.
First of all you have to exclude stuff from rat (as you have seen), then you 
have to exclude stuff from the releases (as you have seen too), but probably 
the most annoying thing is dealing with releasing in GIT.
Having mixed repos, we would have several tags in one repo reflecting releases 
of Edgent and the samples. While I would treat this fact as “annoying” at most, 
the main problem will be merging the parts that are part of the release back to 
the master branch.

If the repos are separate, all you have to do is merge the tagged release 
revision back to master and all is good. In case of a mixed repo, you will have 
to do a lot of manual merging and cherry picking.

So I would opt for splitting up the repos and creating nicely separated build 
configs for both.

Repos are cheap at the ASF :-)

Chris




Am 08.08.17, 15:59 schrieb "Dale LaBossiere" :

That explains the failure in the SVT test in travis.  Ugh.  :-(

I’ll look into it.  By the end of the day I’ll either fix it or temporarily 
disable the SVT test (and add a tracking item to the wiki page).

As I noted in the PR, the top-level pom.xml has comments (3?) related to 
the handling of the samples project.  When you get a chance could you look at 
those and perhaps identify what needs to be done to address them?  Thanks!

— Dale


> On Aug 8, 2017, at 9:36 AM, Christofer Dutz  
wrote:
> 
> Hi all,
> 
> I just pulled in Dales changes to my forks branch. I like excluding the 
examples from the core build. However there is one problem as the test/svt 
project has a test dependency on the samples/apps project. If this is excluded, 
the build will probably fail.
> I would suggest to adjust the test to not rely on a sample. Hereby I 
could remove the top most issue in the “problems” document.
> 
> Should we leave everything the way it currently is, or should I create a 
feature/maven branch in the Edgent repo? I’m fine with both options. If anyone 
else needs write access to my fork, just send me an email. 
> 
> Chris
> 
> 
    > Am 23.07.17, 20:05 schrieb "Christofer Dutz" :
> 
>Hi,
> 
>I just pushed a change that includes my improved jar-free version of 
the maven-wrapper that should be 100% compliant with Apache Release rules.
>It’s currently the exact same version I submitted as pull-request for 
the maven-wrapper project, but as the scripts are duplicated and checked in 
anyway, I thought I’d just go ahead and add them to Edgent.
>My first tests were perfect :-)
> 
>So now, if you checked out Edgent and have JAVA_HOME set all you need 
to do, is run: 
> 
>./mvnw clean install
> 
>and it will download the maven version, install it and use it. So you 
can now reduce the requirements to having Java 8 Installed.
> 
>One thing I noticed today – as I’m currently setting up my new laptop 
– is that it’s no longer trivial to get a Java 7 JDK. 
>I will try to figure out how to setup the toolchain to support 
building Java 7 with only Java 8 in the next few days … hopefully it will be as 
easy as defining a java 7 JDK which points to the Java 8 version.
> 
>Chris
> 
> 
> 
>Am 19.07.17, 11:13 schrieb "Christofer Dutz" 
:
> 
>By the way … my pull request for the maven-wrapper is currently 
being finalized … hopefully this will be finished soon and then it will make 
things even easier ;-)
>https://github.com/takari/maven-wrapper/pull/60
> 
>Chris
> 
>Am 17.07.17, 16:03 schrieb "Dale LaBossiere" 
:
> 
>Sorry for that confusion.  There are so many details to track 
/ deal with.
> 
>The Issues / TODOs in [1] all need to be reviewed and need 
resolutions.  Can we just work from that? (marking done items as such, 
including the resolution, and then just doing a strikethrough it the resolved 
item)
> 
>Right now, I think dealing with the binary release bundle and 
samples are the highest priority / largest unknowns.
> 
>Thanks for all your continued diligence!
> 
>— Dale
> 
>> On Jul 17, 2017, at 2:43 AM, Christofer Dutz  
wrote:
>> 
>> Hi guys,
>> 
>> So right now, I sort of lost track of what’

Re: [ANNOUNCE] New Edgent PPMC member and committer: Christofer Dutz

[Christofer Dutz]

Hi all,

first off all I would like to thank the existing PPMC for the honor of joining 
the team. Your invitation makes me very happy.

I do have great plans for using Edgent together as part of my efforts in 
creating an open-source SCADA system. Being able to contribute directly is a 
great benefit … a benefit for both sides, I hope :-)

Being able to use Edgent inside Maven, is a first, but big step for me. I guess 
we have to take another few steps, but as they say, every journey starts with a 
first step.

So let’s change the industry together. 

Looking forward to working with you guys ;-)

Chris


Am 08.08.17, 17:35 schrieb "Dale LaBossiere" :

All,

Please welcome Chris as the newest member of the Edgent PPMC and committers 
group!

As you’ve probably noticed Chris has been very active driving the 
restructuring of the Edgent build/release tooling.
Edgent application developer benefits are:
- simplify consumption of Edgent as a result of releasing Edgent jars in 
maven central
- improvements for consumption of the Edgent samples
Edgent runtime development benefits are:
- commonality with other project’s use of maven for build/release tooling 
and release management process
We’re not there yet but we continue to make progress and work through the 
issues.

— Dale

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi Dale,

I guess it would be a lot easier to split. This way the work of splitting has 
to be done exactly once and from then on everything is super easy. The other 
way around it doesn’t cost anything to setup, but the costs of releasing 
increase dramatically due to the requirement to cherry pick commits.

Sure, I could request the things needed and handle the execution. But I quess 
that would be a runner-up task after merging back the maven changes first.

Chris


Am 08.08.17, 22:11 schrieb "Dale LaBossiere" :

In the near term I was thinking/hoping that simply separating the samples 
and the core *source release bundles* would be less disruptive than, though a 
necessary precursor to, migrating the samples to a separate repo.

If it’s simply much easier, given maven and the release plugins, to have a 
separate repos to achieve separate core / samples source release bundles, then 
maybe that needs to be considered now.  Chris, would you be able to set that 
up?  Maybe give it a thought while I’m out. 

Thanks!
— Dale

> On Aug 8, 2017, at 10:14 AM, Christofer Dutz  
wrote:
> 
> Hi Dale,
> 
> great you’re looking into this issue … I would have to work myself into 
the topic a little more in order to address that.
> 
> Regarding the samples issues: I would strongly suggest to request a 
separate GIT repo for the samples. While it is possible to keep them in there, 
there are a lot of issues that have to be dealt with this way.
> First of all you have to exclude stuff from rat (as you have seen), then 
you have to exclude stuff from the releases (as you have seen too), but 
probably the most annoying thing is dealing with releasing in GIT.
> Having mixed repos, we would have several tags in one repo reflecting 
releases of Edgent and the samples. While I would treat this fact as “annoying” 
at most, the main problem will be merging the parts that are part of the 
release back to the master branch.
> 
> If the repos are separate, all you have to do is merge the tagged release 
revision back to master and all is good. In case of a mixed repo, you will have 
to do a lot of manual merging and cherry picking.
> 
> So I would opt for splitting up the repos and creating nicely separated 
build configs for both.
> 
> Repos are cheap at the ASF :-)
> 
> Chris
> 
> 
> 
> 
> Am 08.08.17, 15:59 schrieb "Dale LaBossiere" :
> 
>That explains the failure in the SVT test in travis.  Ugh.  :-(
> 
>I’ll look into it.  By the end of the day I’ll either fix it or 
temporarily disable the SVT test (and add a tracking item to the wiki page).
> 
>As I noted in the PR, the top-level pom.xml has comments (3?) related 
to the handling of the samples project.  When you get a chance could you look 
at those and perhaps identify what needs to be done to address them?  Thanks!
> 
>— Dale
> 
> 
>> On Aug 8, 2017, at 9:36 AM, Christofer Dutz  
wrote:
>> 
>> Hi all,
>> 
>> I just pulled in Dales changes to my forks branch. I like excluding the 
examples from the core build. However there is one problem as the test/svt 
project has a test dependency on the samples/apps project. If this is excluded, 
the build will probably fail.
>> I would suggest to adjust the test to not rely on a sample. Hereby I 
could remove the top most issue in the “problems” document.
>> 
>> Should we leave everything the way it currently is, or should I create a 
feature/maven branch in the Edgent repo? I’m fine with both options. If anyone 
else needs write access to my fork, just send me an email. 
>> 
>> Chris
>> 
>> 
>> Am 23.07.17, 20:05 schrieb "Christofer Dutz" :
>> 
>>   Hi,
>> 
>>   I just pushed a change that includes my improved jar-free version of 
the maven-wrapper that should be 100% compliant with Apache Release rules.
>>   It’s currently the exact same version I submitted as pull-request for 
the maven-wrapper project, but as the scripts are duplicated and checked in 
anyway, I thought I’d just go ahead and add them to Edgent.
>>   My first tests were perfect :-)
>> 
>>   So now, if you checked out Edgent and have JAVA_HOME set all you need 
to do, is run: 
>> 
>>   ./mvnw clean install
>> 
>>   and it will download the maven version, install it and use it. So you 
can now reduce the requirements to having Java 8 Installed.
>> 
>>   One thing I noticed today – as I’m currently setting up my new laptop 
– is that it’s no longer trivial to get a Java 7 JDK. 
>>   I will try to fi

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi Dale,

thanks for the summary … highly appreciated ;-)

- I’ll add the console app to my highest priority todo.
- I already started completely re-writing the DEVELOMENT.md
- I’ll take care of Nexus, Jenkins as soon as the PR is merged or at least the 
branch is moved to the core Edgent repo (no longer my fork)
- Java 7 should no longer be a requirement (It’s now optional and only needed 
if the “toolchains” Maven profile is enabled)
- Will re-install Eclipse on my new Mac and see what I can do about the errors
- The maven-release-plugin has a built in dry-run option so we could do such a 
check immediately. If I simply branch off my branch again and change the remote 
maven repo address to my private Artifactory instance, I could even to a full 
release without any issues. We could do that in form of a video conference, if 
you like … then I could demonstrate everything super-easy.

Chris



Am 09.08.17, 21:46 schrieb "Dale LaBossiere" :

Hi Chris/team,

Since I’m going to be on vacation for the next two weeks I wanted to send 
out an update, etc.

I’ve committed a lot of things to the PR over the last couple of days and 
I've updated the wiki so hopefully the TODOs are accurate/complete 
https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 


To restate, IMO we’re working towards getting the PR in a good enough state 
so that we can merge it and then pretty quickly work on making a real release 
(if for no other reason as a forcing function since after the merge the old 
build tooling is broken).

A couple of higher priority TODOs that come to mind:
- Item 15.h getting the Edgent console working 
(HttpServer,war,get-edgent-jars.sh)
- Item 12.d DEVELOPMENT.md
- Item 10.c setup Nexus for receiving Edgent jars?
- Item 18 build requires Java7
- Item 8 Errors importing info Eclipse
- How much of a dry run of a release can be done before merging the PR - in 
order to really assess the state of things?
   Can we get as far as creating and actually staging fake-RC source 
bundles and jars… that can then be evaluated?

As another near-term sanity/progress check it would be great if another 
contributor/user could clone your repo 
(https://github.com/chrisdutz/incubator-edgent 
) and then:
- git checkout feature/maven
- follow the info in the README in the top of the repo (not README.md) to 
build the Edgent SDK and then build and run some samples and report how it went.

It would then be double great if someone could start working on an new 
version of Getting Started 
(https://edgent.apache.org/docs/edgent-getting-started 
) appropriate for the 
new environment :-) … retaining the current version as is.

— Dale

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi Dale,

I think I found out what’s going on:
In general all is setup correctly. The generated console application is also 
build correctly and seems to be functional. 

The only problem is that the example uses the HttpServer class in the server 
module to start the server. In the old build this referenced the war file in 
the servlets module relative to the server module. I cleaned that up. 
Unfortunately now whenever you run the HttpServer, this expects a 
target/war-resources directory containing a servlets.war file inside. 
Eventually we should think of a way to do this bundling of the servlet.war 
differently.

I committed a change to the pom, that ensures all is in place for the 
edgent-sampes-topology example.

All examples I tested seemed to work with that. 

I also added code to throw an exception in case the war file isn’t found at all 
(which was the real problem)

Chris


Am 09.08.17, 21:46 schrieb "Dale LaBossiere" :

Hi Chris/team,

Since I’m going to be on vacation for the next two weeks I wanted to send 
out an update, etc.

I’ve committed a lot of things to the PR over the last couple of days and 
I've updated the wiki so hopefully the TODOs are accurate/complete 
https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 


To restate, IMO we’re working towards getting the PR in a good enough state 
so that we can merge it and then pretty quickly work on making a real release 
(if for no other reason as a forcing function since after the merge the old 
build tooling is broken).

A couple of higher priority TODOs that come to mind:
- Item 15.h getting the Edgent console working 
(HttpServer,war,get-edgent-jars.sh)
- Item 12.d DEVELOPMENT.md
- Item 10.c setup Nexus for receiving Edgent jars?
- Item 18 build requires Java7
- Item 8 Errors importing info Eclipse
- How much of a dry run of a release can be done before merging the PR - in 
order to really assess the state of things?
   Can we get as far as creating and actually staging fake-RC source 
bundles and jars… that can then be evaluated?

As another near-term sanity/progress check it would be great if another 
contributor/user could clone your repo 
(https://github.com/chrisdutz/incubator-edgent 
) and then:
- git checkout feature/maven
- follow the info in the README in the top of the repo (not README.md) to 
build the Edgent SDK and then build and run some samples and report how it went.

It would then be double great if someone could start working on an new 
version of Getting Started 
(https://edgent.apache.org/docs/edgent-getting-started 
) appropriate for the 
new environment :-) … retaining the current version as is.

— Dale

Do we want Edgent stickers?

[Christofer Dutz]

Hi,

I’m currently finishing the work on a 3 day Apache track at a conference in 
Hamburg. Right now we can order project stickers for our projects. I was 
thinking of requesting some Edgent stickers.
Any objections?

Do we have the logo in some SVG like (vector) format?

I always love sticking stickers on things ;-)

Chris

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi all,

today I managed to spare some time to work on the Maven migration topic. 
Here a short summary on what I fixed:
1) The console/server now bundles the servlets.war as a resource inside the 
jar. So now all you need to so is reference the server jar and start the 
server. It did fix all problems with the samples I tried it with.
2) Fix a problem with the JDBC Tests. In my case I couldn’t run the tests, 
because my username contained a special char (“christofer.dutz”), additionally 
I made the tests generate output only inside the target directory.
3) Did some additional cleaning up in the maven build (removed some unneeded 
configuration options, fine-tuned the code coverage report generation)

Today I did a lot of full builds:
- Java8
- Java8, Java7
- Java8, Java7, Android (without toolchain – java7 built with java8 VM)
- Java8, Java7, Android (with toolchain – java7 built with java7 VM)

What I have on my to-do list next:
1) Optimize the Eclipse support
2) Finish the site-generation (Generation of API Docs, Test Reports, Coverage 
Reports, …)

And it did show a problem I had as I was embedding a java8 servlets.war in the 
server and the build didn’t fail without toolchain, but with toolchain it did 
fail. So, it’s a good thing to have that in place as an additional measure of 
security :-)

So far, the update … looking into Eclipse next.

Chris



Am 14.08.17, 15:41 schrieb "Christofer Dutz" :

Hi Dale,

I think I found out what’s going on:
In general all is setup correctly. The generated console application is 
also build correctly and seems to be functional. 

The only problem is that the example uses the HttpServer class in the 
server module to start the server. In the old build this referenced the war 
file in the servlets module relative to the server module. I cleaned that up. 
Unfortunately now whenever you run the HttpServer, this expects a 
target/war-resources directory containing a servlets.war file inside. 
Eventually we should think of a way to do this bundling of the servlet.war 
differently.

I committed a change to the pom, that ensures all is in place for the 
edgent-sampes-topology example.

All examples I tested seemed to work with that. 

I also added code to throw an exception in case the war file isn’t found at 
all (which was the real problem)

Chris


Am 09.08.17, 21:46 schrieb "Dale LaBossiere" :

Hi Chris/team,

Since I’m going to be on vacation for the next two weeks I wanted to 
send out an update, etc.

I’ve committed a lot of things to the PR over the last couple of days 
and I've updated the wiki so hopefully the TODOs are accurate/complete 
https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>

To restate, IMO we’re working towards getting the PR in a good enough 
state so that we can merge it and then pretty quickly work on making a real 
release (if for no other reason as a forcing function since after the merge the 
old build tooling is broken).

A couple of higher priority TODOs that come to mind:
- Item 15.h getting the Edgent console working 
(HttpServer,war,get-edgent-jars.sh)
- Item 12.d DEVELOPMENT.md
- Item 10.c setup Nexus for receiving Edgent jars?
- Item 18 build requires Java7
- Item 8 Errors importing info Eclipse
- How much of a dry run of a release can be done before merging the PR 
- in order to really assess the state of things?
   Can we get as far as creating and actually staging fake-RC source 
bundles and jars… that can then be evaluated?

As another near-term sanity/progress check it would be great if another 
contributor/user could clone your repo 
(https://github.com/chrisdutz/incubator-edgent 
<https://github.com/chrisdutz/incubator-edgent>) and then:
- git checkout feature/maven
- follow the info in the README in the top of the repo (not README.md) 
to build the Edgent SDK and then build and run some samples and report how it 
went.

It would then be double great if someone could start working on an new 
version of Getting Started 
(https://edgent.apache.org/docs/edgent-getting-started 
<https://edgent.apache.org/docs/edgent-getting-started>) appropriate for the 
new environment :-) … retaining the current version as is.

— Dale

Re: What's left to do for Maven migration?

[Christofer Dutz]

Ok …

so yesterday, while watching the latest game of thrones episode, I updated all 
poms and now Eclipse doesn’t complain about anything anymore.
I even fixed some minor issues Eclipse was reporting.

So, you guys satisfied with this?

Next Stop: Repots and Site generation …

Chris 

Am 15.08.17, 22:19 schrieb "Christofer Dutz" :

Hi all,

today I managed to spare some time to work on the Maven migration topic. 
Here a short summary on what I fixed:
1) The console/server now bundles the servlets.war as a resource inside the 
jar. So now all you need to so is reference the server jar and start the 
server. It did fix all problems with the samples I tried it with.
2) Fix a problem with the JDBC Tests. In my case I couldn’t run the tests, 
because my username contained a special char (“christofer.dutz”), additionally 
I made the tests generate output only inside the target directory.
3) Did some additional cleaning up in the maven build (removed some 
unneeded configuration options, fine-tuned the code coverage report generation)

Today I did a lot of full builds:
- Java8
- Java8, Java7
- Java8, Java7, Android (without toolchain – java7 built with java8 VM)
- Java8, Java7, Android (with toolchain – java7 built with java7 VM)

What I have on my to-do list next:
1) Optimize the Eclipse support
2) Finish the site-generation (Generation of API Docs, Test Reports, 
Coverage Reports, …)

And it did show a problem I had as I was embedding a java8 servlets.war in 
the server and the build didn’t fail without toolchain, but with toolchain it 
did fail. So, it’s a good thing to have that in place as an additional measure 
of security :-)

So far, the update … looking into Eclipse next.

Chris



Am 14.08.17, 15:41 schrieb "Christofer Dutz" :

Hi Dale,

I think I found out what’s going on:
In general all is setup correctly. The generated console application is 
also build correctly and seems to be functional. 

The only problem is that the example uses the HttpServer class in the 
server module to start the server. In the old build this referenced the war 
file in the servlets module relative to the server module. I cleaned that up. 
Unfortunately now whenever you run the HttpServer, this expects a 
target/war-resources directory containing a servlets.war file inside. 
Eventually we should think of a way to do this bundling of the servlet.war 
differently.

I committed a change to the pom, that ensures all is in place for the 
edgent-sampes-topology example.

All examples I tested seemed to work with that. 

I also added code to throw an exception in case the war file isn’t 
found at all (which was the real problem)

Chris


Am 09.08.17, 21:46 schrieb "Dale LaBossiere" :

Hi Chris/team,

Since I’m going to be on vacation for the next two weeks I wanted 
to send out an update, etc.

I’ve committed a lot of things to the PR over the last couple of 
days and I've updated the wiki so hopefully the TODOs are accurate/complete 
https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>

To restate, IMO we’re working towards getting the PR in a good 
enough state so that we can merge it and then pretty quickly work on making a 
real release (if for no other reason as a forcing function since after the 
merge the old build tooling is broken).

A couple of higher priority TODOs that come to mind:
- Item 15.h getting the Edgent console working 
(HttpServer,war,get-edgent-jars.sh)
- Item 12.d DEVELOPMENT.md
- Item 10.c setup Nexus for receiving Edgent jars?
- Item 18 build requires Java7
- Item 8 Errors importing info Eclipse
- How much of a dry run of a release can be done before merging the 
PR - in order to really assess the state of things?
   Can we get as far as creating and actually staging fake-RC 
source bundles and jars… that can then be evaluated?

As another near-term sanity/progress check it would be great if 
another contributor/user could clone your repo 
(https://github.com/chrisdutz/incubator-edgent 
<https://github.com/chrisdutz/incubator-edgent>) and then:
- git checkout feature/maven
- follow the info in the README in the top of the repo (not 
README.md) to build the Edgent SDK and then build and run some samples and 
report how it went.

It would then be double great if someone could start working on an 
new version of Getting Started 
(https://edgent.apache.org/docs/edg

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi,

so today I finally got Travis to be green again :-)

So, my next step was to setup a Jenkins build on the ASF Jenkins. That too is 
now done:
https://builds.apache.org/view/E-G/view/Edgent/job/edgent-dev/5/console

All Edgent jobs will be located here:
https://builds.apache.org/view/E-G/view/Edgent/

Right now, it still references the branch in my fork, but as soon as the 
changes get merged back, I’ll adjust that. I’m also not installing or 
publishing any artifacts to Nexus yet. That too is a task I would do as soon as 
we have merged things back.

I also requested Infra to allow us to use the Apache SonarQube instance (I hope 
that was ok). As soon as that’s done, we’ll also be able to get code analysis 
of our code as part of the build.

The difference of the Jenkins to the Travis build is that due to log file size 
restrictions Travis can only build and test the Java 8 version. Jenkins now 
builds Java 8, Java 7 and Android using the “toolchain” profile.

With this we now have a quick check for commits and pull requests from Travis 
and have the full quality assurance on Apache Jenkins.

So far, the update …

Chris


Am 16.08.17, 09:32 schrieb "Christofer Dutz" :

Ok …

so yesterday, while watching the latest game of thrones episode, I updated 
all poms and now Eclipse doesn’t complain about anything anymore.
I even fixed some minor issues Eclipse was reporting.

So, you guys satisfied with this?

Next Stop: Repots and Site generation …

Chris 

Am 15.08.17, 22:19 schrieb "Christofer Dutz" :

Hi all,

today I managed to spare some time to work on the Maven migration 
topic. 
Here a short summary on what I fixed:
1) The console/server now bundles the servlets.war as a resource inside 
the jar. So now all you need to so is reference the server jar and start the 
server. It did fix all problems with the samples I tried it with.
2) Fix a problem with the JDBC Tests. In my case I couldn’t run the 
tests, because my username contained a special char (“christofer.dutz”), 
additionally I made the tests generate output only inside the target directory.
3) Did some additional cleaning up in the maven build (removed some 
unneeded configuration options, fine-tuned the code coverage report generation)

Today I did a lot of full builds:
- Java8
- Java8, Java7
- Java8, Java7, Android (without toolchain – java7 built with java8 VM)
- Java8, Java7, Android (with toolchain – java7 built with java7 VM)

What I have on my to-do list next:
1) Optimize the Eclipse support
2) Finish the site-generation (Generation of API Docs, Test Reports, 
Coverage Reports, …)

And it did show a problem I had as I was embedding a java8 servlets.war 
in the server and the build didn’t fail without toolchain, but with toolchain 
it did fail. So, it’s a good thing to have that in place as an additional 
measure of security :-)

So far, the update … looking into Eclipse next.

Chris



Am 14.08.17, 15:41 schrieb "Christofer Dutz" 
:

Hi Dale,

I think I found out what’s going on:
In general all is setup correctly. The generated console 
application is also build correctly and seems to be functional. 

The only problem is that the example uses the HttpServer class in 
the server module to start the server. In the old build this referenced the war 
file in the servlets module relative to the server module. I cleaned that up. 
Unfortunately now whenever you run the HttpServer, this expects a 
target/war-resources directory containing a servlets.war file inside. 
Eventually we should think of a way to do this bundling of the servlet.war 
differently.

I committed a change to the pom, that ensures all is in place for 
the edgent-sampes-topology example.

All examples I tested seemed to work with that. 

I also added code to throw an exception in case the war file isn’t 
found at all (which was the real problem)

Chris


Am 09.08.17, 21:46 schrieb "Dale LaBossiere" :

Hi Chris/team,

Since I’m going to be on vacation for the next two weeks I 
wanted to send out an update, etc.

I’ve committed a lot of things to the PR over the last couple 
of days and I've updated the wiki so hopefully the TODOs are accurate/complete 
https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle 
<https://cwiki.apache.org/confluence/display/EDGENT/Maven+vs+Gradle>

To restate, IMO we’re working towards getting 

SonarQube analysis online ...

[Christofer Dutz]

Hi all,

after getting the Jenkins build working, now we have our first SonarQube 
analysis online:
https://builds.apache.org/analysis/overview?id=45154

12days of technical debt … that’s almost nothing (in the FlexJS project we have 
more than 1000d)

But beware … ususally SonarQube does show a lot of stuff that’s not really bad. 
Usually we would create an “Edgent Way” quality profile which is forked from 
the “Sonarqube Way” profile, which is currently active. There we can 
enable/disable rules and change the severity.
BUT I always think having a look at the results is a good thing to do. In the 
past, I have found quite a lot of bad things with this.

Chris

Re: Regarding Contribution

[Christofer Dutz]

Hi Amit,

well I got started listening to some discussions on this list … Ok … I agree … 
It’s quite silent here at the moment ;-)
Another option would be to have a look at our Jira: 
https://issues.apache.org/jira/projects/EDGENT and find something you could 
start working on. 
If you found something, the first step would be to fork the Github repo at: 
https://github.com/apache/incubator-edgent and to checkout your fork.
That’s where you would be doing your work on. I would suggest creating a branch 
for every issue you work on. As soon as you have finished something, you simply 
commit and push to your repo and create a pull request for that. 
This will automatically have our Travis CI Server do a first check. If all is 
good we will start reviewing your contribution and as soon as that’s done, your 
changes can get merged.

Right now, I’m working hard on finishing the migration of the build tool from 
Gradle to Maven, so you should prepare for a bigger change in the near future.

Chris



Am 23.08.17, 17:06 schrieb "Amit Tamrakar" :

Hi All,

I am having professional experience of Java application development and
designed & developed many applications.
I want to contribute to the project and looking for the mentor.
I have already proceeding with the "Getting started guide"

It will be great if I get the pointers to proceed.

Thanks in advance.

Regards,
Amit

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi All,

Ok so I double checked the confluence Maven vs Gradle page and think I 
addressed the remaining open issues. 

The Site generation now works fine from my point of view.

So, what’s left to do? Do we need any final changes? 

I think it would be good to do the merge back soon as I do think that it does 
make it more difficult for others to contribute and/or makes it more difficult 
for us to maintain the branch if others started contributing more.

Chris



Am 22.08.17, 15:01 schrieb "Christofer Dutz" :

Hi,

so today I finally got Travis to be green again :-)

So, my next step was to setup a Jenkins build on the ASF Jenkins. That too 
is now done:
https://builds.apache.org/view/E-G/view/Edgent/job/edgent-dev/5/console

All Edgent jobs will be located here:
https://builds.apache.org/view/E-G/view/Edgent/

Right now, it still references the branch in my fork, but as soon as the 
changes get merged back, I’ll adjust that. I’m also not installing or 
publishing any artifacts to Nexus yet. That too is a task I would do as soon as 
we have merged things back.

I also requested Infra to allow us to use the Apache SonarQube instance (I 
hope that was ok). As soon as that’s done, we’ll also be able to get code 
analysis of our code as part of the build.

The difference of the Jenkins to the Travis build is that due to log file 
size restrictions Travis can only build and test the Java 8 version. Jenkins 
now builds Java 8, Java 7 and Android using the “toolchain” profile.

With this we now have a quick check for commits and pull requests from 
Travis and have the full quality assurance on Apache Jenkins.

So far, the update …

Chris


Am 16.08.17, 09:32 schrieb "Christofer Dutz" :

Ok …

so yesterday, while watching the latest game of thrones episode, I 
updated all poms and now Eclipse doesn’t complain about anything anymore.
I even fixed some minor issues Eclipse was reporting.

So, you guys satisfied with this?

Next Stop: Repots and Site generation …

Chris 

Am 15.08.17, 22:19 schrieb "Christofer Dutz" 
:

Hi all,

today I managed to spare some time to work on the Maven migration 
topic. 
Here a short summary on what I fixed:
1) The console/server now bundles the servlets.war as a resource 
inside the jar. So now all you need to so is reference the server jar and start 
the server. It did fix all problems with the samples I tried it with.
2) Fix a problem with the JDBC Tests. In my case I couldn’t run the 
tests, because my username contained a special char (“christofer.dutz”), 
additionally I made the tests generate output only inside the target directory.
3) Did some additional cleaning up in the maven build (removed some 
unneeded configuration options, fine-tuned the code coverage report generation)

Today I did a lot of full builds:
- Java8
- Java8, Java7
- Java8, Java7, Android (without toolchain – java7 built with java8 
VM)
- Java8, Java7, Android (with toolchain – java7 built with java7 VM)

What I have on my to-do list next:
1) Optimize the Eclipse support
2) Finish the site-generation (Generation of API Docs, Test 
Reports, Coverage Reports, …)

And it did show a problem I had as I was embedding a java8 
servlets.war in the server and the build didn’t fail without toolchain, but 
with toolchain it did fail. So, it’s a good thing to have that in place as an 
additional measure of security :-)

So far, the update … looking into Eclipse next.

Chris



    Am 14.08.17, 15:41 schrieb "Christofer Dutz" 
:

Hi Dale,

I think I found out what’s going on:
In general all is setup correctly. The generated console 
application is also build correctly and seems to be functional. 

The only problem is that the example uses the HttpServer class 
in the server module to start the server. In the old build this referenced the 
war file in the servlets module relative to the server module. I cleaned that 
up. Unfortunately now whenever you run the HttpServer, this expects a 
target/war-resources directory containing a servlets.war file inside. 
Eventually we should think of a way to do this bundling of the servlet.war 
differently.

I committed a change to the pom, that ensures all is in place 
for the edgent-sampes-topology example.

All examples I tested seemed to work with that. 

Re: Do we want Edgent stickers?

[Christofer Dutz]

Hehe ... Yeah it would have been too late now, but I already asked Saharan to 
get some. Think I'll get them next week in Hamburg. She took the normal logo 
from the website for that.

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: William Marshall 
Datum: 29.08.17 19:10 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Do we want Edgent stickers?

Little late in replying here, but I'd love Edgent stickers :)

Hm, I don't think we have a SVG formatted logo.

On Mon, Aug 14, 2017 at 7:13 AM, Christofer Dutz 
wrote:

> Hi,
>
> I’m currently finishing the work on a 3 day Apache track at a conference
> in Hamburg. Right now we can order project stickers for our projects. I was
> thinking of requesting some Edgent stickers.
> Any objections?
>
> Do we have the logo in some SVG like (vector) format?
>
> I always love sticking stickers on things ;-)
>
> Chris
>

Jira access

[Christofer Dutz]

Hi all,

seems I don’t have permission to assign, edit or work on Edgent issues.

Chris

Re: Jira access

[Christofer Dutz]

Hi all,

could anyone with admin rights add me to the Edgent Jira project please?
I was told one of: dale, kathey, queenie, susan, or john could do that ;-)

Chris


Am 31.08.17, 20:40 schrieb "Christofer Dutz" :

Hi all,

seems I don’t have permission to assign, edit or work on Edgent issues.

Chris

Re: Jira access

[Christofer Dutz]

Hi John,

thanks … worked :-)

Chris



Am 04.09.17, 09:45 schrieb "John D. Ament" :

I've add "cdutz" as an admin + PMC member, since you're on the PPMC.

John

On Mon, Sep 4, 2017 at 2:01 AM Christofer Dutz 
wrote:

> Hi all,
>
> could anyone with admin rights add me to the Edgent Jira project please?
> I was told one of: dale, kathey, queenie, susan, or john could do that ;-)
>
> Chris
>
>
> Am 31.08.17, 20:40 schrieb "Christofer Dutz" :
>
> Hi all,
>
> seems I don’t have permission to assign, edit or work on Edgent 
issues.
>
> Chris
>
>
>

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi Dale,

last weekend I addressed your open points regarding the source releases. 

One thing I should mention is that the editor files shouldn’t be a problem as 
when using the maven-release-plugin the tagged version of the sources are 
checked out and built. So there shouldn’t be any leftover files. This should 
only be a problem when manually activating the “apache-release” profile.

I also I filled one of the two TODOs in the DEVELOPMENT.md file.

Added inline comments in the confluence page … will have a look at the samples 
next.

Chris


Am 30.08.17, 18:44 schrieb "Dale LaBossiere" :

Chris,

Simply stated: awesome progress while I’ve been out.  Nice work!!!
I’m now starting to review all of the changes to bring myself up to date.
I completely agree with the desire to merge asap… with emphasis on “as 
possible” / “as practical” :-)
Hopefully I’ll reach your conclusion that now is the time.

— Dale

> On Aug 28, 2017, at 4:58 AM, Christofer Dutz  
wrote:
> 
> Hi All,
> 
> Ok so I double checked the confluence Maven vs Gradle page and think I 
addressed the remaining open issues. 
> 
> The Site generation now works fine from my point of view.
> 
> So, what’s left to do? Do we need any final changes? 
> 
> I think it would be good to do the merge back soon as I do think that it 
does make it more difficult for others to contribute and/or makes it more 
difficult for us to maintain the branch if others started contributing more.
> 
> Chris
> 

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi all,

well I did have a more detailed look at the original (Gradle) version and the 
maven version. When running both examples, I could see two requests every 5 
seconds. Both of them are answered by the server. The “jobs” request is 
answered identically. The difference seems to be in the “metrics” response:

The working version of the Gradle version produces this:

{
"jobId": "JOB_0",
"ops": [
{
"opId": "OP_2",
"metrics": [
{
"type": "counter",
"name": "Count",
"value": "219"
}
]
}
]
}

The Maven version however produces this:

{
"jobId": "JOB_0",
"ops": [
{
"opId": "OP_2",
"metrics": [
{
"type": "long",
"name": "Count",
"value": "436"
}
]
}
]
}

The difference seems to be that the “type” is “counter” in the working version 
and “long” in the not working version. I tried debugging this, but I couldn’t 
really understand why this is different.

Any ideas what could be wrong? I guess in general things should be working, all 
we have to do is fix some final mini-quirks.

Chris



Am 18.09.17, 11:02 schrieb "Christofer Dutz" :

Hi Dale,

last weekend I addressed your open points regarding the source releases. 

One thing I should mention is that the editor files shouldn’t be a problem 
as when using the maven-release-plugin the tagged version of the sources are 
checked out and built. So there shouldn’t be any leftover files. This should 
only be a problem when manually activating the “apache-release” profile.

I also I filled one of the two TODOs in the DEVELOPMENT.md file.

Added inline comments in the confluence page … will have a look at the 
samples next.

Chris


Am 30.08.17, 18:44 schrieb "Dale LaBossiere" :

Chris,

Simply stated: awesome progress while I’ve been out.  Nice work!!!
I’m now starting to review all of the changes to bring myself up to 
date.
    I completely agree with the desire to merge asap… with emphasis on “as 
possible” / “as practical” :-)
Hopefully I’ll reach your conclusion that now is the time.

— Dale

> On Aug 28, 2017, at 4:58 AM, Christofer Dutz 
 wrote:
> 
> Hi All,
> 
> Ok so I double checked the confluence Maven vs Gradle page and think 
I addressed the remaining open issues. 
> 
> The Site generation now works fine from my point of view.
> 
> So, what’s left to do? Do we need any final changes? 
> 
> I think it would be good to do the merge back soon as I do think that 
it does make it more difficult for others to contribute and/or makes it more 
difficult for us to maintain the branch if others started contributing more.
> 
> Chris
> 

Re: latest changes and source bundle content?

[Christofer Dutz]

Hi Dale,

you are right … it should have been. Guess my last step to consolidate the file 
names caused the original descriptor to be used instead of my new one. I fixed 
that and re-tested. Now the maven wrapper is excluded. The Gradle wrapper part 
however is still in, but I thought it would be better not to add that to the 
exclusions as the directory will probably be deleted anyway as one of the last 
steps of the migration.

Chris 



Am 18.09.17, 21:37 schrieb "Dale LaBossiere" :

Hi Chris, I pulled the latest pr309 changes.

The source bundle created from “wmvn install -Papache-release -DskipTests” 
still contains:
- maven-wrapper.jar
- samples source

You indicated they should now be excluded. ???

— Dale

Re: servlets.war in distribution bundle?

[Christofer Dutz]

Hi Dale,

Well the reason for double including it, is just that if you want to deploy it 
separately, you can run the servlets.jar in any Servlet engine, the Server.jar 
is just a convenience wrapper to start it without an existing servlet engine.

Chris

Am 19.09.17, 18:30 schrieb "Dale LaBossiere" :

Hi Chris, 

Recently I noted that the servlets.war wasn’t present in the distribution 
and you made the changes to add it.
I just noticed that edgent-console-server.jar has it embedded in it under 
resources/servlets.war.

So do we really need to separately include it in the distribution?
I can undo the changes you made to include if that’s the appropriate thing 
to do.
Sorry about that.

— Dale

Re: servlets.war in distribution bundle?

[Christofer Dutz]

Hi Dale,

It's super easy to change that back. I'll do that right away tomorrow.

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


 Ursprüngliche Nachricht 
Von: Dale LaBossiere 
Datum: 19.09.17 18:46 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: servlets.war in distribution bundle?

Thanks for the clarification, that makes sense.

Followup questions:
I, perhaps naively, liked the fact the edgent-console-server.jar used to be in 
the libs folder with the rest of the edgent jars.
What’s the motivation for it and the war being in a separate console folder?
Does it make sense to move server.jar back?

— Dale

> On Sep 19, 2017, at 12:34 PM, Christofer Dutz  
> wrote:
>
> Hi Dale,
>
> Well the reason for double including it, is just that if you want to deploy 
> it separately, you can run the servlets.jar in any Servlet engine, the 
> Server.jar is just a convenience wrapper to start it without an existing 
> servlet engine.
>
> Chris
>
> Am 19.09.17, 18:30 schrieb "Dale LaBossiere" :
>
>Hi Chris,
>
>Recently I noted that the servlets.war wasn’t present in the distribution 
> and you made the changes to add it.
>I just noticed that edgent-console-server.jar has it embedded in it under 
> resources/servlets.war.
>
>So do we really need to separately include it in the distribution?
>I can undo the changes you made to include if that’s the appropriate thing 
> to do.
>Sorry about that.
>
>— Dale
>

Re: What's left to do for Maven migration?

[Christofer Dutz]

Ok so today I investigated the problem a little more.

Seems to be one of those cool problems you get when operating with a type less 
language ;-)

There is a difference between the original Sankey.js and the Edgent version. 
The difference is that the original uses “source” and “target” while the Edgent 
version uses “sourceIdx” and “targetIdx” so I am getting JavaScript errors in 
the browser. 
I think I should try and find a solution for this and then see if the charts 
come back. If that doesn’t help I can still get my hands dirty by digging in 
the depths of the system.

Chris



Am 19.09.17, 17:59 schrieb "Susan Cline" :

I did look at Chris’s pull request and don’t see any changes to the file 
that is formatting the metrics.

I also looked at the java class that is sending the metrics to the console 
- that code looks unchanged as well, so I am at a loss for what did change.
Perhaps some of the underlying java classes the servlets or utility classes 
accessed changed?

Susan


> On Sep 18, 2017, at 8:02 AM, Dale LaBossiere  wrote:
> 
> Hmm… I’m not familiar with that code but will take a look.
> 
> Susan, any thoughts, including whether or not this sort of difference 
could account for the graph not getting rendered but the console page otherwise 
looking ok?  (reminder this is pr309 and you can clone Chris’s fork to get all 
the changes)
> 
> — Dale
> 
    >> On Sep 18, 2017, at 8:54 AM, Christofer Dutz  
wrote:
>> 
>> Hi all,
>> 
>> well I did have a more detailed look at the original (Gradle) version 
and the maven version. When running both examples, I could see two requests 
every 5 seconds. Both of them are answered by the server. The “jobs” request is 
answered identically. The difference seems to be in the “metrics” response:
>> 
>> The working version of the Gradle version produces this:
>> 
>> {
>>   "jobId": "JOB_0",
>>   "ops": [
>>   {
>>   "opId": "OP_2",
>>   "metrics": [
>>   {
>>   "type": "counter",
>>   "name": "Count",
>>   "value": "219"
>>   }
>>   ]
>>   }
>>   ]
>> }
>> 
>> The Maven version however produces this:
>> 
>> {
>>   "jobId": "JOB_0",
>>   "ops": [
>>   {
>>   "opId": "OP_2",
>>   "metrics": [
>>   {
>>   "type": "long",
>>   "name": "Count",
>>   "value": "436"
>>   }
>>   ]
>>   }
>>   ]
>> }
>> 
>> The difference seems to be that the “type” is “counter” in the working 
version and “long” in the not working version. I tried debugging this, but I 
couldn’t really understand why this is different.
>> 
>> Any ideas what could be wrong? I guess in general things should be 
working, all we have to do is fix some final mini-quirks.
>> 
>> Chris
> 


.

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi all,

ok so finding this little detail got me on the right path. I updated index.js 
to use “source” instead of “sourceIdx” and “target” instead of “targetIdx” and 
now the console is working identically to the original gradle version.

Chris



Am 20.09.17, 13:30 schrieb "Christofer Dutz" :

Ok so today I investigated the problem a little more.

Seems to be one of those cool problems you get when operating with a type 
less language ;-)

There is a difference between the original Sankey.js and the Edgent 
version. The difference is that the original uses “source” and “target” while 
the Edgent version uses “sourceIdx” and “targetIdx” so I am getting JavaScript 
errors in the browser. 
I think I should try and find a solution for this and then see if the 
charts come back. If that doesn’t help I can still get my hands dirty by 
digging in the depths of the system.

Chris



Am 19.09.17, 17:59 schrieb "Susan Cline" :

I did look at Chris’s pull request and don’t see any changes to the 
file that is formatting the metrics.

I also looked at the java class that is sending the metrics to the 
console - that code looks unchanged as well, so I am at a loss for what did 
change.
Perhaps some of the underlying java classes the servlets or utility 
classes accessed changed?

Susan


> On Sep 18, 2017, at 8:02 AM, Dale LaBossiere  
wrote:
> 
> Hmm… I’m not familiar with that code but will take a look.
> 
> Susan, any thoughts, including whether or not this sort of difference 
could account for the graph not getting rendered but the console page otherwise 
looking ok?  (reminder this is pr309 and you can clone Chris’s fork to get all 
the changes)
> 
> — Dale
> 
    >> On Sep 18, 2017, at 8:54 AM, Christofer Dutz 
 wrote:
>> 
>> Hi all,
>> 
>> well I did have a more detailed look at the original (Gradle) 
version and the maven version. When running both examples, I could see two 
requests every 5 seconds. Both of them are answered by the server. The “jobs” 
request is answered identically. The difference seems to be in the “metrics” 
response:
>> 
>> The working version of the Gradle version produces this:
>> 
>> {
>>   "jobId": "JOB_0",
>>   "ops": [
>>   {
>>   "opId": "OP_2",
>>   "metrics": [
>>   {
>>   "type": "counter",
>>   "name": "Count",
>>   "value": "219"
>>   }
>>   ]
>>   }
>>   ]
>> }
>> 
>> The Maven version however produces this:
>> 
>> {
>>   "jobId": "JOB_0",
>>   "ops": [
>>   {
>>   "opId": "OP_2",
>>   "metrics": [
>>   {
>>   "type": "long",
>>   "name": "Count",
>>   "value": "436"
>>   }
>>   ]
>>   }
>>   ]
>> }
>> 
>> The difference seems to be that the “type” is “counter” in the 
working version and “long” in the not working version. I tried debugging this, 
but I couldn’t really understand why this is different.
>> 
>> Any ideas what could be wrong? I guess in general things should be 
working, all we have to do is fix some final mini-quirks.
>> 
>> Chris
> 


.

Re: What's left to do for Maven migration?

[Christofer Dutz]

Hi Dale,

I’m sure that is the reason for this. I did investigate that problem a wile 
yesterday and I found out that the MBean the ConsoleMetricsServlet->MetricsUtil 
(line 105) is using to generate the output returns “long” instead of “counter” 
… but I have to admit that I have no Idea why.

Will investigate tomorrow ;-)

Chris



Am 20.09.17, 18:33 schrieb "Dale LaBossiere" :


> On Sep 20, 2017, at 12:03 PM, Dale LaBossiere  
wrote:
> 
> Yay, we have graphs!
> 
> In briefly playing with the console I noticed a number of things that 
I’ll have to investigate to verify they’re not regressions (others chime in if 
you know them to be present on the master branch):
> 
> - The Pause Graph button does pause graph info update (e.g., metrics on 
hovers), however the metrics charts continue to update.
> - Hover on a Counter or StreamScope oplet doesn’t report streams or 
tuples info like on “normal” oplets.  However that info for them is cleanly 
visible in the All Oplet Properties table
> - The All Oplet Properties table doesn’t refresh.  There’s a Pause 
refresh button on it that doesn’t yield any observable behavior.

Those behaviors are the same / no regression  :-/

But here’s a regression:

When you hover on a stream, in 1.1.0 the bottom of the hover reports a 
tuple count.  On the maven branch the bottom of the hover reports “No value - 
counter not present”

That’s emitted by index.js/1164/renderGraph based on d.derived being true.  
Not yet sure how that comes to be.  Chris, maybe related to your earlier 
observation:

The working version of the Gradle version produces this:

{
   "jobId": "JOB_0",
   "ops": [
   {
   "opId": "OP_2",
   "metrics": [
   {
   "type": "counter",
   "name": "Count",
   "value": "219"
   }
   ]
   }
   ]
}

The Maven version however produces this:

{
   "jobId": "JOB_0",
   "ops": [
   {
   "opId": "OP_2",
   "metrics": [
   {
   "type": "long",
   "name": "Count",
   "value": "436"
   }
   ]
   }
   ]
}
>