The problem with running anything under Ant is that it's not set up for this kind of flow:
- start service - run tests - stop service Ant is about building and is not a sequential language, so starting this under Ant is the wrong idea. Instead, we can invoke scripts at will from within the java test class itself. But we need both Linux and Windows scripts for that, and we download only one ES instance, and therefore we get only one kind of script to go with it. I supposed I can platform-conditionalize the download itself so we get different scripts for different platforms. Since ES doesn't include all script variants in every download we're kind of stuck with this it seems. The other issue we have to address is waiting for ES to actually fully start. I believe the code we had did this via a specific HTTP Get request fired at the instance, so maybe we can reuse that code. There may also be a way to shut ES down via a similar HTTP Get mechanism. Can you verify that waiting for ES to come up and shutting down ES can be done with the same mechanism as is currently in the test code? Karl On Thu, Apr 23, 2020 at 7:44 AM Michael Cizmar <mich...@michaelcizmar.com> wrote: > Karl, > > I found this: > > https://gquintana.github.io/2016/11/30/Testing-a-Java-and-Elasticsearch-50-application.html > > Which should solve the issue of running elastic in the background via Ant. > > I can also provide a simple setup script (sh) if that helps as well. > > Michael > > On Wed, Apr 22, 2020 at 12:35 PM Karl Wright <daddy...@gmail.com> wrote: > > > I looked into trying to get things working under Ant and created a branch > > CONNECTORS-1639 containing some changes relating to download of > > elasticsearch artifacts. I did a little exploration as to whether we > could > > use the Elasticsearch Runner package to start a cluster, but that is > really > > painful because it has a ton of dependencies, so I think I'll just try > > calling the main class that the ES startup script uses and see how we do > > that way. > > > > But I'm snowed under with work related tasks again so it will have to > wait. > > > > Karl > > > > > > On Sat, Apr 18, 2020 at 5:52 PM Michael Cizmar < > mich...@michaelcizmar.com> > > wrote: > > > > > I've updated the ticket with the changes: > > > > > > > > > https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1639 > > > > > > On Sat, Apr 18, 2020 at 3:59 PM Cihad Guzel <cguz...@gmail.com> wrote: > > > > > > > Thanks folks for your information. > > > > > > > > I reviewed the pom.xml of the ES connector. It uses an old elastic > > > search > > > > version as dependency. It misled me. On the other hand, you are > right. > > I > > > > agree with your thoughts on this matter. It is best if we can > rearrange > > > > them. > > > > > > > > Kind regards, > > > > Cihad Guzel > > > > > > > > > > > > Karl Wright <daddy...@gmail.com>, 18 Nis 2020 Cmt, 17:49 tarihinde > > şunu > > > > yazdı: > > > > > > > > > I can help with Ant tasks but I need information as to how you're > > > > supposed > > > > > to start the ES instance. An ant task snippet would be sufficient > I > > > > think. > > > > > > > > > > > > > > > On Sat, Apr 18, 2020 at 10:33 AM Michael Cizmar < > > > > > michael.ciz...@mcplusa.com> > > > > > wrote: > > > > > > > > > > > I believe so. I only modified the Pom in the es connector > project > > > and > > > > > > removed the Node references. I know there is a way to do this in > > ant > > > > as > > > > > > well. I will look it up but may need some guidance on Ant. > > > > > > > > > > > > Get Outlook for iOS<https://aka.ms/o0ukef> > > > > > > ________________________________ > > > > > > From: Karl Wright <daddy...@gmail.com> > > > > > > Sent: Saturday, April 18, 2020 9:15:15 AM > > > > > > To: dev <dev@manifoldcf.apache.org> > > > > > > Subject: Re: Release schedule > > > > > > > > > > > > Hi Michael, > > > > > > This has to run under Ant as well. Any way to make that happen? > > > > > > > > > > > > Karl > > > > > > > > > > > > > > > > > > On Sat, Apr 18, 2020 at 9:49 AM Michael Cizmar < > > > > > michael.ciz...@mcplusa.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > I've got a fix for this. I switched to using a Maven plugin > that > > > > spins > > > > > > up > > > > > > > an Elasticsearch instance. With this, you need only to remove > > the > > > > Node > > > > > > > code in the integration tests. Tested with 6.x client and 7.x > > > > > > > elasticsearch. > > > > > > > > > > > > > > There are more things we can do with this output plugin in the > > > future > > > > > > like > > > > > > > moving to the SDK. > > > > > > > > > > > > > > M > > > > > > > > > > > > > > On 4/18/20, 8:32 AM, "Karl Wright" <daddy...@gmail.com> > wrote: > > > > > > > > > > > > > > Thanks for the quick reply. > > > > > > > I agree we don't want to turn off the ES connector itself, > > but > > > > yes > > > > > we > > > > > > > will > > > > > > > need to shut down the tests. Cihad, would you like to > > propose > > > a > > > > > > > strategy > > > > > > > for that? I think for now just marking them with @Ignore > > > should > > > > be > > > > > > OK, > > > > > > > since the tests don't have compile time dependencies on > > missing > > > > > > > classes. > > > > > > > What do you think? > > > > > > > > > > > > > > Upgrading to ES 6.x is obviously the right thing to do but > > who > > > > here > > > > > > > has the > > > > > > > knowledge to do a good job with this? I am certain there > > are a > > > > > > number > > > > > > > of > > > > > > > ES users lurking on this list. Please volunteer if so. > > > > > > > > > > > > > > Karl > > > > > > > > > > > > > > > > > > > > > On Sat, Apr 18, 2020 at 9:15 AM Furkan KAMACI < > > > > > > furkankam...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > There is a compatibility matrix for Elasticsearch. We > need > > to > > > > > > > support at > > > > > > > > least Elasticsearch 6.5.x for Java 11 support. You can > > check > > > it > > > > > > from > > > > > > > here: > > > > > > > > https://www.elastic.co/de/support/matrix#matrix_jvm > > > > > > > > > > > > > > > > @Cihad > > > > > > > > > > > > > > > > As far as I know, current support is not 2.0.0. It is > > 5.5.2: > > > > > > > > > > > > > https://github.com/apache/manifoldcf-integration-elasticsearch-5.5 > > > > > > > > > > > > > > > > @Karl Wright <daddy...@gmail.com> > > > > > > > > > > > > > > > > So, such an upgrade from 5.5.2 to 6.5.x may not be so > > > painful. > > > > > > > Committers > > > > > > > > who use ES can comment on this. > > > > > > > > > > > > > > > > My comments: > > > > > > > > > > > > > > > > +1 to temporarily turning those tests off > > > > > > > > -1 to temporarily turning the connector off > > > > > > > > > > > > > > > > Kind Regards, > > > > > > > > Furkan KAMACI > > > > > > > > > > > > > > > > On Sat, Apr 18, 2020 at 3:27 PM Cihad Guzel < > > > cguz...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Hi Karl, > > > > > > > >> > > > > > > > >> MFC ES Connector uses the Elastic Search 2.0.0 . It's an > > > > ancient > > > > > > > version. > > > > > > > >> The latest version is 7.6.2 . So, I agree with you and I > > > think > > > > > we > > > > > > > can > > > > > > > >> temporarily turn the connector off. > > > > > > > >> > > > > > > > >> +1 > > > > > > > >> > > > > > > > >> Kind Regards, > > > > > > > >> Cihad Güzel > > > > > > > >> > > > > > > > >> > > > > > > > >> Karl Wright <daddy...@gmail.com>, 18 Nis 2020 Cmt, > 11:41 > > > > > > tarihinde > > > > > > > şunu > > > > > > > >> yazdı: > > > > > > > >> > > > > > > > >> > Hi all, > > > > > > > >> > > > > > > > > >> > We're due to release ManifoldCF 2.16 by April 30th. > The > > > > major > > > > > > > work for > > > > > > > >> > this release was adoption of Java 11, and that work is > > > > > > incomplete > > > > > > > >> because > > > > > > > >> > of ElasticSearch incompatibilities. I'm therefore > > tempted > > > > to > > > > > > > hold the > > > > > > > >> > release until we at least have a plan for dealing with > > ES > > > > > going > > > > > > > forward. > > > > > > > >> > > > > > > > > >> > It's not clear that our ES connector support is > > affected, > > > > but > > > > > > > certainly > > > > > > > >> our > > > > > > > >> > integration tests are, because Java 11 isn't supported > > in > > > > any > > > > > of > > > > > > > the ES > > > > > > > >> > versions we run for those tests. So at the least we > > need > > > to > > > > > > > decide to > > > > > > > >> turn > > > > > > > >> > those off. And indeed, we really need to have someone > > > with > > > > ES > > > > > > > >> experience > > > > > > > >> > map a strategy for getting our ES support back into > > > > compliance > > > > > > > with > > > > > > > >> what's > > > > > > > >> > out in the world at large now. Cihad Guzel did much > > work > > > on > > > > > > Java > > > > > > > 11 but > > > > > > > >> > stumbled over the Elastic Search problem. Any of our > > > > > committers > > > > > > > who > > > > > > > >> know > > > > > > > >> > ES and are stuck inside at the moment, please speak > up. > > > > > > > >> > > > > > > > > >> > Thanks in advance, > > > > > > > >> > Karl > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
