Hello I have been able to resolve the build issue in teamcity and was able to pass the build failures.
https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build The issue that was causing failure was that I had put ignite.zip as artifact dependency and it was downloading the artifacts including pom file inside the build checkout folder of ignite-extensions and as a result the pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml. This was resulting in missing flink-ext module error as it was not even present in the ignite main pom.xml. To resolve the issue I just had to download and extract artifact of ignite.zip in a separate directory. If I can get a +1 on this PR then I can go ahead and merge the changes. Regards, Saikat On Tue, Dec 10, 2019 at 8:26 PM Saikat Maitra <saikat.mai...@gmail.com> wrote: > Hi Ivan, > > Thank you for sharing your feedback. I will try to install all modules in > local agent's repository and run the build. > > I will share updates. > > Regards, > Saikat > > On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin <vololo...@gmail.com> > wrote: > >> Saikat, >> >> Let me share my understanding how do we run the majority of our jobs >> executing test suites. Hope it will help. >> 1. ~Build Apache Ignite~ in a separate run and publish everything as a >> single artifact ignite.zip (it contains ignite working directory with >> compiled modules, e.g. modules/core/target and others). >> 2. Download and extract ignite.zip and run particular test suites >> against extracted files. >> >> In a such setup test suites run the same as you do it on your local >> machine and it always works with a single multi-module maven project >> (apache-ignite). So, all dependencies are resolved in the working >> directory. >> >> For a separate IgniteExtensions maven project you need to install all >> dependencies from downloaded ignite.zip into .m2/repository local to >> the agent. I suppose we can do it as follows: >> 1. Download ignite.zip and extract it into "ignite" subfolder. >> 2. Run a maven command installing all modules into local maven >> repository. Something like "mvn install" but here we need to make sure >> that nothing is compiled once again. >> 3. Build IgniteExtensions. >> >> There might be a better way to do the same, I just tried to describe a >> schema how it can be done. >> >> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra <saikat.mai...@gmail.com>: >> > >> > Hi Ivan, Ilya >> > >> > Thank you for your reply. I have added you as dev in IgniteExtensions >> > project and you should be able to check the build configurations. >> > >> > Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar >> to >> > other projects then I receive below error >> > >> > [ERROR] [ERROR] Could not find the selected project in the reactor: >> > :ignite-flink-ext @ >> > >> > I do not see the same error when I am running the build in local and >> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my >> > local .m2 repository but it is failing in teamcity. >> > >> > Regards, >> > Saikat >> > >> > >> > >> > >> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev < >> ilya.kasnach...@gmail.com> >> > wrote: >> > >> > > Hello! >> > > >> > > I think that your build should depend on an Apache Ignite build(s) or >> just >> > > use an already released version. >> > > >> > > In the same fashion as all our teamcity depend on "Build Apache >> Ignite" and >> > > use its artifacts. >> > > >> > > Here you want Ignite snapshot but Teamcity does not know where to >> take it >> > > from. >> > > >> > > Regards, >> > > -- >> > > Ilya Kasnacheev >> > > >> > > >> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <saikat.mai...@gmail.com>: >> > > >> > > > Hello, >> > > > >> > > > I am running into a problem specific to teamcity build for Ignite >> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I >> am >> > > > getting an error message during build as below >> > > > >> > > > [06:24:12][Step 4/5] Failed to execute goal on project >> ignite-flink-ext: >> > > > Could not resolve dependencies for project >> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The >> following >> > > > artifacts could not be resolved: >> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, >> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, >> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, >> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find >> > > artifact >> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com >> ( >> > > > https://h2database.com/m2-repo) >> > > > >> > > > >> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I >> receive >> > > > below error >> > > > >> > > > [ERROR] [ERROR] Could not find the selected project in the reactor: >> > > > :ignite-flink-ext @ >> > > > >> > > > Build url >> > > > >> > > > >> > > >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv >> > > > >> > > > Can you please let me know if you faced similar problem with >> teamcity >> > > > build? >> > > > >> > > > I can set Ignite Extensions dependencies to released ignite-core >> > > artifacts >> > > > version like 2.7.6 and build works fine. >> > > > >> > > > Regards, >> > > > Saikat >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra < >> saikat.mai...@gmail.com> >> > > > wrote: >> > > > >> > > > > Hello Denis, >> > > > > >> > > > > Thank you for your email and sharing your thoughts on the release >> > > > process. >> > > > > I will update the artifact id and dependencies for >> ignite-extensions >> > > > > accordingly. >> > > > > >> > > > > I had created Ignite-Extensions project as a root level project >> and in >> > > > > teamcity I was facing issues pulling dependencies for >> 2.8.0-SNAPSHOT >> > > > > whereas I was able to pull dependencies for ignite-core 2.7.6 from >> > > maven >> > > > > central. I will look into it further why teamcity build was not >> able to >> > > > > pull snapshot dependencies. >> > > > > >> > > > > I will also create "ignite-core-2.9+" branch for the upcoming >> release >> > > > > process. >> > > > > >> > > > > Thank you, >> > > > > Saikat >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dma...@apache.org> >> wrote: >> > > > > >> > > > >> Hi Saikat, >> > > > >> >> > > > >> Thanks for driving this activity forward and raising the >> question. Let >> > > > me >> > > > >> share my thoughts below and let's see what the broader community >> > > thinks. >> > > > >> >> > > > >> Each extension needs to have its own version unrelated to the >> core and >> > > > >> Maven's groupId parameter for extension artifacts should be >> > > > >> "org.apache.ignite.ext". For instance, the very first release of >> Flink >> > > > in >> > > > >> the form of extension should be pulled from Maven this way >> > > > >> >> > > > >> <dependency> >> > > > >> <groupId>org.apache.ignite.ext</groupId> >> > > > >> <artifactId>ignite-flink</artifactId> >> > > > >> <version>1.0.0</version> >> > > > >> </dependency> >> > > > >> >> > > > >> When it comes to the releases, all the extensions need to be >> verified >> > > > for >> > > > >> an upcoming release and updated if needed (with the version >> increase >> > > > only >> > > > >> for those updated). Thus, looks like the extensions master needs >> to be >> > > > >> linked to the latest Ignite core snapshot. Whenever the core >> will be >> > > > being >> > > > >> prepared and any extensions need to be modified we can take this >> > > > approach: >> > > > >> >> > > > >> - Create a branch of extensions for the upcoming core >> release. For >> > > > >> instance, "ignite-core-2.9+" branch. That's just the branch >> name >> > > (and >> > > > >> not >> > > > >> any Maven artifact name) with "+" sign implying that the >> updated >> > > > >> extensions >> > > > >> will work for Ignite 2.9 and later until we need to update >> them >> > > again >> > > > >> creating a release branch like "ignite-core-2.14+" >> > > > >> - If only a subset of the extensions was updated then we need >> to >> > > > >> release >> > > > >> those extensions to Maven. The goal is to avoid the practice >> of >> > > > >> publishing >> > > > >> Flink or any other extension to Maven for every core release >> if >> > > there >> > > > >> are >> > > > >> no changes. >> > > > >> - As for a ZIP archive, we should prepare the archive for a >> > > download >> > > > >> with the name like "ignite-core-2.9+" >> > > > >> >> > > > >> >> > > > >> - >> > > > >> Denis >> > > > >> >> > > > >> >> > > > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra < >> > > saikat.mai...@gmail.com> >> > > > >> wrote: >> > > > >> >> > > > >> > Hello, >> > > > >> > >> > > > >> > I wanted to connect and discuss on the release process for >> > > > >> > ignite-extensions. As of today all our integrations since >> released >> > > > >> together >> > > > >> > were able to run build based on latest snapshot for example the >> > > > current >> > > > >> > build depends on 2.8.0-SNAPSHOT. If we are making >> ignite-extensions >> > > as >> > > > >> > separate project with different release cycle then it make >> sense to >> > > > have >> > > > >> > dependencies on core modules based on released artifact for >> example >> > > > the >> > > > >> > dependency for ignite-core would be 2.7.6 >> > > > >> > >> > > > >> > Please review and share your thoughts. >> > > > >> > >> > > > >> > PR https://github.com/apache/ignite-extensions/pull/1 >> > > > >> > >> > > > >> > Regards >> > > > >> > Saikat >> > > > >> > >> > > > >> >> > > > > >> > > > >> > > >> >> >> >> -- >> Best regards, >> Ivan Pavlukhin >> >
