Using Kotlin we no longer need templates, as any build configuration (as object) can be heavily customised, parametrised and reused as many times as you like.
However, if we are going to separate build settings per project, in me experience — we may need some kind of lib with common customisations and custom build settings objects, like maven or script step runners. Otherwise there will be lots of duplicate code. But syncing secrets will be a bit tricky. Currently custom UID should be created for the project with secret, which can be later reused in Kotlin DSL as reference. Secret is indeed kept on the hard drive of the server with TC instance. And I am not sure that we can add the same UID on the second instance. > On 17 Dec 2021, at 15:32, Виталий Осилов <osipov.wita...@gmail.com> wrote: > >> I thought we can store 3.x settings in 3.x repo and so on. > > It's not a good idea to use different repositories. > > Ignite also has many different modules, but all are situated in the same > project. > > > If we use different repositories, we can get a situation when a template > from another project will be deleted. > > Best regards, > Osipov Vitaliy > osipov.wita...@gmail.com > > > пт, 17 дек. 2021 г. в 15:14, Anton Vinogradov <a...@apache.org>: > >> Petr, >> >>> I strongly suggest avoiding a separate repository for project settings. >>> Let's store them in https://github.com/apache/ignite >> >> Sounds good, but we must avoid dozens of additional commits in this case. >> Each commit should be properly formalized and related to the issue. >> We may create a special Jira project for CI issues to separate commits. >> >> On Fri, Dec 17, 2021 at 2:46 PM Pavel Tupitsyn <ptupit...@apache.org> >> wrote: >> >>> Petr, >>> >>>> why should I edit code in Apache Ignite 2.x repo to introduce new >> changes >>> in Apache Ignite 3.x build settings >>> >>> I thought we can store 3.x settings in 3.x repo and so on. >>> >>> Looks like it does not work as I hoped it would. >>> Thanks for your answers. >>> >>> On Fri, Dec 17, 2021 at 2:35 PM Petr Ivanov <mr.wei...@gmail.com> wrote: >>> >>>> Pavel, >>>> >>>> >>>> If you are referring to this paragraph >>>> >>>> • If you are using TeamCity feature branches, you can define a >>>> branch specification when creating a project from URL (Git only) or in >>> the >>>> VCS root used for versioned settings. TeamCity will run a build in a >>> branch >>>> using the settings from this branch. >>>> >>>> then it is only for new projects. >>>> After setting current one all settings will be acquired from the chosen >>>> branch and it will not be able to change (only create new project from >>> new >>>> branch). >>>> >>>> >>>> And it still can be done from separate repository. >>>> >>>> >>>> Choosing one of the project's repo to host settings for every other >>>> project seems non-logical — why should I edit code in Apache Ignite 2.x >>>> repo to introduce new changes in Apache Ignite 3.x build settings? >>>> >>>>> On 17 Dec 2021, at 14:28, Pavel Tupitsyn <ptupit...@apache.org> >> wrote: >>>>> >>>>> Petr, >>>>> >>>>>> you cannot run new suite added in branch because it just won't be >>>> visible >>>>> from UI >>>>> >>>>> That's unfortunate. >>>>> However, a more important scenario is "make changes to the existing >>>> project >>>>> in a branch", which is supported, as I understand [1] >>>>> >>>>> [1] >>>>> >>>> >>> >> https://www.jetbrains.com/help/teamcity/storing-project-settings-in-version-control.html#Defining+Settings+to+Apply+to+Builds >>>>> >>>>> On Fri, Dec 17, 2021 at 2:19 PM Petr Ivanov <mr.wei...@gmail.com> >>> wrote: >>>>> >>>>>> TeamCity DSL just does not work as you wish it should. >>>>>> >>>>>> >>>>>> Changes from branches are not visible: you cannot run new suite >> added >>> in >>>>>> branch because it just won't be visible from UI because project UI >> is >>>>>> rendered from default branch only), >>>>>> and at least snapshot dependencies are taken from default branch >> only: >>>>>> even if you will add dependency on already visible Run All >>>> configuration, >>>>>> it will be ignored. >>>>>> >>>>>> >>>>>> Also about storing in already existing apache/ignite repository — >> that >>>>>> will break the model about current separation on ignite and >> ignite-3, >>>>>> ignite-extensions and so-on. >>>>>> As we have different repositories for different parts of our project >>> and >>>>>> TeamCity unites them all — repository should be separate. >>>>>> And when JB will fix the branches issues — we still will be able to >>> use >>>> it >>>>>> with creating branches with the same name on both project and TC >>>> repository. >>>>>> >>>>>>> On 17 Dec 2021, at 14:06, Pavel Tupitsyn <ptupit...@apache.org> >>> wrote: >>>>>>> >>>>>>> Witaliy, >>>>>>> >>>>>>>> repository is created >>>>>>>> only in the master branch >>>>>>> >>>>>>> I strongly suggest avoiding a separate repository for project >>> settings. >>>>>>> Let's store them in https://github.com/apache/ignite >>>>>>> >>>>>>> *1. We should be able to test code changes together with CI/CD >>>> changes.* >>>>>>> *2. We should be able to have different project settings in >> different >>>>>>> branches.* >>>>>>> >>>>>>> For example, I may remove or rename a module in ignite/master >> branch >>>> and >>>>>>> update TC project accordingly, >>>>>>> while it continues to work as before in ignite-2.12 branch where we >>>>>> prepare >>>>>>> the release with older code. >>>>>>> >>>>>>> This is super important and this is how most other projects do it >>> (from >>>>>> my >>>>>>> experience). >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Dec 17, 2021 at 11:12 AM Виталий Осилов < >>>>>> osipov.wita...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Dear Ignite Community! >>>>>>>> >>>>>>>> I propose for discussion the conception of using two TeamCity >>> servers >>>>>> with >>>>>>>> a roadmap. >>>>>>>> https://ci.ignite.apache.org/ >>>>>>>> https://ci2.ignite.apache.org/ >>>>>>>> >>>>>>>> Storing project settings. >>>>>>>> Servers synchronize configurations between themselves using the >>>> version >>>>>>>> control-storing DSL (repository at https://github.com/apache). >>>> TeamCity >>>>>>>> allows to store the settings in the DSL based on the kotlin >>> language. >>>>>>>> You can read more about the version control-storing DSL here >>>>>>>> >>>>>>>> >>>>>> >>>> >>> >> https://www.jetbrains.com/help/teamcity/2021.2/kotlin-dsl.html#Getting+Started+with+Kotlin+DSL >>>>>>>> This scheme will allow to maintain a single configuration for both >>> TC >>>>>> and >>>>>>>> update configuration after a code review. >>>>>>>> Changes in the suite should be made only in the master branch of >> the >>>>>> stored >>>>>>>> configuration. WebUI should be disabled on both TC. >>>>>>>> The code-modified PR should be tested for compatibility before >>>> approval. >>>>>>>> >>>>>>>> Configuring authentication. >>>>>>>> It is proposed to switch sign in to TeamCity with using GitHub.com >>>>>> account >>>>>>>> ( >>>>>>>> >>>>>>>> >>>>>> >>>> >>> >> https://www.jetbrains.com/help/teamcity/configuring-authentication-settings.html#GitHub.com >>>>>>>> ) >>>>>>>> and restriction of access for the organization (https : // >>>>>>>> >>>>>>>> >>>>>> >>>> >>> >> docs.github.com/en/organizations/collaborating-with-groups-in-organizations/about-organizations >>>>>>>> .) >>>>>>>> User rights are changed on each server by request in Jira. >>>>>>>> >>>>>>>> Storing security credentials. >>>>>>>> All credentials are stored on each server in the file >>>>>> "<TEAMCITY_DATA_DIR> >>>>>>>> / config / projects / <PROJECT_ID> >>>> /pluginData/secure/credentials.json". >>>>>>>> That is why it is supposed to configure the synchronization of >> this >>>> file >>>>>>>> between two servers. >>>>>>>> >>>>>>>> Maintenance. >>>>>>>> All change requests of server's maintenance should be created in >>> Jira >>>>>>>> https://issues.apache.org/jira/projects/IGNITE >>>>>>>> <https://issues.apache.org/jira/projects/IGNITE/summary> >>>>>>>> Responsible for the servers and agents - TC1 Petr Ivanov and TC2 >>>> Vitaly >>>>>>>> Osipov. They are responsible for the health of the servers >> (backups, >>>>>>>> updates, agents, etc.) >>>>>>>> Both TC servers must be updated sequentially with a minimum time >>>>>> interval >>>>>>>> by request in Jira. >>>>>>>> --- >>>>>>>> At the first stage, it is required to synchronize the differences >> in >>>>>>>> configurations. The configuration from TC1 is uploaded to TC2. >>>>>>>> If changes are required in this configuration, tasks are created >> in >>>>>> Jira to >>>>>>>> change settings in TC1, then new configuration is re-uploaded to >>> TC2. >>>>>>>> Repository is created at https://github.com/apache. Tech-user is >>>>>> created >>>>>>>> for connecting to this repository. Tech-user should have RW >> access >>> in >>>>>> the >>>>>>>> master branch. >>>>>>>> Checking TC2 functionality for several weeks. >>>>>>>> Stage 2 - The option of "synchronization of the project settings >>> with >>>>>> the >>>>>>>> version control system" must be allowed for the root-project on >> TC1 >>>> and >>>>>>>> TC2. >>>>>>>> The option "*Allow editing project settings via UI" must be >> allowed >>>> on >>>>>> TC1 >>>>>>>> and disabled on TC2.* >>>>>>>> Checking TC2 functionality for several weeks. >>>>>>>> Stage 3 - Switch sign in to TeamCity with using GitHub with >> linking >>>> TC >>>>>>>> users to their accounts on the github.com. >>>>>>>> Stage 4 - Configuring synchronization of secret files >>>>>>>> "secure/credentials.json". Synchronization is one-way, so security >>>> creds >>>>>>>> can only be made on the TC1 server. Configuring upload and >> download >>>>>> files >>>>>>>> if they change. >>>>>>>> Stage 5 - Setting up of checking PR before approval. >>>>>>>> Stage 6 - Disabling the ability to edit configurations via WebUI >> on >>>> TC1 >>>>>>>> >>>>>> >>>>>> >>>> >>>> >>> >>
