Question posted to Apache Arrow at https://lists.apache.org/thread/t6598v7lsdb74q65x9grwhxqt38ntq2s
On Tue, Jan 24, 2023 at 9:00 PM Matthew Benedict de Detrich < [email protected]> wrote: > > The Apache Arrow project publishes nightlies here. It's where the > current nightlies GHA was copied from. > > > https://nightlies.apache.org/arrow/java/org/apache/arrow/ > > Thanks for pointing this out, I must have missed it since it was nested in > a java folder. I will contact the Apache Arrow project to ask them if they > had specific reasons as to why they ended up using the nightlies folder, > there might be something I am missing. > > Also just want to mention that regardless of what was done, the work put > into this isn't wasted as we need to use the same delivery mechanism for > official Apache release. Likewise I imagine that publishing documentation > would work similarly. > > On Tue, Jan 24, 2023 at 5:23 PM Sean Glover <[email protected]> wrote: > >> I'll defer to whatever Apache policies we need to adhere to and/or what >> this group decides, but to address the point: >> >> > In fact, after >> manually checking each Apache project directory in the nightlies >> directories at https://nightlies.apache.org there is no single other >> project that is publishing library jars here. >> >> The Apache Arrow project publishes nightlies here. It's where the current >> nightlies GHA was copied from. >> >> https://nightlies.apache.org/arrow/java/org/apache/arrow/ >> >> On Tue, Jan 24, 2023 at 10:23 AM Keith Wansbrough >> <[email protected]> wrote: >> >>> Aha, thanks - I see what you mean now! >>> >>> From: Matthew Benedict de Detrich <[email protected]> >>> Sent: Tuesday, January 24, 2023 3:21 PM >>> To: [email protected] >>> Subject: Re: Process for snapshots and nightlies >>> >>> > If using Maven, do we still need to control the number of snapshots? >>> >>> There is an already existing job in Nexus which automates this for all >>> Apache projects with the configuration that I mentioned earlier. >>> >>> On Tue, Jan 24, 2023 at 4:18 PM Keith Wansbrough >>> <[email protected]<mailto: >>> [email protected]>> wrote: >>> I agree, let's put snapshots in Maven rather than Nexus. Thanks for the >>> detailed analysis. >>> >>> If using Maven, do we still need to control the number of snapshots? >>> >>> Doing snapshots on a per-merged-PR basis rather than nightly makes a lot >>> of sense, both for the reasons you state (most nightlies would be identical >>> just now!) and because it would deliver a snapshot sooner when there is a >>> change. Of course we should still run tests nightly, to catch any >>> regressions in dependencies etc. >>> >>> Cheers, >>> >>> --KW 8-) >>> >>> -----Original Message----- >>> From: PJ Fanning <[email protected]<mailto:[email protected]>> >>> Sent: Tuesday, January 24, 2023 1:41 PM >>> To: [email protected]<mailto:[email protected]> >>> Cc: [email protected]<mailto:[email protected]> >>> Subject: Re: Process for snapshots and nightlies >>> >>> Since maven snapshots appear to be the norm - my view is that if we can >>> get the maven snapshot working then we can switch off the nightlies (after >>> we see the snapshots are publishing ok). >>> >>> >>> On Tue, 24 Jan 2023 at 13:22, Matthew Benedict de Detrich < >>> [email protected]<mailto:[email protected]>.invalid> >>> wrote: >>> > >>> > As a result of the ongoing work that's been happening to get Pekko >>> > ready for release, more concretely >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884428450%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QP65FTIdi1yjb4SEq9%2BK4Iu2isZGNSiqOQ6775O48sw%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884428450%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DKaAbpiW4fLkSBsDiYsAWmP3foqOWDnxlQq77%2BoR39Y%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko%2Fpull%2F105%23discussion_r107191293 >>> > 8&data=05%7C01%7Ckeith.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884428450%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uDaSO6e9joUMpG6xSPZbhITD1tVX%2FB7AxUra5Vfxldw%3D&reserved=0>%7C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MlgROoPKh9o0KVldS9rsHYIDaDJgjg1fXqODfuvquAY%3D&reserved=0 >>> I came to the conclusion that I think its a smart idea to clarify what our >>> process is going to be in regards to snapshots/nightlies. >>> > >>> > The current state of affairs is that we have a nightly job which >>> > manually packages Pekko and then pushes the jar to the Apache >>> > nightlies repository ( >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnigh >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnigh%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884428450%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=te9KMyZddBTT0O5ZmfaevdICwVk6g446fFOjZlSeVXI%3D&reserved=0 >>> > >>> > tlies.apache.org< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftlies.apache.org%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1UGYxWPWwGOCllqCVYEcVTpZRCqAebzDDVPv%2F6zrDiQ%3D&reserved=0 >>> >%2Fpekko%2F&data=05%7C01%7Ckeith.wansbrough%40microsof >>> > t.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Ft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mcvAPJKK5Hvqj%2Btadiul7Kc1XLTPdqPJY4wB8oEveqk%3D&reserved=0 >>> >%7C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd01 >>> > 1db47%7C1%7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4 >>> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7 >>> > C&sdata=qLuz83av%2FK84wZd4DF7BbbVNaMcsIgxb0y9p0ebj5EM%3D&reserved=0) >>> > via rsync, see >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7fUEccieWJqRtcPZp4eHc6yR8AxAMqJy7SqRrS5ouf0%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V9ZAynvlQfU2Vb3DuSP0O3p2h9OF9J%2F8YDI%2FNwIyLC4%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko%2Fblob%2Fmain%2F.github%2Factions%2F >>> > sync-nightlies%2Faction.yml&data=05%7C01%7Ckeith.wansbrough%40microsof >>> > t.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Ft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mcvAPJKK5Hvqj%2Btadiul7Kc1XLTPdqPJY4wB8oEveqk%3D&reserved=0 >>> >%7C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd01 >>> > 1db47%7C1%7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4 >>> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7 >>> > C&sdata=XV9v3OgmtfYswuagSWmhj8jFBVxA%2FbPeI79X01BuzOM%3D&reserved=0 >>> > and >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7fUEccieWJqRtcPZp4eHc6yR8AxAMqJy7SqRrS5ouf0%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V9ZAynvlQfU2Vb3DuSP0O3p2h9OF9J%2F8YDI%2FNwIyLC4%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko%2Fblob%2Fmain%2F.github%2Fworkflows% >>> > 2Fnightly-builds.yml&data=05%7C01%7Ckeith.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5g8rhx0WhOhaFIaH24RezMqE%2FW7etFTtwlPLo%2Bbm%2Fv4%3D&reserved=0 >>> >%7 >>> > C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd011db47%7 >>> > C1%7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD >>> > AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata >>> > =EFguvIuKu%2Fa7dWLbsoAcJn11QneGzXNjyRW3dq9StWM%3D&reserved=0 >>> > for more details. >>> > >>> > While it's fantastic that we have a nightly system setup (especially >>> > since it unblocks other Pekko modules from being able to update the >>> > package to >>> > org.apache.pekko) the current implementation of it raises a couple of >>> > questions. The first more technical one is that we are using the >>> > nightlies directory as a pseudo maven repository even though Apache >>> > has its own Nexus repo that supports snapshots (see >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Finfr >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Finfr%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fO%2Bw5S0WtHC5iogYJZy7JSMvrJlv8jkMvuFkDy%2FXC5c%3D&reserved=0 >>> > >>> > a.apache.org< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fa.apache.org%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BQZ1KAcatdAAvA5E9FmId58hqcRnAX7ibUKaW%2FbZkLc%3D&reserved=0 >>> >%2Fpublishing-maven-artifacts.html&data=05%7C01%7Ckeith.wa >>> > nsbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5g8rhx0WhOhaFIaH24RezMqE%2FW7etFTtwlPLo%2Bbm%2Fv4%3D&reserved=0 >>> >%7C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf >>> > 86f141af91ab2d7cd011db47%7C1%7C0%7C638101645113113343%7CUnknown%7CTWFp >>> > bGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn >>> > 0%3D%7C3000%7C%7C%7C&sdata=%2Fvfw6niA0D0sH39B8mQisfD3NYphl%2BoQFnK4vY% >>> > 2BzkuI%3D&reserved=0 and >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepo >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepo%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ck2qdhMmz5qDs2AfnVQwuLxHS5zOKf5e97CnvdGG0JY%3D&reserved=0 >>> > >>> > sitory.apache.org< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsitory.apache.org%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884584668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AhSTIqBNOQpOM%2BrijR6edG4v1ktILzWTPbbH4ibC0XY%3D&reserved=0 >>> >%2Fcontent%2Fgroups%2Fsnapshots%2F&data=05%7C01%7Ckei >>> > th.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0 >>> >%7C5fc708f926284ce5593d08dafe10bbcd%7C72f >>> > 988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101645113113343%7CUnknown%7 >>> > CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV >>> > CI6Mn0%3D%7C3000%7C%7C%7C&sdata=82xrBQWLHnwDYAcnPIOa6QH2%2FV9isqCBFR38 >>> > 9QzlPqA%3D&reserved=0). In fact, after manually checking each Apache >>> > project directory in the nightlies directories at >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnigh >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnigh%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HsoIFmoJmbW6uCT8IG%2FScmlyCtfxVlYbfNvEFfd0O0w%3D&reserved=0 >>> > >>> > tlies.apache.org< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftlies.apache.org%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Sx%2BLEg%2F2qFuxCf3ywFxuGcIK5x6MTERXCzMlzE%2FJlZs%3D&reserved=0 >>> >%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0 >>> >%7C >>> > 5fc708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd011db47%7C >>> > 1%7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDA >>> > iLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata= >>> > %2F1tzQycyROl8Mg0Nl0gJxJw0MhBBDX72pJCsiieKrWE%3D&reserved=0 there is >>> > no single other project that is publishing library jars here. The >>> > content that appears to be published there roughly falls under one of >>> > the following >>> > >>> > >>> > * Build results, either executables, fatjars or some other output of a >>> > build >>> > >>> > * Documentation >>> > >>> > * Microsite >>> > >>> > >>> > This means from what I can tell, as of now we are the only project >>> > that is publishing snapshots this way. I asked some various related >>> > questions in the #asfinfra slack channel and they also said that any >>> > kind of JVM snapshot/nightlies should be published to the maven repo, >>> > not the nightlies directory (see >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fthe- >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fthe-%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BhlwTLR1asUFgNfC4MIfhbrS5dKX1mwRUxyTYSVG%2Ffw%3D&reserved=0 >>> > >>> > asf.slack.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fasf.slack.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bBOYl59fju0d6aCNC4NlBRw7%2BpAwM7OWxjyPhDXO6PI%3D&reserved=0 >>> >%2Farchives%2FCBX4TSBQ8%2Fp1674497939153149&data=05%7C01% >>> > 7Ckeith.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0 >>> >%7C5fc708f926284ce5593d08dafe10bbcd% >>> > 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101645113113343%7CUnkn >>> > own%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwi >>> > LCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d0A25pkoFSu9k6b1ONE8BFw0%2Fow3MhK >>> > %2BFvP7MMOSF08%3D&reserved=0 for thread trail). The only other >>> > critical point here is regarding snapshot expiry (current nightlies >>> > setup has a 30 day expiry) but they also confirmed that there already >>> > exists a job on the Apache Nexus repo which performs snapshot expiry >>> > with the following relevant settings >>> > >>> > >>> > * Minimum snapshot count: 2 >>> > >>> > * Snapshot retention (days): 2 >>> > >>> > * Recurrence: Weekly >>> > >>> > >>> > The main thing that was also mentioned in that Slack channel was when >>> > using the Apache snapshot Nexus repo we should keep snapshots to >>> > minimum required which then leads to the second point of how we should >>> > handle snapshot creation that is useful/brings value (note that to me >>> > snapshots are analogous to nightlies, it's just the latter describes a >>> > specific way of creating snapshots). Since Pekko (and its related >>> > modules) is a JVM library, not a >>> > package/application/executable/database I think that nightly snapshot >>> > creation is excessive. Nightlies make sense for projects that need to >>> > be run because often the process of packaging/building it is >>> > non-trivial and a large portion of users are not developers. On the >>> > other hand with Pekko being a library, almost all of our users are >>> > developers which use Pekko by adding it as a dependency to their build >>> tool (maven, gradle, sbt etc etc). >>> > >>> > Due to this, one of the main problems that nightlies are solving (i.e. >>> > testing a change which hasn't been released yet) can be trivially done >>> > by just pointing a checked out Pekko git repo to a specific point in >>> > time and doing publishLocal which publishes the entire Pekko modules >>> > to local ivy repo which can then be picked up by other JVM build tools >>> > on that local machine (note that publishLocal was also added to the >>> > useful sbt commands, see >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jP4iPybBsRmOFlkWJBXJS45uFCHtxWroe4ASkOSLjR8%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aihB2erTRLkcSFvea%2BhAOwdcs4z6J0Bj%2FWSphMw2HUA%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko%2Fpull%2F83&data=05%7C01%7Ckeith.wan >>> > sbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0 >>> >%7C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf8 >>> > 6f141af91ab2d7cd011db47%7C1%7C0%7C638101645113113343%7CUnknown%7CTWFpb >>> > GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0 >>> > %3D%7C3000%7C%7C%7C&sdata=YHT9o%2B22u3wyKn%2B%2FgKMvZzI8M2z9IIp2FdRuZo >>> > mNarI%3D&reserved=0). The other typical use case that needs to be >>> > handled is when someone needs to test a specific version/revision of >>> > Pekko with a feature in a pre-production/production setting (i.e. not >>> > a local machine) to see if its working as intended, snapshots in a >>> > remote repository are helpful here. Snapshots are also handy when >>> > having to deal with Pekko modules that require Pekko (some Pekko >>> > modules such as pekko-http happen to have custom sbt code to lookup >>> > the latest snapshot, see >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jP4iPybBsRmOFlkWJBXJS45uFCHtxWroe4ASkOSLjR8%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aihB2erTRLkcSFvea%2BhAOwdcs4z6J0Bj%2FWSphMw2HUA%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko-http%2Fblob%2Fmain%2Fproject%2FAkkaD >>> > ependency.scala&data=05%7C01%7Ckeith.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0 >>> >%7C5fc7 >>> > 08f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C >>> > 0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJ >>> > QIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2Fw >>> > d0h7KzDMj8qaYQ%2B1GziIS0PFnxy76ryR9DGfjn4fw%3D&reserved=0 >>> > and >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jP4iPybBsRmOFlkWJBXJS45uFCHtxWroe4ASkOSLjR8%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aihB2erTRLkcSFvea%2BhAOwdcs4z6J0Bj%2FWSphMw2HUA%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko-http%2Fblob%2Fmain%2Fproject%2FVersi >>> > onGenerator.scala&data=05%7C01%7Ckeith.wansbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0 >>> >%7C5f >>> > c708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd011db47%7C1% >>> > 7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL >>> > CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1s >>> > D7BN5wjXaXMZ6x1NocUESJvUcJ9o%2B1de%2FTy%2FsXbaQ%3D&reserved=0, >>> > although modifying that custom snapshot lookup code to also check >>> > local ivy repositories also seems sensible). >>> > >>> > Given this, what to me personally makes sense is that snapshots should >>> > be published when a PR is merged to main, at least for now since we >>> > don't have that many PR's being merged, the amount of snapshots being >>> > created is likely going to be less than nightly (on average) which >>> > should satisfy the "minimum necessary". However I am perfectly happy >>> > with keeping the current nightly build because there are other >>> > considerations as well(i.e. should we deploy a snapshot only when a >>> > nightly build + test runs?) and in the future when Pekko picks up >>> steam this can change. >>> > >>> > >>> > On a similar note, a proposed ideal nice to have in the future would >>> > be to implement a GitHub bot where a maintainer can trigger the bot >>> > via a GitHub comment on a PR to trigger a snapshot of that specific PR >>> > before it gets merged, this can then handle the use case of "I want to >>> > test these changes on actual use case scenario before merging the PR". >>> > Since Pekko build currently distinguishes between clean and unclean >>> > artifacts via a timestamp (see >>> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith >>> < >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jP4iPybBsRmOFlkWJBXJS45uFCHtxWroe4ASkOSLjR8%3D&reserved=0 >>> > >>> > ub.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fub.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aihB2erTRLkcSFvea%2BhAOwdcs4z6J0Bj%2FWSphMw2HUA%3D&reserved=0 >>> >%2Fapache%2Fincubator-pekko%2Fpull%2F125&data=05%7C01%7Ckeith.wa >>> > nsbrough%40microsoft.com< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884740897%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WMB6VKgF20F25k6XwumldyN270xrHz%2BkWypuOGHzF4M%3D&reserved=0>%7C5fc708f926284ce5593d08dafe10bbcd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101645113113343%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KwXIqCcMKyp%2FR%2BxTp1MyGbwZvnW4Gu6zhJwJx6BYQVM%3D&reserved=0 >>> for a recent PR that fixes this) this also means that we can also separate >>> snapshots generated from PR's from snapshots generated by the main job >>> (whether that be nightly or after merge into main). >>> > >>> > What are everyones thoughts on this? >>> > >>> > -- >>> > >>> > Matthew de Detrich >>> > >>> > *Aiven Deutschland GmbH* >>> > >>> > Immanuelkirchstraße 26, 10405 Berlin >>> > >>> > Amtsgericht Charlottenburg, HRB 209739 B >>> > >>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen >>> > >>> > *m:* +491603708037 >>> > >>> > *w:* aiven.io< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Faiven.io%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884897118%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WoT73jmrWClF7rWL78N%2Fd4D6JMfa%2BsnfVsTO4a0RKXw%3D&reserved=0> >>> *e:* [email protected]<mailto:[email protected]> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected]<mailto: >>> [email protected]> For additional commands, e-mail: >>> [email protected]<mailto:[email protected]> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected]<mailto: >>> [email protected]> >>> For additional commands, e-mail: [email protected]<mailto: >>> [email protected]> >>> >>> >>> -- >>> >>> Matthew de Detrich >>> >>> Aiven Deutschland GmbH >>> >>> Immanuelkirchstraße 26, 10405 Berlin >>> >>> Amtsgericht Charlottenburg, HRB 209739 B >>> >>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen >>> >>> m: +491603708037 >>> >>> w: aiven.io< >>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Faiven.io%2F&data=05%7C01%7Ckeith.wansbrough%40microsoft.com%7C63133f336219439d40f108dafe1ea6c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638101704884897118%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WoT73jmrWClF7rWL78N%2Fd4D6JMfa%2BsnfVsTO4a0RKXw%3D&reserved=0> >>> e: [email protected]<mailto:[email protected]> >>> >> > > -- > > Matthew de Detrich > > *Aiven Deutschland GmbH* > > Immanuelkirchstraße 26, 10405 Berlin > > Amtsgericht Charlottenburg, HRB 209739 B > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > *m:* +491603708037 > > *w:* aiven.io *e:* [email protected] > -- Matthew de Detrich *Aiven Deutschland GmbH* Immanuelkirchstraße 26, 10405 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen *m:* +491603708037 *w:* aiven.io *e:* [email protected]
