Tison, > On Apr 28, 2024, at 12:10 PM, tison <wander4...@gmail.com> wrote: > > Thank you, Dave :D > > It gives a reason. I'm OK with this explanation now so that I won't bring > it to the INFRA. > > Back to the original purpose of this thread, I suggest: > > 1. Go through our Incubator Guide and find if we have some references to > this release distribution policy (maybe in [1][2]). Then, make this > requirement clear and discoverable. > 2. Try to implement the preview feature and add the verify items in one > project so later we use it as a reference. > > [1] https://incubator.apache.org/guides/distribution.html > [2] https://incubator.apache.org/guides/releasemanagement.html
FYI - The incubator clutch analysis includes generated release download sections for all releases it finds in the incubator directory on the dist.apache.org svn repository: For example: https://incubator.apache.org/clutch/streampark.html See https://svn.apache.org/repos/asf/incubator/public/trunk/clutch2status.py: print("== Releases",file=f) print("",file=f) print("=== Current",file=f) print("",file=f) hasKeys = False url1 = urls['keys'] if len(url1) > 0: hasKeys = True print("*{0}[Signing Keys]*".format(url1),file=f) url1 = "/".join(url1.split("/")[:-1])+"/" print("",file=f) print("It is essential that you verify the integrity of release downloads. See https://www.apache.org/dyn/closer.cgi#verify[instructions here]",file=f) else: print("*No PGP Signing Keys*",file=f) jj = 0 fsize = 0 if len(projects[k]['releases']) > 0 and hasKeys: j = 1 for r in sorted(projects[k]['releases']): path = "{0}/{1}/{2}".format(k,projects[k]['releases'][r]['folder'],r) print("",file=f) print("==== {0}: {1}".format(j,r),file=f) print("",file=f) print("| *{0}{1}[Download]* | *{2}{3}.asc[Signature]* | *{4}{5}.{6}[Hash]* | {7}".format( "https://www.apache.org/dyn/closer.lua/incubator/";, path, "https://downloads.apache.org/incubator/";, path, "https://downloads.apache.org/incubator/";, path, projects[k]['releases'][r]['hash'], r),file=f) print("-- Filesize: {0}".format(humanbytes(float(projects[k]['releases'][r]['size']))),file=f) print("-- Released: {0} by {1} in {2}".format( projects[k]['releases'][r]['dtm'], projects[k]['releases'][r]['user'], projects[k]['releases'][r]['revision']),file=f) j += 1 fsize += int(projects[k]['releases'][r]['size']) print("",file=f) print("Total size of all downloads = {0}".format(humanbytes(float(fsize))),file=f) elif hasKeys: print(NO_RELEASES.format(projects[k]['fullName']),file=f) print("",file=f) And https://svn.apache.org/repos/asf/incubator/public/trunk/clutch2.py with osPopen(['svn', 'ls', '-Rv', 'https://dist.apache.org/repos/dist/release/incubator']) as s: for line in s.stdout: line = line.strip() if line[-1:] == '/': # skip directories continue listing = line.split(' ') revision = "r{0}".format(listing[0]) user = listing[1] if listing[-6] == '': dtm1 = datetime.datetime.strptime(" ".join(listing[-4:-2])+" "+str(gatherYear),"%b %d %Y") if dtm1 > gatherDate: dtm1 = datetime.datetime.strptime(" ".join(listing[-4:-2])+" "+str(gatherYear-1),"%b %d %Y") fsize = listing[-5] else: dtm1 = datetime.datetime.strptime(" ".join(listing[-5:-1]),"%b %d %Y") fsize = listing[-6] dtm = dtm1.strftime("%m/%d/%Y") line = listing[-1] releasesListing[line] = { 'user': user, 'revision': revision, 'dtm': dtm, 'size': fsize } fields = line.split('/') podling = fields[0] distareas[podling] = True file = fields[-1] if file: if re.search('KEYS(\.txt)?$', file): keysList[podling] = "{0}/{1}".format("https://downloads.apache.org/incubator";, line) if re.search('\.(asc|sig)$', file, flags=re.IGNORECASE): path = "/".join(fields[1:]) if optionVerbose: print("DEBUG: {0} - {1}".format(podling,path)) try: if distributions[podling]: distributions[podling].append(path) except: distributions[podling] = [] distributions[podling].append(path) if re.search('incubat(ing|or)', file, flags=re.IGNORECASE): releases[podling] = True else: releasesBadName[podling] = True if podling in projects and podling != "netbeans": # netbeans has over 600 release parts and had legacy naming requirements errorMsg = "WARN: {0}: Release is missing incubator/incubating in file name {1}".format(podling,file) print(errorMsg) errorMsg = ["42","Distribution is missing incubator/incubating in file name {0}".format(file)] projects[podling]['zFixMeList'].append(errorMsg) if re.search('\.(sha512|sha1|sha256|sha|md5)$', file, flags=re.IGNORECASE): path = "/".join(fields[1:]) if optionVerbose: print("DEBUG: {0} - {1}".format(podling,path)) try: # some projects have done a hash on the detached signature. Identify and ignore. part0 = ".".join(path.split('.')[-2:-1]) if part0 == "asc": continue # see if this is not the first hash for this release # sha512 is preferred and it will be the last provided. part1 = ".".join(distribHash[podling][-1].split('.')[:-1]) part2 = ".".join(path.split('.')[:-1]) if part1 == part2: distribHash[podling][-1] = path else: distribHash[podling].append(path) except: distribHash[podling] = [] distribHash[podling].append(path) Best, Dave > > I'll try my best to find time to work on it, but I don't declare it an > assignment. > > Best, > tison. > > > Dave Fisher <w...@apache.org> 于2024年4月29日周一 02:46写道： > >> >> >>> On Apr 28, 2024, at 9:58 AM, tison <wander4...@gmail.com> wrote: >>> >>> Thank you! >>> >>> I found there is no rationale for these links, and thus, it's quite a bit >>> challenging in memory. >>> >>> IIRC the closer.lua script is for selecting the most proper CDN for >>> source/binary bundles in use. They can, technically, work for SHASUM and >>> signatures also. Why do we use https://downloads.apache.org for the >> latter >>> two? >> >> Historically we had a mirror network and closer.lua picked out a mirror >> near you. In order to be sure that the download source or binary on the >> mirror was not altered on (or on its way to or from) the mirror, the >> detached signature and checksums must be served from ASF controlled >> resources. >> >> Whether or not this still makes sense is a discussion for Infra since they >> are charged with enforcing and supporting the release distribution policy. >> >> Best, >> Dave >> >>> >>> Best, >>> tison. >>> >>> >>> sebb <seb...@gmail.com> 于2024年4月29日周一 00:34写道： >>> >>>> On Sun, 28 Apr 2024 at 15:38, tison <wander4...@gmail.com> wrote: >>>>> >>>>> Yeah. I support that we always need to release sources on our platform. >>>>> >>>>> Given the links to downloads.apache.org, archive.apache.org, >>>>> https://www.apache.org/dyn/closer.lua, can be unintuitive for users, I >>>>> agree that we can have a simple Download page for such library-only >>>>> projects. >>>> >>>> The download page can also be used for links to release notes, and to >>>> provide other support information. >>>> >>>>> Here is a patch to cover a minimal download page [1], which is derived >>>> from >>>>> OpenDAL's download page [2]. Welcome to leave comments if you find any >>>>> issues or things we can improve on. >>>>> >>>>> [1] https://github.com/apache/datafusion/pull/10271 >>>> >>>> The closer.lua script is only intended for the source and binary >> bundles. >>>> >>>> The sigs and hashes (and KEYS) should link directly to >>>> https://downloads.apache.org/datafusion/... >>>> >>>>> [2] https://opendal.apache.org/download >>>>> >>>>> Best, >>>>> tison. >>>>> >>>>> >>>>> Justin Mclean <jus...@classsoftware.com> 于2024年4月28日周日 10:02写道： >>>>> >>>>>> Hi, >>>>>> >>>>>> Projects need to make source releases on ASF infrastructure and have a >>>>>> download page for good reasons. Some users need a place to verify and >>>>>> download a trusted release. Having it hosted on ASF infrastructure >>>> means >>>>>> people can 100% trust it, unlike 3rd party providers. 3rd party >>>> providers >>>>>> have gone rogue in the past (e.g . Source Forge), disappeared (e.g. >>>> Google >>>>>> Code), or had multiple serious issues (e.g. NPM). Also by placing a >>>> release >>>>>> in the ASF distribution area / a project download page gives >> confidence >>>>>> that the ASF release process has been followed and that it is not a >>>> release >>>>>> by a 3rd party or an unofficial release of some sort. IMO, all >>>> projects >>>>>> need to have a download page, even if it may not be used by the >>>> majority of >>>>>> users. >>>>>> >>>>>> Kind Regards, >>>>>> Justin >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>>>>> For additional commands, e-mail: general-h...@incubator.apache.org >>>>>> >>>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>>> For additional commands, e-mail: general-h...@incubator.apache.org >>>> >>>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> For additional commands, e-mail: general-h...@incubator.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
