Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-13 Thread Sarah Gilmore
Hi Khou and Raúl,

> Yes. We should use apache/arrow's GitHub Releases.

We agree using apache/arrow's GitHub Releases the right thing to do.

​> GitHub releases do have a prerelease/rc status that can be activated. Maybe
> that could be used as an indicator to not include theses on the exchange
> site?

We just got confirmation from the File Exchange development team that the File 
Exchange-GitHub Integration ignores pre-releases. So marking release candidates 
as pre-releases should prevent the File Exchange entry from linking to them.

>  Does it just use "polling"? Or do we need to install any
> GitHub App, set secret variable or something on
> apache/arrow? If the latter, we need to ask INFRA to do it.

The current version of the File Exchange GitHub integration relies on polling. 
If this changes, we'll followup.

Based on the discussion so far, it seems like we have a clear path forward for 
integrating the MATLAB Interface into the Apache Arrow release process via 
GitHub Releases.

Thanks again for all the advice!

Best,
Sarah Gilmore


From: Sutou Kouhei 
Sent: Sunday, November 12, 2023 4:25 PM
To: dev@arrow.apache.org 
Cc: Lei Hou ; Sarah Gilmore 
Subject: Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the 
MATLAB interface

Yes. We should use apache/arrow's GitHub Releases.

In 
"Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB 
interface" on Fri, 10 Nov 2023 19:11:16 +0100,
Raúl Cumplido  wrote:

> In case it was not clear, even though the binary job is run on
> ursacomputing/crossbow when we upload the binaries and create the
> Release that should be, at least in my opinion, an apache/arrow
> release.
>
> Both for the steps:
> 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> and
> 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
>
> El vie, 10 nov 2023 a las 19:02, Raúl Cumplido () escribió:
>>
>> Hi Sara,
>>
>> El vie, 10 nov 2023 a las 18:48, Sarah Gilmore
>> () escribió:
>> >
>> > Hi Kou,
>> >
>> > > We can use apache/arrow's GitHub Releases. The release
>> > > distribution document says that we can use GitHub as a
>> > > release platform:
>> > > https://infra.apache.org/release-distribution.html#other-platforms<https://infra.apache.org/release-distribution.html#other-platforms>
>> > >
>> > > apache/arrow doesn't use GitHub Releases yet but
>> > > apache/arrow-adbc and apache/arrow-flight-sql-postgresql
>> > > already use GitHub Releases. (We just use "gh release
>> > > upload" to upload our artifacts to GitHub Releases.)
>> >
>> > Thank you for clarifying that we can use apache/arrow's GitHub Releases 
>> > area for hosting the MLTBX file. We assumed we couldn't use the main 
>> > repository, but it's great to hear we can!
>> >
>> > > BTW, how does File Exchange "Connecting to GitHub Repositories"?
>> > > https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
>> > >
>> > > Does it just use "polling"? Or do we need to install any
>> > > GitHub App, set secret variable or something on
>> > > apache/arrow? If the latter, we need to ask INFRA to do it.
>> >
>> > We are currently consulting with the development team responsible for the 
>> > GitHub <-> File Exchange integration. We'll send a followup email with a 
>> > concrete answer once we know more.
>> >
>> > > If we use GitHub Releases on apache/arrow, we can use the
>> > > following workflow. We don't need to use JFrog.
>> > >
>> > > 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
>> > > 2. Release: Run a post release script that would:
>> > > 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
>> > > 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
>> > > 2.3 Linked File Exchange entry will be automatically updated
>> >
>> > This seems like a much more streamlined approach. Not having to upload to 
>> > JFrog will make things easier. Thanks for the suggestion!
>> >
>> > To clarify, in step 1, would we upload the MLTBX to 
>> > ursacomputing/crossbow's GitHub Releases area [1]? Or, would we upload to 
>> > apache/arrow's GitHub Releases area? If we upload release candidates to 
>> > apache/arrow's GitHub Releases area, they would get automatically linked 
>> > to the File Exchange. Ideally, we wouldn&#

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-12 Thread Sutou Kouhei
Yes. We should use apache/arrow's GitHub Releases.

In 
  "Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB 
interface" on Fri, 10 Nov 2023 19:11:16 +0100,
  Raúl Cumplido  wrote:

> In case it was not clear, even though the binary job is run on
> ursacomputing/crossbow when we upload the binaries and create the
> Release that should be, at least in my opinion, an apache/arrow
> release.
> 
> Both for the steps:
> 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> and
> 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
> 
> El vie, 10 nov 2023 a las 19:02, Raúl Cumplido () escribió:
>>
>> Hi Sara,
>>
>> El vie, 10 nov 2023 a las 18:48, Sarah Gilmore
>> () escribió:
>> >
>> > Hi Kou,
>> >
>> > > We can use apache/arrow's GitHub Releases. The release
>> > > distribution document says that we can use GitHub as a
>> > > release platform:
>> > > https://infra.apache.org/release-distribution.html#other-platforms
>> > >
>> > > apache/arrow doesn't use GitHub Releases yet but
>> > > apache/arrow-adbc and apache/arrow-flight-sql-postgresql
>> > > already use GitHub Releases. (We just use "gh release
>> > > upload" to upload our artifacts to GitHub Releases.)
>> >
>> > Thank you for clarifying that we can use apache/arrow's GitHub Releases 
>> > area for hosting the MLTBX file. We assumed we couldn't use the main 
>> > repository, but it's great to hear we can!
>> >
>> > > BTW, how does File Exchange "Connecting to GitHub Repositories"?
>> > > https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
>> > >
>> > > Does it just use "polling"? Or do we need to install any
>> > > GitHub App, set secret variable or something on
>> > > apache/arrow? If the latter, we need to ask INFRA to do it.
>> >
>> > We are currently consulting with the development team responsible for the 
>> > GitHub <-> File Exchange integration. We'll send a followup email with a 
>> > concrete answer once we know more.
>> >
>> > > If we use GitHub Releases on apache/arrow, we can use the
>> > > following workflow. We don't need to use JFrog.
>> > >
>> > > 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
>> > > 2. Release: Run a post release script that would:
>> > > 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
>> > > 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
>> > > 2.3 Linked File Exchange entry will be automatically updated
>> >
>> > This seems like a much more streamlined approach. Not having to upload to 
>> > JFrog will make things easier. Thanks for the suggestion!
>> >
>> > To clarify, in step 1, would we upload the MLTBX to 
>> > ursacomputing/crossbow's GitHub Releases area [1]? Or, would we upload to 
>> > apache/arrow's GitHub Releases area? If we upload release candidates to 
>> > apache/arrow's GitHub Releases area, they would get automatically linked 
>> > to the File Exchange. Ideally, we wouldn't want users to download release 
>> > candidates.
>> >
>>
>> Currently all the binaries are generated on the third step of the
>> Release process [1] when we run `03-binary-submit.sh`. The crossbow
>> job could build the MLTBX artifact and then when we do download the
>> other binaries (`04-binary-download.sh`) we should also download the
>> MTLBX and when we submit the rest to jfrog (`05-binary-upload.sh`) we
>> could Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN.
>>
>> Once the release is approved and we do the post-release tasks to
>> "officially" release, we would download the MLTBX and upload to the
>> new GitHub Releases for apache-arrow-X.Y.Z this can be done as another
>> step on our post-release tasks (post-xx-matlab.sh)
>>
>> [1] 
>> https://arrow.apache.org/docs/developers/release.html#build-source-and-binaries-and-submit-them
>>
>> > > We can use GitHub Releases as I said. But if we use GitHub
>> > > Releases, the release notes on GitHub Releases may include
>> > > not only the MATLAB interface but also all
>> > > implementations. It may not be useful for this use case.
>> > >
>> > > FYI: The R bindings have their release notes under
>> > > https://arrow.apache.o

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-10 Thread Jacob Wujciak-Jens
GitHub releases do have a prerelease/rc status that can be activated. Maybe
that could be used as an indicator to not include theses on the exchange
site?

Sarah Gilmore  schrieb am Fr., 10. Nov.
2023, 21:06:

> Hi Raúl,
>
> > Currently all the binaries are generated on the third step of the
> > Release process [1] when we run `03-binary-submit.sh`. The crossbow
> > job could build the MLTBX artifact and then when we do download the
> > other binaries (`04-binary-download.sh`) we should also download the
> > MTLBX and when we submit the rest to jfrog (`05-binary-upload.sh`) we
> > could Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN.
>
> Thanks for clarifying how these scripts work together. This all makes
> sense. Our one concern is that the Arrow-MATLAB File Exchange entry would
> be automatically updated to show release candidates that have been uploaded
> to apache/arrow's GitHub Releases area. We're looking into how to prevent
> this from happening.
>
> Best,
>
> Sarah Gilmore
>
> 
> From: Raúl Cumplido 
> Sent: Friday, November 10, 2023 1:11 PM
> To: Raúl Cumplido 
> Cc: Sutou Kouhei ; dev@arrow.apache.org <
> dev@arrow.apache.org>; Lei Hou ; Sarah Gilmore <
> sgilm...@mathworks.com>
> Subject: Re: [DISCUSS][MATLAB] Proposal for incremental point releases of
> the MATLAB interface
>
> In case it was not clear, even though the binary job is run on
> ursacomputing/crossbow when we upload the binaries and create the
> Release that should be, at least in my opinion, an apache/arrow
> release.
>
> Both for the steps:
> 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> and
> 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
>
> El vie, 10 nov 2023 a las 19:02, Raúl Cumplido ()
> escribió:
> >
> > Hi Sara,
> >
> > El vie, 10 nov 2023 a las 18:48, Sarah Gilmore
> > () escribió:
> > >
> > > Hi Kou,
> > >
> > > > We can use apache/arrow's GitHub Releases. The release
> > > > distribution document says that we can use GitHub as a
> > > > release platform:
> > > > https://infra.apache.org/release-distribution.html#other-platforms<
> https://infra.apache.org/release-distribution.html#other-platforms>
> > > >
> > > > apache/arrow doesn't use GitHub Releases yet but
> > > > apache/arrow-adbc and apache/arrow-flight-sql-postgresql
> > > > already use GitHub Releases. (We just use "gh release
> > > > upload" to upload our artifacts to GitHub Releases.)
> > >
> > > Thank you for clarifying that we can use apache/arrow's GitHub
> Releases area for hosting the MLTBX file. We assumed we couldn't use the
> main repository, but it's great to hear we can!
> > >
> > > > BTW, how does File Exchange "Connecting to GitHub Repositories"?
> > > >
> https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
> > > >
> > > > Does it just use "polling"? Or do we need to install any
> > > > GitHub App, set secret variable or something on
> > > > apache/arrow? If the latter, we need to ask INFRA to do it.
> > >
> > > We are currently consulting with the development team responsible for
> the GitHub <-> File Exchange integration. We'll send a followup email with
> a concrete answer once we know more.
> > >
> > > > If we use GitHub Releases on apache/arrow, we can use the
> > > > following workflow. We don't need to use JFrog.
> > > >
> > > > 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> > > > 2. Release: Run a post release script that would:
> > > > 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
> > > > 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
> > > > 2.3 Linked File Exchange entry will be automatically updated
> > >
> > > This seems like a much more streamlined approach. Not having to upload
> to JFrog will make things easier. Thanks for the suggestion!
> > >
> > > To clarify, in step 1, would we upload the MLTBX to
> ursacomputing/crossbow's GitHub Releases area [1]? Or, would we upload to
> apache/arrow's GitHub Releases area? If we upload release candidates to
> apache/arrow's GitHub Releases area, they would get automatically linked to
> the File Exchange. Ideally, we wouldn't want users to download release
> candidates.
> > >
> >
> > Currently all the binaries ar

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-10 Thread Sarah Gilmore
Hi Raúl,

> Currently all the binaries are generated on the third step of the
> Release process [1] when we run `03-binary-submit.sh`. The crossbow
> job could build the MLTBX artifact and then when we do download the
> other binaries (`04-binary-download.sh`) we should also download the
> MTLBX and when we submit the rest to jfrog (`05-binary-upload.sh`) we
> could Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN.

Thanks for clarifying how these scripts work together. This all makes sense. 
Our one concern is that the Arrow-MATLAB File Exchange entry would be 
automatically updated to show release candidates that have been uploaded to 
apache/arrow's GitHub Releases area. We're looking into how to prevent this 
from happening.

Best,

Sarah Gilmore


From: Raúl Cumplido 
Sent: Friday, November 10, 2023 1:11 PM
To: Raúl Cumplido 
Cc: Sutou Kouhei ; dev@arrow.apache.org 
; Lei Hou ; Sarah Gilmore 

Subject: Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the 
MATLAB interface

In case it was not clear, even though the binary job is run on
ursacomputing/crossbow when we upload the binaries and create the
Release that should be, at least in my opinion, an apache/arrow
release.

Both for the steps:
1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
and
2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z

El vie, 10 nov 2023 a las 19:02, Raúl Cumplido () escribió:
>
> Hi Sara,
>
> El vie, 10 nov 2023 a las 18:48, Sarah Gilmore
> () escribió:
> >
> > Hi Kou,
> >
> > > We can use apache/arrow's GitHub Releases. The release
> > > distribution document says that we can use GitHub as a
> > > release platform:
> > > https://infra.apache.org/release-distribution.html#other-platforms<https://infra.apache.org/release-distribution.html#other-platforms>
> > >
> > > apache/arrow doesn't use GitHub Releases yet but
> > > apache/arrow-adbc and apache/arrow-flight-sql-postgresql
> > > already use GitHub Releases. (We just use "gh release
> > > upload" to upload our artifacts to GitHub Releases.)
> >
> > Thank you for clarifying that we can use apache/arrow's GitHub Releases 
> > area for hosting the MLTBX file. We assumed we couldn't use the main 
> > repository, but it's great to hear we can!
> >
> > > BTW, how does File Exchange "Connecting to GitHub Repositories"?
> > > https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
> > >
> > > Does it just use "polling"? Or do we need to install any
> > > GitHub App, set secret variable or something on
> > > apache/arrow? If the latter, we need to ask INFRA to do it.
> >
> > We are currently consulting with the development team responsible for the 
> > GitHub <-> File Exchange integration. We'll send a followup email with a 
> > concrete answer once we know more.
> >
> > > If we use GitHub Releases on apache/arrow, we can use the
> > > following workflow. We don't need to use JFrog.
> > >
> > > 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> > > 2. Release: Run a post release script that would:
> > > 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
> > > 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
> > > 2.3 Linked File Exchange entry will be automatically updated
> >
> > This seems like a much more streamlined approach. Not having to upload to 
> > JFrog will make things easier. Thanks for the suggestion!
> >
> > To clarify, in step 1, would we upload the MLTBX to 
> > ursacomputing/crossbow's GitHub Releases area [1]? Or, would we upload to 
> > apache/arrow's GitHub Releases area? If we upload release candidates to 
> > apache/arrow's GitHub Releases area, they would get automatically linked to 
> > the File Exchange. Ideally, we wouldn't want users to download release 
> > candidates.
> >
>
> Currently all the binaries are generated on the third step of the
> Release process [1] when we run `03-binary-submit.sh`. The crossbow
> job could build the MLTBX artifact and then when we do download the
> other binaries (`04-binary-download.sh`) we should also download the
> MTLBX and when we submit the rest to jfrog (`05-binary-upload.sh`) we
> could Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN.
>
> Once the release is approved and we do the post-release tasks to
> "officially" release, we would download the MLTBX and upload to the
> new GitHub Releases for apache-arrow-X.Y.Z this can be done as anothe

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-10 Thread Raúl Cumplido
In case it was not clear, even though the binary job is run on
ursacomputing/crossbow when we upload the binaries and create the
Release that should be, at least in my opinion, an apache/arrow
release.

Both for the steps:
1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
and
2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z

El vie, 10 nov 2023 a las 19:02, Raúl Cumplido () escribió:
>
> Hi Sara,
>
> El vie, 10 nov 2023 a las 18:48, Sarah Gilmore
> () escribió:
> >
> > Hi Kou,
> >
> > > We can use apache/arrow's GitHub Releases. The release
> > > distribution document says that we can use GitHub as a
> > > release platform:
> > > https://infra.apache.org/release-distribution.html#other-platforms
> > >
> > > apache/arrow doesn't use GitHub Releases yet but
> > > apache/arrow-adbc and apache/arrow-flight-sql-postgresql
> > > already use GitHub Releases. (We just use "gh release
> > > upload" to upload our artifacts to GitHub Releases.)
> >
> > Thank you for clarifying that we can use apache/arrow's GitHub Releases 
> > area for hosting the MLTBX file. We assumed we couldn't use the main 
> > repository, but it's great to hear we can!
> >
> > > BTW, how does File Exchange "Connecting to GitHub Repositories"?
> > > https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
> > >
> > > Does it just use "polling"? Or do we need to install any
> > > GitHub App, set secret variable or something on
> > > apache/arrow? If the latter, we need to ask INFRA to do it.
> >
> > We are currently consulting with the development team responsible for the 
> > GitHub <-> File Exchange integration. We'll send a followup email with a 
> > concrete answer once we know more.
> >
> > > If we use GitHub Releases on apache/arrow, we can use the
> > > following workflow. We don't need to use JFrog.
> > >
> > > 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> > > 2. Release: Run a post release script that would:
> > > 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
> > > 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
> > > 2.3 Linked File Exchange entry will be automatically updated
> >
> > This seems like a much more streamlined approach. Not having to upload to 
> > JFrog will make things easier. Thanks for the suggestion!
> >
> > To clarify, in step 1, would we upload the MLTBX to 
> > ursacomputing/crossbow's GitHub Releases area [1]? Or, would we upload to 
> > apache/arrow's GitHub Releases area? If we upload release candidates to 
> > apache/arrow's GitHub Releases area, they would get automatically linked to 
> > the File Exchange. Ideally, we wouldn't want users to download release 
> > candidates.
> >
>
> Currently all the binaries are generated on the third step of the
> Release process [1] when we run `03-binary-submit.sh`. The crossbow
> job could build the MLTBX artifact and then when we do download the
> other binaries (`04-binary-download.sh`) we should also download the
> MTLBX and when we submit the rest to jfrog (`05-binary-upload.sh`) we
> could Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN.
>
> Once the release is approved and we do the post-release tasks to
> "officially" release, we would download the MLTBX and upload to the
> new GitHub Releases for apache-arrow-X.Y.Z this can be done as another
> step on our post-release tasks (post-xx-matlab.sh)
>
> [1] 
> https://arrow.apache.org/docs/developers/release.html#build-source-and-binaries-and-submit-them
>
> > > We can use GitHub Releases as I said. But if we use GitHub
> > > Releases, the release notes on GitHub Releases may include
> > > not only the MATLAB interface but also all
> > > implementations. It may not be useful for this use case.
> > >
> > > FYI: The R bindings have their release notes under
> > > https://arrow.apache.org/docs/r/ . See
> > > https://arrow.apache.org/docs/r/news/ .
> >
> > We think it would still be useful to link to the GitHub release notes from 
> > the File Exchange entry even if it includes notes for all language 
> > bindings. The File Exchange <-> GitHub integration just includes a link to 
> > the GitHub release notes under the Version History tab. If we find having a 
> > more focused version of the release notes would be useful, then we can 
> > create a markdown file analogous to the NEWS.md for the R b

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-10 Thread Raúl Cumplido
Hi Sara,

El vie, 10 nov 2023 a las 18:48, Sarah Gilmore
() escribió:
>
> Hi Kou,
>
> > We can use apache/arrow's GitHub Releases. The release
> > distribution document says that we can use GitHub as a
> > release platform:
> > https://infra.apache.org/release-distribution.html#other-platforms
> >
> > apache/arrow doesn't use GitHub Releases yet but
> > apache/arrow-adbc and apache/arrow-flight-sql-postgresql
> > already use GitHub Releases. (We just use "gh release
> > upload" to upload our artifacts to GitHub Releases.)
>
> Thank you for clarifying that we can use apache/arrow's GitHub Releases area 
> for hosting the MLTBX file. We assumed we couldn't use the main repository, 
> but it's great to hear we can!
>
> > BTW, how does File Exchange "Connecting to GitHub Repositories"?
> > https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
> >
> > Does it just use "polling"? Or do we need to install any
> > GitHub App, set secret variable or something on
> > apache/arrow? If the latter, we need to ask INFRA to do it.
>
> We are currently consulting with the development team responsible for the 
> GitHub <-> File Exchange integration. We'll send a followup email with a 
> concrete answer once we know more.
>
> > If we use GitHub Releases on apache/arrow, we can use the
> > following workflow. We don't need to use JFrog.
> >
> > 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> > 2. Release: Run a post release script that would:
> > 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
> > 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
> > 2.3 Linked File Exchange entry will be automatically updated
>
> This seems like a much more streamlined approach. Not having to upload to 
> JFrog will make things easier. Thanks for the suggestion!
>
> To clarify, in step 1, would we upload the MLTBX to ursacomputing/crossbow's 
> GitHub Releases area [1]? Or, would we upload to apache/arrow's GitHub 
> Releases area? If we upload release candidates to apache/arrow's GitHub 
> Releases area, they would get automatically linked to the File Exchange. 
> Ideally, we wouldn't want users to download release candidates.
>

Currently all the binaries are generated on the third step of the
Release process [1] when we run `03-binary-submit.sh`. The crossbow
job could build the MLTBX artifact and then when we do download the
other binaries (`04-binary-download.sh`) we should also download the
MTLBX and when we submit the rest to jfrog (`05-binary-upload.sh`) we
could Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN.

Once the release is approved and we do the post-release tasks to
"officially" release, we would download the MLTBX and upload to the
new GitHub Releases for apache-arrow-X.Y.Z this can be done as another
step on our post-release tasks (post-xx-matlab.sh)

[1] 
https://arrow.apache.org/docs/developers/release.html#build-source-and-binaries-and-submit-them

> > We can use GitHub Releases as I said. But if we use GitHub
> > Releases, the release notes on GitHub Releases may include
> > not only the MATLAB interface but also all
> > implementations. It may not be useful for this use case.
> >
> > FYI: The R bindings have their release notes under
> > https://arrow.apache.org/docs/r/ . See
> > https://arrow.apache.org/docs/r/news/ .
>
> We think it would still be useful to link to the GitHub release notes from 
> the File Exchange entry even if it includes notes for all language bindings. 
> The File Exchange <-> GitHub integration just includes a link to the GitHub 
> release notes under the Version History tab. If we find having a more focused 
> version of the release notes would be useful, then we can create a markdown 
> file analogous to the NEWS.md for the R bindings as you suggested (thanks or 
> pointing this out).
>
> [1] https://github.com/ursacomputing/crossbow/releases
>
> Thanks for all your help!
>
> Best,
>
> Sarah Gilmore
> 
> From: Sutou Kouhei 
> Sent: Thursday, November 9, 2023 7:50 PM
> To: dev@arrow.apache.org 
> Cc: Sarah Gilmore ; Lei Hou 
> Subject: Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the 
> MATLAB interface
>
> Hi,
>
> > One open question about this approach: which GitHub
> > repository should we use for hosting the MLTBX via GitHub
> > Releases?
> >
> > We don't think using the main apache/arrow GitHub Releases
> > area is the right approach. So, would it make sense to
> > creat

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-10 Thread Sarah Gilmore
Hi Kou,

> We can use apache/arrow's GitHub Releases. The release
> distribution document says that we can use GitHub as a
> release platform:
> https://infra.apache.org/release-distribution.html#other-platforms
>
> apache/arrow doesn't use GitHub Releases yet but
> apache/arrow-adbc and apache/arrow-flight-sql-postgresql
> already use GitHub Releases. (We just use "gh release
> upload" to upload our artifacts to GitHub Releases.)

Thank you for clarifying that we can use apache/arrow's GitHub Releases area 
for hosting the MLTBX file. We assumed we couldn't use the main repository, but 
it's great to hear we can!

> BTW, how does File Exchange "Connecting to GitHub Repositories"?
> https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub
>
> Does it just use "polling"? Or do we need to install any
> GitHub App, set secret variable or something on
> apache/arrow? If the latter, we need to ask INFRA to do it.

We are currently consulting with the development team responsible for the 
GitHub <-> File Exchange integration. We'll send a followup email with a 
concrete answer once we know more.

> If we use GitHub Releases on apache/arrow, we can use the
> following workflow. We don't need to use JFrog.
>
> 1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
> 2. Release: Run a post release script that would:
> 2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
> 2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
> 2.3 Linked File Exchange entry will be automatically updated

This seems like a much more streamlined approach. Not having to upload to JFrog 
will make things easier. Thanks for the suggestion!

To clarify, in step 1, would we upload the MLTBX to ursacomputing/crossbow's 
GitHub Releases area [1]? Or, would we upload to apache/arrow's GitHub Releases 
area? If we upload release candidates to apache/arrow's GitHub Releases area, 
they would get automatically linked to the File Exchange. Ideally, we wouldn't 
want users to download release candidates.

> We can use GitHub Releases as I said. But if we use GitHub
> Releases, the release notes on GitHub Releases may include
> not only the MATLAB interface but also all
> implementations. It may not be useful for this use case.
>
> FYI: The R bindings have their release notes under
> https://arrow.apache.org/docs/r/ . See
> https://arrow.apache.org/docs/r/news/ .

We think it would still be useful to link to the GitHub release notes from the 
File Exchange entry even if it includes notes for all language bindings. The 
File Exchange <-> GitHub integration just includes a link to the GitHub release 
notes under the Version History tab. If we find having a more focused version 
of the release notes would be useful, then we can create a markdown file 
analogous to the NEWS.md for the R bindings as you suggested (thanks or 
pointing this out).

[1] https://github.com/ursacomputing/crossbow/releases

Thanks for all your help!

Best,

Sarah Gilmore
____________
From: Sutou Kouhei 
Sent: Thursday, November 9, 2023 7:50 PM
To: dev@arrow.apache.org 
Cc: Sarah Gilmore ; Lei Hou 
Subject: Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the 
MATLAB interface

Hi,

> One open question about this approach: which GitHub
> repository should we use for hosting the MLTBX via GitHub
> Releases?
>
> We don't think using the main apache/arrow GitHub Releases
> area is the right approach. So, would it make sense to
> create a separate "bridge" repository just for hosting the
> latest MLTBX files? Should this be an ASF associated
> repository like apache/arrow-matlab or would a MathWorks
> associated repository like mathworks/arrow-matlab be OK?
> We aren't sure what makes the most sense here, but welcome
> any suggestions.

We can use apache/arrow's GitHub Releases. The release
distribution document says that we can use GitHub as a
release platform:
https://infra.apache.org/release-distribution.html#other-platforms<https://infra.apache.org/release-distribution.html#other-platforms>

apache/arrow doesn't use GitHub Releases yet but
apache/arrow-adbc and apache/arrow-flight-sql-postgresql
already use GitHub Releases. (We just use "gh release
upload" to upload our artifacts to GitHub Releases.)

BTW, how does File Exchange "Connecting to GitHub Repositories"?
https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub

Does it just use "polling"? Or do we need to install any
GitHub App, set secret variable or something on
apache/arrow? If the latter, we need to ask INFRA to do it.

If we use GitHub Releases on apache/arrow, we can use the
following workflow. We don't need to use JF

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-09 Thread Sutou Kouhei
Hi,

> One open question about this approach: which GitHub
> repository should we use for hosting the MLTBX via GitHub
> Releases?
> 
> We don't think using the main apache/arrow GitHub Releases
> area is the right approach. So, would it make sense to
> create a separate "bridge" repository just for hosting the
> latest MLTBX files? Should this be an ASF associated
> repository like apache/arrow-matlab or would a MathWorks
> associated repository like mathworks/arrow-matlab be OK?
> We aren't sure what makes the most sense here, but welcome
> any suggestions.

We can use apache/arrow's GitHub Releases. The release
distribution document says that we can use GitHub as a
release platform:
https://infra.apache.org/release-distribution.html#other-platforms

apache/arrow doesn't use GitHub Releases yet but
apache/arrow-adbc and apache/arrow-flight-sql-postgresql
already use GitHub Releases. (We just use "gh release
upload" to upload our artifacts to GitHub Releases.)

BTW, how does File Exchange "Connecting to GitHub Repositories"?
https://www.mathworks.com/matlabcentral/content/fx/about.html#Why_GitHub

Does it just use "polling"? Or do we need to install any
GitHub App, set secret variable or something on
apache/arrow? If the latter, we need to ask INFRA to do it.

If we use GitHub Releases on apache/arrow, we can use the
following workflow. We don't need to use JFrog.

1. RC: Upload MLTBX to GitHub Releases for apache-arrow-X.Y.Z-rcN
2. Release: Run a post release script that would:
2.1 Download MLTBX from GitHub Releases for apache-arrow-X.Y.Z-rcN
2.2 Upload it to GitHub Releases for apache-arrow-X.Y.Z
2.3 Linked File Exchange entry will be automatically updated


> File Exchange entries have a "Version History" which
> includes release notes from the "backing" GitHub Releases
> area. So, this would probably be a sensible location to
> put the release notes.

We can use GitHub Releases as I said. But if we use GitHub
Releases, the release notes on GitHub Releases may include
not only the MATLAB interface but also all
implementations. It may not be useful for this use case.

FYI: The R bindings have their release notes under
https://arrow.apache.org/docs/r/ . See
https://arrow.apache.org/docs/r/news/ .

> Also, including MATLAB updates in
> Apache Arrow release blog posts
> (e.g. https://arrow.apache.org/blog/2023/11/01/14.0.0-release/)
> may also be helpful.

Yes. We should do it. :-)


Thanks,
-- 
kou

In 

  "Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB 
interface" on Wed, 8 Nov 2023 20:44:10 +,
  Kevin Gurney  wrote:

> Hi Kou and Dewey,
> 
> Thank you very much for your very thorough and detailed responses to all of 
> our questions. This is extremely valuable feedback and the points that you 
> made make alot of sense.
> 
> Sarah and I talked this over a bit more and we think that sticking with the 
> overall apache/arrow project release cycle (i.e. stay in line with 15.0.0) 
> makes the most sense in the long term.
> 
> @Dewey - thanks very much for highlighting the pros and cons of creating a 
> separate repository. We also really appreciate the community being willing to 
> try and support our development needs. That being said, we think it is 
> probably best to stay in-model with the main apache/arrow release process for 
> the time being rather than creating a separate repository for the MATLAB 
> interface.
> 
> To address some related points and questions:
> 
>> Can we just mention "This is not stable yet!!!" in the documentation instead 
>> of using isolated version?
> 
> Yes. This is good point and we already have a disclaimer in the README.md [1] 
> for the MATLAB interface which says: "Warning The MATLAB interface is under 
> active development and should be considered experimental."
> 
>> It's better that we use CI for this like other binary packages such as 
>> .deb/.rpm/.wheel/.jar/...
> 
> This makes sense and we agree. We will follow up with PRs to add the 
> necessary MATLAB packaging scripts and CI workflow files.
> 
>> Does the MLTBX file include Apache Arrow C++ binaries too like .wheel/.jar?
> 
> Yes. The MLTBX file will package the Apache Arrow C++ binaries, similar to 
> the Java JARs / Python wheels.
> 
>> MATLAB doesn't provide the official package repository such as PyPI for 
>> Python and https://rubygems.org/ for Ruby, right?
> 
> The equivalent to pypi.org or rubygems.org for MATLAB would be the MathWorks 
> File Exchange [2].
> 
>> If the official package repository for MATLAB doesn't exist, JFrog is better 
>> because the MLTBX file will be large (Apache Arrow C++ binaries are large).
>

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-08 Thread Kevin Gurney
main/docs/source/matlab>
> .
>
> > 3. Is there a recommended cadence for incremental point releases?
>
> I suggest avoiding separated release as above.
>
> > 4. Are there any notable ASF procedures [4] [5] (e.g. voting on a new 
> > release proposal) that we should be aware of as we consider creating an 
> > initial release?
>
> We don't need additional task for an initial release.
>
> > 5. How should the Arrow project release (i.e. 14.0.0)
> > relate to the MATLAB interface version (i.e. 0.1)? As a
> > point of reference, we noticed that PyArrow is on
> > version 14.0.0, but it feels "misleading" to say that
> > the MATLAB interface is at version 14.0.0 when we
> > haven't yet implemented or stabilized all core Arrow
> > APIs. Is there any precedent for using independent
> > release versions for language bindings which are not
> > fully stabilized and are also part of the main
> > apache/arrow repository?
>
> We don't have any precedent for using independent release
> versions for language bindings. All language bindings used
> the same version.
>
> Apache Arrow JavaScript isn't a language bindings but it
> used separated release and isolated versions before
> 0.4.1. It joined apache/arrow release after 0.4.1. (The next
> version of Apache Arrow JavaScript 0.4.1 is 13.0.0.)
>
> > We've noticed that Arrow-related projects which are not
> > part of the main apache/arrow GitHub repository
> > (e.g. DataFusion) follow a mailing list-based voting and
> > release process. However, it's not clear whether it makes
> > sense to follow this process for the MATLAB interface
> > since it is part of the main apache/arrow repository.
>
> If we want to use separated release for the MATLAB
> interface, we should follow the same release process as
> apache/arrow and other apache/arrow-* because it's the
> standard ASF release process.
>
>
> Thanks,
> --
> kou
>
> In 
> 
> "[DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB 
> interface" on Tue, 7 Nov 2023 20:31:31 +,
> Kevin Gurney  wrote:
>
> > Hi All,
> >
> > A considerable amount of new functionality has been added to the MATLAB 
> > interface over the last few months. We appreciate all the community's 
> > support in making this possible and are happy to see all the progress that 
> > is being made.
> >
> > At this point, we would like to create an initial "0.1" release of the 
> > MATLAB interface. Incremental point releases will enable MATLAB users to 
> > provide early feedback. In addition, learning how to navigate the release 
> > process is an important step towards eventually releasing a stable 1.0 
> > version of the MATLAB interface.
> >
> > Our proposed approach to creating an initial release would be to:
> >
> > 1. Manually build the MATLAB interface on Windows, macOS, and Linux
> > 2. Combine all of the cross platform build artifacts into a single MLTBX 
> > file [1] for distribution
> > 3. Host the MLTBX somewhere that is easliy accessible for download
> >
> > For reference - MLTBX is a standard packaging format for MATLAB which 
> > enables simple "one-click" installation - analogous to a Python pip package 
> > or a Ruby gem.
> >
> > Creating an MLTBX file manually should be relatively low effort. However, 
> > in the long term, we would love to enable semi-automated "push button" 
> > releases via GitHub Actions (and possibly even "nightly builds").
> >
> > Since this is our first time creating a release of the MATLAB interface, we 
> > wanted to draw on the community's expertise to answer a few questions:
> >
> > 1. Is there a recommended location where we can host the MLTBX file? e.g. 
> > GitHub Releases [2], JFrog [3], etc.?
> > 2. Is there a recommended location for hosting release notes?
> > 3. Is there a recommended cadence for incremental point releases?
> > 4. Are there any notable ASF procedures [4] [5] (e.g. voting on a new 
> > release proposal) that we should be aware of as we consider creating an 
> > initial release?
> > 5. How should the Arrow project release (i.e. 14.0.0) relate to the MATLAB 
> > interface version (i.e. 0.1)? As a point of reference, we noticed that 
> > PyArrow is on version 14.0.0, but it feels "misleading" to say that the 
> > MATLAB interface is at version 14.0.0 when we haven't yet implemented or 
> > stabilized all core Arrow APIs. Is there any precedent for using 
> > independent release

Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-07 Thread Dewey Dunnington
elease.
>
> > 5. How should the Arrow project release (i.e. 14.0.0)
> >relate to the MATLAB interface version (i.e. 0.1)? As a
> >point of reference, we noticed that PyArrow is on
> >version 14.0.0, but it feels "misleading" to say that
> >the MATLAB interface is at version 14.0.0 when we
> >haven't yet implemented or stabilized all core Arrow
> >APIs. Is there any precedent for using independent
> >release versions for language bindings which are not
> >fully stabilized and are also part of the main
> >apache/arrow repository?
>
> We don't have any precedent for using independent release
> versions for language bindings. All language bindings used
> the same version.
>
> Apache Arrow JavaScript isn't a language bindings but it
> used separated release and isolated versions before
> 0.4.1. It joined apache/arrow release after 0.4.1. (The next
> version of Apache Arrow JavaScript 0.4.1 is 13.0.0.)
>
> > We've noticed that Arrow-related projects which are not
> > part of the main apache/arrow GitHub repository
> > (e.g. DataFusion) follow a mailing list-based voting and
> > release process. However, it's not clear whether it makes
> > sense to follow this process for the MATLAB interface
> > since it is part of the main apache/arrow repository.
>
> If we want to use separated release for the MATLAB
> interface, we should follow the same release process as
> apache/arrow and other apache/arrow-* because it's the
> standard ASF release process.
>
>
> Thanks,
> --
> kou
>
> In 
> 
>   "[DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB 
> interface" on Tue, 7 Nov 2023 20:31:31 +,
>   Kevin Gurney  wrote:
>
> > Hi All,
> >
> > A considerable amount of new functionality has been added to the MATLAB 
> > interface over the last few months. We appreciate all the community's 
> > support in making this possible and are happy to see all the progress that 
> > is being made.
> >
> > At this point, we would like to create an initial "0.1" release of the 
> > MATLAB interface. Incremental point releases will enable MATLAB users to 
> > provide early feedback. In addition, learning how to navigate the release 
> > process is an important step towards eventually releasing a stable 1.0 
> > version of the MATLAB interface.
> >
> > Our proposed approach to creating an initial release would be to:
> >
> > 1. Manually build the MATLAB interface on Windows, macOS, and Linux
> > 2. Combine all of the cross platform build artifacts into a single MLTBX 
> > file [1] for distribution
> > 3. Host the MLTBX somewhere that is easliy accessible for download
> >
> > For reference - MLTBX is a standard packaging format for MATLAB which 
> > enables simple "one-click" installation - analogous to a Python pip package 
> > or a Ruby gem.
> >
> > Creating an MLTBX file manually should be relatively low effort. However, 
> > in the long term, we would love to enable semi-automated "push button" 
> > releases via GitHub Actions (and possibly even "nightly builds").
> >
> > Since this is our first time creating a release of the MATLAB interface, we 
> > wanted to draw on the community's expertise to answer a few questions:
> >
> > 1. Is there a recommended location where we can host the MLTBX file? e.g. 
> > GitHub Releases [2], JFrog [3], etc.?
> > 2. Is there a recommended location for hosting release notes?
> > 3. Is there a recommended cadence for incremental point releases?
> > 4. Are there any notable ASF procedures [4] [5] (e.g. voting on a new 
> > release proposal) that we should be aware of as we consider creating an 
> > initial release?
> > 5. How should the Arrow project release (i.e. 14.0.0) relate to the MATLAB 
> > interface version (i.e. 0.1)? As a point of reference, we noticed that 
> > PyArrow is on version 14.0.0, but it feels "misleading" to say that the 
> > MATLAB interface is at version 14.0.0 when we haven't yet implemented or 
> > stabilized all core Arrow APIs. Is there any precedent for using 
> > independent release versions for language bindings which are not fully 
> > stabilized and are also part of the main apache/arrow repository?
> >
> > We've noticed that Arrow-related projects which are not part of the main 
> > apache/arrow GitHub repository (e.g. DataFusion) follow a mailing 
> > list-based voting and release process. However, it's not clear whether it 
> > makes sense to follow this process for the MATLAB interface since it is 
> > part of the main apache/arrow repository.
> >
> > We sincerely appreciate the community's help and guidance on this topic!
> >
> > Please let us know if you have any questions.
> >
> > [1] 
> > https://www.mathworks.com/help/matlab/creating-help.html?s_tid=CRUX_lftnav
> > [2] https://github.com/apache/arrow/releases
> > [3] https://apache.jfrog.io/ui/native/arrow/
> > [4] https://www.apache.org/foundation/voting.html
> > [5] https://www.apache.org/legal/release-policy.html#release-approval
> >
> > Best Regards,
> >
> > Kevin Gurney


Re: [DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-07 Thread Sutou Kouhei
Hi,

> As a point of reference, we noticed that PyArrow is on
> version 14.0.0, but it feels "misleading" to say that the
> MATLAB interface is at version 14.0.0 when we haven't yet
> implemented or stabilized all core Arrow APIs.

I can understand this but I suggest that we use the same
version as other packages in apache/arrow. Because:

* Using isolated version increases release complexity.
* Using isolated version may introduce another
  "misleading"/"confusion": For example, "the MATLAB
  interface 1.0.0 uses Apache Arrow C++ 20.0.0" may be
  misleading/confused:
  * The MATLAB interface 1.0.0 doesn't use Apache Arrow C++
1.0.0.
  * It may be difficult to find the corresponding
Apache Arrow C++ version from the MATLAB interface
version.

Can we just mention "This is not stable yet!!!" in the
documentation instead of using isolated version?

We may want to use the status page for it:
https://arrow.apache.org/docs/status.html

> 1. Manually build the MATLAB interface on Windows, macOS, and Linux

It's better that we use CI for this like other binary
packages such as .deb/.rpm/.wheel/.jar/...

If we release the MATLAB interface separately, which Apache
Arrow C++ version is used? If we release the MATALB
interface right now, is Apache Arrow C++ 14.0.0 (the latest
release) used or is Apache Arrow C++ main (not released yet)
used? The MATLAB interface on main will depend on Apache
Arrow C++ main, we may not be able to use the latest release
for the MATLAB interface on main.

> 2. Combine all of the cross platform build artifacts into
>a single MLTBX file [1] for distribution

Does the MLTBX file include Apache Arrow C++ binaries too
like .wheel/.jar?

> 3. Host the MLTBX somewhere that is easliy accessible for download

MATLAB doesn't provide the official package repository such
as PyPI for Python and https://rubygems.org/ for Ruby, right?

> 1. Is there a recommended location where we can host the MLTBX file? e.g. 
> GitHub Releases [2], JFrog [3], etc.?

If the official package repository for MATLAB doesn't exist,
JFrog is better because the MLTBX file will be large (Apache
Arrow C++ binaries are large).

> 2. Is there a recommended location for hosting release notes?

How about creating https://arrow.apache.org/docs/matlab/ ?
We can use Sphinx like the Python docs
https://arrow.apache.org/docs/python/ or another
documentation tools like the R docs
https://arrow.apache.org/docs/r/ .
If we use Sphinx, we can create
https://github.com/apache/arrow/tree/main/docs/source/matlab/
.

> 3. Is there a recommended cadence for incremental point releases?

I suggest avoiding separated release as above.

> 4. Are there any notable ASF procedures [4] [5] (e.g. voting on a new release 
> proposal) that we should be aware of as we consider creating an initial 
> release?

We don't need additional task for an initial release.

> 5. How should the Arrow project release (i.e. 14.0.0)
>relate to the MATLAB interface version (i.e. 0.1)? As a
>point of reference, we noticed that PyArrow is on
>version 14.0.0, but it feels "misleading" to say that
>the MATLAB interface is at version 14.0.0 when we
>haven't yet implemented or stabilized all core Arrow
>APIs. Is there any precedent for using independent
>release versions for language bindings which are not
>fully stabilized and are also part of the main
>apache/arrow repository?

We don't have any precedent for using independent release
versions for language bindings. All language bindings used
the same version.

Apache Arrow JavaScript isn't a language bindings but it
used separated release and isolated versions before
0.4.1. It joined apache/arrow release after 0.4.1. (The next
version of Apache Arrow JavaScript 0.4.1 is 13.0.0.)

> We've noticed that Arrow-related projects which are not
> part of the main apache/arrow GitHub repository
> (e.g. DataFusion) follow a mailing list-based voting and
> release process. However, it's not clear whether it makes
> sense to follow this process for the MATLAB interface
> since it is part of the main apache/arrow repository.

If we want to use separated release for the MATLAB
interface, we should follow the same release process as
apache/arrow and other apache/arrow-* because it's the
standard ASF release process.


Thanks,
-- 
kou

In 

  "[DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB 
interface" on Tue, 7 Nov 2023 20:31:31 +,
  Kevin Gurney  wrote:

> Hi All,
> 
> A considerable amount of new functionality has been added to the MATLAB 
> interface over the last few months. We appreciate all the community's support 
> in making this possible and are happy to see all the progress that is being 
> made.
> 
>

[DISCUSS][MATLAB] Proposal for incremental point releases of the MATLAB interface

2023-11-07 Thread Kevin Gurney
Hi All,

A considerable amount of new functionality has been added to the MATLAB 
interface over the last few months. We appreciate all the community's support 
in making this possible and are happy to see all the progress that is being 
made.

At this point, we would like to create an initial "0.1" release of the MATLAB 
interface. Incremental point releases will enable MATLAB users to provide early 
feedback. In addition, learning how to navigate the release process is an 
important step towards eventually releasing a stable 1.0 version of the MATLAB 
interface.

Our proposed approach to creating an initial release would be to:

1. Manually build the MATLAB interface on Windows, macOS, and Linux
2. Combine all of the cross platform build artifacts into a single MLTBX file 
[1] for distribution
3. Host the MLTBX somewhere that is easliy accessible for download

For reference - MLTBX is a standard packaging format for MATLAB which enables 
simple "one-click" installation - analogous to a Python pip package or a Ruby 
gem.

Creating an MLTBX file manually should be relatively low effort. However, in 
the long term, we would love to enable semi-automated "push button" releases 
via GitHub Actions (and possibly even "nightly builds").

Since this is our first time creating a release of the MATLAB interface, we 
wanted to draw on the community's expertise to answer a few questions:

1. Is there a recommended location where we can host the MLTBX file? e.g. 
GitHub Releases [2], JFrog [3], etc.?
2. Is there a recommended location for hosting release notes?
3. Is there a recommended cadence for incremental point releases?
4. Are there any notable ASF procedures [4] [5] (e.g. voting on a new release 
proposal) that we should be aware of as we consider creating an initial release?
5. How should the Arrow project release (i.e. 14.0.0) relate to the MATLAB 
interface version (i.e. 0.1)? As a point of reference, we noticed that PyArrow 
is on version 14.0.0, but it feels "misleading" to say that the MATLAB 
interface is at version 14.0.0 when we haven't yet implemented or stabilized 
all core Arrow APIs. Is there any precedent for using independent release 
versions for language bindings which are not fully stabilized and are also part 
of the main apache/arrow repository?

We've noticed that Arrow-related projects which are not part of the main 
apache/arrow GitHub repository (e.g. DataFusion) follow a mailing list-based 
voting and release process. However, it's not clear whether it makes sense to 
follow this process for the MATLAB interface since it is part of the main 
apache/arrow repository.

We sincerely appreciate the community's help and guidance on this topic!

Please let us know if you have any questions.

[1] https://www.mathworks.com/help/matlab/creating-help.html?s_tid=CRUX_lftnav
[2] https://github.com/apache/arrow/releases
[3] https://apache.jfrog.io/ui/native/arrow/
[4] https://www.apache.org/foundation/voting.html
[5] https://www.apache.org/legal/release-policy.html#release-approval

Best Regards,

Kevin Gurney