Hello MXNet community,
After the Julia package has been imported into the main repo,
the next unresolved issue is establishing the releasing process for Julia.
The PR #12845, which is for upgrading Julia package to work with Julia v0.7+,
is ready for review at this moment.
I want to release the Julia part along with the next mxnet major release (v1.5,
I think).
We imported the Julia package to make it part of the CI validation chain.
And merging code into main repo makes Julia release process different from
normal.
At first, let me describe a usual release process for a Julia package.
I will try to explain it with corresponding ideas in Python.
1.Package manager in Julia:
There are a module from stdlib named `Pkg` served as the package manager in
Julia,
like Python's `pip`.
2. The package registry and package source:
The package manager will download the desired packages from a default
registry.
In Python, pip will conduct two actions before installing packags:
(1) Search the metadata (like pkg name, version, ...etc)
from the CheeseShop (a.k.a PyPI) for users.
(2) Fetch them from the CheeseShop if matched.
In Julia, there is a default registry to hold the packages metadata,
hosted on GitHub:
https://github.com/JuliaLang/METADATA.jl
And where to fetch the package? The aren't a center for download packages.
It's unlike Python's idea, we don't upload package to a center server in
case of Julia.
The package manager will fetch stuffs from the URL listed in the metadata.
It should be a valid git URL for git-clone.
e.g. For the package `Distributions`:
https://github.com/JuliaLang/METADATA.jl/blob/metadata-v2/Distributions/url
Most of the packages use GitHub for hosting code.
3. A valid package directory structure
In the process of package installation,
Python's pip will run `./setup.py` from unpacked package dir.
This is kind of protocol between package developer and package manager.
In Julia, the package manager run `./deps/build.jl` from the git-cloned dir.
Here comes an issue of mxnet's main repo.
The Julia package is collected under the subdir `./julia/`, so the setup
script
in our case is `./julia/deps/build.jl`.
This breaks the protocol.
Also, in runtime, there is another protocol for phase of module loading.
The entry point of a module must be `./src/PackageName.jl`.
In our repo, it's `./julia/src/MXNet.jl`.
So, the directory structure of mxnet main repo is invalid for Julia package
manager.
Putting the url of main repo as package metadata will not work.
The proposed solution:
To provide a valid dir structure, it need a standalone git repo.
We can just treat this standalone repo as the place to "upload" package code,
just like uploading Python package to PyPI in every release.
I want to discuss about the home of this standalone repo.
1) Reuse the old repo: https://github.com/dmlc/MXNet.jl
It's under DMLC. I have the committer bit of this repo.
2) A new repo under ASF's organization?
3) Anything else?
I will vote for option (1).
----
The actual process in detail:
1) Split the julia dir out via git-subtree
```
cd /path/to/mxnet/
git subtree split -P julia -b MXNet_jl
```
2) Push the branch `MXNet_jl` to the standalone git repo mentioned in
previous section.
3) Push a new tag for MXNet.jl. The version number is same as the mxnet
main repo.
4) Make a release page via GitHub release button.
5) Then, updating metadata to Julia package registry will be automated by
the bot.
https://github.com/attobot/attobot
6) Relax!
--
Iblis Lin
