> I think trying to package CUDA is the wrong way to think about it. Instead, perhaps you should try to make the package compatible with system CUDA installs.
I agree in principle. The problem fundamentally stems from user expectation. In my ~6+ years of supporting Torch and PyTorch, installing CUDA on a system can take days, with a user mean approximately half a day. It might be userland incompetence, or that CUDA is a magical snowflake, but the reality is that installing CUDA is never great. So, a huge amount of issues reported by userland are side-effects from broken CUDA installs. It doesn't help that the PyPI user expectations of "my package should just work after a pip install". If we can reliably install an up-to-date CUDA in a standardized way, and NVIDIA simply doesn't sidestep the userland issues by saying "user our docker", or "our PPA is 100% reliable", we would've been in a better state. Until then, I think it's best that we find a solution for PyPI users that can work out of box with PyPI. On Mon, Feb 4, 2019 at 12:52 PM Antoine Pitrou <solip...@pitrou.net> wrote: > On Tue, 5 Feb 2019 01:45:34 +0800 > Jason Zaman <ja...@perfinion.com> wrote: > > On Tue, 5 Feb 2019 at 01:30, soumith <soum...@gmail.com> wrote: > > > > > > Unfortunately I'll be on a long flight, and cannot make it to the > SIGBuild meeting. > > > I'm definitely interested in the meeting notes and any follow-up > meeting. > > > > > > > I think we should leave CUDA out of the > > > discussion initially and see if we can get the cpu-only wheel working > > > correctly. Hopefully cpu-only is viable on manylinux2014 then we can > > > tackle CUDA afterwards. > > > > > > 50% of the complexity is in the CUDA packaging. > > > The other 50% is in shipping a more modern libstdc++.so > > > I believe we'll make progress if we ignore CUDA, but we'll not address > half of the issue. > > > > Yeah, we'll definitely need both to solve it fully. My thinking is > > that all packages need at least C++11 but only some need CUDA. Or > > might we end up where the libstcc++.so is incompatible with CUDA if we > > don't work on everything together? > > I think trying to package CUDA is the wrong way to think about it. > Instead, perhaps you should try to make the package compatible with > system CUDA installs. > > For example, the Numba pip wheel almost works out-of-the-box with a > system CUDA install on Ubuntu 18.04. I say "almost" because I had to > set two environment variables: > https://github.com/numba/numba/issues/3738 > > Regards > > Antoine. > > >
