Re: [Distutils] Which commercial vendor?

[Ben Finney]

Thomas Güttler  writes:

> Let's look back. How was this in the past? Maybe five years ago?

That's a very vague question. What kind of answer do you want? Is it one
you have an answer for already; and if so, what is the point of your
question here?

I don't doubt that you *have* a point. What I doubt is the value to
others here of asking something that demands a lot of to even find out
what you mean.

-- 
 \ “[T]he question of whether machines can think … is about as |
  `\ relevant as the question of whether submarines can swim.” |
_o__)  —Edsger W. Dijkstra |
Ben Finney

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Thomas Güttler]

Am 08.04.2017 um 01:51 schrieb Wes Turner:


AFAIU, the only way to read the package version from the {git, hg, } source 
repository is to run the setup.py.


I see. This is the only way at the moment.

Let's look back. How was this in the past? Maybe five years ago?



Regards,
  Thomas Güttler


--
Thomas Guettler http://www.thomas-guettler.de/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Wes Turner]

On Fri, Apr 7, 2017 at 4:58 PM, Chris Barker  wrote:

> On Thu, Apr 6, 2017 at 5:34 PM, Wes Turner  wrote:
>
>
>> Chances are, there will be a package or two that you rely on that is not
>>> in conda defaults (maintained by Continuum) or currently in conda-forge. So
>>> you can pip-install those few -- but what if they aren't on PyPi either? or
>>> are hard to compile and install with ugly dependencies?  You can contribute
>>> build recipes to conda-forge, and then have it for you, and all your users,
>>> and the rest of the world to access. Much better than hand maintaining
>>> stuff yourself.
>>>
>>
>> Someone still needs to commit to maintaining the conda package; otherwise
>> who knows whether this is the latest  stable  release?
>>
>
> Indeed. and it it's a not-that-widely-used package, then you will have to
> do that yourself -- but using the conda-forge infrastructure makes that
> (relatively) easy. In contrast -- who knows whether the package on PyPi is
> the latest stable release? Hopefully the maintainer is keeping it up, but
> if not, you're kinda dead in the water.
>

So then there's pulling from a specific source rev:

  pip install -e git+ssh://g...@github.com/pypa/pip@9.0.1#egg=pip


There was a discussion about adding the git/hg/svn/vcs source URI to each
package's metadata:
- "add "sourceURL" to the metadata 3.0 PEP."
  https://www.mail-archive.com/distutils-sig@python.org/msg25836.html
  https://www.mail-archive.com/distutils-sig@python.org/msg25833.html
  - Project-URL
  - Source-URL (metadata 2.0)

AFAIU, the only way to read the package version from the {git, hg, } source
repository is to run the setup.py.

There's a semver way to specify the vcs revision ("git short SHA") in the
package identifier:
https://github.com/openstack-dev/pbr/pull/14/commits/5b7a619046eb10ed3fa7bb987be95208faf2fda3


>
>
>>> My pain point now is still full multi-platform support. conda has
>>> package versions that are platform independent, but it can still be hard to
>>> get everything built  in the same version on all platforms, so it does get
>>> a bit ugly.
>>>
>>
>> Docker images are reproducible and archivable:
>>
>
> In a way Docker, as I understand it, is orthogonal to this conversation.
> And when I talk about "all platforms", I mean running natively on all
> platforms -- I can't give my Windows users a Linux VM and expect them to
> know what the heck to do with it.
>

IIUC, this should work w/ Docker for {Linux, Windows, OSX, }:

docker run -i -t continuumio/miniconda3 /bin/bash

AFAIU, If you instead wanted to run Windows containers on a Windows host,
you'd need Windows Server 2016:
https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-server

You can run "provisioner(s)" (shell script, salt states, ansible playbooks)
at image build time with e.g. Packer:
https://www.packer.io/docs/basics/terminology.html

With a configuration management tool like salt or ansible, you can define
configuration policies with conditionals for whichever given platform specs
(and see it all in one place as "infrastructure as code").



>
> Not that Docker isn't a really useful tool to htlep address some of these
> problems...
>
> -CHB
>
>
>
>> What platforms does conda-forge auto-build for?
>> - [x] x86[-64]
>>
>
>  linux-64
>  win-32
>  win-64
>  osx-64
>
> (all Intel)
>
>
>
> - [ ] linux-armv7l
>>   - https://github.com/conda-forge/conda-forge.github.io/issues/269
>>
>
>
> looks like folks are trying, but it's not really there yet -- mostly due
> to the lack of easy to access CI services for armv7I
>
> It's an open-source project -- if it's important to someone, it will get
> done.
>

Raspberry Pi support for conda-forge would probably be really useful for
education.
IDK what sort of resources would be needed to add Pi2's to the CI build
farm?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Chris Barker]

On Thu, Apr 6, 2017 at 5:34 PM, Wes Turner  wrote:


> Chances are, there will be a package or two that you rely on that is not
>> in conda defaults (maintained by Continuum) or currently in conda-forge. So
>> you can pip-install those few -- but what if they aren't on PyPi either? or
>> are hard to compile and install with ugly dependencies?  You can contribute
>> build recipes to conda-forge, and then have it for you, and all your users,
>> and the rest of the world to access. Much better than hand maintaining
>> stuff yourself.
>>
>
> Someone still needs to commit to maintaining the conda package; otherwise
> who knows whether this is the latest  stable  release?
>

Indeed. and it it's a not-that-widely-used package, then you will have to
do that yourself -- but using the conda-forge infrastructure makes that
(relatively) easy. In contrast -- who knows whether the package on PyPi is
the latest stable release? Hopefully the maintainer is keeping it up, but
if not, you're kinda dead in the water.


>> My pain point now is still full multi-platform support. conda has package
>> versions that are platform independent, but it can still be hard to get
>> everything built  in the same version on all platforms, so it does get a
>> bit ugly.
>>
>
> Docker images are reproducible and archivable:
>

In a way Docker, as I understand it, is orthogonal to this conversation.
And when I talk about "all platforms", I mean running natively on all
platforms -- I can't give my Windows users a Linux VM and expect them to
know what the heck to do with it.

Not that Docker isn't a really useful tool to htlep address some of these
problems...

-CHB



> What platforms does conda-forge auto-build for?
> - [x] x86[-64]
>

 linux-64
 win-32
 win-64
 osx-64

(all Intel)



- [ ] linux-armv7l
>   - https://github.com/conda-forge/conda-forge.github.io/issues/269
>


looks like folks are trying, but it's not really there yet -- mostly due to
the lack of easy to access CI services for armv7I

It's an open-source project -- if it's important to someone, it will get
done.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Wes Turner]

On Thursday, April 6, 2017, Chris Barker  wrote:

> On Wed, Apr 5, 2017 at 6:41 PM, Nick Coghlan  > wrote:
>
>
>> PayPal Engineering put together a decent write-up of their path
>> towards adopting that model last year:
>> https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/
>
>
> Thanks for that link.
>

+1


>
> We're a much smaller shop, but have had pretty much the same experience --
> also really good to see them mention miniconda at the end -- I think that's
> a much better way to go for most folks than the whole Anaconda pile.
>
> It's definitely a reasonable way to go for organisational
>> infrastructure, but even conda doesn't cover all the potential use
>> cases that are out there
>
>
> of course not -- nothing does, but I would add to the contents of that
> post:
>
> The conda-forge project is a Major boon to the conda infrastructure --
> there is now a robust way for the community to expand the available number
> of packages (and keep up more recent versions). And anyone can take
> advantage of that infrastructure for their own (selfish?) needs:
>
> Chances are, there will be a package or two that you rely on that is not
> in conda defaults (maintained by Continuum) or currently in conda-forge. So
> you can pip-install those few -- but what if they aren't on PyPi either? or
> are hard to compile and install with ugly dependencies?  You can contribute
> build recipes to conda-forge, and then have it for you, and all your users,
> and the rest of the world to access. Much better than hand maintaining
> stuff yourself.
>

Someone still needs to commit to maintaining the conda package; otherwise
who knows whether this is the latest  stable  release?


>
> My pain point now is still full multi-platform support. conda has package
> versions that are platform independent, but it can still be hard to get
> everything built  in the same version on all platforms, so it does get a
> bit ugly.
>

Docker images are reproducible and archivable:

https://github.com/ContinuumIO/docker-images
-
https://github.com/ContinuumIO/docker-images/blob/master/miniconda3/Dockerfile
-
https://github.com/ContinuumIO/docker-images/blob/master/anaconda3/Dockerfile

https://github.com/jupyter/docker-stacks
-
https://github.com/jupyter/docker-stacks/blob/master/README.md#visual-overview
-
https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile
  - a diff miniconda setup

-
https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile.ppc64le
  - ARMv would be cool too (e.g. for raspberry pi)

https://github.com/Kaggle/docker-python
- https://github.com/Kaggle/docker-python/blob/master/Dockerfile
  - everything and the kitchen sink


What platforms does conda-forge auto-build for?
- [x] x86[-64]
- [ ] linux-armv7l
  - https://github.com/conda-forge/conda-forge.github.io/issues/269
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Chris Barker]

On Wed, Apr 5, 2017 at 6:41 PM, Nick Coghlan  wrote:


> PayPal Engineering put together a decent write-up of their path
> towards adopting that model last year:
> https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/


Thanks for that link.

We're a much smaller shop, but have had pretty much the same experience --
also really good to see them mention miniconda at the end -- I think that's
a much better way to go for most folks than the whole Anaconda pile.

It's definitely a reasonable way to go for organisational
> infrastructure, but even conda doesn't cover all the potential use
> cases that are out there


of course not -- nothing does, but I would add to the contents of that post:

The conda-forge project is a Major boon to the conda infrastructure --
there is now a robust way for the community to expand the available number
of packages (and keep up more recent versions). And anyone can take
advantage of that infrastructure for their own (selfish?) needs:

Chances are, there will be a package or two that you rely on that is not in
conda defaults (maintained by Continuum) or currently in conda-forge. So
you can pip-install those few -- but what if they aren't on PyPi either? or
are hard to compile and install with ugly dependencies?  You can contribute
build recipes to conda-forge, and then have it for you, and all your users,
and the rest of the world to access. Much better than hand maintaining
stuff yourself.

My pain point now is still full multi-platform support. conda has package
versions that are platform independent, but it can still be hard to get
everything built  in the same version on all platforms, so it does get a
bit ugly.

But no other solution makes that better anyway.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Nick Coghlan]

On 6 April 2017 at 01:52, Chris Barker - NOAA Federal
 wrote:
>> On Mar 30, 2017, at 1:53 AM, Thomas Güttler  
>> wrote:
>>
>>
>> My frustration has reached a limit. Yes, I am willing to pay money.
>>
>> Which vendor do you suggest to give me a reliable package management?
>
> You may want conda -- it's an open source project, and you can get
> commercial support through Continuum Analytics.
>
> Though conda, and Continuum was born in the data science / scientific
> computing world, there is nothing about conda specific to data
> science. It's just that computational computing poses extra challenges
> for packaging -- but the easy stuff is still easy.

PayPal Engineering put together a decent write-up of their path
towards adopting that model last year:
https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/

It's definitely a reasonable way to go for organisational
infrastructure, but even conda doesn't cover all the potential use
cases that are out there (something I discuss a bit in
http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html
)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Chris Barker - NOAA Federal]

> On Mar 30, 2017, at 1:53 AM, Thomas Güttler  
> wrote:
>
>
> My frustration has reached a limit. Yes, I am willing to pay money.
>
> Which vendor do you suggest to give me a reliable package management?

You may want conda -- it's an open source project, and you can get
commercial support through Continuum Analytics.

Though conda, and Continuum was born in the data science / scientific
computing world, there is nothing about conda specific to data
science. It's just that computational computing poses extra challenges
for packaging -- but the easy stuff is still easy.

And the conda-forge project provides an open source way to support a
larger community.

-CHB
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Which commercial vendor?

[Nick Coghlan]

On 30 March 2017 at 18:53, Thomas Güttler  wrote:
> Am 29.03.2017 um 10:27 schrieb Nick Coghlan:
>> The Python ecosystem *does* include commercial vendors that offer to
>> make opinionated technical decisions on behalf of their customers, as
>> well as providing a single point of contact for support questions and
>> feature requests, but beyond that, offering an overwhelming array of
>> confusing choices is pretty much the way open source *works*.
>
> My frustration has reached a limit. Yes, I am willing to pay money.
>
> Which vendor do you suggest to give me a reliable package management?

For cross-platform use cases, the best known options are ActiveState,
Enthought, and Continuum Analytics (with the latter two focusing
primarily on data analysis tasks).

Another option if you're looking to bundle your own applications is
PyRun, from eGenix: http://www.egenix.com/products/python/PyRun/

Finally, if you're solely interested in Linux, then Python runtimes
are generally covered by commercial Linux support agreements.

However, exactly which of the available commercial support options
will be the best fit for your needs will depend on what you're aiming
to do, and how it aligns with their product offerings.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Which commercial vendor?

[Thomas Güttler]

Am 29.03.2017 um 10:27 schrieb Nick Coghlan:

On 29 March 2017 at 17:51, Paul Moore  wrote:

On 29 March 2017 at 06:29, Thomas Güttler  wrote:

I am stupid and missing a guiding hand which gives me simple straight forward 
step by step instruction.


To do what?


As far as I can tell, to get a customer experience instead of a
prospective co-contributor one.

I'm sorry Thomas, as long as you continue looking for a coherent
customer experience from a collaborative collection of volunteer-run
community projects, you're going to continually be confused and
disappointed.

The Python ecosystem *does* include commercial vendors that offer to
make opinionated technical decisions on behalf of their customers, as
well as providing a single point of contact for support questions and
feature requests, but beyond that, offering an overwhelming array of
confusing choices is pretty much the way open source *works*.



My frustration has reached a limit. Yes, I am willing to pay money.

Which vendor do you suggest to give me a reliable package management?

Regards,
  Thomas Güttler



--
Thomas Guettler http://www.thomas-guettler.de/
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig