Bug#1054647: ITP: asdf-unit-schemas -- ASDF schemas for validating unit tags

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: asdf-unit-schemas
  Version : 0.1.0
  Upstream Author : The ASDF Developers
* URL : https://github.com/asdf-format/asdf-unit-schemas
* License : BSD-3-Clause
  Programming Lang: Python
  Description : ASDF schemas for validating unit tags

This package provides ASDF schemas for validating unit tags. Users
should not need to install this directly; instead, install an
implementation package such as asdf-astropy, which will include
asdf-unit-schemas as a recommendation. ASDF (Advanced Scientific Data
Format) is a proposed next generation interchange format for scientific
data, mainly used by the Space Telescope Science Institude (STScI).

It is a build dependency of the asdf-astropy package. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/asdf-unit-schemas

Best regards

Ole

Bug#1038969: ITP: ewah-bool-utils -- EWAH Bool Array compression for Python

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org, debian-scie...@lists.debian.org

* Package name: ewah-bool-utils
  Version : 1.0.2
  Upstream Author : Matthew Turk, Meagan Lang, Navaneeth Suresh
* URL : https://github.com/yt-project/ewah_bool_utils
* License : BSD-3-Clause
  Programming Lang: Python
  Description : EWAH Bool Array compression for Python

EWAH Bool Array compression is a repackaging and Python-exposed version
of "Enhanced Word-Aligned Hybrid" (EWAH) compression. It's a python
wrapper to a compressed bitarray method, for storing large bitsets.

It is a new build dependency of the "yt" package. I will maintain it
within the Debian Python team. Salsa dir is

https://salsa.debian.org/python-team/packages/ewah-bool-utils

Best regards

Ole

Re: Different behavior of package with pyproject.toml

[Ole Streicher]

Scott Kitterman  writes:
> On December 4, 2022 3:42:15 PM UTC, Ole Streicher  wrote:
>>So I would suspect that it is already the build does something wrong
>>here. Which package is that? pybuild-plugin-pyproject?
>
>
> python3-installer


Thank you, I filed a bug.

Best

Ole

Re: Different behavior of package with pyproject.toml

[Ole Streicher]

Hi Scott,

thanks for the hint.

Scott Kitterman  writes:
> My first guess is that in some circumstances setuptools is doing the 
> installing 
> and in others it's the dh-python pyproject plugin (using the installer 
> module).  Looking at the build log on buildd.d.o, I can see that build is 
> using the plugin/installer.
>
> I think it would be useful to stop the build right before the install step 
> starts [1], manually unpack the wheel (it's a zip file) and see if it has all 
> the files in it.  If it does, then plugin/build is doing the right thing and 
> it's an issue with either the plugin or the installer module.

I couldn't manage this; however when looking into the log, I see

-8<-
  dh_auto_build -O--buildsystem=pybuild
I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build" 
module
I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation 
--wheel --outdir 
/builds/debian-astro-team/asdf-astropy/debian/output/source_dir/.pybuild/cpython3_3.11
 
/usr/lib/python3/dist-packages/setuptools/config/pyprojecttoml.py:108: 
_BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is 
still *beta*.
  warnings.warn(msg, _BetaConfiguration)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/asdf_astropy
copying asdf_astropy/_version.py -> build/lib/asdf_astropy
[... no asdf_astropy/io files here ...]
* Building wheel...
Successfully built asdf_astropy-0.3.0-py3-none-any.whl
I: pybuild plugin_pyproject:118: Unpacking wheel built for python3.11 with 
"installer" module
-8<-

So I would suspect that it is already the build does something wrong
here. Which package is that? pybuild-plugin-pyproject?

Best

Ole

Different behavior of package with pyproject.toml

[Ole Streicher]

Hi,

I need some help with a package that switched to using pyproject.toml
only. The package is asdf-astropy, and the problem I have that it does
not package all Python files recursively: for example astropy_asdf.io
files are missing (as seen in unstable now).

https://tracker.debian.org/pkg/asdf-astropy
https://salsa.debian.org/debian-astro-team/asdf-astropy

What I do not understand is that this happens "sometimes":

* when I build the package in a clean schroot with "debuild", files in
  subpackages are missing (but no error message in the log)
  
* when I install git, then a debuild results in a complete package

* when I cut the command from the build log [1], the copy process
  always happens completely, independent of whether git was installed or
  not

* when I use pbuilder, files of subpackages are never package, with our
  without git in the build dependencies

[1] python3.11 -m build --skip-dependency-check --no-isolation --wheel --outdir 
.pybuild/cpython3_3.11

Since the package "sometimes" is built correctly, I think that upstream
did it right; however I have no idea where the problem could be. Does
anyone have an idea?

Best

Ole

Re: pybuild pyproject.toml problem

[Ole Streicher]

Scott Talbert  writes:
> On Fri, 12 Aug 2022, Ole Streicher wrote:
>> Scott Talbert  writes:
>>> On Thu, 11 Aug 2022, Ole Streicher wrote:
>>>> E: pybuild pybuild:369: build: plugin pyproject failed with: UNKNOWN
>>>> wheel found: UNKNOWN-1.0.3-py3-none-any.whl. Does pyproject.toml
>>>> specify a build-backend?
>>>
>>> I *think* the issue might be that our setuptools is too old to
>>> understand how to get project metadata from pyproject.toml (PEP 621).
>>
>> Since setuptools 63 is already in experimental, I will probably just
>> wait until it makes its way to unstable (and file a bug for it).
>
> Does building with setuptools from experimental resolve the issue?

Yes, I can confirm that it builds with the version from experimental
(63.2.0-1) without other changes.

Cheers

Ole

Re: pybuild pyproject.toml problem

[Ole Streicher]

Scott Talbert  writes:
> On Thu, 11 Aug 2022, Ole Streicher wrote:
>> ---8<---
>> * Building wheel...
>> Successfully built UNKNOWN-1.0.3-py3-none-any.whl
>> I: pybuild plugin_pyproject:118: Unpacking wheel built for
>> python3.10 with "installer" module
>> E: pybuild pybuild:369: build: plugin pyproject failed with: UNKNOWN
>> wheel found: UNKNOWN-1.0.3-py3-none-any.whl. Does pyproject.toml
>> specify a build-backend?
>> dh_auto_build: error: pybuild --build -i python{version} -p 3.10
>> returned exit code 13
>> make: *** [debian/rules:5: binary] Error 25
>> ---8<---
>
> I *think* the issue might be that our setuptools is too old to
> understand how to get project metadata from pyproject.toml (PEP 621).
> This seems to indicate that it was added in setuptools 61.0.0:
>
> https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
>
> As a workaround, you might try to patch the metadata back into setup.cfg?

Since setuptools 63 is already in experimental, I will probably just
wait until it makes its way to unstable (and file a bug for it).

Cheers

Ole

pybuild pyproject.toml problem

[Ole Streicher]

Hi,

I have a package

https://salsa.debian.org/debian-astro-team/asdf-standard

that changed its build process to pyproject.toml. As hinted by pybuild,
I added pybuild-plugin-pyproject to the build dependencies; however it
still does not build properly. The build fails with the following
output:

---8<---
* Building wheel...
Successfully built UNKNOWN-1.0.3-py3-none-any.whl
I: pybuild plugin_pyproject:118: Unpacking wheel built for python3.10 with 
"installer" module
E: pybuild pybuild:369: build: plugin pyproject failed with: UNKNOWN wheel 
found: UNKNOWN-1.0.3-py3-none-any.whl. Does pyproject.toml specify a 
build-backend?
dh_auto_build: error: pybuild --build -i python{version} -p 3.10 returned exit 
code 13
make: *** [debian/rules:5: binary] Error 25
---8<---

which I do not understand. The build backend specified in pyproject.toml is

build-backend = "setuptools.build_meta"

What goes wrong here?

Full build log:

https://salsa.debian.org/debian-astro-team/asdf-standard/-/jobs/3095942

Cheers

Ole

Bug#1015838: ITP: specreduce-data -- Test and reference data for the specreduce package

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-as...@lists.debian.org, 
debian-python@lists.debian.org
Control: block 1012441 by -1

* Package name: specreduce-data
  Version : 0+git2021.11.18
  Upstream Author : Astropy Specreduce contributors
* URL : https://github.com/astropy/specreduce-data
* License : BSD-3-Clause
  Programming Lang: Python 3
  Description : Test and reference data for the specreduce package

This package contains general reference data for spectroscopic data
reduction (e.g. standard star spectra, atmospheric extinction curves,
line lists for calibration lamps) and test data for the specreduce
package.

This is a dependency of the upcoming specreduce package (ITP #1012441).

I will maintain the package within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/specreduce-data

Best regards

Ole

Bug#1015324: ITP: synphot -- Simulate photometric data ans spectra in astronomy

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-as...@lists.debian.org, 
debian-python@lists.debian.org
Control: block 1012441 by -1

* Package name: synphot
  Version : 1.1.2
  Upstream Author : Space Telescope Science Institute
* URL : https://github.com/spacetelescope/synphot_refactor
* License : BSD-3-Clause
  Programming Lang: Python 3
  Description : Simulate photometric data ans spectra in astronomy

synphot simulates photometric data and spectra, observed or otherwise.
It can incorporate external filters, spectra, and data. One can also use
a pre-defined standard star (Vega), bandpass, or extinction law.
Furthermore, it allows one to:

 * Construct complicated composite spectra using different models.
 * Simulate observations.
 * Compute photometric properties such as count rate, effective
   wavelength, and effective stimulus.
 * Manipulate a spectrum; e.g., applying redshift or normalize it to a
   given flux value in a given bandpass.
 * Sample a spectrum at given wavelengths.
 * Plot a quick-view of a spectrum.
 * Perform repetitive operations such as simulating the observations of
   multiple type of sources through multiple bandpasses.

This is a dependency of the upcoming specreduce package (ITP #1012441).

I will maintain the package within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/synphot

Best regards

Ole

Bug#1012441: ITP: specreduce -- Astropy coordinated package to reduce and calibrate spectroscopic astronomical data

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-as...@lists.debian.org, 
debian-python@lists.debian.org

* Package name: specreduce
  Version : 1.0.0
  Upstream Author : Astropy Specreduce contributors 

* URL : http://astropy.org/
* License : BSD-3-Clause
  Programming Lang: Python 3
  Description : Astropy package to reduce and calibrate astronomical spectra

The specreduce package aims to provide a data reduction toolkit for
optical and infrared spectroscopy, on which applications such as
pipeline processes for specific instruments can be built. The scope of
its functionality is limited to basic spectroscopic reduction, with
basic image processing steps (such as bias subtraction) instead
covered by ccdproc and other packages, data analysis by specutils and
visualization by specviz or cubeviz. A few examples will nevertheless
be provided of its usage in conjunction with these complementary
packages.

I will maintain the package within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/specreduce

Best regards

Ole

Bug#1006200: ITP: asdf-standard -- Standards document describing ASDF

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: asdf-standard
  Version : 1.6.0
  Upstream Author : The ASDF Developers
* URL : https://github.com/asdf-format/asdf-standard
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Standards document describing ASDF

This document describes the Advanced Scientific Data Format (ASDF).
ASDF is a proposed next generation interchange format for scientific
data. ASDF aims to exist in the same middle ground that made FITS so
successful, by being a hybrid text and binary format: containing
humanditable metadata for interchange, and raw binary data that is fast
to load and use. Unlike FITS, the metadata is highly structured and is
designed up-front for extensibility.

It is a build dependency of the asdf-astropy package. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/asdf-standard

Best regards

Ole

Bug#1004960: ITP: cmyt -- Matplotlib colormaps from the yt project

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org, debian-scie...@lists.debian.org

* Package name: cmyt
  Version : 1.0.4
  Upstream Author : Clement Robert
* URL : https://github.com/yt-project/cmyt
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Matplotlib colormaps from the yt project

cmyt integrates with matplotlib in a similar fashion to cmocean or cmasher

It is a new build dependency of the "yt" package. I will maintain it
within the Debian Python team. Salsa dir is

https://salsa.debian.org/python-team/packages/cmyt

Best regards

Ole

Bug#1003331: ITP: asdf-wcs-schemas -- World Coordinate System (WCS) ASDF schemas

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: asdf-wcs-schemas
  Version : 0.1.1
  Upstream Author : The ASDF Developers
* URL : https://github.com/asdf-format/asdf-wcs-schemas
* License : BSD-3-Clause
  Programming Lang: Python
  Description : World Coordinate System (WCS) ASDF schemas

This package provides ASDF schemas for validating WCS tags.
Users should not need to install this directly; instead, install an
implementation package such as asdf-astropy, which will include
asdf-coordinates-schemas as a dependency. ASDF (Advanced Scientific Data
Format) is a proposed next generation interchange format for scientific
data,mainly used by the Space Telescope Science Institude (STScI).

It is a build dependency of the gwcs package. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/asdf-gwcs-schemas

Best regards

Ole

Bug#1003049: ITP: asdf-coordinates-schemas -- ASDF schemas for validating coordinates tags

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: asdf-coordinates-schemas
  Version : 0.1.0
  Upstream Author : The ASDF Developers
* URL : https://github.com/asdf-format/asdf-coordinates-schemas
* License : BSD-3-Clause
  Programming Lang: Python
  Description : ASDF schemas for validating coordinates tags

This package provides ASDF schemas for validating coordinates tags.
Users should not need to install this directly; instead, install an
implementation package such as asdf-astropy, which will include
asdf-coordinates-schemas as a dependency. ASDF (Advanced Scientific Data
Format) is a proposed next generation interchange format for scientific
data,mainly used by the Space Telescope Science Institude (STScI).

It is a build dependency of the asdf-astropy package. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/asdf-coordinates-schemas

Best regards

Ole

Bug#1003048: ITP: asdf-transmorm-schemas -- ASDF schemas for validating transform tags

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: asdf-transform-schemas
  Version : 0.2.0
  Upstream Author : The ASDF Developers
* URL : https://github.com/asdf-format/asdf-transform-schemas
* License : BSD-3-Clause
  Programming Lang: Python
  Description : ASDF schemas for validating transform tags

This package provides ASDF schemas for validating transform tags. Users
should not need to install this directly; instead, install an
implementation package such as asdf-astropy, which will include
asdf-transform-schemas as a dependency. ASDF (Advanced Scientific Data
Format) is a proposed next generation interchange format for scientific
data,mainly used by the Space Telescope Science Institude (STScI).

It is a build dependency of the asdf-astropy package. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/asdf-transform-schemas

Best regards

Ole

Bug#1003040: ITP: asdf-astropy -- ASDF serialization support for astropy

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: asdf-astropy
  Version : 0.1.2
  Upstream Author : The ASDF Developers
* URL : https://github.com/astropy/asdf-astropy
* License : BSD-3-Clause
  Programming Lang: Python
  Description : ASDF serialization support for astropy

This package includes plugins that provide ASDF serialization support
for astropy objects. The plugins are automatically enabled when the
package is installed. ASDF (Advanced Scientific Data Format) is a
proposed next generation interchange format for scientific data,
mainly used by the Space Telescope Science Institude (STScI).

It is a new build dependency of the gwcs package. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/asdf-astropy

Best regards

Ole

Bug#998234: ITP: mpl-animators: An interative animation framework for matplotlib

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, 
debian-python@lists.debian.org, debian-as...@lists.debian.org


* Package name: mpl-animators
  Version : 1.0.0
  Upstream Author : Sunpy Developers
* URL : https://github.com/sunpy/mpl-animators
* License : BSD-3-Clause
  Programming Lang: Python
  Description : An interative animation framework for matplotlib

The mpl_animators package provides a set of classes which allow the easy
construction of interactive matplotlib widget based animations. “Out of
the box” classes are provided for making line or image plots from numpy
arrays, with sliders to control the animation automatically added for
all dimensions not on the axes of the plot.

It is a new build dependency of the sunpy and ndcube packages. I will
maintain it within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/mpl-animators

Best regards

Ole

Re: platform.machine() on salsa i386 build?

[Ole Streicher]

Hi Nilesh, Andrey,

Nilesh Patra  writes:
> On 10/30/21 5:50 PM, Ole Streicher wrote:
>> Hi,
>> https://salsa.debian.org/debian-astro-team/pyraf/-/jobs/2130878#L2062
>> How can one find out the correct machine here?
>
> I just committed something to salsa -- sort of a hack, but it works
> (i386 pipeline passes). I do this
> for one of my other package as well. Please consider taking a look.

Thank you, but I think I will not use it: This requires dpkg-dev (which
is the package with dpkg-architecture) being installed, which I would
not like to depend on just to have the possibility to test.

I will rather follow Stefanos hint to use sys.implementation._multiarch,
which is however hackish as well, since variables starting with "_"
should not considered as part of the API, and this may break without
warning.

Optimal would be a way to detect whether the internal floating point
processing happens with 80 bit FPU registers (i386) or 64 bit FPU
registers (other systems), since this is the relevant difference here.

Best regards

Ole

platform.machine() on salsa i386 build?

[Ole Streicher]

Hi,

I have a package (pyraf) where I need to switch off some tests for i386
(but not for other 32-bit platforms). I do this by

import platform
is_i386 = platform.machine() in ('i386', 'i486', 'i586', 'i686')

[...]
@pytest.mark.skipif(is_i386, reason='diff is not identical on i386')
 def test_gki_single_prow():

https://salsa.debian.org/debian-astro-team/pyraf/-/blob/master/debian/patches/Skip-tests-that-are-too-tight-for-i386.patch

However, on Salsa this fails; and it appears that this test is
ignored. Further investigation gave me that there -- during the regular
(pybuild) test -- platform.machine() gives "x86_64", despite of the
32-bit platform.

https://salsa.debian.org/debian-astro-team/pyraf/-/jobs/2130878#L2062

How can one find out the correct machine here?

Best regards

Ole

Bug#995696: ITP: casa-formats-io -- Code to handle I/O from/to data in CASA format

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: casa-formats-io
  Version : 0.1
  Upstream Author : Thomas Robitaille
* URL : http://casa-formats-io.readthedocs.org
* License : LGPL
  Programming Lang: Python
  Description : Code to handle I/O from/to data in CASA format

The casa-formats-io package is a small package which implements functionality 
to read data stored in CASA formats (such as .image datasets). This 
implementation is independent of and does not use casacore. The motivation for 
this package is to provide efficient data access via dask arraye, 
cross-platform data access, and data access with all modern Python versions.

It is a new build dependency of the "spectral-cube" package. I will maintain it
within the Debian Astro team in Salsa:

https://salsa.debian.org/debian-astro-team/casa-formats-io

Best regards

Ole

Bug#992999: ITP: unyt -- Python package for handling numpy arrays with units.

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org, 
debian-as...@lists.debian.org, debian-scie...@lists.debian.org

* Package name: unyt
  Version : 2.8.0
  Upstream Author : Nathan Goldbaum
* URL : debian-de...@lists.debian.org
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Pyton package for handling numpy arrays with units.

Often writing code that deals with data that has units can be confusing.
A function might return an array but at least with plain NumPy arrays,
there is no way to easily tell what the units of the data are without
somehow knowing a priori.

The unyt package (pronounced like “unit”) provides a subclass of NumPy’s
ndarray class that knows about units.

It is a new build dependency of the "yt" package. I will maintain it
within the Debian Python team. Salsa dir is

https://salsa.debian.org/python-team/packages/unyt

Best regards

Ole

Re: Generic Python packages which don’t work on all architectures

[Ole Streicher]

Paul Wise  writes:
> On Thu, Jan 21, 2021 at 3:42 PM Ole Streicher wrote:
>
>> But this is also the case for all packages which implicitly depend on
>> other packages which are not available on some architectures.
>
> This is true, but it doesn't make for a good user experience.

Yes, but this is a separate problem, and this IMO cannot be reliably
solved by adding a simple arch-specific declaration to the source
package: If we would do (require) f.e. an additional "installs-on" field
in d/control, then each maintainer would be requested to monitor all
dependencies on all archs to find out where the package could be
installable, and to re-upload the package with adjustments whenever one
of the dependencies changes the list of supported architectures.

IMO, a useful solution would be to parse the dependency chain on the
client (aptitude, synaptic or whatever) and to list only those packages
that can be installed.

If we would introduce arch specific marker packages, we wouldn't solve
this ofcourse, but we would solve the problem that you can install a
package that is actually not working on the architecture, which is even
a worse user experience. And we would solve the problem that one can't
specify f.e. all little-endian systems, or all 64-bit systems as build
dependency, except for explicitly listing all of them (which is IMO a
nightmare for ports or for derivatives that have additional
architectures).

It is not perfect, but a pragmatic step forward.

Best

Ole

Re: Generic Python packages which don’t work on all architectures

[Ole Streicher]

Paul Wise  writes:
> On Thu, Jan 21, 2021 at 1:12 PM Ole Streicher wrote:
>
>> This would be a quite flexible and extendible approach to have packages
>> installable only where they work.
>
> There is precedent for this sort of thing in the isa-support source
> package, which fails installation when your CPU doesn't support
> particular ISA extensions like SSE or NEON.
>
> It is a reasonable workaround for now, but the packages using the
> pseudo-packages would still be available, causing confusion when
> people try to install them.

But this is also the case for all packages which implicitly depend on
other packages which are not available on some architectures. This is
not very rare:

 * Python arch:all packages may depend on arch:* Python packages which
   are not available everywhere

 * Packages written in an interpreted language are arch:all, but depend
   on the interpreter and can only installed on archs where the
   interpreter exists (like gnudatalanguage or iraf from my portfolio)

Generally, arch:all packages depend on a lot of architecture dependent
packages, and unless one resolves the whole dependency chain, there is
no way to check whether a package is actually installable.

> It would be better to just not have them available where they aren't
> going to work. I also wonder how multi-arch stuff would interact with
> the pseudo-packages idea.

IMO this is a problem that already exists there and is not
solved. Imagine the following constellation:

 * I have a python3-foo package with arch:all.
 * this depends on python3-foo-obj that exists only for i386.

Now I have a x86_64 machine that gets i386 added. On this machine, I
could install python3-foo (since the dependency is resolvable), but
'import foo' with the x86_64 interpreter would fail.

Best regards

Ole

Re: Generic Python packages which don’t work on all architectures

[Ole Streicher]

Ansgar  writes:
> On Thu, 2021-01-21 at 13:46 +0100, Ole Streicher wrote:
>> I am still wondering why we don't have just empty some pseudo-
> packages that are available only on specific architectures
>> (or  groups of them, like linux, or little endian, or 64 bit or so).
>
> To solve which problem?

For example, the problem that a certain package is working properly only
on big endian systems. Or on Linux.

Currently, there is (as discussed here) no way to tell that an arch:all
package is not working on a 32-bit system. And for architecture
dependent builds, one needs to specify every single architecture where
it is supposed to build. How do you currently otherwise would specify
"all little endian systems" as build dependency?

> Packages being installable don't mean that they can do anything useful
> as they might, for example, require special hardware.  We don't have a
> way to express "requires device ${X}" on a package level.

Special hardware is another topic, which is surely not solved here (and
not solvable at all in general, since hardware today may be
hot-pluggable, and the hardware configuration at install time does not
need to be the configuration at run time).

However, the proposal to have architecture (group) specific empty
packages would already solve some of the problems if this is limited to
architecture specification.

Best regards

Ole

Re: Generic Python packages which don’t work on all architectures

[Ole Streicher]

Paul Wise  writes:
> This is what I eventually chose for iotop. At the time I wanted dpkg
> and dak to support something like Architecture: linux-all, which would
> build arch: all packages, but only put them in the Packages files for
> the Linux architectures.
>
> I am now thinking that a more generic solution than Architecture:
> linux-all is needed, in order to cover your case as well. Perhaps
> something like Available-Architecures or Runtime-Architectures or
> Architecture-all-Architectures: or similar.

I am still wondering why we don't have just empty some pseudo-packages
that are available only on specific architectures (or groups of them,
like linux, or little endian, or 64 bit or so). If a package works (or
builds) only on a specific arch (or an arch with specific properties),
it can be set dependent (or built-dependent) on that.

This would be a quite flexible and extendible approach to have packages
installable only where they work.

Best regards

Ole

Bug#978432: ITP: astroalign -- Astrometric registration of images when no WCS info is available

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-as...@lists.debian.org, 
debian-python@lists.debian.org

* Package name : astroalign
  Version  : 2.3.1
  Upstream Author  : Martin Beroiz 
* URL  : http://astroalign.readthedocs.io/
* License  : Expat
  Programming lang : Python
  Upstream git : https://github.com/quatrope/astroalign
  Pypi URL : https://pypi.python.org/pypi/astroalign
  Description  : Astrometric registration of images

ASTROALIGN is a python module that will try to register (align) two stellar
astronomical images, especially when there is no WCS information available.

It does so by finding similar 3-point asterisms (triangles) in both images
and estimating the affine transformation between them.

This is a popular astropy affiliated package. It will be maintained under the
umbrella of the Debian Astro team. A salsa repo was created at

https://salsa.debian.org/debian-astro-team/astroalign

Best regards

Ole

Bug#960314: ITP: tifffile -- Read and write TIFF(r) files with Python

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name : tifffile
  Version : TBD
  Upstream Author : Christoph Gohlke <mailto:cgoh...@uci.edu>
* URL : https://pypi.org/project/tifffile/#description
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Read and write TIFF(r) files with Python

Tifffile is a Python library to

 1. store numpy arrays in TIFF (Tagged Image File Format) files, and
 2. read image and metadata from TIFF-like files used in bioimaging.

Image and metadata can be read from TIFF, BigTIFF, OME-TIFF, STK, LSM,
SGI, NIHImage, ImageJ, MicroManager, FluoView, ScanImage, SEQ, GEL, SVS,
SCN, SIS, ZIF, QPTIFF, NDPI, and GeoTIFF files.

Numpy arrays can be written to TIFF, BigTIFF, and ImageJ hyperstack
compatible files in multi-page, memory-mappable, tiled, predicted, or
compressed form.
This is a new dependency of the scikit-image (skimage) package. I will
maintain it within the Debian Python team. Salsa dir is

https://salsa.debian.org/python-team/modules/tifffiles

Best regards

Ole

Bug#960142: ITP: pyerfa -- Python bindings for ERFA routines

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: pyerfa
  Version : TBD
  Upstream Author : Astropy Developers
* URL : https://github.com/liberfa/pyerfa
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Python bindings for ERFA routines

PyERFA is the Python wrapper for the ERFA library (Essential Routines
for Fundamental Astronomy), a C library containing key algorithms for
astronomy, which is based on the SOFA library published by the
International Astronomical Union (IAU). All C routines are wrapped as
Numpy universal functions, so that they can be called with scalar or
array inputs.

The project is a split of astropy._erfa module, developed in the context
of Astropy project, into a standalone package, and therefore will be a
dependency of upcoming Astropy releases.

The packages will be maintained under Debian Astro team maintainance,
with the repository at

https://salsa.debian.org/debian-astro-team/pyerfa

Best regards

Ole

Bug#959778: ITP: pybdsf -- Python Blob Detection and Source Finder

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: pybdfs
  Version : 1.9.2-1
  Upstream Author : LOFAR/Astron
* URL : https://www.astron.nl/citt/pybdsf/
* License : GPLv2
  Programming Lang: Python
  Description : Python Blob Detection and Source Finder

PyBDSF (the Python Blob Detection and Source Finder)
is a tool designed to decompose radio interferometry images into source
and make available their properties for further use. PyBDSF can
decompose an image into a set of Gaussians, shapelets, or wavelets as
well as calculate spectral indices and polarization properties of
sources and measure the psf variation across an image. PyBDSF uses an
interactive environment based on CASA that will be familiar to most
radio astronomers. Additionally, PyBDSF may also be used in Python
scripts.

The packages will be maintained under Debian Astro team maintainance,
with the repository at

https://salsa.debian.org/debian-astro-team/pybdfs

Best regards

Ole

Bug#956048: ITP: extension-helpers -- Utilities for building and installing packages

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org,
 debian-de...@lists.debian.org

* Package name: extension-helpers
  Version : 0.1
  Upstream Author : The Astropy Developers
* URL : https://pypi.org/project/extension-helpers/
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Utilities for building and installing packages

The extension-helpers package includes convenience helpers to assist
with building Python packages with compiled C/Cython extensions. It is
developed by the Astropy project but is intended to be general and
usable by any Python package.

This is not a traditional package in the sense that it is not intended
to be installed directly by users or developers. Instead, it is meant to
be accessed when the setup.py command is run and should be defined as a
build-time dependency in pyproject.toml files.

This package is a new dependency of many astropy affiliated packages. I
intend to maintain it under the DPMT hood, so the salsa git dir will be

https://salsa.debian.org/python-team/modules/extension-helpers

Best regards

Ole

pybuild with more than one binary package

[Ole Streicher]

Hi,

we have a source package (older packaging attempt in gavodachs [1],
Arch:all) that builds a number of binary packages. Specifically:

python3-gavo-stc gavo/stc/
python3-gavo-utils   gavo/utils/, gavo/imp
python3-gavo-votable gavo/votable
python3-gavo all other gavo/* + 2 entry points

How do I do that? The pybuild documentation states that it is possible
to magically build several binary packages without the need of *.install
files, but this seems not to be the case here: all goes into
python3-gavo, the other packages are empty.

Is there a trick to do the magic? And if not, what should one specify in
the *.install files in that case?

Best

Ole

[1] https://salsa.debian.org/debian-astro-team/gavodachs

Bug#949470: ITP: pytest-filter-subpackage -- Allow sub-packages to test to be selected with e.g. -P io.fits,modeling

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: pytest-filter-subpackage
  Version : 0.1.1
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astropy/pytest-filter-subpackage
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Allow sub-packages to test to be selected with -P

This package contains a simple plugin for the pytest framework that
provides a shortcut to testing all code and documentation for a given
sub-package. It is a new build dependency of pytest-astropy.

I will maintain it within the Debian Python team. Salsa dir is

https://salsa.debian.org/python-team/modules/pytest-filter-subpackage

Best regards

Ole

Re: Understanding pybuild

[Ole Streicher]

Hi Dmitry,

Dmitry Shachnev  writes:
> On Thu, Dec 19, 2019 at 01:09:40PM +0100, Ole Streicher wrote:
>> 2. dh_auto_test then tries runs the tests in .pybuild. If the test need
>> data files installed, they must be copied manually (right?).
>
> You can use debian/pybuild.testfiles to copy the test files. See pybuild(1)
> for details.

That would formalize it a bit, but still does not solve the underlying
problem: it requires that I maintain the list of these files manually,
while upstream already does this (by adding them to the list of data
files).

And also, the built time tests are with pybuild not done with the files
(anc compilations) that are finally installed, but the installed files
(especially Python extensions) are created again from scratch, doubling
the compile time and introducing another difficile source of error.

Best regards

Ole

Understanding pybuild

[Ole Streicher]

Hi,

I am currently updating scikit-image (skimage) to its newest version,
and I have some difficulties to understand how pybuild works (for
setup.py packages):

1. first, dh_auto_build calls "python3.X setup.py build" to create the
packages (Python files and compiled extensions) in
..pybuild/cpython3_3.X/. This however does not include the copying of any
data files.

2. dh_auto_test then tries runs the tests in .pybuild. If the test need
data files installed, they must be copied manually (right?).

3. dh_auto_install finally calls "python3.X setup.py install ..." to
install the complete packages (Python files, extensions, and data files)
from their original place to debian/tmp/. This includes a re-compilation
of the extensions.

What I do not understand now is: what is the purpose of step 1? Except
for the tests, these files seem to be unused. And for the tests, they
are not complete, since the data files are missing.

And what is the canonical way to get the data files copied to
..pybuild/...? They are already marked as such in setup.py. Doing this
manually by the Debian maintainer seems a constant source of error.

Best

Ole

Python 3.8 transition: CMake problems

[Ole Streicher]

Hi,

I am currently updating the "casacore" package to the latest upstream
version. Doing this, I discovered that it is not marked as part of the
Python 3.8 transition, and was also not binNMUed for this, although it
has the package libcasa-python3-3 (to be renamed to libcasa-python3-4)
that depends on libpython3.7. Was this somehow forgotten?

Nevertheless, when I now try to build the (updated) package, I get the
following error:

8<-
  Could NOT find Python3 (missing: Python3_LIBRARY Python3_INCLUDE_DIR
  Development NumPy) (found version "3.8.0")
Call Stack (most recent call first):
  /usr/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:378 
(_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.15/Modules/FindPython/Support.cmake:1653 
(find_package_handle_standard_args)
  /usr/share/cmake-3.15/Modules/FindPython3.cmake:215 (include)
  python3/CMakeLists-cmake3.14.txt:3 (find_package)
8<-

The corresponding line in python3/CMakeLists-cmake3.14.txt is

find_package(Python3 REQUIRED COMPONENTS Interpreter Development NumPy)

which looks normal to me. Any idea why CMAKE does not find Python 3.8?

Best regards

Ole

Bug#943940: ITP: pytest-astropy-header -- Include basic system dependencies in the header of pytest output

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org, 
debian-as...@lists.debian.org

* Package name: pytest-astropy-header
  Version : 0.1.1
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astropy/pytest-astropy-header
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Include basic sysdeps in the header of pytest output

This plugin package provides a way to include information about the
system, Python installation, and select dependencies in the header of
the output when running pytest. It can be used with packages that are
not affiliated with the Astropy project, but is optimized for use
It is a new build dependency of pytest-astropy.

I will maintain it within the Debian Astro team. Salsa dir is

https://salsa.debian.org/debian-astro-team/pytest-astropy-header

Best regards

Ole

Re: package_data in python package

[Ole Streicher]

Félix Sipma  writes:
> They just run "python3 setup.py install" 
> (https://github.com/scheibler/khard/issues/231):
> [...]
>
> For my part, I "just" build the debian package with "gbp buildpackage",
> and with "dh $@ --with python3 --buildsystem=pybuild" in debian/rules.
>
> The solutions I say are: I can keep the patch in the package (without
> really understanding the need for it) or try to convince upstream to
> merge it (but they can't reproduce the problem, so why would they do
> that?). In both cases, probably others will encounter the same problem
> in other packages... It would be nice to fix the problem once for all
> (if possible :)).

Just FYI, https://bugs.debian.org/940728 describes a similar problem,
although there the unhandled files are Python source files. pybuild
seems to handle things different from plain setup.py.

Cheers

Ole

Re: pybuild mysteriously does not copy all testfiles

[Ole Streicher]

Hi IOhannes,

I have basically the same problem, funnily with almost the same subject:

https://lists.debian.org/debian-python/2019/09/msg00040.html

The difference is, that the "external" subdirectory is not copied. So,
it seems to be a bug in dh-python.

Cheers

Ole

"IOhannes m zmölnig (Debian/GNU)"  writes:
> updating one of my packages (python-can), the test-suite started to
> fail, with an error-code 5, indicating that "no tests have been run".
>
> since there *are* tests to be run, i investigated and - after scratching
> my head - found that 'pybuild' did not fully copy the test-directory
> from the source-directory to the build directory.
>
> The sources "test/" directory contains:
> ```
> test/serial_test.py
> test/test_slcan.py
> test/test_cyclic_socketcan.py
> test/data/logfile.blf
> test/data/__init__.py
> test/data/example_data.py
> test/data/logfile.asc
> test/test_bit_timing.py
> test/zero_dlc_test.py
> test/test_systec.py
> test/open_vcan.sh
> test/message_helper.py
> test/contextmanager_test.py
> test/test_detect_available_configs.py
> test/test_message_sync.py
> test/notifier_test.py
> test/config.py
> test/simplecyclic_test.py
> test/test_scripts.py
> test/logformats_test.py
> test/listener_test.py
> test/back2back_test.py
> test/test_message_class.py
> test/__init__.py
> test/test_message_filtering.py
> test/test_socketcan.py
> test/test_kvaser.py
> test/test_viewer.py
> test/network_test.py
> test/test_vector.py
> test/test_socketcan_helpers.py
> test/test_load_file_config.py
> ```
>
> However, ".pybuild/cpython3_3.7_can/build/test/" contains only: (after
> dh_auto_test failed):
> ```
> test/data/__init__.py
> test/data/example_data.py
> ```
>
> d/rules is pretty standard:
> ```
> #! /usr/bin/make -f
> export PYBUILD_NAME=can
> include /usr/share/dpkg/default.mk
> export DEB_VERSION_UPSTREAM
> %:
> dh $@ --with python3,sphinxdoc --buildsystem=pybuild
> override_dh_clean:
> dh_clean
> rm -f test.asc test.csv
> rm -rf .pytest_cache/
> override_dh_auto_build:
> dh_auto_build
> python3 -m sphinx -E -T -b html doc .pybuild/docs/html/
> override_dh_auto_install:
> dh_auto_install
> -for f in debian/*/usr/bin/*.py; do mv $$f $${f%.py}; done
> -rm -f debian/*/usr/lib/python*/dist-packages/test.???
> ```
>
> and the package does not contain d/pybuild.testfiles or similar.
>
> Now if I add a file from tests (say "test/network_test.py") to
> d/pybuild.testfiles, that file is copied (although into
> .pybuild/cpython3_3.7_can/build/ rather than into
> .pybuild/cpython3_3.7_can/build/test/) and runs correctly.
> adding "test/" to d/pybuild.testfiles (with or without the trailing
> slash) did not do anything (apart from the original behaviour of only
> copying 2 files).
> doing something like "find test -type f > d/pybuild.testfiles" copied
> all files, however the directory structure was lost (all files ended up
> in .pybuild/cpython3_3.7_can/build/), and the tests started to fail as
> they didn't find files in the subdirectories as expected.
>
> i'm totally stumified.
>
> an intermediate hack i found to work is to manually copy the "test/"
> directoy into the build-dir, using:
> export PYBUILD_BEFORE_TEST=cp -r {dir}/test {build_dir}
>
> but that seems extraordinarily clumsy. esp. since pybuild(1) says:
>> By default only test and tests directories are copied to build
>> directory.
>
> any ideas what does wrong?
> mgfdsar
> IOhannes

pybuild mysteriously does not copy all files

[Ole Streicher]

Hi,

I am not sure whether this is a bug in dh-python or if something is
wrong with my package (python-asdf). The package has an (almost)
standard setup, so dh_build is not overridden.

However, when I try to package, some subdirectories are not copied;
namely `asdf/external`:

8<--
   dh_auto_build -O--buildsystem=pybuild
I: pybuild base:217: /usr/bin/python3 setup.py build 
running build
running build_py
creating /build/python-asdf-2.4.2/.pybuild/cpython3_3.7_asdf/build/asdf
[...]
8<--

The subdir `asdf/external` which is in the source is not created and not
filled during this process.

However, when I run `python3 setup.py build` or `pybuild --build`
manually, everything is correct:

8<--
$ python3 setup.py build -vvv
running build
running build_py
creating build
creating build/lib
creating build/lib/asdf
copying asdf/yamlutil.py -> build/lib/asdf
[...]
creating build/lib/asdf/extern
copying asdf/extern/RangeHTTPServer.py -> build/lib/asdf/extern
copying asdf/extern/__init__.py -> build/lib/asdf/extern
copying asdf/extern/atomicfile.py -> build/lib/asdf/extern
copying asdf/extern/decorators.py -> build/lib/asdf/extern
[...]
8<--

Is there any idea what happens here? Why does pybuild (or setup.py)
behave differently when called from dh_python than directly?

How can I ensure that all files are copied during dh_auto_build?

Best regards

Ole

Pybuild: build time test requiring registration of entry point

[Ole Streicher]

Hi,

I have a package (sunpy) that in the latest registers an entry point for
another package (asdf):

[options.entry_points]
asdf_extensions =
  sunpy = sunpy.io.special.asdf.extension:SunpyExtension

This functionality is also used during some tests, which I would like to
keep.

However, pybuild runs the test before "setup.py install" is called, so
the extension is not there at this time. I would not like to disable the
test, since it checks some important functionality.

Is there a way to solve this?

Best regards

Ole

Bug#929438: ITP: parfive -- An asyncio based parallel file downloader for Python

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-as...@lists.debian.org, debian-de...@lists.debian.org, 
debian-python@lists.debian.org

* Package name : parfive
  Version  : 1.0.0
  Upstream Author  : Stuart Mumford
* URL  : https://github.com/Cadair/parfive
* License  : MIT
  Programming lang : Python
  Description  : A parallel file downloader using asyncio.

Parfive is a small library for downloading files, its objective is to
provide a simple API for queuing files for download and then providing
excellent feedback to the user about the in progress downloads. It also
aims to provide a clear interface for inspecting any failed downloads.

The parfive package was motivated by the needs of SunPy’s net submodule,
but should be generally applicable to anyone who wants a user friendly
way of downloading multiple files in parallel.

Parfive supports downloading files over either HTTP or FTP using aiohttp
and aioftp aioftp is an optional dependency, which does not need to be
installed to download files over HTTP.

The Debian package will be maintained by the Debian Astro team. A git
repository will be created at

https://salsa.debian.org/debian-astro-team/astropy-sphinx-theme

Best regards

Ole

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:
> On Mon, Jan 07, 2019 at 09:03:11AM +0100, Ole Streicher wrote:
>> The problem is that aplpy uses matplotlib, and the old matplotlib uses
>> the deprecated numpy function np.asscalar(), which leads to a
>> DeprecationWarning, which is (on purpose, by upstream) thrown as error.
>> 
>> https://ci.debian.net/data/autopkgtest/testing/amd64/a/aplpy/1658948/log.gz
>> 
>> This does only happen in the combination "new numpy + old matplotlib",
>> but this is the one that is tested for the CI test.
>
> mhh… for this, indeed a breaks numpy -> matplotlib could be used, but it
> feels a bit too strong to me, as nothing was really broken since it's
> just deprecations.
>
> elbrus; this is a case where I think I could use your input (tl;dr: new
> numpy causes deprecations in testing's version of matplotlib (fixed in
> unstable), which in turn causes failures in new aplpy, how to make the
> stack happy?).

BTW, same is true for astropy-healpix and pyregion (both not maintained
by me). Just to shorten the list of relevant CI failures.

Cheers

Ole

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:
> On Sun, Jan 06, 2019 at 05:07:41PM +0100, Ole Streicher wrote:
>> Now it turns out that there is a new migration problem, which is aplpy:
>> Current aplpy (2.0~rc2-2) CI test works well
>
> You probably mean aplpy 1.1.1-4.

No, I meant the one above (although the unstable test was not done on
ci.d.n when I wrote my last mail).

>>  - with numpy-1.15.4-2 (testing) and matplotlib-2.2.2-4 (testing),
>>  - with numpy-1.16.0~rc2-1 (unstable) and matplotlib-3.0.2-2 (unstable).
>> 
>> However it does not work well when combining numpy-1.16.0~rc2 (unstable)
>> and matplotlib-2.2.2-4 (testing), which is the combination that is
>> tested for migration of numpy. Needless to say that matplotlib migrates
>> only after numpy. What should one do here? Declaring another
>> "Breaks: matplotlib (<< 3.0)" in numpy?
>
> Well, to me it seems it's python3-astropy that is missing a dependency
> on python3-skimage there, to be honest:
> |>   from skimage.measure import block_reduce
> |E   ModuleNotFoundError: No module named 'skimage'
> |
> |/usr/lib/python3/dist-packages/astropy/nddata/utils.py:370: 
> ModuleNotFoundError
>
> However I wouldn't be able to tell you why it would pass with numpy and
> matplotlib from unstable, given that neither pulls in skimage…

astropy needs askimage only in some cases, so it is (should be) a
suggestion and not a recommendation. aplpy is however one of these
cases -- but this is not the problem here, since aplpy 2.0~rc2-2 has
skimage in its test dependency.

The problem is that aplpy uses matplotlib, and the old matplotlib uses
the deprecated numpy function np.asscalar(), which leads to a
DeprecationWarning, which is (on purpose, by upstream) thrown as error.

https://ci.debian.net/data/autopkgtest/testing/amd64/a/aplpy/1658948/log.gz

This does only happen in the combination "new numpy + old matplotlib",
but this is the one that is tested for the CI test.

best

Ole

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:

> On Sat, Jan 05, 2019 at 08:30:28PM +0100, Ole Streicher wrote:
>> This would remove one dependent party (release team) from the chain of
>> blocking causes for the migration.
>
> Given your email on -mentors a few minutes ago I see there are troubles
> on removing python-astropy from unstable (I'll reply to that in a bit),
> but for testing is quite easy.
> Just file a bug against release.debian.org asking to remove it from
> testing, and one against python-astropy at RC-level with "not release
> with buster", and it will be out of testing beofore you even notice.

The reverse build deps of python-astropy in testing are pyregion and
veusz. Veusz has the build-dep removed in unstable, but didn't migrate
since 192 days. Pyregion just got an upload with a fix, but will
probably only migrate after numpy (which will not migrate until
python-astropy is removed).

Now it turns out that there is a new migration problem, which is aplpy:
Current aplpy (2.0~rc2-2) CI test works well

 - with numpy-1.15.4-2 (testing) and matplotlib-2.2.2-4 (testing),
 - with numpy-1.16.0~rc2-1 (unstable) and matplotlib-3.0.2-2 (unstable).

However it does not work well when combining numpy-1.16.0~rc2 (unstable)
and matplotlib-2.2.2-4 (testing), which is the combination that is
tested for migration of numpy. Needless to say that matplotlib migrates
only after numpy. What should one do here? Declaring another
"Breaks: matplotlib (<< 3.0)" in numpy?

Best

Ole

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:

> On Sat, Jan 05, 2019 at 04:02:35PM +0100, Ole Streicher wrote:
>> I'll do tonight. It however looks a bit suboptimal: when the CI test
>> with a new version fails for an old reverse dependency, then the new
>> version obviously breaks that old package. So, the breakage could be
>> detected (and taken into account) automatically. Why do we need a manual
>> action then?
>
> Let me try to suggest you see from another side: the goal of the Breaks
> field is to prevent a given combination of known broken packages to be
> installed.
> Currently autopkgtest blocks the migration of numpy, but if there wasn't
> that block a Breaks should still be added, as the astropy in testing is
> not compatible with the new numpy.  The fact that it hints britney to
> trigger the right tests is just a "side effects", the Breaks should be
> added nonetheless, theoretically; in practice, we rarely did it before
> autopkgtest started blocking the testing migration.

I agree (with the explanation in your other mail) here; it just states
the fact that they really don't go together very well (at least in
situation similar to the specific CI test).

>> Python-astropy is however going to be removed completely; it has
>> however some cruft rdeps left in unstable. So, it cannot removed from
>> unstable now, and therefore still remains in testing and
>> (unnecessarily) blocks the numpy migration.  Python-astropy already
>> has an RC bug, but its autoremoval from testing is still not even
>> announced yet.
>
> Maybe you could ask the release team to hasten the removal of
> python-astropy and rdeps from testing?  If the plan is to not relase
> it in buster I don't see a reason to keep it further.

Couldn't we just also add a "breaks" to numpy? The important fact here
is, that the new numpy and the current python-astropy don't work
together; and this is independent of whether a fixed python-astropy
version exists.

This would remove one dependent party (release team) from the chain of
blocking causes for the migration.

>> The migration blocking CI tests seem to cause much more headaches than
>> just "fix your bugs"...
>
> Well, from what I see, it has helped a lot in this half a year detect a
> ton of regressions that would have otherwise bothered several people in
> a later moment…

I usually regularly look into my packages and file bugs if a CI test
fails caused by a buggy updated dependency. This works well without the
need of blocks. It would also work, when a failing CI test would
automatically trigger a bug report against the updated package, which
then could be re-assigned to the rdep in case the problem was there.

So, I value very much the CI tests by themself: they are the greatest
invention since sliced bread, but I still dislike the inflexible
handling of blocks. Often, a failing CI test makes a package only buggy
with respect to a certain feature, so it would correspond to a
"important" or "normal" bug severity. They are however always handled
like a "serious" bug by preventing migration. I miss the "That was not a
serious failure" button.

Thanks fur your help!

Best

Ole

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:
> The way forward in cases like these is for the package that originally
> cuased the breakage (i.e. numpy) to declare a versioned Breaks on the
> borken and now fixed package (i.e. astropy (<< 3.1-1)). This way
> britney and debci will know they have to test numpy and astropy
> together, and will be able to correctly migrate to testing at the same
> time, and properly avoid a situation when two incompatible packages
> are installed.
> Maybe you could open a bug on numpy to get the maintainer to add the breaks.

There is one more problem, which are transitional dependencies:

The new python3-numpy version breaks (f.e.) python3-pyregion because of
the problem in python3-astropy. The new upload of python3-astropy fixes
this, so in unstable everything is OK. Even when one now adds a

Breaks: python3-astropy (<< fixed-version)

to python-numpy, it will not migrate due to the test failure of
python3-pyregion in testing. Adding a "Breaks: python3-pyregion" is
however wrong, since pyregion is perfect for the new numpy version.

So, how to handle this?

Cheers

Ole

Re: Numpy migration?

[Ole Streicher]

Mattia Rizzolo  writes:
> The way forward in cases like these is for the package that originally
> cuased the breakage (i.e. numpy) to declare a versioned Breaks on the
> borken and now fixed package (i.e. astropy (<< 3.1-1)).  This way
> britney and debci will know they have to test numpy and astropy
> together, and will be able to correctly migrate to testing at the same
> time, and properly avoid a situation when two incompatible packages
> are installed.
> Maybe you could open a bug on numpy to get the maintainer to add the
> breaks.

I'll do tonight. It however looks a bit suboptimal: when the CI test
with a new version fails for an old reverse dependency, then the new
version obviously breaks that old package. So, the breakage could be
detected (and taken into account) automatically. Why do we need a manual
action then?

>> Another CI problem is python-astropy, which is the Python 2 version of
>> Astropy. python-astropy is going to be removed as soon as there are no
>> backward dependencies left; however there is still some cruft in
>> unstable that depends on python-astropy. But this should not hinder
>> numpy to migrate.
>
> I don't understand this, I don't see any python2-related issue right
> now.  Could you please expand?

The new python-numpy breaks python-astropy in testing. Python-astropy is
however going to be removed completely; it has however some cruft rdeps
left in unstable. So, it cannot removed from unstable now, and therefore
still remains in testing and (unnecessarily) blocks the numpy migration.
Python-astropy already has an RC bug, but its autoremoval from testing
is still not even announced yet.

The migration blocking CI tests seem to cause much more headaches than
just "fix your bugs"...

Best

Ole

Numpy migration?

[Ole Streicher]

Hi all,

Since a few days, a new RC is stalled in unstable:

 testing: 1:1.15.4-2
unstable: 1:1.16.0~rc1-3

The reason that it can't migrate is that it breaks the CI tests of
several packages in testing. For example astropy (where I am the
uploader), and its dependencies. The CI test failures were caused by
astropy (#917466) and are fixed since a week in unstable.

However, astropy cannot migrate now, to testing, since it depends on the
new numpy version (and therefore can only migrate after numpy). And
numpy is blocked by the CI failure of astropy ...

Looks like a deadlock. Which will be resolved before the migration delay
ends. Which is in the second half of february, and therefore will cause
all packages that depend on numpy and are not in testing yet to be kept
outside of Buster (due to the release timeline).

What is the way to resolve those problems? These deadlocks probably
appear regularly when a package gets a major update: package A breaks
the CI test of the rdep B in testing, and a rebuild of B in unstable
depends on the newer version of A.

Another CI problem is python-astropy, which is the Python 2 version of
Astropy. python-astropy is going to be removed as soon as there are no
backward dependencies left; however there is still some cruft in
unstable that depends on python-astropy. But this should not hinder
numpy to migrate.

How to solve this?

Best

Ole

Bug#915786: ITP: sphinx-automodapi -- Sphinx extension for generating API documentation

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher 
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name : sphinx-automodapi
  Version  : 0.9
  Upstream Author  : Thomas Robitaillie
* URL  : https://astropy.org/sphinx-automodapi
* License  : BSD 3-Clause
  Programming lang : Python
  Upstream git : https://github.com/astropy/sphinx-automodapi
  Pypi URL : https://pypi.python.org/pypi/sphinx-automodapi
  Description  : Sphinx extensions for generating API documentation

The sphinx-automodapi package provides Sphinx directives that help
facilitate the automatic generation of API documentation pages for
Python package modules. It was originally developed for the Astropy
project, but is now developed as a standalone package that can be used
for any project.

I will maintain it within the Debian Python team.

Best regards

Ole

Re: [Python-modules-team] Bug#914592: Upcoming pytest-4 problems

[Ole Streicher]

On 27.11.18 09:35, Ondrej Novy wrote:
> út 27. 11. 2018 v 8:46 odesílatel Ole Streicher  <mailto:oleb...@debian.org>> napsal:
> 
> Thanks, however it turns out that this will unfortunately not work for
> me: from astropys POV, pytest seems to be broken in versions 3.7 - 3.10;
> see
> 
> https://github.com/astropy/astropy/issues/8177
> 
> 
> according to linked issue, that bug is fixed in 3.10.1, which is in
> Debian unstable now.

Thanks for updating, but I still get failures. And I asked pytest
upstream about possible fixes of the 3.X branch, which they are not
going to do:

https://github.com/astropy/astropy/pull/8173#issuecomment-441959185

> I also observed diverse failures on other (astropy dependent) packages
> with 3.10. Which basically means that I need to switch off the tests,
> unless pytests upstream provides a bugfixed 3.X version.
> 
> maybe just disable some of that tests? Not whole testsuite?

The current failure in python-astropy is

E   UsageError: Plugin specs must be a ','-separated string or a
list/tuple of strings for plugin names. Given: 

which blocks the whole suite. Other packages have errors that really
look strange (data files are missing, submodules not found etc). Sure,
that all could be workarounded somehow, but in a few weeks the next
update of a dependency may make trouble again.

We will publish Debian Buster not earlier than summer/fall next year,
and this is just ~6-9 months before Python 2 EOL. I don't want to put
too many efforts in keeping this alive.

Best

Ole

Re: [Python-modules-team] Bug#914592: Upcoming pytest-4 problems

[Ole Streicher]

Hi Ondrej,

On 26.11.18 21:29, Ondrej Novy wrote:
> my plan is:
> 
>   * keep 3.x for Buster
>   * upgrade to >= 4.x just after Buster release
> 
> Because we are going to drop Python after Buster release, I think it's
> perfectly fine that Astropy (Py2 version) will not support Pytest 4.x.

Thanks, however it turns out that this will unfortunately not work for
me: from astropys POV, pytest seems to be broken in versions 3.7 - 3.10;
see

https://github.com/astropy/astropy/issues/8177

I also observed diverse failures on other (astropy dependent) packages
with 3.10. Which basically means that I need to switch off the tests,
unless pytests upstream provides a bugfixed 3.X version.

However, it seems that I need to start with the removal of the Python 2
astropy ecosystem, which may be in any case not the worst decision.
Personally, I like to put pressure to move towards Python 3.

Cheers

Ole

Upcoming pytest-4 problems

[Ole Streicher]

Hi all,

since the EOL of Python 2 is approaching, the (python-) "astropy"
upstream has split its development into a Python 2.7 legacy branch
(2.0.X) and the normal development (3.X), where the legacy branch does
get only maintenance now. Since many Python 2 packages still depend on
astropy, I split the upstream source into a 2.X legacy package
(src:python-astropy) and a 3.X standard package (src:astropy).

Yesterday, upstream decided that the legacy package will not support
pytest 4:

https://github.com/astropy/astropy/pull/8173

Pytest 4 is however the upcoming version, and I could imagine that
others may have similar problems with Python 2 legacy. Therefore, my
question is how these problems should be solved:

* creating a legacy version package for pytest

* remove the tests for these packages

* remove the legacy packages in question completely

For us, the third option would basically mean to remove the whole Python
2 astronomy ecosystem, and we would need to hurry in making some general
packages Python 3 compatible (mainly astrometry.net).

I opened a bug on pytest to discuss this:

https://bugs.debian.org/914592

Best regards

Ole

dh_python3-ply lintian warning

[Ole Streicher]

Hi,

when I use dh_python3-ply in d/rules like this:

--8<---
override_dh_python3:
dh_python3-ply $$(find $(PYBUILD_DESTDIR_python3) -name '*tab.py')
dh_python3
--8<---

I get the following lintian warnings about the generated files:

W: python3-astropy: python-module-in-wrong-location 
usr/lib/python3.7/dist-packages/astropy/ usr/lib/python3/dist-packages/astropy/
W: python3-astropy: python-module-in-wrong-location 
usr/lib/python3.7/dist-packages/astropy/coordinates/ 
usr/lib/python3/dist-packages/astropy/coordinates/
W: python3-astropy: python-module-in-wrong-location 
usr/lib/python3.7/dist-packages/astropy/coordinates/angle_lextab.py 
usr/lib/python3/dist-packages/astropy/coordinates/angle_lextab.py

How can I fix that?

Best regards

Ole

Re: Matplotlib 3.0 - update ok?

[Ole Streicher]

ghisv...@gmail.com writes:
> Indeed. Note that NumPy has already published plans to become Python 3
> only in the near future, so the deprecation of Python 2 in the
> scientific stack will happen eventually.
>
> I just don't think it should be rushed into the Buster release cycle.

If we really want to then have a Python-2-numpy, why can't there be a
separate Python-2 legacy numpy source package? I do the same for
astropy.

Holding back normal updates especially for science packages just because
we don't want to use a modern numpy/scipy/matplotlib stack is not really
friendly to our users, which (in science) rely on at least "somehow"
modern software. And astropy that is more than a 1.5 years old already
at the release of Buster would not be accepted by the users, and
patching it to use an older sw stack is also impractical.

Even when we /still/ support Python 2, our focus and preference should
be clearly Python 3.

Best regards

Ole

Re: Problem with pybuild and pytest (repost)

[Ole Streicher]

Piotr Ożarowski  writes:
>> >From the pybuild wiki page, I learned to do instead the override:
>> 
>> export PYBUILD_TEST_ARGS=-k "not figure and not online"
>> 
>> However, then the tests are not found anymore and the test step fails:
>
> 1) remove HOME line from debian/rules, pybuild sets one
> 2) you have ".." in PYBUILD_TEST_ARGS (in the git repo) - remove it
> 3) for python2.7 (and 2.7 only, 3.X works fine), cmdclassd somehow
>disables distutils' settings from .pydistutils.cfg (probably
>overwrites too much or doesn't invoke parent's methods).
>It works for 3.X because "from sunpy.tests.setup_command import SunPyTest"
>fails on 3.X and thus SunPyTest is not used at all.
>Disabling SunPyTest fixes it for me

Thanks. Just removing the cmdclass option did the trick, and it works
now.

Best

Ole

Problem with pybuild and pytest (repost)

[Ole Streicher]

Hi,

I already posted this two weeks ago, but didn't get a response yet, and
thus it remains unsolved:

For one of my packages (sunpy), I want to complete the switch to pybuild
by removing the custom (and quite unperfect) dh_override_auto_test rule.

Currently, I have the following lines in d/rules:

override_dh_auto_test:
python -m pytest sunpy -k "not figure and not online"
python3 -m pytest sunpy -k "not figure and not online"

This gives successfull tests with the following output in the log:


--8<---
   debian/rules override_dh_auto_test
make[1]: Entering directory '/<>'
pyversions: missing X(S)-Python-Version in control file, fall back to 
debian/pyversions
pyversions: missing debian/pyversions file, fall back to supported versions
pyversions: missing X(S)-Python-Version in control file, fall back to 
debian/pyversions
pyversions: missing debian/pyversions file, fall back to supported versions
python -m pytest sunpy -k "not figure and not online"
= test session starts ==
platform linux2 -- Python 2.7.15, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
rootdir: /<>, inifile: setup.cfg
plugins: mock-1.7.1, hypothesis-3.44.1
collected 1448 items / 34 deselected
[...]
--8<---


>From the pybuild wiki page, I learned to do instead the override:

export PYBUILD_TEST_ARGS=-k "not figure and not online"

However, then the tests are not found anymore and the test step fails:


--8<---
   dh_auto_test -O--buildsystem=pybuild
I: pybuild base:217: cd /build/sunpy-0.9.3/.pybuild/cpython2_2.7_sunpy/build; 
python2.7 -m pytest -k "not figure and not online"
= test session starts ==
platform linux2 -- Python 2.7.15+, pytest-3.6.4, py-1.6.0, pluggy-0.6.0
rootdir: /build/sunpy-0.9.3, inifile: setup.cfg
plugins: mock-1.7.1, hypothesis-3.44.1
collected 0 items

= no tests ran in 0.00 seconds =
E: pybuild pybuild:338: test: plugin distutils failed with: exit code=5: cd 
/build/sunpy-0.9.3/.pybuild/cpython2_2.7_sunpy/build; python2.7 -m pytest -k 
"not figure and not online"
dh_auto_test: pybuild --test --test-pytest -i python{version} -p 2.7 returned 
exit code 13
--8<---


I don't understand why the tests fail now? How do I correctly specify
them?

Best regards

Ole

Re: Upstreams dropping Python 2 support

[Ole Streicher]

Dmitry Shachnev  writes:
> Sphinx 2.0 will also drop Python 2.7 support, so I am going to make the
> “sphinx” source package ship only python3-sphinx, and introduce the new
> source package (“sphinx-python2”?) which will ship only python-sphinx.
>
> Eventually I will also do a mass bug filing, but with the current 314
> python-sphinx reverse build-depends it is clear that I will have to ship
> sphinx-python2 for quite some time.

Is there a reason why one would use Python2-sphinx instead of the Python
3 version? How often in these 314 cases it is not just used as an
executable in d/rules?

Cheers

Ole

PLY problem

[Ole Streicher]

Hi,

I have (still) problems with PLY generated files in the astropy package:
they are just not generated, and I have no idea how to generate them
correctly. For example, there are the files

astropy/coordinates/angle_lextab.py
astropy/coordinates/angle_parsetab.py

which are now cleaned up in d/clean, and they are somehow cached
regenerated from

astropy/coordinates/angle_utilities.py

The latter file contains the hint

# This module includes files automatically generated from ply (these end in
# _lextab.py and _parsetab.py). To generate these files, remove them from this
# folder, then build astropy and run the tests in-place:
#
#   python setup.py build_ext --inplace
#   pytest astropy/coordinates

but this seems not to fit very well into the pybuild procedure. And, I
have a few more of them (in astropy/units/format).

What would be the best way to regenerate them?

Best regards

Ole

Extra debian.blends Python package

[Ole Streicher]

Hi,

I am currently rewriting part of the blends-dev package which is used
for maintaining Debian Pure Blends into Python. This however comes out
some abstractions of Blends (namely the tasks infrastructure) that may
be re-usable on other places, f.e. in the blends.d.o web service. So it
is probably a good idea to create a separate Python package from it.

Since this package deals with tasks sources, it sounds natural to put
this as a sub-package of the "debian" package. I would however not like
to integrate the sources to python-debian. First, it is very new and
likely to change API quickly in the next time. But also, the blends
subpackage is quite special; there is no need to have it installed in
such a broad manner as python-debian.

And finally, I do not care about Python 2 compatibility here.

What I could do easily is to create a python3-debian-blends binary
package (maintained in the "blends" source package) for this subpackage,
just installs a "blends.py" into ../dist-packages/debian/

Since this is however somehow hijacking (part of) the namespace of
python-debian, I'd like to know the policy here.

Are there any problems with that approach?

Best regards

Ole

apt.Cache.update with alternative sources.list

[Ole Streicher]

Hi,

I need some access (as normal user) to an apt cache with an alternative
sources.list (those in /etc/blends/ installed by blends-dev), but I have
problems to find out how to use it.

I first tried to just specify the alternative source file:

```
>>> import apt
>>> c = apt.Cache()
>>> c.update('sources_list='/etc/blends/sources.list.testing')
LockFailedException: Failed to lock /var/lib/apt/lists/lock
```

Then I tried to use an alternative root directory:

```
>>> import apt
>>> c = apt.Cache(rootdir='/tmp/myapttmp')
>>> c.update('sources_list='/etc/blends/sources.list.testing')
FetchFailedException: W:GPG error: http://ftp.debian.org/debian testing 
InRelease: The following signatures couldn't be verified because the public key 
is not available: NO_PUBKEY 7638D0442B90D010, E:The repository 
'http://ftp.debian.org/debian testing InRelease' is not signed.
```

To which place in the new root directory do I need to copy the keyring?
I tried <>/usr/share/keyrings/ but this didn't work.

And/or how can I disable authentication (--allow-unauthenticated,
resp. APT::Get::AllowUnauthenticated)?

Version of python3_apt is 1.4.0~beta3+b1

Best regards

Ole

Re: pybuild: how to get the build directory name

[Ole Streicher]

Dear Piotr,

Piotr Ożarowski  writes:
> or even better (without using pybuild's internal paths):
>
>   override_dh_auto_test:
>   dh_auto_test -- --system=custom --test-args '{interpreter} 
> {dir}/debian/tests/pyraf-test.py'

That is optimal for me. Thank you very much for the hint!

Best regards

Ole

pybuild: how to get the build directory name

[Ole Streicher]

Hi,

one of my Python packages has a test script under debian/tests that I
want to run at build time. The problem is now how to get the name of the
actual build directory, which is needed to be put into the PYTHONPATH:

Up to now, I did the following in debian/rules:

test-python%:
HOME=/tmp/ PYTHONPATH=$(CURDIR)/.pybuild/pythonX.Y_$*/build python$* 
debian/tests/pyraf-test.py

override_dh_auto_test: $(PYTHON2:%=test-python%) $(PYTHON3:%=test-python%)

This however does not work anymore, since the subdir under .pybuild is
now changed to cpython2_2.7_pyraf resp. cpython3_3.6_pyraf.

To be backward compatible, I would however not like to just follow this
change. How can I get the correct build dir here?

Best regards

Ole

Re: Where to put docs of a -doc package for python 2 + 3 modules?

[Ole Streicher]

Thomas Goirand  writes:
> On 03/13/2018 12:29 AM, Ghislain Vaillant wrote:
>> Imo, we should just make it clear in policy that source packages
>> should be named `foo` or `python-foo`,
>> and corresponding doc packages should be named `foo-doc` or `python-foo-doc`.
>
> Very often, "foo" is already taken by another package, and we have to
> fallback to python-foo. Think about generic libs for compression,
> images, network standards...
>
> Which is why I think we should have standardize on python-foo for the
> source package (which is what I do).

I disagree. Often packages already signalize their Python origin:
astropy, sunpy, pyephem etc. Or relate to pytest (pytest-mpl for
example). Or provide a Python package just as one of many other packages
(plplot). Prefixing them with "python-" increases useless redundancy,
and makes it harder for external people to find the package.

I would deviate from the rule of thumb "stick with the upstream name
unless there is a (potential) conflict". And if there is a conflict,
talk to upstream to find a solution -- it is quite probable that it will
lead to file conflicts as well (tools/apps in /usr/bin).

Best

Ole

Re: Move to salsa? Merge modules and apps team?

[Ole Streicher]

Piotr Ożarowski  writes:
> [W. Martin Borgert, 2018-02-07]
>> And how about merging the modules and apps teams into one?
>
> I don't understand this, though. Why you want to merge them?
> Sure, packaging Python applications is very similar to packaging
> libraries but the difference is very significant - unfortunately many
> application maintainers don't know or care about it and pollute global
> Python namespace with private libraries.

So, why would you like to keep them separate?

> If not separated at team level, I definitely want to have them somehow
> separated at repository level so that it's clear which package is which
> type or to easily checkout libraries only.

At least in the field where I package (astronomy), this does not work:
many of our packages are libraries *and* apps. Often they come as a big
(public) library with a small startup wrapper, and it is a matter of
taste (and of how deep an astronomer digs into Python) whether it is
used from the shell or from a Jupyter notebook.

The rules for both are (should be) the same, right? So, what is the
difference? 

Best regards

Ole

Bug#889054: ITP: pytest-astropy -- Metapackage to resolve pytest dependencies for Astropy

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name: pytest-astropy
  Version : 0.2.1
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astropy/pytest-astropy
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Pytest dependencies for Astropy & Co.

This is a meta-package that pulls in the dependencies that are used by
`astropy` and some `affiliated packages` for testing. It can also be
used for testing packages that are not affiliated with the Astropy
project. It is a new build dependency of astropy 3.0.

I will maintain it within the Debian Astro team.

Best regards

Ole

Bug#887024: ITP: pytest-arraydiff -- Pytest plugin that enables the generation and comparison of data arrays

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name: pytest-arraydiff
  Version : 0.1
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astrofrog/pytest-arraydiff
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Pytest plugin that enables the generation and comparison of 
data arrays

This is a py.test plugin to facilitate the generation and comparison
of data arrays produced during tests. The basic idea is that you can
write a test that generates a Numpy array (or other related objects
depending on the format). You can then either run the tests in a mode
to generate reference files from the arrays, or you can run the tests
in comparison mode, which will compare the results of the tests to the
reference ones within some tolerance.

It is a new build dependency of astropy 3.0.

I will maintain it within the Debian Python team.

Best regards

Ole

Bug#887023: ITP: pytest-openfiles -- Pytest plugin to check for files left open at the end of a test run

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name: pytest-openfiles
  Version : 0.2.0
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astropy/pytest-openfiles
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Pytest plugin to check for files left open at the end of a 
test run

This package provides a plugin for the pytest framework that allows
developers to detect whether any file handles or other file-like
objects were inadvertently left open at the end of a unit test. It is
a new build dependency of astropy 3.0.

I will maintain it within the Debian Python team.

Best regards

Ole

Bug#887021: ITP: pytest-doctestplus -- Pytest plugin providing advanced doctest features

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name: pytest-doctestplus
  Version : 0.1.2
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astropy/pytest-doctestplus
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Pytest plugin providing advanced doctest features

This package contains a plugin for the pytest framework that provides
advanced doctest support and enables the testing of reStructuredText
(".rst") files. It is a new build dependency of astropy 3.0.

I will maintain it within the Debian Python team.

Best regards

Ole

Bug#887020: ITP: pytest-remotedata -- Pytest plugin to control access to online data files

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-python@lists.debian.org, debian-de...@lists.debian.org

* Package name: pytest-remotedata
  Version : 0.1
  Upstream Author : Thomas Robitaille
* URL : https://github.com/astropy/pytest-remotedata
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Pytest plugin to control access to online data files

This package provides a plugin for the pytest framework that allows
developers to control unit tests that require access to data from the
internet. It is a new build dependency of astropy 3.0.

I will maintain it within the Debian Python team.

Best regards

Ole

Python 2 removal strategy

[Ole Streicher]

Hi,

I am the maintainer of the "python-astropy" package, that currently
creates packages for both Python 2 and Python 3. Both packages have a
number of reverse dependencies.

Recently, upstream announced a new version 3.0 of astropy, which
supports Python 3 only, and I think of the best mid-term strategy: The
old version 2.0 is supported upstream for ~2 years, and I want to have a
smooth migration path. I checked the wiki, but could not find good
information about migration.

I thought of a temporary package split: create a new source package
"astropy" that inherits of the current python-astropy package, but only
builds python3-astropy (and the utils + doc, which depend on
python3-astropy), and update this to version 3.0. Then I would remove
these binary packages from the python-astropy package. In parallel, I
would file bugs (severity: important) to remove the reverse dependencies
of the Python 2 packages (many of them are mine, but also may have
reverse dependencies).

As long as there are substantial problems with the removal of the Python
2 support, I then keep the "old" python-astropy package updated. Once
everything is figured out and we decide to finally kick out Python 2
support (from Debian-Astro, or from Debian), I would set the remaining
bugs as RC, and (once they are solved) remove the "python-astropy"
package.

Does this sound reasonable? And how should I do this technically?

Best regards

Ole

Packaging with 2to3

[Ole Streicher]

Hi,

I want to create a package (pyraf, #711038) for both Python2 and
Python3. While the package itself is still not Python 3 compatible,
upstream says that running through 2to3 will make it; however the
resulting code is not Python 2 compatible anymore.

Is there a way to instruct pybuild to run the Python 3 code through
2to3?

Best regards

Ole

Bug#884598: ITP: stsci.tools -- Miscellaneous Python tools for astrophysics from STScI

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-as...@lists.debian.org,debian-python@lists.debian.org
Control: block 711038 by -1

* Package name: stsci.tools
  Version : 3.4.3
  Upstream Author : Space Telescope Science Institute
* URL : 
http://www.stsci.edu/resources/software_hardware/stsci_python
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Miscellaneous Python tools for astrophysics from STScI

This package provides a selection of tools for FITS and WCS handling
image handling, and data analysis, used by other STScI packages.

The package is a dependency of the pyraf package. It will be maintained
by the Debian Astro team. A git repository will be created at

https://anonscm.debian.org/git/debian-astro/packages/stsci.tools.git

Best regards

Ole

Bug#879683: ITP: radio-beam -- Tools for radio beam io and manipulation

[Ole Streicher]

Package: wnpp
Severity: wishlist
Owner: Ole Streicher <oleb...@debian.org>
X-Debbugs-Cc: debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: radio-beam
  Version : 0.1
  Upstream Author : Eric Koch, Adam Ginsburg et al.
* URL : https://github.com/radio-astro-tools/radio-beam
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Tools for Radio Beam IO and Manipulation

Radio Beam is a simple toolkit for reading beam information from FITS
headers and manipulating beams. Some example applications include
Convolution/deconvolution and Unit conversion (Jy to/from K).

The package is a new dependency of the spectral-tools package. It will
be maintained by the Debian Astro team. A git repository will be created
at

https://anonscm.debian.org/git/debian-astro/packages/radio-beam.git

Best regards

Ole

Re: MBF for deprecating Python2 usage

[Ole Streicher]

Diane Trout  writes:
>> What I am opposing is the suggestion to install, in the near to
>> medium
>> term, a command of exactly the same name that has subtly similar but
>> incompatible behaviour, when that behaviour *already* has a command –
>> ‘python3’ – that is widely used by those who need it.
>> 
>
> my problem with that plan is all of the printed documentation saying to
> learn python, type "python".

All *actual* printed documentation? Which refers to Python 3? Could you
give an example here?

Otherwise, (if using a Python 2 book), one of the next steps would be
"print 'hello world'", and nothing would prevent from the horrible
experience that the book is outdated anyway.

Sorry, I don't see the problem with "printed documentation".

Best regards

Ole

Re: MBF for deprecating Python2 usage

[Ole Streicher]

ba...@debian.org writes:
> On Aug 3, 2017, at 23:23, Scott Kitterman  wrote:
> You’re right that folks will still need Python 2 after 2020, and that
> their best option may be to build it themselves, but I don’t think
> they’ll be building and installing some old Debian package.  Instead
> they may build from source, which means they’ll be installing it into
> /usr/local/bin not /usr/bin, and they’ll have to change their shebang
> lines anyway.

It is very usual to use "#!/usr/bin/env python" as shebang, exactly for
the case that python is not installed in /usr/bin.

>> Such a change would be actively user hostile.
>> 
>> When python2.7 goes away, /usr/bin/python should go too.
>
> Maybe in the short term (as Matthias suggests), but I’m not so sure
> it’s best to disappear /usr/bin/python forever.  /usr/bin/python is
> not just the common shebang (and we should be actively transitioning
> to /usr/bin/python2 for that), but it’s also the command line UI for
> people wanting to learn Python, or just wanting to get an interactive
> interpreter to play with.

That is -- at least in my environmet -- "ipython", or (for Python 3)
"ipython3".

> There will be a day when, if they get a command not found when typing
> `python` at a shell prompt, they will wonder why Debian is broken.

IMO it would be better to communicate that the best way for an
interactive session is "ipython3" (or "python3", if you insist). I would
wonder when todays tutorials (that cover Python 3) recommend to use
plain "python".

> I think the transition should roughly be:
>
> * Update policy and tooling so that any scripts that require Python 2
> use /usr/bin/python2 in their shebang.
>
> * Educate our users that they should be using `python2` for anything
> that has not been ported.
>
> * Port as much as possible to Python 3 (eventually, everything
> maintained in Debian) and /usr/bin/python3 in their shebang.

I disagree here, and I don't see an advantage over letting
/usr/bin/python just die with Python 2.

I would also like to point PEP 394, which has a number of arguments why
/usr/bin/python should remain as Python 2.

https://www.python.org/dev/peps/pep-0394/

Best regards

Ole

How to package for jython?

[Ole Streicher]

Hi,

I have a large Java package (as a jar in /usr/share/java) that is
accompanied with some Jython intergration (Python file). Upstream provides
a small script that adds the jar to the classpath, and starts jython
with a local python.path.

How should I package this for Jython?

 * where should the .py go? I assume /usr/share/jython/Lib/ ?

 * how should I set the classpath that the jar is found?

If you are curious, it is this:

http://www.star.bristol.ac.uk/~mbt/stilts/sun256/jystilts.html

Best regards

Ole

Re: PyPI source or github source?

[Ole Streicher]

Brian May  writes:
> Should we be using PyPI as our source of packages? Or github?

I already had packages that use a helper to generate the version
information from the git repository, and therefore need the git metadata
to build from source. I could solve this with a "fake" helper that
uses debian/changelog instead, but this looks more like a hack than a
clean solution. Unfortunately, they did't provide pypi source; otherwise
I would have used them.

Use case: 

https://github.com/spacetelescope/pysynphot

The originally "relic" package which extracts the info from git:

https://github.com/jhunkeler/relic

The "fake" relic package is in a Debian patch:

https://sources.debian.net/src/pysynphot/0.9.8.5%2Bdfsg-1/debian/patches/Don-t-use-relic-package-for-version-info.patch

Best

Ole

Bug#848112: Python-skimage depends on unavailable package python-dask

[Ole Streicher]

package: python-skimage
version: 0.12.3-2
severity: serious

The Python 2 version of skimage depends on a package "python-dask" that
is not available in Debian.

There is a patch that make the dependency optional; however the
dependency was not removed afterwards. For Python 3, this seems to work.

Since skimage is one of the central packages, I would again ask to put
it under science|python team maintenance. Especially when under some
time pressure (upcoming freeze, combined with autoremovals of packages)
it would help a lot if the problems could be debugged within a standard
Debian developer workflow, without the need to switch to github or so.

Best regards

Ole

Bug#823598: ITP: ginga -- Astronomical FITS file viewer

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: ginga
  Version : 2.5.20160420043855-1
  Upstream Author : Eric R. Jeschke
* URL : https://ejeschke.github.com/ginga
* License : BSD-3-Clause
  Programming Lang: Python
  Description : The Ginga astronomical FITS file viewer
 Ginga is a toolkit designed for building viewers for scientific image
 data in Python, visualizing 2D pixel data in numpy arrays.
 It can view astronomical data such as contained in files based on the
 FITS (Flexible Image Transport System) file format.  It is written and
 is maintained by software engineers at the Subaru Telescope, National
 Astronomical Observatory of Japan.
 .
 The Ginga toolkit centers around an image display object which supports
 zooming and panning, color and intensity mapping, a choice of several
 automatic cut levels algorithms and canvases for plotting scalable
 geometric forms.  In addition to this widget, a general purpose
 "reference" FITS viewer is provided, based on a plugin framework.
 A fairly complete set of standard plugins are provided for features
 that is expected from a modern FITS viewer: panning and zooming windows,
 star catalog access, cuts, star pick/fwhm, thumbnails, etc.

It will maintained within the Debian Astronomy Working Group. A git
repository will be created on alioth [2].

Best regards

Ole

[1] http://www.astropy.org/affiliated/index.html
[2] https://anonscm.debian.org/cgit/debian-astro/packages/ginga.git

Bug #820811: ITP: python-fitsio -- Python wrapper on the CFITSIO library

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: python-fitsio
  Version : 0.9.8
  Upstream Author : Eric Sheldon
* URL : https://github.com/esheldon/fitsio
* License : GPL-2+
  Programming Lang: Python
  Description :  Python library to read from and write to FITS files
 Fitsio is a Python wrapper on the CFITSIO library. It allows direct
 access to the columns of a FITS binary table which can be useful for
 reading large fits files.

This package will maintained within the Debian Astronomy Working Group.
A git repository will be created on alioth:

https://anonscm.debian.org/cgit/debian-astro/packages/python-fitsio.git

Best regards

Ole

Bug#813896: ITP: asdf -- Python tools to handle ASDF files

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: asdf
  Version : 1.0.1
  Upstream Author : Erik Bray, Michael Droettboom
* URL : https://github.com/spacetelescope/pyasdf
* License : BSD
  Programming Lang: Python
  Description : Python tools to handle ASDF files
 ASDF (Advanced Scientific Data Format) is a proposed 
 next generation interchange format for scientific data.

This package will maintained within the Debian Astronomy Working Group.
A git repository is created on alioth [1].

Best regards

Ole

[1] https://anonscm.debian.org/cgit/debian-astro/packages/asdf.git

Bug#813733: ITP: pysynphot -- Python Synthetic Photometry Utilities

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: pysynphot
  Version : 0.9.7
  Upstream Author : Vicki Laidler et al.
* URL : http://ssb.stsci.edu/pysynphot/docs/
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Python Synthetic Photometry Utilities
 pysynphot simulates photometric data and spectra as they are observed
 with the Hubble Space Telescope (HST). Passbands for standard
 photometric systems are available, and users can incorporate their
 own filters, spectra, and data.

This package will maintained within the Debian Astronomy Working Group.
A git repository is created on alioth [1].

Best regards

Ole

[1] https://anonscm.debian.org/cgit/debian-astro/packages/pysynphot.git

Bug#813147: ITP: emcee -- Kick ass affine-invariant ensemble MCMC sampling for Python

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: emcee
  Version : 2.1.0
  Upstream Author : Dan Foreman-Mackey <https://github.com/dfm>
* URL : http://dan.iel.fm/emcee/
* License : MIT
  Programming Lang: Python
  Description :  Kick ass affine-invariant ensemble MCMC sampling for Python
 emcee is an extensible, pure-Python implementation of Goodman &
 Weare's Affine Invariant Markov chain Monte Carlo (MCMC) Ensemble
 sampler. It's designed for Bayesian parameter estimation.

This package will maintained within the Debian Astronomy Working Group.
A git repository will be created on alioth [1].

Best regards

Ole

[1] https://anonscm.debian.org/cgit/debian-astro/packages/emcee.git

Request to join DPMT

[Ole Streicher]

Hello all,

I would like to join the DPMT. I already have packages some Python
packages (python-astropy and many of its affiliated packages,
python-pywcs, python-astroml) under the hood of the Debian-Astro team.
In future I probably need to package dependencies like Bottleneck that
are not specific to astronomy.

My Alioth account is: olebole

I have read and accept the Debian Python Modules Team's Policy.

I already applied directly in alioth, but didn't get a response.

Best regards

Ole

Re: Request to join DPMT

[Ole Streicher]

Scott Kitterman <deb...@kitterman.com> writes:
> On Wednesday, January 06, 2016 04:11:18 PM Ole Streicher wrote:
>> I already applied directly in alioth, but didn't get a response.
> According to Alioth, you're in the team already.

Oh, I see. I didn't check it again before I processed the point "send
reminder" of my own todo-list :-) Sorry, and thanks.

Cheers

Ole

Bug#807739: ITP: sunpy -- Software library for solar physics based on Python

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: sunpy
  Version : 0.6.1
  Upstream Author : Sunpyy Developers
* URL : https://sunpy.org
* License : BSD-2-Clause
  Programming Lang: Python
  Description : Software library for solar physics based on Python
 SunPy is a community-developed free and open-source software package
 for solar physics. SunPy is meant to be a free alternative to the
 SolarSoft data analysis environment which is based on the IDL
 scientific programming language sold by Exelis. Though SolarSoft is
 open-source IDL is not and can be prohibitively expensive. The aim
 of the SunPy project is to provide the software tools necessary so
 that anyone can analyze solar data.

This package will maintained within the Debian Astronomy Working Group.
A git repository is created on alioth [1].

Best regards

Ole

[1] https://anonscm.debian.org/cgit/debian-astro/packages/sunpy.git

Bug#807778: ITP: pyephem -- Compute positions of the planets and stars

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: pyephem
  Version : 3.7.6.0
  Upstream Author : Brandon Rhodes
* URL : http://rhodesmill.org/pyephem/
* License : LGPL-3
  Programming Lang: Python
  Description : Compute positions of the planets and stars
 PyEphem provides an ephem Python package for performing high-precision
 astronomy computations. The underlying numeric routines are coded in C
 and are the same ones that drive the popular XEphem astronomy
 application, whose author, Elwood Charles Downey, generously gave
 permission for their use in PyEphem. The name ephem is short for the
 word ephemeris, which is the traditional term for a table giving the
 position of a planet, asteroid, or comet for a series of dates.

This package will maintained within the Debian Astronomy Working Group.
A git repository is created on alioth [1].

Best regards

Ole

[1] https://anonscm.debian.org/cgit/debian-astro/packages/pyephem.git

Bug#807309: ITP: montage-wrapper -- Python wrapper for the Montage mosaicking toolkit

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: montage-wrapper
  Version : 0.9.8
  Upstream Author :  Thomas Robitaille
* URL : hhttp://astropy.org/montage-wrapper/
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Python wrapper for the Montage mosaicking toolkit
 The montage-wrapper module (formerly python-montage) is an Astropy
 affiliated package that provides a way to run the Montage Astronomical
 Image Mosaic Engine from Python, including both functions to access
 individual Montage commands, and high-level functions to facilitate
 mosaicking and reprojecting.

This is an Astropy Affiliated Package [1]. It will maintained
within the Debian Astronomy Working Group. A git repository is
created on alioth [2].

Best regards

Ole

[1] http://www.astropy.org/affiliated/index.html
[2] https://anonscm.debian.org/cgit/debian-astro/packages/montage-wrapper.git

Bug#806694: ITP: spectutils -- Base classes and utilities for astronomical spectra in Python

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: specutils
  Version : 0.1 (to be updated)
  Upstream Author : Wolfgang Kerzendorf
* URL : https://github.com/astropy/specutils
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Base classes and utilities for astronomical spectra
 The specutils package implements base classes and utilities for
 interacting with astronomical spectra in Python and the Astropy
 project. It is intended for eventual merger with the astropy package,
 but for now is being developed independently.

This is an Astropy Affiliated Package [1]. It will maintained
within the Debian Astronomy Working Group. A git repository is
created on alioth [2].

Best regards

Ole

[1] http://www.astropy.org/affiliated/index.html
[2] https://anonscm.debian.org/cgit/debian-astro/packages/specutils.git

Bug#805853: ITP: photutils -- Astropy affiliated package for image photometry

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: photutils
  Version : 0.1
  Upstream Author : Larry Bradley and Brigitta Sipocz
* URL : https://github.com/astropy/photutils
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Astropy affiliated package for image photometry
 Photutils contains functions for:
  * estimating the background and background rms in astronomical images
  * detecting sources in astronomical images
  * estimating morphological parameters of those sources (e.g., centroid
and shape parameters)
  * performing aperture and PSF photometry

This is an Astropy Affiliated Package [1]. It will maintained
within the Debian Astronomy Working Group. A git repository is
created on alioth [2].

Best regards

Ole

[1] http://www.astropy.org/affiliated/index.html
[2] https://anonscm.debian.org/cgit/debian-astro/packages/photutils.git

Bug#804649: ITP: astroml-addons -- Python Machine Learning library for astronomy (performance addons)

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: astroml-addons
  Version : 0.2.2
  Upstream Author : Jake VanderPlas
* URL : http://www.astroml.org/
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Python Machine Learning library for astronomy (performance 
addons)
 AstroML is a Python module for machine learning and data mining built on
 numpy, scipy, scikit-learn, and matplotlib. It contains a growing library of
 statistical and machine learning routines for analyzing astronomical data in
 python, loaders for several open astronomical datasets, and a large suite of
 examples of analyzing and visualizing astronomical datasets.
 .
 This package contains the performance add-ons

This package will be created as an extension of the "astroml" package [1].
It will maintained within the Debian Astronomy Working Group. A git
repository is created on alioth [2].

Best regards

Ole

[1] https://bugs.debian.org/756994
https://anonscm.debian.org/cgit/debian-astro/packages/astroml.git
[2] https://anonscm.debian.org/cgit/debian-astro/packages/astroml-addons.git

Bug#804520: ITP: astroscrappy -- Optimized Cosmic Ray Annihilation in Python

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: astroscrappy
  Version : 1.0.3
  Upstream Author : Curtis McCully
* URL : https://github.com/astropy/astroscrappy
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Optimized Cosmic Ray Annihilation in Python
 Astro-SCRAPPY is designed to detect cosmic rays in images (numpy
 arrays), based on Pieter van Dokkum's L.A.Cosmic algorithm.
 Parallelization is achieved using OpenMP.

This is an Astropy Affiliated Package [1]. It will maintained
within the Debian Astronomy Working Group. A git repository is
created on alioth [2].

Best regards

Ole

[1] http://www.astropy.org/affiliated/index.html
[2] https://anonscm.debian.org/cgit/debian-astro/packages/astroscrappy.git

Bug#803038: ITP: ccdproc -- Basic data reductions of astronomy CCD images

[Ole Streicher]

Package: wnpp
Owner: Ole Streicher <oleb...@debian.org>
Severity: wishlist
X-Debbugs-Cc: 
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: ccdproc
  Version : 0.3.3
  Upstream Author : Steve Crawford
* URL : http://ccdproc.readthedocs.org/
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Basic data reductions of astronomy CCD images

The ccdproc package provides many of the necessary tools for processing
of ccd images built on a framework to provide error propagation and bad
pixel tracking throughout the reduction process.

It will maintained within the Debian Astronomy Working Group. A git
repository will be created on alioth [2].

Best regards

Ole

[1] http://www.astropy.org/affiliated/index.html
[2] https://anonscm.debian.org/cgit/debian-astro/packages/ccdproc.git

Re: [Debian-astro-maintainers] glueviz_0.5.2+dfsg-1_amd64.changes ACCEPTED into unstable

[Ole Streicher]

Hi Josue,

Debian FTP Masters ftpmas...@ftp-master.debian.org writes:
 Changes:
  glueviz (0.5.2+dfsg-1) unstable; urgency=medium
* debian/control: Added Conflicts section in order to avoid overwrite
  files from glue-sprite (Closes: #793931).

One comment on this: I think it is just too simple here to just add a
Conflicts here. Did you investigate the other optione?

* Try to convince glue-sprite to change their Python package name. This
could work since their package seem to be used just internally.

* Try to convince glueviz upstream to change their package name (to
glueviz?)

Having two packages with the same name is at least very unfortunate.

Best regards

Ole

Re: [Debian-astro-maintainers] glueviz_0.5.2+dfsg-1_amd64.changes ACCEPTED into unstable

[Ole Streicher]

Dmitry Shachnev mity...@debian.org writes:
 On Sun, 16 Aug 2015 15:12:34 +0200, Ole Streicher wrote:
 One comment on this: I think it is just too simple here to just add a
 Conflicts here. Did you investigate the other optione?

 * Try to convince glue-sprite to change their Python package name. This
 could work since their package seem to be used just internally.

 * Try to convince glueviz upstream to change their package name (to
 glueviz?)

 Having two packages with the same name is at least very unfortunate.

 It is not just unfortunate, it is also a violation of Policy §10.1,
 which says:

   Two different packages must not install programs with different
   functionality but with the same filenames.

Strictly speacking, these are Python packages and not programs, so I was
not sure whether this applies here. Therefore I used the phrasing at
least :-)

Best

Ole

Fwd: ITP: Sherpa -- Modeling and fitting in Python

[Ole Streicher]

X-Debbugs-Cc is quite picky, therefore a manual forward to the relevant 
mailing lists:


 Weitergeleitete Nachricht 
Betreff:Sherpa -- Modeling and fitting in Python
Datum:  Thu, 13 Aug 2015 15:09:03 +0200
Von:Christoph Deil deil.christ...@gmail.com
An: sub...@bugs.debian.org



Package: wnpp
Owner: Christoph Deil deil.christ...@gmail.com 
mailto:deil.christ...@gmail.com

Severity: wishlist
X-Debbugs-Cc: debian-de...@lists.debian.org
mailto:debian-de...@lists.debian.org , debian-as...@lists.debian.org
mailto:debian-as...@lists.debian.org , debian-python@lists.debian.org
mailto:debian-python@lists.debian.org

* Package name: sherpa
  Version : 4.7
  Upstream Author : Smithsonian Astrophysical Observatory / Chandra 
X-Ray Center

* URL : https://github.com/sherpa/sherpa
* License : GPLv3+
  Programming Lang: Python
  Description : Modeling and fitting in Python

Sherpa is a Python package for modeling and fitting. It enables the user 
to construct complex models from simple definitions and fit those models 
to data, using a variety of statistics and optimization methods.


It was originally developed by the Smithsonian Astrophysical Observatory 
/ Chandra X-Ray Center as part of the larger CIAO package for X-ray data 
analysis [1].

Recently it became on open-source, open-development project [2] and
development was moved to Github [3].

It will maintained within the Debian Astronomy Working Group.
A git repository will be created on alioth [4].

I’ll attend the sprint [5] tomorrow and will try to follow the
instructions at [6] to get this packaged for Debian.

Best regards,
Christoph


[1] http://cxc.cfa.harvard.edu/sherpa/
[2] http://pysherpa.blogspot.de/2015/04/sherpa-becomes-open-project.html
[3] https://github.com/sherpa/sherpa
[4] https://anonscm.debian.org/cgit/debian-astro/packages/sherpa.git
[5] https://wiki.debian.org/Sprints/2015/Astro
[6] https://wiki.debian.org/DebianAstro/AstropyPackagingTutorial/Preparation

Fwd: ITP: yt -- Astrophysical Simulation Analysis and Visualization package

[Ole Streicher]

This ITP did not make its way to the mailing lists. Re-sending...

 Forwarded Message 
Subject: 	ITP: yt -- Astrophysical Simulation Analysis and Visualization 
package

Date:   Thu, 25 Jun 2015 15:02:56 -0500
From:   Kacper Kowalik xarthisius...@gmail.com
To: sub...@bugs.debian.org



Package: wnpp
Owner: Kacper Kowalik xarthisius...@gmail.com
Severity: wishlist
X-Debbugs-Cc:
debian-de...@lists.debian.org,debian-as...@lists.debian.org,debian-python@lists.debian.org

* Package name: yt
  Version : 3.1
  Upstream Author : yt devs yt-...@lists.spacepope.org
* URL : http://yt-project.org/
* License : BSD-3-Clause
  Programming Lang: Python
  Description : Framework for analyzing and visualizing simulation data

yt is an analysis and visualization system written in Python, designed
for use with Astrophysical Simulation codes. yt can process data in a
myriad of ways and make a variety of visualizations. It is designed to
be portable across simulation codes, and it is developed by developers
from a variety of institutions and backgrounds.

It will maintained within the Debian Astronomy Working Group. A git
repository will be created on alioth [1].

Best regards
Kacper Kowalik

[1] https://anonscm.debian.org/cgit/debian-astro/packages/yt.git






--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/559f6944.2090...@debian.org

Astropy affiliated packages tutorial

[Ole Streicher]

Hi all,

the Astropy affiliated packages [1] are a great opportunity to get
people involved in Debian packaging: They follow some common rules, tend
to have a similar structure (since there is a template for these
packages), and the project has a good popularity. Also, all these
packages would add a good value to Debian.

To make this easier, I started a tutorial that is specific for these
packages, using wcsaxes as an example [2]. The third part is not
complete yet. Since it is my first tutorial, it would be great if I
could get some review before I announce it on the astropy-devel mailing
list.

Best regards

Ole

[1] http://www.astropy.org/affiliated/
[2] https://wiki.debian.org/DebianAstro/AstropyPackagingTutorial/Preparation
https://wiki.debian.org/DebianAstro/AstropyPackagingTutorial/Packaging
https://wiki.debian.org/DebianAstro/AstropyPackagingTutorial/Advanced


-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/55768f7e.4070...@debian.org

pybuild (Re: image-file-in-usr-lib)

[Ole Streicher]

Ben Finney ben+deb...@benfinney.id.au writes:
 Ole Streicher oleb...@debian.org writes:
 Installing the non-program files to ‘/usr/share/foopackage/images/’ (for
 example) is simple enough, using ‘dh_install’.

I am using pybuild, and I am wondering why it doesn't do this
automatically (maybe with some help)?

Generally: pybuild seem to install all files into /usr/lib/? As far as I
understand the FHS [1], architecture independent files should go to
/usr/share. But, almost all python files (except maybe compiled .so
extensions) are architecture independent?

* .py sources
* .pyc bytecode
* .egg-info
* helper files in png, xml, txt, ...

What is the rationale between having all this in /usr/lib? And what
should I do with lintian-warnings like the one above? Just silence them?

I am asking a bit rigide here since I wanted to write a small newcomers
tutorial to package astropy affiliated packages [2]. These packages are
quite standard and therefore a good start for newcomers -- but then the
final package should be lintian-clean with not too much efforts.

The package I am using as example is wcsaxes [3], and it ends up in
quite a couple of /usr/lib/.png lintian warnings. I don't want to
explain to a newcomer just ignore these, or make them silent.

The Debian Policy [4] states (9.1.1):

| 1. The FHS requirement that architecture-independent application-
| specific static files be located in /usr/share is relaxed to a
| suggestion. In particular, a subdirectory of /usr/lib may be used by a
| package (or a collection of packages) to hold a mixture of
| architecture-independent and architecture-dependent files. However,
| when a directory is entirely composed of architecture-independent
| files, it should be located in /usr/share.

which is a bit contradicting to the current practice: If it is just a
suggestion, then maybe the lintian warning has not right severity
warning and should be lowered? And are pure python packages
(Architecure: all) not covered by the last sentence and should *not* be
in /usr/lib/ by policy?

What is the best way here, also to explain this in a tutorial to an
absolute newcomer?

Best regards

Ole

[1] http://www.pathname.com/fhs/
[2] http://www.astropy.org/affiliated/index.html
[3] https://pypi.python.org/pypi/wcsaxes/0.3
[4] https://www.debian.org/doc/debian-policy/ch-opersys.html#s9.1


-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/87fv738ujq.fsf...@debian.org