Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[Simon McVittie]

On Wed, 20 Feb 2019 at 06:19:56 +0100, Adam Borowski wrote:
> On Tue, Feb 19, 2019 at 11:52:49PM +0100, W. Martin Borgert wrote:
> > It's also a practical matter: Can I run autopkgtests, when my
> > computer is disconnected from the internet?

According to the autopkgtest specification: maybe, but not necessarily.
https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst
(the title of the last section is "Network Access")

> At the very least, I'd require a "restriction" to be set -- autopkgtest
> has a system for facets like this, it'd be a good fit here.

It's been proposed in the past, but the autopkgtest maintainers
considered and rejected it. 


Maybe "Features: works-offline" (with the opposite meaning) would be
another way to represent this?

game-data-packager (in contrib) is an example of a package that tries to
test itself by downloading non-Free data from the internet, although it
looks like it currently skips the relevant test on ci.debian.net because
the testbed doesn't have the freedesktop.org DOWNLOADS directory
(usually ~/Downloads) configured.

smcv

Re: Solved: Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

On 2019-02-21 22:15, Jonas Smedegaard wrote:
> Depending on the kinds of tests you need, there is also the option of 
> telling ffmpeg to generate some data for you.  Saves space :-)

I will look into that!

Re: Solved: Use of FATE (FFmpeg Automated Testing Envionment) data?

[Jonas Smedegaard]

Quoting W. Martin Borgert (2019-02-21 21:52:03)
> upstream confirms that the files are mainly non-free.
> For my purpose, I will just search free replacements.

Depending on the kinds of tests you need, there is also the option of 
telling ffmpeg to generate some data for you.  Saves space :-)

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Solved: Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

Hi,

upstream confirms that the files are mainly non-free.
For my purpose, I will just search free replacements.

Thanks & Cheers

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

On 2019-02-19 23:41, Holger Levsen wrote:
> On Wed, Feb 20, 2019 at 12:06:30AM +0100, Chris Lamb wrote:
> > Noted, but it is not the size that is the concern, but rather the
> > licensing.
>
> once those (licencing concerns) are resolved, I wonder whether it makes
> sense to provide a fate-testdata package which then other packages can
> build-depend on?

Yes. That's mainly something the maintainers of ffmpeg and
depending packages need to decide.

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

On 2019-02-20 06:19, Adam Borowski wrote:
> On Tue, Feb 19, 2019 at 11:52:49PM +0100, W. Martin Borgert wrote:
> > In my case, it's only 3.3 MiB.  And it is only in the source package
> > anyway.
>
> So you can just ship it then.

After clarifying the license(s).

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[Adam Borowski]

On Tue, Feb 19, 2019 at 11:52:49PM +0100, W. Martin Borgert wrote:
> On 2019-02-19 22:55, Chris Lamb wrote:
> > (Naturally, downloading them during the build would not be
> > acceptable, but I wonder what the consensus would be on the ethics
> > of downloading "contrib"-like data during autopkgtests?)
> 
> It's also a practical matter: Can I run autopkgtests, when my
> computer is disconnected from the internet? Can I run them, if
> their server is down? Will they always serve exactly the same
> files or will I have to deal with random test results?

At the very least, I'd require a "restriction" to be set -- autopkgtest
has a system for facets like this, it'd be a good fit here.  It explicitly
allows downloading packages during test, but that's via apt rather than
some random website.

> In my case, it's only 3.3 MiB.  And it is only in the source package
> anyway.

So you can just ship it then.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ Have you accepted Khorne as your lord and saviour?
⠈⠳⣄

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[Holger Levsen]

On Wed, Feb 20, 2019 at 12:06:30AM +0100, Chris Lamb wrote:
> Noted, but it is not the size that is the concern, but rather the
> licensing.

once those (licencing concerns) are resolved, I wonder whether it makes
sense to provide a fate-testdata package which then other packages can
build-depend on?


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

On 2019-02-20 00:06, Chris Lamb wrote:
> Noted, but it is not the size that is the concern, but rather the
> licensing.

Sure! If this cannot be clarified, I'll try to either find
similar files with a suitable licsense, or even produce them
myself. Like with the "(Ma)Lena, the cat" photo :-)

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[Chris Lamb]

W. Martin Borgert wrote:

> > Naturally, downloading them during the build would not be
> > acceptable, but I wonder what the consensus would be on the ethics
> > of downloading "contrib"-like data during autopkgtests?
> 
> It's also a practical matter: Can I run autopkgtests, when my
> computer is disconnected from the internet? Can I run them, if
> their server is down? Will they always serve exactly the same
> files or will I have to deal with random test results?

Indeed, this would definitely be a vastly inferior fallback option.

As someone who has encountered these issues countless times via the
Reproducible Builds effort, I might have assumed these highly-salient
practical concerns as a given and focused on the free software
question.

(However, on the practical side, I believe the "flaky" Restriction
could, potentially, be suitable.)
 
> Maybe we don't need a huge amount of test data, i.e. not the
> full set that is used by upstream. A small subset might be
> sufficient for at least some packages. In my case, it's only
> 3.3 MiB.

Noted, but it is not the size that is the concern, but rather the
licensing.


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org  chris-lamb.co.uk
   `-

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

On 2019-02-19 22:55, Chris Lamb wrote:
> Dumb question: have you tried consulting upstream?

Not yet. If this question has been clarified inside Debian
before, I'm reluctant pestering upstream again. I know, that
some upstreams get shirty about our insistence on such matters.
But if not, I'll contact upstream of course.

> (Naturally, downloading them during the build would not be
> acceptable, but I wonder what the consensus would be on the ethics
> of downloading "contrib"-like data during autopkgtests?)

It's also a practical matter: Can I run autopkgtests, when my
computer is disconnected from the internet? Can I run them, if
their server is down? Will they always serve exactly the same
files or will I have to deal with random test results?

Maybe we don't need a huge amount of test data, i.e. not the
full set that is used by upstream. A small subset might be
sufficient for at least some packages. In my case, it's only
3.3 MiB. And it is only in the source package anyway.

Cheers

Re: Use of FATE (FFmpeg Automated Testing Envionment) data?

[Chris Lamb]

W. Martin Borgert wrote:

> I can easily add the small number of needed files to debian/fate-
> suite/, but I'm not even sure about the license of the files.

Dumb question: have you tried consulting upstream? Getting a
definitive statement from them on the licensing of these files
would probably be most helpful and possibly even required here, as
well as assisting potential future users of this test data.

(Naturally, downloading them during the build would not be
acceptable, but I wonder what the consensus would be on the ethics
of downloading "contrib"-like data during autopkgtests?)


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org  chris-lamb.co.uk
   `-

Use of FATE (FFmpeg Automated Testing Envionment) data?

[W. Martin Borgert]

Hi,

a package I'm working on makes use of eight files from the FATE
project for unit testing. The files are multimedia files (image,
audio, video, subtitles). By default it downloads files during
build, but it's easy to just provide them locally. So far, I
could not find any FATE files in Debian. To enable unit tests
during build, I can easily add the small number of needed files
to debian/fate-suite/, but I'm not even sure about the license
of the files. (Lintian complained about Lena Söderberg, which I
replaced with an AGPL3ed cat photo.)

How is this handled in other multimedia packages, esp. ffmpeg?
Is FATE data used somewhere, somehow by Debian?
Are some files packaged, which I missed?

TIA & Cheers

PS: This is the list of files needed by me:

http://fate.ffmpeg.org/fate-suite/aac/latm_stereo_to_51.ts
http://fate.ffmpeg.org/fate-suite/audio-reference/chorusnoise_2ch_44kHz_s16.wav
http://fate.ffmpeg.org/fate-suite/h264/interlaced_crop.mp4
http://fate.ffmpeg.org/fate-suite/mpeg2/mpeg2_field_encoding.ts
http://fate.ffmpeg.org/fate-suite/mxf/track_01_v02.mxf
http://fate.ffmpeg.org/fate-suite/png1/lena-rgb24.png
http://fate.ffmpeg.org/fate-suite/sub/MovText_capability_tester.mp4
http://fate.ffmpeg.org/fate-suite/sub/vobsub.sub