Re: Added channel parsers for DVB-S2 and DVB-T2 at libdvbv5 and found some issues at dtv-scan-tables

2014-09-15 Thread Jonathan McCrohan
Hi Mauro,

On Thu, Sep 04, 2014 at 11:56:03PM -0300, Mauro Carvalho Chehab wrote:
> I added some improvements at libdvbv5 to parse the DVB-T2 and DVB-S2 lines
> at dtv-scan-tables.
> 
> It can now successully parse all correct files there. The patches for
> it are already merged upstream at:
>   http://git.linuxtv.org/cgit.cgi/v4l-utils.git/
> 
> I also added there the Makefile from Jonathan that adds support
> for converting from DVBv3 to DVBv5, and added myself some logic
> there to convert back from DVBv5 into DVBv3 format.

I've been away from my mail for a few days; many thanks for this work. I
hope to upload a new version of dtv-scan-tables which incorporates these
changes to Debian soon. Gregor has already uploaded the new version of
v4l-utils [1].

Jon

[1] https://packages.qa.debian.org/v/v4l-utils/news/20140909T180008Z.html


pgpja7t5ZXGo5.pgp
Description: PGP signature


Added channel parsers for DVB-S2 and DVB-T2 at libdvbv5 and found some issues at dtv-scan-tables

2014-09-04 Thread Mauro Carvalho Chehab
Hi Oliver,

I added some improvements at libdvbv5 to parse the DVB-T2 and DVB-S2 lines
at dtv-scan-tables.

It can now successully parse all correct files there. The patches for
it are already merged upstream at:
http://git.linuxtv.org/cgit.cgi/v4l-utils.git/

I also added there the Makefile from Jonathan that adds support
for converting from DVBv3 to DVBv5, and added myself some logic
there to convert back from DVBv5 into DVBv3 format.

With that, we can compare and check if everything is being properly
parsed on both ways.

My hope is to remove the DVBv3 files, keeping everything stored at the
DVBv5 format, adding a target to generate the DVBv3 format for the
ones that need it.

While doing that, I noticed that there are some troubles at some of
the existing tables.

I fixed most of them and added on my experimental dtv-scan-tables:
http://git.linuxtv.org/cgit.cgi/mchehab/dtv-scan-tables.git/

The errors I found are:

1) The dvb-t/ca-AB-Calgary file has an invalid line:

http://git.linuxtv.org/cgit.cgi/mchehab/dtv-scan-tables.git/commit/?id=b3eeab8ff0f87f0dd7207ff80aaed4e098052bab

2) One FEC on dvb-t/sk-Bratislava is marked as 2/1 (very likely
   instead of 1/2):

http://git.linuxtv.org/cgit.cgi/mchehab/dtv-scan-tables.git/commit/?id=795628c5ea7b4f99e932b5b243b6ee3a8ebecf0c

3) The dvb-s/Astra-31.5E file has two extra parameters for a DVB-S 
   stream:

http://git.linuxtv.org/cgit.cgi/mchehab/dtv-scan-tables.git/commit/?id=9b6ba6016d092ce823a60b2eb537dd3ac2efa825

4) Most of the DVB-T2 lines put the PLP at the end, but one of them put
   it, together with another field (system ID?), at the beginning:

http://git.linuxtv.org/cgit.cgi/mchehab/dtv-scan-tables.git/commit/?id=3c3cd4befd73a5e7fc906224082d197d6b510ea3
 

With regards to this extra "system ID" with is always 1 on this file,
I'm assuming that this is bogus. Anyway, dvb-format-tool won't
keep it (see dvb-t/ug-All diff):

-T2 53800 8MHz AUTO AUTO AUTO  32k 1/16 AUTO 1 1
-T2 58600 8MHz AUTO AUTO AUTO  32k 1/16 AUTO 9 1
+T2 53800 8MHz AUTO AUTO AUTO 32k 1/16 AUTO 1
+T2 58600 8MHz AUTO AUTO AUTO 32k 1/16 AUTO 9

There is still one remaining issue affecting two DVB-S files: 
dvb-s/Hotbird-13.0E and dvb-s/Atlantic-Bird-3-5.0W.

There, the DVB-S2 have one (or two) fields that are not part of
the DVBv5 API:

One extra field at Hotbird 13E:


# EUTELSAT SkyPlex, Hotbird 13E
# std freq pol sr fec rolloff mod is_id pls_code pls_mode

S2 11432000 V 2750 2/3 AUTO 8PSK 1 8
S2 11432000 V 2750 2/3 AUTO 8PSK 2 8

And two extra fields at Atlantic Bird 3:

# Atlantic Bird 3 @ 5.0W
# std freq pol sr fec rolloff mod is_id pls_code pls_mode

S2 11012000 V 3000 AUTO AUTO 8PSK 2 16416
S2 11012000 V 3000 AUTO AUTO 8PSK 3 16416
...

As those parameters don't exist at DVBv5 API, nor at libdvbv5,
a conversion on those files will lose them.

While it shouldn't be hard to add it to libdvbv5, letting it to
parse back and forth, I'm not sure if this is the best way to
proceed, as I'm not sure if they're useful and what program
uses it, if any.

Comments?

Regards,
Mauro


---

As reference, those are the diffs between the original and the parsed
files, after removing all comments with this small script:

$ for i in atsc/* dvb-?/*; do echo $i; perl -ne 's/\s*#.*//; print $_;' <$i >a 
&& mv a $i; done

--- dvb-s/Atlantic-Bird-3-5.0W  2014-09-04 23:48:54.359100590 -0300
+++ dvbv3/dvb-s/Atlantic-Bird-3-5.0W2014-09-04 23:49:02.740096400 -0300
@@ -1,33 +1,30 @@
-
-
-
-S2 11012000 V 3000 AUTO AUTO 8PSK 2 16416
-S2 11012000 V 3000 AUTO AUTO 8PSK 3 16416
-S2 11012000 V 3000 AUTO AUTO 8PSK 11 16416
-S2 11179000 V 3000 AUTO AUTO 8PSK 4 16416
-S2 11179000 V 3000 AUTO AUTO 8PSK 5 16416
-S2 11179000 V 3000 AUTO AUTO 8PSK 12 16416
-S2 11637000 V 3000 AUTO AUTO 8PSK 1 16416
-S2 11637000 V 3000 AUTO AUTO 8PSK 2 16416
-S2 11637000 V 3000 AUTO AUTO 8PSK 3 16416
-S2 11637000 V 3000 AUTO AUTO 8PSK 4 16416
-S2 11637000 V 3000 AUTO AUTO 8PSK 5 16416
-S2 11637000 V 3000 AUTO AUTO 8PSK 6 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 7 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 8 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 9 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 10 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 11 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 12 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 13 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 14 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 15 16416
-S2 11675000 H 3000 AUTO AUTO 8PSK 16 16416
-S2 12585000 H 2750 AUTO AUTO 8PSK 1 8
-S2 12585000 H 2750 AUTO AUTO 8PSK 2 8
-S2 12606000 V 2990 AUTO AUTO 8PSK 1 8
-S2 12606000 V 2990 AUTO AUTO 8PSK 2 8
-S2 12648000 V 2950 AUTO AUTO 8PSK 1 121212 1
-S2 12648000 V 2950 AUTO AUTO 8PSK 2 121212 1
-S2 12669000 H 3008 AUTO AUTO 8PSK 1 8
-S2 12669000 H 3008 AUTO AUTO 8PSK 2 8
+S2 11012000 V 3000 AUTO AUTO 8PSK 2
+S2 11012000