Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

2015-10-03 0:31 GMT+02:00 Simon McVittie :
> On 02/10/15 20:55, Markus Koschany wrote:
>> P.S.: g-d-p is currently broken in Git master. :>
> ...
>> line 829, in load_file_data
>> ), (self.shortname, wanted.name)
>> AssertionError: ('enemy-territory', 'license.rtf')
>
> Should now be fixed in git. I had split out detailed information about
> the required files for each package into separate files, which don't
> need to be loaded until g-d-p starts working on specific packages, but
> then forgot to tell the packaging to install them.
>
> That feature could still do with some improvement - some of the *.json
> that are loaded for things like --help are still rather large, and it
> might be better to install a .zip rather than > 600 smallish files,
> after benchmarking whether it's faster compressed or uncompressed - but
> it's a start.

Hi,

I've just done that, it's about as fast on the Rpi2 and a bit faster
(~5%) on the desktop PC.

The out/vfs.zip is defined as .PHONY rule that only get build when
building the .deb;
./run will continue to use the unpacked files.

$ git pull ; make
$ time ./run > /dev/null
real0m8.119s
user0m7.850s
sys 0m0.210s
$ make out/vfs.zip
$ echo 'USE_VFS = True' >> game_data_packager/paths.py
$ time ./run > /dev/null
real0m8.148s
user0m7.880s
sys 0m0.240s
$ git checkout game_data_packager/paths.py

It should be reproducible, but I didn't went through the whole
testing procedure.

https://wiki.debian.org/ReproducibleBuilds/Howto#Testing_procedure

Greets,

Alexandre

Bug#800713: Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Control: tags -1 patch pending

Added support for Icewind Dale 1.

Markus



signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Control: tags -1 patch pending

Am 08.10.2015 um 16:10 schrieb Alexandre Detiste:
> 2015-10-07 14:01 GMT+02:00 Markus Koschany :
>>>
>>> It's important to specify the english archive first; this one will be 
>>> considered the "original"
>>> version, and the other will have the ?lang prefix appended to files with 
>>> the same
>>> name but different size/md5.
>>
>> I tried the exact same command with the German and English windows
>> installer and it works but some files from the English version also get
>> the ?en suffix which is probably unnecessary?
>>
>> make-template outputs for example:
>> chitin.key
>> chitin.key?en
>>
>> If the English version is always considered the "base" version only
>> chitin.key should suffice.
> 
> This happens because this English archive was likely unpacked without
> using then new "--collisions=rename" innoextract flag.
> You end up with two different files names "chitin.key" & "Chitin.key".

Hi,

ok I give up for now because I can't get it to work with more than one
language version. I have pushed baldurs-gate-1.yaml to git master which
supports the English Linux installer from gog.com.

The debug information from g-d-p are very sparse. I only see

INFO:game-data-packager.build:identifying app1/movies/MOVIECD1.BIF
INFO:game-data-packager.build:identifying app1/movies/MOVIECD4.BIF
INFO:game-data-packager.build:identifying app1/data/GUI.bif
INFO:game-data-packager.build:identifying app1/data/CHAANIM.bif
ERROR:game-data-packager.build:Unable to complete any packages.

That's it and then it fails. The --verbose flag doesn't add any extra
information.

Please correct me if I am wrong: At first I used

game-data-packager make-template setup_baldurs_gate_2.0.0.20.exe
setup_baldurs_gate_german_2.0.0.20.exe > bg_en_de.txt

this produces output like chitin.key?en. How can I use g-d-p with the
--collisions=rename option of innoextract?

When I manually unpack both windows installers with innoextract
--collisions=rename and then run make-template on both unpacked
directories, I get something which looks similar to your last e-mail
attachment but it is still not the same. I've pushed my bg_en_de.txt
file to the baldurs-gate branch, in case you want to have look at it.


This whole renaming is somehow crazy. What do you think about the
following design: just declare all files with size and hashsums per
package and enforce that the user passes a language option to
game-data-packager like

game-data-packager baldurs-gate-1 --lang de 


baldurs-gate-1.yaml

size_and_md5:
  lang: en
  baldurs-gate-original-saga-en-data:
  .
  .
  .
  all detected files from the English Linux Installer with size and
  hashsums

  lang: de
  baldurs-gate-original-saga-de-data:
  .
  .
  .
  all detected files from the German Linux Installer with size and
  hashsums


I would only have to trim the raw output from make-template, remove some
files and that's it. No renaming necessary. There will be duplicate
information regarding file names, sizes and hashsums but this design is
much less error prone and simpler to maintain, IMO.

Cheers,

Markus











signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

Le samedi 10 octobre 2015, 17:16:51 Markus Koschany a écrit :
> Control: tags -1 patch pending
> 
> Hi,
> 
> ok I give up for now because I can't get it to work with more than one
> language version. I have pushed baldurs-gate-1.yaml to git master which
> supports the English Linux installer from gog.com.

Thanks

I commited these tow more thins:

- I liked the shorter packager names in the test branch better
("baldurs-gate-1-data" or "baldurs-gate1-data");
I picked that up from there; the 'marketing name' remain
in the package short description.

- I don't yet know if gemreb is case-sensitive;
but for the documentation in /usr/share/doc/ that should
never matter. If these files were needed at runtime they would
need to go in /usr/share/games anyway.

> The debug information from g-d-p are very sparse. I only see

with "DEBUG=1 ./run " you'll get pages & pages of log,
maybe some of this was needed when building the core
functionality and could be disabled now.

> That's it and then it fails. The --verbose flag doesn't add any extra
> information.

The --verbose flag show output from the external tools.

Showing the missing files when --package is explicitely defined
would be a plus !

> Please correct me if I am wrong: At first I used
> 
> game-data-packager make-template setup_baldurs_gate_2.0.0.20.exe
> setup_baldurs_gate_german_2.0.0.20.exe > bg_en_de.txt
> 
> this produces output like chitin.key?en. How can I use g-d-p with the
> --collisions=rename option of innoextract?

That's not yet implemented & this is the exactly only
game that would need this and can't be packaged with innoextract 1.4-1
& I'll have to recheck all the other archives with this --collisions=rename 
flag.

> When I manually unpack both windows installers with innoextract
> --collisions=rename and then run make-template on both unpacked
> directories, I get something which looks similar to your last e-mail
> attachment but it is still not the same. I've pushed my bg_en_de.txt
> file to the baldurs-gate branch, in case you want to have look at it.

Thanks

> This whole renaming is somehow crazy. What do you think about the
> following design: just declare all files with size and hashsums per
> package and enforce that the user passes a language option to
> game-data-packager like
> 
> game-data-packager baldurs-gate-1 --lang de 
> I would only have to trim the raw output from make-template, remove some
> files and that's it. No renaming necessary. There will be duplicate
> information regarding file names, sizes and hashsums but this design is
> much less error prone and simpler to maintain, IMO.

I think it should be easier & less error-prove to package Baldur's Gate the
current way than changing everything else; make-template is already quite 
improved.


Greets,

Alexandre



signature.asc
Description: This is a digitally signed message part.

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

2015-10-07 14:01 GMT+02:00 Markus Koschany :
>>
>> It's important to specify the english archive first; this one will be 
>> considered the "original"
>> version, and the other will have the ?lang prefix appended to files with the 
>> same
>> name but different size/md5.
>
> I tried the exact same command with the German and English windows
> installer and it works but some files from the English version also get
> the ?en suffix which is probably unnecessary?
>
> make-template outputs for example:
> chitin.key
> chitin.key?en
>
> If the English version is always considered the "base" version only
> chitin.key should suffice.

This happens because this English archive was likely unpacked without
using then new "--collisions=rename" innoextract flag.
You end up with two different files names "chitin.key" & "Chitin.key".

>From attachment to mail #130:

  2175596882b361d7e1ef4e0de5a25481754a6a chitin.key
  23849620e12866311a215e236b8eb6fa91405c chitin.key$0
  23849620e12866311a215e236b8eb6fa91405c chitin.key?de

I don't know how gemrb works and which file should be used.

The way G-D-P calls innoextract, both files will always be unpacked
& checksummed & then if the 217559 bytes version is not used at all,
it'll be ignored.

It's ok to document hashes for useless files to silence warnings
about bad files, e.g. "doom_wad?shareware" in doom.yaml.

"distinctive_name: false" silence all warnings, so that's a bit
more dangerous to use.

> Some remarks / improvement suggestions:
>
> I think make-template should add all files to the Provides: fields
> automatically. I mean the tool already knows what files should be
> installed into each package when it is run with the setup_*.exe files,
> and in this case install: and provides: can be handled identically. It
> saves one copy step.

I first thought it was tedious to remove extraneous files there too;
they must already be removed from packages:, size_and_md5: and sha1: ;
but deleting lines is easy; so ok to do this.

What I did now the last ecoquest games is

./run make-template /tmp/en /tmp/french | grep -v '\.drv' > data/ecoquest1.yaml

that works as well to remove a bunch of "*.drv" files not needed for this game.



That, or a new syntax that says that setup_...exe provides all the files
needed to build package X. That would become usefull if GOG
starts to update their archives regularly.


> The remaining issues are:
> The windows installer can't be downloaded automatically from gog.com.
>
> The command
>
> game-data-packager baldurs-gate-1 setup_baldurs_gate_german_2.0.0.20.exe
>
> does unpack and verify all files but it does not automatically rename
> the ?de files hence it won't work and eventually fails.

I'll try that with the fake_lgogdonwloader. To debug I use:

DEBUG=1 ./run baldurs-gate-1 setup_baldurs_gate_german_2.0.0.20.exe
2>&1 | grep IMPOSSIBLE | sort -u

There should be a way to tell users that x files are missing; but that open
the door to many problems: which files from which packages ?
When is this information worthy & not even more confusing ?
I would fit better in a GUI with green & red leds.

> It also triggers a loop whereby all files are unpacked and verified three 
> times in a row.

I'll have a look a that tomorrow.

> What works is to manually unpack the exe files with innoextract and to
> rename the files by hand. How did you do the renaming for all the other
> dubbed games?

The idea is to have the smallest working .yaml files, so all the
renaming is implicit
with the new '?lang' syntax.

What makes it difficult to handle is that Baldur Gate is a huge game
with many files.
(and that chitin.key file !?#)

Here's an example of an other dubbed game sold on GOG:
 
http://anonscm.debian.org/cgit/pkg-games/game-data-packager.git/tree/data/goblins3.yaml


Greets

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Am 06.10.2015 um 23:47 schrieb Alexandre Detiste:
> Hi,
> 
> 
> Most game definitions have be rewriten with the shorter "?xx" syntax.

Hi,

yes, I saw that. Good job!

> make-template should now be able to mostly automatically process dubbed games 
> too.
> 
> $ ./run make-template ../../gdp/baldur/setup_baldurs_gate_2.0.0.20.exe 
> 
> ../../Downloads/setup_baldurs_gate_german_2.0.0.20.exe > baldur_de_en.txt 


That's a major improvement of the make-template feature and saves a lot
of time. Thanks.

> (raw ouput attached)
> 
> It's important to specify the english archive first; this one will be 
> considered the "original"
> version, and the other will have the ?lang prefix appended to files with the 
> same
> name but different size/md5.

I tried the exact same command with the German and English windows
installer and it works but some files from the English version also get
the ?en suffix which is probably unnecessary?

make-template outputs for example:
chitin.key
chitin.key?en

If the English version is always considered the "base" version only
chitin.key should suffice.


> Even for these Polich, Czech, French games were the English version
> is not really the original one, English is considered the original one for 
> consistency.
> ... like strings in LibreOffice are translated from English
> and not German like maybe it could had been in original StarOffice.
> 

I will focus on the German and English versions first, this is already
enough work but after the other GemRB games are supported too, I will
have a look into this again.

Some remarks / improvement suggestions:

I think make-template should add all files to the Provides: fields
automatically. I mean the tool already knows what files should be
installed into each package when it is run with the setup_*.exe files,
and in this case install: and provides: can be handled identically. It
saves one copy step.

The remaining issues are:
The windows installer can't be downloaded automatically from gog.com.

The command

game-data-packager baldurs-gate-1 setup_baldurs_gate_german_2.0.0.20.exe

does unpack and verify all files but it does not automatically rename
the ?de files hence it won't work and eventually fails. It also triggers
a loop whereby all files are unpacked and verified three times in a row.
What works is to manually unpack the exe files with innoextract and to
rename the files by hand. How did you do the renaming for all the other
dubbed games?

Cheers,

Markus








signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

Le mardi 6 octobre 2015, 23:47:55 Alexandre Detiste a écrit :
> Language autodetection works by analyzing the filename,
> so it doesn't work currently with unpacked archives.

Fixed, one just need to name the directory "$game_$lang"

./run make-template -l /tmp/ecoquest_english/ /tmp/ecoquest_french/ > 
data/ecoquest1.yaml


signature.asc
Description: This is a digitally signed message part.

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Am 03.10.2015 um 22:04 schrieb Simon McVittie:
[...]
> We don't currently have a good way to spell "the file with sha1sum
> 1234... should be installed as foo1.dat in the English data but foo2.dat
> in German". Is that what you're asking for?
> 
> If some files are identical, there's syntax for adding symlinks (see
> tyrian.yaml).

Hi,

I have pushed my latest revision to branch baldurs-gate. My problem is
that I have seven files with identical names but different sizes and
hashsums. e.g.

 # English Linux installer
  842   e5526c907bb56ec72f8e6cf4eb30fc2e Baldur.ini

 # German Linux installer
  905   2dc7b81c210ed13ea88f509c765d4b76 Baldur.ini

My first attempt was to list all files from both installers under
size_and_md5: and sha1sums: but obviously g-d-p doesn't like this scheme
and fails with an assertion error

AssertionError: ('trying to set size of "%s" to both %d and %d',
'Baldur.ini', 842, 905)


I would like to express:

If Baldur.ini in GameData with size 905 and md5sum
2dc7b81c210ed13ea88f509c765d4b76
is_valid: true
install_to baldurs-gate1-de package

If Baldur.ini in directory with size 905 and md5sum
2dc7b81c210ed13ea88f509c765d4b76 AND Baldur.ini in directory with size
842 e5526c907bb56ec72f8e6cf4eb30fc2e Baldur.ini
raise exception ("That doesn't look like a valid archive")

If Baldur.ini with size and hashsum as declared in baldurs-gate-1.yaml
NOT IN GameData then give up

The current workaround seems to be to rename those files and to append a
language specific suffix. Then one has to declare another stanza where
"file x" is installed as "file y". (e.g. dig.yaml) Baldur's Gate from
gog requires a manual repack, so this technique is quite annoying.

Another problem: the English version ships

data/AREA1700.BIF

the German version

Data/AREA1700.BIF

Both files are identical. My hope was that g-d-p would recognize the
hashsums for both of them when I declare

  25300442  496788a2ce730353acf26d2bebf0471c AREA1700.BIF

and accepts that I install the same file into the data directory
(English version) or the Data directory (German version). Unfortunately
this fails with another error message at build time.
For the last issue symlinking would be an option but this is rather
inconvenient when you have to process a lot of files like that.

Markus






signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 04/10/15 12:27, Markus Koschany wrote:
> If Baldur.ini in GameData with size 905 and md5sum
> 2dc7b81c210ed13ea88f509c765d4b76
>   is_valid: true
>   install_to baldurs-gate1-de package
> 
> If Baldur.ini in directory with size 905 and md5sum
> 2dc7b81c210ed13ea88f509c765d4b76 AND Baldur.ini in directory with size
> 842 e5526c907bb56ec72f8e6cf4eb30fc2e Baldur.ini
>   raise exception ("That doesn't look like a valid archive")
...
> The current workaround seems to be to rename those files and to append a
> language specific suffix. Then one has to declare another stanza where
> "file x" is installed as "file y". (e.g. dig.yaml)

Yes. The "filenames" are almost more like opaque identifiers for a
particular file than anything else: as far as g-d-p is concerned, you
could call them anything, but it's a lot easier for humans to understand
the .yaml if you call the file Baldur.ini_de than if you call it
2dc7b81c210ed13ea88f509c765d4b76.

The look_for and install_as options default to the g-d-p-readable name
of the file, so that in the common case where the file's content is
unique, you don't have to do anything else. For the less common case
where the file's content varies, you have to tell g-d-p how to install
it, and whether it should look for any additional filenames (but the
default is look_for: [name, install_as] so that is often unnecessary).

We could have a heuristic like stripping some "extension" automatically
when determining look_for and install_as, but we'd have to use something
that is unlikely to appear in a genuine filename. One possibility would
be one of the characters that are syntactically invalid on Windows,
which are:

\ / : * ? " < > |

(). Obviously we can't
use / or \, because we need / for relative paths, and \ is the YAML
escape character. To minimize awkwardness in the YAML syntax, we'd
ideally use a character that isn't special in YAML, but unfortunately
that rules out all of the Windows non-filename characters except "<",
and I don't like the look of that.

I'd vaguely lean towards "?", which is vaguely reminiscent of HTTP
query-strings, is relatively obscure in YAML (unlike ":" which we use
all the time), and is only special in YAML if followed by whitespace; if
we constrain the variation selectors (version and/or language) to not
start with whitespace, then we don't need special quoting. So we could
say that mentioning

doom.wad?1.1

in a file list implies the equivalent of this:

files:
  doom.wad?1.1:
install_as: doom.wad

unless a different look_for or install_as value is configured?

In the unlikely event that a game's data includes the special "variation
selector" character, you'd have to specifically write

  # yes, really this filename
  files: why this silly filename?.txt
install_as: why this silly filename?.txt

which doesn't seem too onerous.

> Another problem: the English version ships
> 
>   data/AREA1700.BIF
> 
> the German version
> 
>   Data/AREA1700.BIF

g-d-p searches case-insensitively anyway, because the 99% case is that
the game was designed to run on Windows or DOS.

> I install the same file into the data directory
> (English version) or the Data directory (German version).

Does gemrb really require you to do this? If you install either language
into the other directory (I'd use all-lower-case and move the German
version), does it work?

> For the last issue symlinking would be an option but this is rather
> inconvenient when you have to process a lot of files like that.

Or if gemrb is case-sensitive, can you install everything into 'data',
and use a single symlink Data -> data to accommodate the German version?

S

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Am 04.10.2015 um 18:22 schrieb Simon McVittie:
[...]
> So we could say that mentioning
> 
> doom.wad?1.1
> 
> in a file list implies the equivalent of this:
> 
> files:
>   doom.wad?1.1:
> install_as: doom.wad
> 
> unless a different look_for or install_as value is configured?

That would be a nice shortcut.

[...]

>> I install the same file into the data directory
>> (English version) or the Data directory (German version).
> 
> Does gemrb really require you to do this? If you install either language
> into the other directory (I'd use all-lower-case and move the German
> version), does it work?

You may well ask. I'll try this.

>> For the last issue symlinking would be an option but this is rather
>> inconvenient when you have to process a lot of files like that.
> 
> Or if gemrb is case-sensitive, can you install everything into 'data',
> and use a single symlink Data -> data to accommodate the German version?

That would probably work as well. I'll report back.

Markus



signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

Le dimanche 4 octobre 2015, 21:11:56 Markus Koschany a écrit :
> Am 04.10.2015 um 18:22 schrieb Simon McVittie:
> [...]
> > So we could say that mentioning
> > 
> > doom.wad?1.1
> > 
> > in a file list implies the equivalent of this:
> > 
> > files:
> >   doom.wad?1.1:
> > install_as: doom.wad
> > 
> > unless a different look_for or install_as value is configured?
> 
> That would be a nice shortcut.

And it's a one-line change...

I pushed this + updated definition for Soltys game as a sample.

--- a/game_data_packager/__init__.py
+++ b/game_data_packager/__init__.py
@@ -54,7 +54,7 @@ class WantedFile(HashedFile):
 self.distinctive_name = True
 self.distinctive_size = False
 self.download = None
-self.install_as = name
+self.install_as = name.split('?')[0]

signature.asc
Description: This is a digitally signed message part.

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

> 2015-10-04 18:22 GMT+02:00 Simon McVittie :
> g-d-p searches case-insensitively anyway, because the 99% case is that
> the game was designed to run on Windows or DOS.

maybe make-template heuristic should be refined to be lowercase by default,
except with HTML + images documentation sets that goes in /usr/share/doc/.

I vaguely remember that when browsing some documentation from
a Quake or RTCW game; some image links were broken,
because the files hadn't the right case; I'll have a look with a link checker.

very few engines really except uppercase files (I can think of ROTT)

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

2015-10-04 18:22 GMT+02:00 Simon McVittie :
> We could have a heuristic like stripping some "extension" automatically
> when determining look_for and install_as, but we'd have to use something
> that is unlikely to appear in a genuine filename.
>
> ...
>
> I'd vaguely lean towards "?", which is vaguely reminiscent of HTTP
> query-strings, is relatively obscure in YAML (unlike ":" which we use
> all the time)
>
> doom.wad?1.1

That would be great. The '?' is already special-handled elsewhere
in choose_mirror() to means that URL with this character can't be
'solid' file names,
only a transaction that'll give a real file, with an other name.

For example, 
https://docs.google.com/uc?export=download=0ByCRmO4G6HOoRWNjaS04VTlMbWc
will give us "iortcw-1.42b-linux-x86.zip", but there will never be
a file named "uc?export=download=0ByCRmO4G6HOoRWNjaS04VTlMbWc"
in a local stash of patches/game demos that can be referenced with GDP_MIRROR.

As these '?' are only visible in the "download:" tag,
there's not conflict with the automatic stripping.

>> Another problem: the English version ships
>>
>>   data/AREA1700.BIF
>>
>> the German version
>>
>>   Data/AREA1700.BIF
>
> g-d-p searches case-insensitively anyway, because the 99% case is that
> the game was designed to run on Windows or DOS.

maybe make-template heuristic should be refined to be lowercase by default,
except with HTML + images documentation sets that goes in /usr/share/doc/.

I vaguely remeber that when browsing some documentation from
a Quake or RTCW game; some image links were broken,
because the files hadn't the right case; I have


>
>> I install the same file into the data directory
>> (English version) or the Data directory (German version).
>
> Does gemrb really require you to do this? If you install either language
> into the other directory (I'd use all-lower-case and move the German
> version), does it work?
>
>> For the last issue symlinking would be an option but this is rather
>> inconvenient when you have to process a lot of files like that.
>
> Or if gemrb is case-sensitive, can you install everything into 'data',
> and use a single symlink Data -> data to accommodate the German version?
>
> S
>

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Am 03.10.2015 um 00:31 schrieb Simon McVittie:
> On 02/10/15 20:55, Markus Koschany wrote:
>> P.S.: g-d-p is currently broken in Git master. :>
> ...
>> line 829, in load_file_data
>> ), (self.shortname, wanted.name)
>> AssertionError: ('enemy-territory', 'license.rtf')
> 
> Should now be fixed in git. I had split out detailed information about
> the required files for each package into separate files, which don't
> need to be loaded until g-d-p starts working on specific packages, but
> then forgot to tell the packaging to install them.
> That feature could still do with some improvement - some of the *.json
> that are loaded for things like --help are still rather large, and it
> might be better to install a .zip rather than > 600 smallish files,
> after benchmarking whether it's faster compressed or uncompressed - but
> it's a start.

Great.

> (FYI, you can run g-d-p with "make && ./run enemy-territory ..." to use
> the version under development without installing it.)

Thanks, Alexandre pointed me in the same direction, that's very useful
for development.

I'm currently struggling with the German version of Baldur's Gate 1. A
lot of identical files are named differently like


fa670e7810253e8dfba283f844d450b8c5f429e4  Music/Bd2/Bd2endb2.ACM
fa670e7810253e8dfba283f844d450b8c5f429e4  Music/Bf2/Bf2endd4.ACM

then there are many files which are different but share the same name

and of course there are also new ones. If I understand the current
work flow correctly, files with identical names should be renamed (for
ScummVM games there is also a "prefix" mechanism) and then installed
into the corresponding binary package with "install_as". That may be a
suitable approach for games which only provide few files but with a lot
of files this approach becomes quickly painful.

Perhaps I miss something. Is it already possible to assign md5 and sha
sums to binary packages like that:

Assumption: There is a directory baldurs_gate_1 which contains the
German version of Baldur's Gate 1:

baldurs-gate-1.yaml:

baldurs-gate1-de-data:
  install_to 
  install
  - file1
  - file2
  sha1sums:
  90deca7b26d7f477de5667236973fbcdd3f1e626 file1
  dc9b2590ae6a7576a4e12d9b8ab5ade231517e09 file2

baldurs-gate1-en-data:
  install_to 
  install
  - file1
  - file2
  sha1sums:
   e7bab3db4e542a32a601a164c666956441d10ef0 file1
   2f6378980d1af3e5a35f28d648a0fe7f6fca9ed9 file2

and g-d-p would try to detect whether the directory baldurs_gate_1
contains all necessary files for either the en or de version and then
builds the corresponding binary package?

Cheers,

Markus









signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 03/10/15 11:38, Markus Koschany wrote:
> Perhaps I miss something. Is it already possible to assign md5 and sha
> sums to binary packages like that:
[...]
> and g-d-p would try to detect whether the directory baldurs_gate_1
> contains all necessary files for either the en or de version and then
> builds the corresponding binary package?

That's how the data-driven g-d-p always works. It follows this pseudocode:

* list all the files in the GameData, regardless of package;

* for each file, see whether it is in the input, or not in the input
  but obtainable by downloading or unpacking, or entirely unobtainable;

* for each package, if all files are obtainable, do all the necessary
  downloading/unpacking/conversion and build the package.

For instance, English RTCW and French RTCW only differ by one file.

We don't currently have a good way to spell "the file with sha1sum
1234... should be installed as foo1.dat in the English data but foo2.dat
in German". Is that what you're asking for?

If some files are identical, there's syntax for adding symlinks (see
tyrian.yaml).

S

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

Hi,

[Markus]
>I'm currently struggling with the German version of Baldur's Gate 1. A
>lot of identical files are named differently like
>
>fa670e7810253e8dfba283f844d450b8c5f429e4  Music/Bd2/Bd2endb2.ACM
>fa670e7810253e8dfba283f844d450b8c5f429e4  Music/Bf2/Bf2endd4.ACM

[Simon]
> We don't currently have a good way to spell "the file with sha1sum
> 1234... should be installed as foo1.dat in the English data but foo2.dat
> in German". Is that what you're asking for?

Maybe this is not a problem, there happens the same with Dreamweb for example:

  1040570   18fbb58f8621ee93b4045556cad447c6 dreamweb.v02
  1040570   18fbb58f8621ee93b4045556cad447c6 dreamwfr.v02
  1040570   18fbb58f8621ee93b4045556cad447c6 dreamwsp.v02

This is ok as long as these files doesn't share any common "look_for" name;
if this was the case, a file would be identified at random;
and G-D-P would randomly fail to create a package.

This problem did happened with Arx Fatalis game:

 
http://anonscm.debian.org/cgit/pkg-games/game-data-packager.git/commit/?id=c14cbf6522f1ff3e9c38a87a9a4bf781b9a546d6

There's not yet an assertion against this in the code.

> For instance, English RTCW and French RTCW only differ by one file.

I had to change this into two full-blown packages
because GOG only sell the English version and a German dub (but not for people 
living in Germany);
so French user using a to be "--language=fr" arg won't get what they expect.

Now something really weird:

Someone told me I should contact GOG to propose them this one file
that's missing; because for many old games dubs they have the rights to sell,
but not the game itself.

> If some files are identical, there's syntax for adding symlinks (see
> tyrian.yaml).
That won't work accross packages, or one need to mess with a baldur-common 
package
and have the various dubs depend on this one; like it's done for 
spear-of-destiny.

Maybe these dubs aren't even co-installable; I too have never used gemrb.


PS: I've added a "step 0" in adding_a_game.mdwn to explain how to work from
the git version.

Alexandre



signature.asc
Description: This is a digitally signed message part.

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Thanks for the help. As discussed via private e-mail, I switched to the
English Linux installer and executed the Mojo-Installer manually. All
files were unpacked into a separate directory and that's what g-d-p can
use to repack everything into a *.deb file now.

I also use compression now which reduces the size of the archive by 1 GB
(2,6 GB uncompressed) but it is terribly slow on my machine. The game
works and I have tested the result. I intend to add the German version
too before I push to Git master.

I think it makes sense to clone this bug report into one for each
supported gemrb game. Do you agree?

Cheers,

Markus



signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 02/10/15 19:25, Markus Koschany wrote:
> I also use compression now which reduces the size of the archive by 1 GB
> (2,6 GB uncompressed) but it is terribly slow on my machine.

You could try gzip, like comi.yaml?

# gzip -1 saves significant size; gzip -6 only saves an extra 10M and
# takes 1.5x time; xz is really slow
compress_deb: [-Zgzip, -z1]

> I think it makes sense to clone this bug report into one for each
> supported gemrb game. Do you agree?

Yes if there a reasonable number (up to 10, say). If there are lots, I
would suggest not bothering with bug reports for any that you don't have
or don't intend to package.

(I know nothing about gemrb.)

S

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Am 02.10.2015 um 21:12 schrieb Simon McVittie:
> On 02/10/15 19:25, Markus Koschany wrote:
>> I also use compression now which reduces the size of the archive by 1 GB
>> (2,6 GB uncompressed) but it is terribly slow on my machine.
> 
> You could try gzip, like comi.yaml?
> 
> # gzip -1 saves significant size; gzip -6 only saves an extra 10M and
> # takes 1.5x time; xz is really slow
> compress_deb: [-Zgzip, -z1]

Oh yeah, much better and only +170 MB. Thanks.

>> I think it makes sense to clone this bug report into one for each
>> supported gemrb game. Do you agree?
> 
> Yes if there a reasonable number (up to 10, say). If there are lots, I
> would suggest not bothering with bug reports for any that you don't have
> or don't intend to package.

GemRB supports: Baldurs Gate 1 + 2, Icewind Dale 1+2 and Planescape:
Torment. I intend to package the data of the other four games too but
this requires more time. So if you want to release a new version of
g-d-p you can include Baldur's Gate 1 but don't have to wait for the
other ones.

P.S.: g-d-p is currently broken in Git master. :>
Compilation works but when I run for instance

game-data-packager enemy-territory


Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
  File
"/usr/share/games/game-data-packager/game_data_packager/__main__.py",
line 21, in 
run_command_line()
  File
"/usr/share/games/game-data-packager/game_data_packager/__init__.py",
line 1054, in run_command_line
with game.construct_task() as task:
  File
"/usr/share/games/game-data-packager/game_data_packager/__init__.py",
line 832, in construct_task
self.load_file_data()
  File
"/usr/share/games/game-data-packager/game_data_packager/__init__.py",
line 829, in load_file_data
), (self.shortname, wanted.name)
AssertionError: ('enemy-territory', 'license.rtf')


Markus



signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 02/10/15 20:55, Markus Koschany wrote:
> P.S.: g-d-p is currently broken in Git master. :>
...
> line 829, in load_file_data
> ), (self.shortname, wanted.name)
> AssertionError: ('enemy-territory', 'license.rtf')

Should now be fixed in git. I had split out detailed information about
the required files for each package into separate files, which don't
need to be loaded until g-d-p starts working on specific packages, but
then forgot to tell the packaging to install them.

That feature could still do with some improvement - some of the *.json
that are loaded for things like --help are still rather large, and it
might be better to install a .zip rather than > 600 smallish files,
after benchmarking whether it's faster compressed or uncompressed - but
it's a start.

(FYI, you can run g-d-p with "make && ./run enemy-territory ..." to use
the version under development without installing it.)

S

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

I've added the "--collisions=rename" arg to make-template
when innoextract >= 1.5 is detected.



size_and_md5: |
  18966 e4af432b9f2658b694865c8ed06c1ef0 beyond zork/beyondzo.mg1
  276480743c4a97b9c85d89d85d4f1e8e8bfe85 beyond zork/data/beyondzo.dat
  15878883c222e625ba2ef900462bd52277a7b3 beyond zork/gfw_high.ico
  1279488   f03981332cf9b4284a26784a82e60080 graphic mode setup.exe#g2
  1279488   f03981332cf9b4284a26784a82e60080 graphic mode setup.exe#g3
  1279488   f03981332cf9b4284a26784a82e60080 graphic mode setup.exe#g4
  1279488   f03981332cf9b4284a26784a82e60080 graphic mode setup.exe#g5
  1279488   f03981332cf9b4284a26784a82e60080 graphic mode setup.exe#g6

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

Le mardi 29 septembre 2015, 20:13:30 Markus Koschany a écrit :
> Am 29.09.2015 um 13:05 schrieb Alexandre Detiste:
> > 2015-09-29 11:21 GMT+02:00 Markus Koschany :
> [...]
> > I own these from GOG too. When the packaging of the GOG version is
> > done it should be tested against the original physical media too;
> > to avoid surprises; like GOG version of Zork Inquisitor shipping the
> > obsolete Z-Engine and some bits of it's assets slipping in package 
> > definition;
> > or GOG skipping some *.bak from the retail CD of Theme Hospital...
> 
> Absolutely agreed. This should be tested against "real" physical media
> releases of the games. I intend to provide patches for the gog.com
> versions first. Let's see if there is demand and feedback for more.
 
It's a so-called long-tail problem; packaging the GOG + Steam
version would please most people & with a bit of luck work for the original 
media too.

Packaging only the English version at first will also please most people.

> >> Unfortunately g-d-p hardcodes the platform-priority for lgogdownloader
> >> to linux,windows.
> > 
> > I don't understand why you feel this is unfortunate, maybe you mistook
> > this parameter with " --platform" which does something else.
> 
> Providing a sane default is fine. I only find it unfortunate that I
> can't override the values in my yaml file.
>
> I would like to add
> 
> platform: windows

I'm still thinking of use case for this : 
- sometimes the windows archive is smaller that the linux one
- user may want to --save-dowloads /media/win/something
  & install the game again on his windows system ?

For the rest a .zip file is better because 
we don't need to call an external program.

> language: de
> 
> to the specific yaml file. But this is apparently not implemented yet.
> (Just a finding, nothing serious :) )

For now there's always a 1:1 match between the language of
the Debian package and the one of the GOG package; 
the 'de' .deb is built from the setup_*_german.exe etc...

I can't see how one e. g. could build a "fr" .deb from a "de" gog archive (?)

> Ok. I appreciate that we prioritize linux versions. But there is no
> runtime option to override the lgogdownloader behaviour, isn't it?
> I can't use
> game-data-packager baldurs-gate-1 --platform windows --language de

"--language xx" should well be added to G-D-P; having
to list possible package with --help & the run again
with --package game-$LANG-date is a bit cumbersome.

"--language xx" would have the seame meaning than for
lgogdownloader: "create package in this language or do nothing",
but would also work with freebies like Soltys or Dreamweb
but also Steam games. G-D-P would fail instead of packaging
an unwanted english version

This way someone can override the automagical behaviour of
LANGUAGE & LANG.

---

"--platform":
G-D-P should not show, but hide the complexity of all the
external tools that all have tens and tens of command line switches
("man arj" is nightmarish)

> > Now for the rest:
> > 
> > - G-D-P has not yet support for multipart GOG archives used for huge games,
> >   multipart archive are a lot of pain (see ARJ, unshield handling) by
> > themselves;
> >   I will add this soon.
done

> >   To avoid to download the same game again & again & again for tests I wrote
> >   a fake lgogdownloader, that can be symlinked in ~/bin .
> 
> Thanks. I usually download the provided archive once and then I run
> 
> game-data-packager  
> 
> and then I hope g-d-p will do the right thing. :)
> 
> Although I have 120 MBit to my disposal, I think this is more sane. :P

I don't like to waste either.

> When I run
> 
> game-data-packager baldurs-gate-1 setup_baldurs_gate_2.0.0.20.exe
> 
> I get this now:
> 
> game-data-packager baldurs-gate-1 setup_baldurs_gate_2.0.0.20.exe
> INFO:game-data-packager:identifying setup_baldurs_gate_2.0.0.20.exe
> WARNING:game-data-packager:found possible chitin.key
> but its size does not match:
>   file:
> /mnt/data/gog/baldurs_gate_the_original_saga/gdptmp/gdptmp.qfsdo9pg/tmp/setup_baldurs_gate_2.0.0.20.exe.d/app/Chitin.key
>   expected: 217559 bytes
>   found   : 238496 bytes
> INFO:game-data-packager:identifying
> /mnt/data/gog/baldurs_gate_the_original_saga/gdptmp/gdptmp.qfsdo9pg/tmp/setup_baldurs_gate_2.0.0.20.exe.d/app/movies/MOVIECD1.BIF
> INFO:game-data-packager:identifying
> /mnt/data/gog/baldurs_gate_the_original_saga/gdptmp/gdptmp.qfsdo9pg/tmp/setup_baldurs_gate_2.0.0.20.exe.d/app/movies/MOVIECD4.BIF
> WARNING:game-data-packager:found possible data/npcsound.bif
> but its size does not match:

I remember having read somewhere that someone else had a similar problem with 
this exact same file:

https://github.com/dscharrer/innoextract/issues/6#issuecomment-11473169
>>> In fact, if I don't use -L or manually delete the files only present in the 
>>> innoextract'd copy,
>>> GemRB prefers chitin.key over Chitin.key and segfaults while starting up.

There is a "chitin.key" & a "Chitin.key" ; but 

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

PS:

Innoextract v1.5 provides a solution for this.

This baldur gate game definitively needs custom coding
& to check for innoextract 1.5 at run-time;
either via some conditional check or aclass function that
get overriden in "game_data_packager/games/gemrb_common"
to create.


chitin.key = Baldur's Gate
Chitin.key = Baldur's Gate - Tales of the Sword Coast

tchet@antec:/tmp$ innoextract --version --quiet
innoextract 1.5

tchet@antec:/tmp$ innoextract -l  -I chitin.key 
/home/tchet/gdp/baldur/Baldur\'s\ Gate\ -\ The\ Original\ Saga\ 
\(1999\)/setup_baldurs_gate.exe
Listing "Baldur's Gate" - setup data version 5.2.3
 - "app/chitin.key" (212 KiB) - overwritten
 - "app/Chitin.key" (232 KiB)
Done.

tchet@antec:/tmp$ innoextract -l --collisions=rename -I chitin.key 
/home/tchet/gdp/baldur/Baldur\'s\ Gate\ -\ The\ Original\ Saga\ 
\(1999\)/setup_baldurs_gate.exe 
Listing "Baldur's Gate" - setup data version 5.2.3
 - "app/chitin.key" (212 KiB)
 - "app/Chitin.key#g2" (232 KiB)
Done.


from the man page:

"rename"  Rename  files  in the collision set by appending 
"#component", "@language" and/or "$id" where component and language are the
  file's unique component and language and id is the lowest number 
to make the filename unique. If the --default-language  is  specified,  the
  "@language" component is omitted if it matches the default 
language

signature.asc
Description: This is a digitally signed message part.

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 29/09/15 12:05, Alexandre Detiste wrote:
> Ultimately, this means G-D-P is the _one_ package that
> most needs to be team-maintained, as no-one can own
> all those games at once.

This is why I have tried to encourage bug reporters to provide full
information about their games, including multiple hashes (for better
ability to match up files between multiple reports), and all the files
that might possibly be useful in the CD/installation/whatever. If we
have more information than we strictly need from someone who owns
Hypothetical Monkey Island in Esperanto on 5½" disks, we can leave some
of it out of the .yaml; but if we don't have enough information, we
can't discover it without access to that (version of the) game.

Regards,
S

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

2015-09-29 14:21 GMT+02:00 Simon McVittie :
> This is why I have tried to encourage bug reporters to provide full
> information about their games, including multiple hashes (for better
> ability to match up files between multiple reports), and all the files
> that might possibly be useful in the CD/installation/whatever. If we
> have more information than we strictly need from someone who owns
> Hypothetical Monkey Island in Esperanto on 5½" disks, we can leave some
> of it out of the .yaml; but if we don't have enough information, we
> can't discover it without access to that (version of the) game.

Hi

That's right, that's why make-template is shipped in the .deb;
so end-users can use it without having to mess with git.

I'm sorry, but I feel I'm missing something between the lines.
Do you mean it's wrong to drop the *.exe in make-template ?

Regards,

Alexandre

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

2015-09-29 11:21 GMT+02:00 Markus Koschany :
>  On Thu, 29 Jan 2015 07:40:22 +0100 Alexandre Detiste
>  wrote:
>> Package: game-data-packager
>> Version: 39
>> Severity: wishlist
>> Tags: newcomer
>>
>> Supported games: Baldur's Gate and Icewind Dale series, Planescape:Torment.
>>
>
> I own all supported gemrb games from GOG. My first try was Baldur's Gate
> 1 and I pushed the .yaml file to a new branch "baldurs-gate".

I own these from GOG too. When the packaging of the GOG version is
done it should be tested against the original physical media too;
to avoid surprises; like GOG version of Zork Inquisitor shipping the
obsolete Z-Engine and some bits of it's assets slipping in package definition;
or GOG skipping some *.bak from the retail CD of Theme Hospital...

That also likely means to find a way to patch old CD's.

The people from the http://wiki.dotslashplay.it/ project seems
to be huge fan of this game, I'll ask them to test this.


Ultimately, this means G-D-P is the _one_ package that
most needs to be team-maintained, as no-one can own
all those games at once.


> GOG provides 5 different windows installers and 5 linux installers.
> Since "game-data-packager make-template" didn't work for the linux
> installer out-of-the-box, I used setup_baldurs_gate_2.0.0.20.exe and
> extracted the files with innoextract.

I'll try make-template too.


> Unfortunately g-d-p hardcodes the platform-priority for lgogdownloader
> to linux,windows.

I don't understand why you feel this is unfortunate, maybe you mistook
this parameter with " --platform" which does something else.


Using "--platform windows" was the easiest solution;
but some marketeer at GOG should know that we like
that they support linux even if we nitpick about
'details' like the chmod 666 stuff in /opt ;
or them providing 'i386' instead of 'all' packages
than can't be used on ARM computers etc...


GOG.is_native() is used in a way to avoid to request users to install
innoextract and then not even using it anyway because the linux
packages provided by GOG can be read as simple zip archives.


> I would like to override this option with a new tag like platform

An other reason for not having the platform: tag in the yaml
file is that GOG.com may decide to release more & more
linux packages after each GDP release,
so this checked at runtime.


> language

This is copied from package.lang in the python GameDataPackage
object; which already default to "en".

It's maybe not exactly the same flavor of ISO 639, so things
like pt_BR or en_GB may need some translation/lower/replace
if these were ever sold by GOG.


> Is there another way to
> force the download of the English windows installer

G-D-P will read the contents of $LANGUAGE
and the $LANG to pickup the best match;
and then call "lgogdownloader --language XX"

The only way to override this for now is this way:
   LANGUAGE=en game-data-packager kyrandia



Now for the rest:

- G-D-P has not yet support for multipart GOG archives used for huge games,
  multipart archive are a lot of pain (see ARJ, unshield handling) by
themselves;
  I will add this soon.

  To avoid to download the same game again & again & again for tests I wrote
  a fake lgogdownloader, that can be symlinked in ~/bin .

  
http://anonscm.debian.org/cgit/pkg-games/game-data-packager.git/tree/tools/fake_lgog.py

  I need to add multipart archive support here too & replace "cp"  by
a "ln" to avoid to fill up /tmp.

  I'll test this against Feeble Files

- baldurs-gate1-data should be a virtual package provided by
baldurs-gate1-en-data
baldurs-gate1-de-data
baldurs-gate1-fr-data
baldurs-gate1-pl-data
baldurs-gate1-ru-data

here is an example of another multilingual GOG game:
  
http://anonscm.debian.org/cgit/pkg-games/game-data-packager.git/tree/data/simon2.yaml

Having at first a yaml with only baldurs-gate1-en-data is OK ! ;
other languages can be added "later"; and there is the "missing_langs: []"
tag to make it show up on the dashboard.

"later": For example there seems to exist a russian version of "I have
no mouth..."
not sold by Steam/GOG. As I can't browse the web in russian; I'll
never locate it myself.


- There are dubious files in the package definition that can be trimmed:
   ie-ddraw-fix-bg.cmd
   sdb.cmd
   sdbu.cmd
   ddrawfix/readme.html

  these icons always show up as tiny & ugly

  map.pdf should be optional, unless if shown ingame (unlikely)

  gog*.ico should be optional or trimmed out


PS: nice idea to work on a branch for this.

Alexandre Detiste

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

 On Thu, 29 Jan 2015 07:40:22 +0100 Alexandre Detiste
 wrote:
> Package: game-data-packager
> Version: 39
> Severity: wishlist
> Tags: newcomer
> 
> Supported games: Baldur's Gate and Icewind Dale series, Planescape:Torment.
> 

I own all supported gemrb games from GOG. My first try was Baldur's Gate
1 and I pushed the .yaml file to a new branch "baldurs-gate".

GOG provides 5 different windows installers and 5 linux installers.
Since "game-data-packager make-template" didn't work for the linux
installer out-of-the-box, I used setup_baldurs_gate_2.0.0.20.exe and
extracted the files with innoextract.

Unfortunately g-d-p hardcodes the platform-priority for lgogdownloader
to linux,windows. I would like to override this option with a new tag
like platform or platform-priority and language. Is there another way to
force the download of the English windows installer or might it be even
better to add support for unpacking the linux installer?

Regards,

Markus




signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Simon McVittie]

On 29/09/15 13:47, Alexandre Detiste wrote:
> I'm sorry, but I feel I'm missing something between the lines.
> Do you mean it's wrong to drop the *.exe in make-template ?

I think it's fine to drop files that are identifiably junk, but if in
doubt we should include the files.

For example, a Windows .exe is usually unwanted - except when it's a
self-extracting archive (or other file with a readable payload) that
contains something we want :-)

Perhaps we should include the files but "comment them out"?

S

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Markus Koschany]

Am 29.09.2015 um 13:05 schrieb Alexandre Detiste:
> 2015-09-29 11:21 GMT+02:00 Markus Koschany :
[...]
> I own these from GOG too. When the packaging of the GOG version is
> done it should be tested against the original physical media too;
> to avoid surprises; like GOG version of Zork Inquisitor shipping the
> obsolete Z-Engine and some bits of it's assets slipping in package definition;
> or GOG skipping some *.bak from the retail CD of Theme Hospital...

Absolutely agreed. This should be tested against "real" physical media
releases of the games. I intend to provide patches for the gog.com
versions first. Let's see if there is demand and feedback for more.

[...]

>> Unfortunately g-d-p hardcodes the platform-priority for lgogdownloader
>> to linux,windows.
> 
> I don't understand why you feel this is unfortunate, maybe you mistook
> this parameter with " --platform" which does something else.

Providing a sane default is fine. I only find it unfortunate that I
can't override the values in my yaml file.

I would like to add

platform: windows
language: de

to the specific yaml file. But this is apparently not implemented yet.
(Just a finding, nothing serious :) )

[...]
> An other reason for not having the platform: tag in the yaml
> file is that GOG.com may decide to release more & more
> linux packages after each GDP release,
> so this checked at runtime.

Ok. I appreciate that we prioritize linux versions. But there is no
runtime option to override the lgogdownloader behaviour, isn't it?
I can't use
game-data-packager baldurs-gate-1 --platform windows --language de

or something like that?

[...]
>> Is there another way to
>> force the download of the English windows installer
> 
> G-D-P will read the contents of $LANGUAGE
> and the $LANG to pickup the best match;
> and then call "lgogdownloader --language XX"
> 
> The only way to override this for now is this way:
>LANGUAGE=en game-data-packager kyrandia

I see. I would prefer if this could be adjusted with yaml parameters.


> Now for the rest:
> 
> - G-D-P has not yet support for multipart GOG archives used for huge games,
>   multipart archive are a lot of pain (see ARJ, unshield handling) by
> themselves;
>   I will add this soon.
> 
>   To avoid to download the same game again & again & again for tests I wrote
>   a fake lgogdownloader, that can be symlinked in ~/bin .

Thanks. I usually download the provided archive once and then I run

game-data-packager  

and then I hope g-d-p will do the right thing. :)

Although I have 120 MBit to my disposal, I think this is more sane. :P

>   
> http://anonscm.debian.org/cgit/pkg-games/game-data-packager.git/tree/tools/fake_lgog.py
> 
>   I need to add multipart archive support here too & replace "cp"  by
> a "ln" to avoid to fill up /tmp.

I experienced the same issue. I use export TMP=path-to-more-free-space

[...]
> Having at first a yaml with only baldurs-gate1-en-data is OK ! ;
> other languages can be added "later"; and there is the "missing_langs: []"
> tag to make it show up on the dashboard.

I agree. My goal is to provide a working English version first.


> 
> PS: nice idea to work on a branch for this.

Yeah, I feel more comfortable with this solution. I only push to master
when I am sure that the work is in an acceptable state or can be fixed
in no time.

When I run

game-data-packager baldurs-gate-1 setup_baldurs_gate_2.0.0.20.exe

I get this now:

game-data-packager baldurs-gate-1 setup_baldurs_gate_2.0.0.20.exe
INFO:game-data-packager:identifying setup_baldurs_gate_2.0.0.20.exe
WARNING:game-data-packager:found possible chitin.key
but its size does not match:
  file:
/mnt/data/gog/baldurs_gate_the_original_saga/gdptmp/gdptmp.qfsdo9pg/tmp/setup_baldurs_gate_2.0.0.20.exe.d/app/Chitin.key
  expected: 217559 bytes
  found   : 238496 bytes
INFO:game-data-packager:identifying
/mnt/data/gog/baldurs_gate_the_original_saga/gdptmp/gdptmp.qfsdo9pg/tmp/setup_baldurs_gate_2.0.0.20.exe.d/app/movies/MOVIECD1.BIF
INFO:game-data-packager:identifying
/mnt/data/gog/baldurs_gate_the_original_saga/gdptmp/gdptmp.qfsdo9pg/tmp/setup_baldurs_gate_2.0.0.20.exe.d/app/movies/MOVIECD4.BIF
WARNING:game-data-packager:found possible data/npcsound.bif
but its size does not match:


Apparently the files don't match, although I used the make-template
feature of g-d-p. It seems like innoextract runs are not reproducible...

Markus






signature.asc
Description: OpenPGP digital signature

Bug#776540: game-data-packager: please add support for games working with gemrbd engine

[Alexandre Detiste]

Package: game-data-packager
Version: 39
Severity: wishlist
Tags: newcomer

Supported games: Baldur's Gate and Icewind Dale series, Planescape:Torment.

copied from GemRB description:



GemRB (Game Engine Made with preRendered Background) is a new implementation of
the original Infinity Engine (the one of Baldur's Gate, Icewind Dale,
Planescape: Torment, ...) to Linux/Unix, MacOS X and Windows with some
enhancements.

It means that you either need some of the original game's data somewhere on
your harddisk, or you can try to use the data from the Dragonlance Total
Conversion project.

The original game data has to be installed on a windows partition and mounted
to your Linux/Unix filesystem, installed on windows and then copied to your
filesystem, installed with WINE or extracted manually from the CDs using the
tool `unshield'.



-- System Information:
Debian Release: 8.0
  APT prefers testing
  APT policy: (501, 'testing'), (450, 'unstable'), (400, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.18.0-trunk-amd64 (SMP w/6 CPU cores)
Locale: LANG=fr_BE.UTF-8, LC_CTYPE=fr_BE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages game-data-packager depends on:
ii  fakeroot1.20.2-1
ii  python3 3.4.2-2
ii  python3-debian  0.1.25
ii  python3-yaml3.11-2
pn  python3:any none

game-data-packager recommends no packages.

Versions of packages game-data-packager suggests:
ii  binutils  2.25-4
ii  cdparanoia3.10.2+debian-11
ii  dynamite  0.1.1-2
ii  gcc   4:4.9.2-1
pn  lgc-pgnone
ii  lhasa [lzh-archiver]  0.2.0+git3fe46-1
ii  make  4.0-8.1
ii  unzip 6.0-14
ii  vorbis-tools  1.4.0-6

-- no debconf information


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org