Re: [pygame] Closing issue 211 with big-endian CPU test

[Lenard Lindstrom]

Hi,

On 15-10-25 04:19 PM, Ian Mallett wrote:
On Sun, Oct 25, 2015 at 4:27 PM, Lenard Lindstrom > wrote:


Is there a need to continue big-endian support?

​Bi-endian architectures are still around (ARM, MIPS in particular). 
AFAICT little endian won the war, but big endian is still around 
mainly for compatibility. The Raspberry Pi, for example, is almost 
universally configured to little endian, despite being a bi processor.
If Pygame is built for one of these platforms configured as big-endian, 
its big-endian code will be tested then.




Also, notionally, programs could swap endian-ness when operating on 
certain memory regions for performance improvements. Though, I don't 
know of any modern programs that do that.

Pygame's endian support is set at compile time.


​

I mean, we leave in the big-endian specific code, but just leave
it untested. Worry about it only if the need arises.

I like this solution. Making endianness go both ways with a compile 
flag isn't really extra effort.


Lenard Lindstrom

​Ian

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

Ok. We can leave big endian support until we get testing up and going again
(if someone cares to test it).

One of the main reasons I spent so long on the launchpad stuff is to get
PPAs working.


So, I closed the bug with a remark about not supporting big endian.​

Re: [pygame] Closing issue 211 with big-endian CPU test

[Ian Mallett]

On Sun, Oct 25, 2015 at 4:27 PM, Lenard Lindstrom  wrote:

> Is there a need to continue big-endian support?

​Bi-endian architectures are still around (ARM, MIPS in particular). AFAICT
little endian won the war, but big endian is still around mainly for
compatibility. The Raspberry Pi, for example, is almost universally
configured to little endian, despite being a bi processor.

Also, notionally, programs could swap endian-ness when operating on certain
memory regions for performance improvements. Though, I don't know of any
modern programs that do that.
​

> I mean, we leave in the big-endian specific code, but just leave it
> untested. Worry about it only if the need arises.
>
I like this solution. Making endianness go both ways with a compile flag
isn't really extra effort.

Lenard Lindstrom

​Ian

Re: [pygame] Closing issue 211 with big-endian CPU test

[Lenard Lindstrom]

Is there a need to continue big-endian support? I mean, we leave in the 
big-endian specific code, but just leave it untested. Worry about it 
only if the need arises.


Lenard Lindstrom

On 15-10-23 08:07 AM, René Dudfield wrote:

Hi,

Interesting. I didn't know you could do that ( and that, and that).

Thanks for the information.

By the way, the webhooks can be added through the bitbucket admin 
interface. Any of the admins can add them in there.


Am I able to add other users to the launchpad pygame team? I'd 
probably want a separate "pygame Robot" user which is able to upload 
to launchpad repos+PPAs.


I think for now, the code is building and all but one test is running. 
For me, this is enough for now. I'll concentrate on fixing the failing 
test in the lib/compat.py and getting a qemu build going. I have a 
mate running a freebsd PPC laptop that might be able to give pygame a 
build... I'll see how that goes. According to him the more recent 
debian/ubuntu releases don't work so well, so he had to change to freebsd.



best,






On Fri, Oct 23, 2015 at 11:12 AM, Thomas Kluyver > wrote:


On 23 October 2015 at 07:33, René Dudfield mailto:ren...@gmail.com>> wrote:

I guess we should incorporate the Debian changes, but I wanted
to check with Thomas if what I did was ok? (see revs 14-19).


Fine by me. I just copied the packaging from Debian/Ubuntu
originally anyway, so bringing in the updates is probably useful.

They use an interesting method to do a hg clone. I'm not sure
how to actually use this though... (see get-orig-source-hg in
rules). In rev 19 I changed that to get the latest rev from
hg. But I guess they want to specify a specific version.


AFAIK, the builders never actually run that target (and even if
they did, they don't have outside network access). Unlike
everything else in debian/rules, it's there for human packagers to
run to get the data and then upload it. Debian packaging can be
pretty ugly at times.

Since you already have a machine running a custom webhook when
there are new commits, perhaps it's best to use that to upload
directly to Launchpad for PPA builds, bypassing Launchpad's code
import and recipe builds:
https://help.launchpad.net/Packaging/PPA/Uploading

You'd need to work out how to get from a source tree to a Debian
source package locally, though. I'm sure GPG signatures are
involved somehow.

Best wishes,
Thomas

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

Hi,

Interesting. I didn't know you could do that ( and that, and that).

Thanks for the information.

By the way, the webhooks can be added through the bitbucket admin
interface. Any of the admins can add them in there.

Am I able to add other users to the launchpad pygame team? I'd probably
want a separate "pygame Robot" user which is able to upload to launchpad
repos+PPAs.

I think for now, the code is building and all but one test is running. For
me, this is enough for now. I'll concentrate on fixing the failing test in
the lib/compat.py and getting a qemu build going. I have a mate running a
freebsd PPC laptop that might be able to give pygame a build... I'll see
how that goes. According to him the more recent debian/ubuntu releases
don't work so well, so he had to change to freebsd.


best,






On Fri, Oct 23, 2015 at 11:12 AM, Thomas Kluyver  wrote:

> On 23 October 2015 at 07:33, René Dudfield  wrote:
>
>> I guess we should incorporate the Debian changes, but I wanted to check
>> with Thomas if what I did was ok? (see revs 14-19).
>>
>
> Fine by me. I just copied the packaging from Debian/Ubuntu originally
> anyway, so bringing in the updates is probably useful.
>
>
>> They use an interesting method to do a hg clone. I'm not sure how to
>> actually use this though... (see get-orig-source-hg in rules). In rev 19 I
>> changed that to get the latest rev from hg. But I guess they want to
>> specify a specific version.
>>
>
> AFAIK, the builders never actually run that target (and even if they did,
> they don't have outside network access). Unlike everything else in
> debian/rules, it's there for human packagers to run to get the data and
> then upload it. Debian packaging can be pretty ugly at times.
>
> Since you already have a machine running a custom webhook when there are
> new commits, perhaps it's best to use that to upload directly to Launchpad
> for PPA builds, bypassing Launchpad's code import and recipe builds:
> https://help.launchpad.net/Packaging/PPA/Uploading
>
> You'd need to work out how to get from a source tree to a Debian source
> package locally, though. I'm sure GPG signatures are involved somehow.
>
> Best wishes,
> Thomas
>

Re: [pygame] Closing issue 211 with big-endian CPU test

[Thomas Kluyver]

On 23 October 2015 at 07:33, René Dudfield  wrote:

> I guess we should incorporate the Debian changes, but I wanted to check
> with Thomas if what I did was ok? (see revs 14-19).
>

Fine by me. I just copied the packaging from Debian/Ubuntu originally
anyway, so bringing in the updates is probably useful.


> They use an interesting method to do a hg clone. I'm not sure how to
> actually use this though... (see get-orig-source-hg in rules). In rev 19 I
> changed that to get the latest rev from hg. But I guess they want to
> specify a specific version.
>

AFAIK, the builders never actually run that target (and even if they did,
they don't have outside network access). Unlike everything else in
debian/rules, it's there for human packagers to run to get the data and
then upload it. Debian packaging can be pretty ugly at times.

Since you already have a machine running a custom webhook when there are
new commits, perhaps it's best to use that to upload directly to Launchpad
for PPA builds, bypassing Launchpad's code import and recipe builds:
https://help.launchpad.net/Packaging/PPA/Uploading

You'd need to work out how to get from a source tree to a Debian source
package locally, though. I'm sure GPG signatures are involved somehow.

Best wishes,
Thomas

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

So this is a pretty long, boring, and quite frankly disturbing story on one
dark late night...

tldr; launchpad builds are working again, but it's still failing a test.
Also, we have arm builds now on the PPA :) However, launchpad does not do
PPC builds for PPAs yet... So qemu it is!






Well, now the long version.

I tried to get the launchpad building again... and it did start to build,
but kept failing. Then I tried to get the latest debian files from debian
(control/rules etc), and merged+committed+pushed them to the packaging bzr
repo.

https://code.launchpad.net/~pygame/+recipe/pygame-daily

Unfortunately this didn't help either.


Then I noticed it was the upload failing, and the version numbers were
different. It thought the new package had an older version than the new
one. I couldn't figure out how to change this, so I deleted the old
packages in the PPA. Then it started building again! The source package at
least was uploaded, and x86+amd64 began to build.


Then I reverted packaging back to rev 13, and tried to build again. So now
there are log files for both types of builds.

I guess we should incorporate the Debian changes, but I wanted to check
with Thomas if what I did was ok? (see revs 14-19).

...

Then I tried the build again, and it wouldn't upload, because the version
number was the same. So I once again deleted the packages in the PPA, and
went back to rev 19 with the Debian changes.

Note the latest Debian changes made the version name like this with the
repo rev and the packaging rev:
"pygame - 1.9.2~pre~r3348+1+21~ubuntu15.04.1"

But the older one (rev 13) does not have the repo version in it, which of
course means that we need to delete the packages every time, otherwise they
won't upload.
"pygame - 1.9.2~pre~ubuntu+1+22~ubuntu15.04.1"



But there were LOTS of tests failures with how I integrated the Debian
changes. I likely did lots wrong.

I tried to fix the lib/compat.py test error in pygame hg, and try to
rebuild with rev 13 of packaging.

"



Interestingly Debian also builds for powerpc with the experimental build.
But that uses an older hg revision.

https://packages.debian.org/source/experimental/pygame
https://packages.debian.org/experimental/python-pygame


They use an interesting method to do a hg clone. I'm not sure how to
actually use this though... (see get-orig-source-hg in rules). In rev 19 I
changed that to get the latest rev from hg. But I guess they want to
specify a specific version.


Note, it's a *horrible and long build process* at the moment. Because the
code has to go -> bitbucket -> web hook -> pull from bitbucket -> create
new local git repo -> push git repo to github -> manually tell launchpad to
pull from github -> delete packages -> tell launchpad to build again ->
wait for source build -> wait for binary builds. (15-20 minute turn
around). Oh, and the delete package thing doesn't always work right away,
so that can add another 30-45 minutes on top.


Then I found out that resetting the git repo to have only one commit makes
the bzr import always have a revno of 1. So this is the cause of it
thinking the files are always the same. A horrible hack would be to
automagically add N number of commits extra on top of the git... um, ewww.
Or some other hack perhaps.

Then I found you could use {revdate}+{revtime} in the recipe. (actually the
docs are wrong and you can use {date}+{time}, but {time} contains the date).
https://help.launchpad.net/Packaging/SourceBuilds/Recipes


I'm not sure how to emulate the fs encoding as ANSI_X3.4-1968 on ubuntu.
But that might make debugging easier.

Then I fall asleep finally after coffee wears off...


After waking up this morning I noticed this message from someone in
Launchpad support.
"I've enabled armhf builds on that recipe's PPA (
https://launchpad.net/~pygame/+archive/ubuntu/pygame-dev
). powerpc
builds aren't available for PPAs yet, though we're currently testing
infrastructure to provide them."


best,








# the updated mirror script that actually works.​..
cd /home/pygame/pygame
echo "sleeping 2"
#sleep 2
#hg pull && hg update && hg bookmark -r default master && hg push git+ssh://
g...@github.com/illume/pygame.git

hg pull && hg update && hg bookmark -r default master && hg bookmarks hg

cd /home/pygame/pygamegit/
rm -rf pygame
mkdir pygame
cd pygame
git init

cd /home/pygame/pygame
hg push ../pygamegit/pygame
cd ../pygamegit/pygame
git checkout hg
git checkout -b master
git checkout hg
git checkout master

git-truncate.sh `git log --all -n 1 --pretty=format:"%H"`

git push git+ssh://g...@github.com/illume/pygame.git --force

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

I asked here for PPC and ARM builds:
https://answers.launchpad.net/launchpad/+question/272739

There's other people asking for builds there... so maybe they'll enable it.
(like here: https://answers.launchpad.net/launchpad/+question/272599 )


best,



On Thu, Oct 22, 2015 at 4:15 PM, René Dudfield  wrote:

> Oh damn.
>
> I went through a fair bit of trouble to get launchpad working again so it
> could even pull from github. It turns out no one is maintaining bzr bazaar
> anymore, and it does not handle the hg to git extra fields.
>
> I had to truncate the git history, which removes any HG:rename fields,
> which launchpad chokes on.
>
> Here's the scripts in case anyone else needs them.
>
>
>
> #git-truncate.sh
> #!/bin/bash
> git log -1 > /tmp/git-log-msg
> git checkout --orphan temp $1
> #git commit -m "Truncated history"
> git commit -F /tmp/git-log-msg
> git rebase --onto temp $1 master
> git branch -D temp
>
>
>
> #mirror-pygame.sh
> cd /home/pygame/pygame
> echo "sleeping 2"
> #sleep 2
> #hg pull && hg update && hg bookmark -r default master && hg push
> git+ssh://g...@github.com/illume/pygame.git
>
>
> cd /home/pygame/pygamegit/
> rm -rf pygame
> mkdir pygame
> cd pygame
> git init
>
> cd /home/pygame/pygame
> hg push ../pygamegit/pygame
> cd ../pygamegit/pygame
> git checkout hg
> git checkout -b master
> git checkout hg
> git checkout master
>
> git-truncate.sh `git log --all -n 1 --pretty=format:"%H"`
>
> git push git+ssh://g...@github.com/illume/pygame.git --force
>
>
>
> On Thu, Oct 22, 2015 at 2:54 PM, Thomas Kluyver  wrote:
>
>> On 22 October 2015 at 13:07, René Dudfield  wrote:
>>
>>> Launchpad builds many different architectures, including on powerpc.
>>> https://launchpad.net/builders
>>>
>>
>> PPA builds are only on amd64 and i386 by default. The checkboxes for
>> other architectures are greyed out in the PPA admin page. I guess you can
>> get an admin to approve powerpc builds, because some other PPA have powerpc
>> builds, but I have no idea who to contact about that.
>>
>> Thomas
>>
>
>

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

Oh damn.

I went through a fair bit of trouble to get launchpad working again so it
could even pull from github. It turns out no one is maintaining bzr bazaar
anymore, and it does not handle the hg to git extra fields.

I had to truncate the git history, which removes any HG:rename fields,
which launchpad chokes on.

Here's the scripts in case anyone else needs them.



#git-truncate.sh
#!/bin/bash
git log -1 > /tmp/git-log-msg
git checkout --orphan temp $1
#git commit -m "Truncated history"
git commit -F /tmp/git-log-msg
git rebase --onto temp $1 master
git branch -D temp



#mirror-pygame.sh
cd /home/pygame/pygame
echo "sleeping 2"
#sleep 2
#hg pull && hg update && hg bookmark -r default master && hg push git+ssh://
g...@github.com/illume/pygame.git


cd /home/pygame/pygamegit/
rm -rf pygame
mkdir pygame
cd pygame
git init

cd /home/pygame/pygame
hg push ../pygamegit/pygame
cd ../pygamegit/pygame
git checkout hg
git checkout -b master
git checkout hg
git checkout master

git-truncate.sh `git log --all -n 1 --pretty=format:"%H"`

git push git+ssh://g...@github.com/illume/pygame.git --force



On Thu, Oct 22, 2015 at 2:54 PM, Thomas Kluyver  wrote:

> On 22 October 2015 at 13:07, René Dudfield  wrote:
>
>> Launchpad builds many different architectures, including on powerpc.
>> https://launchpad.net/builders
>>
>
> PPA builds are only on amd64 and i386 by default. The checkboxes for other
> architectures are greyed out in the PPA admin page. I guess you can get an
> admin to approve powerpc builds, because some other PPA have powerpc
> builds, but I have no idea who to contact about that.
>
> Thomas
>

Re: [pygame] Closing issue 211 with big-endian CPU test

[Thomas Kluyver]

On 22 October 2015 at 13:07, René Dudfield  wrote:

> Launchpad builds many different architectures, including on powerpc.
> https://launchpad.net/builders
>

PPA builds are only on amd64 and i386 by default. The checkboxes for other
architectures are greyed out in the PPA admin page. I guess you can get an
admin to approve powerpc builds, because some other PPA have powerpc
builds, but I have no idea who to contact about that.

Thomas

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

Oh!  Of course... I forgot... Before someone goes off to try to build with
qemu...

Launchpad builds many different architectures, including on powerpc.
https://launchpad.net/builders

There's currently a problem with launchpad importing the git branch... that
I'm trying to fix right now.
https://code.launchpad.net/~pygame/pygame/trunk


best,



On Thu, Oct 22, 2015 at 1:57 PM, René Dudfield  wrote:

> Hi,
>
> you can test with qemu emulating a powerpc machine running debian.
> http://create.stephan-brumme.com/big-endian/
> http://wiki.scummvm.org/index.php/HOWTO-Debug-Endian-Issues
>
> I haven't tried it in a few years, but it did work quite easily.
>
> cu
>
>
>

Re: [pygame] Closing issue 211 with big-endian CPU test

[René Dudfield]

Hi,

you can test with qemu emulating a powerpc machine running debian.
http://create.stephan-brumme.com/big-endian/
http://wiki.scummvm.org/index.php/HOWTO-Debug-Endian-Issues

I haven't tried it in a few years, but it did work quite easily.

cu

Re: [pygame] Closing issue 211 with big-endian CPU test

[Jake b]

Or /r/pygame

-- 
Jake

Re: [pygame] Closing issue 211 with big-endian CPU test

[Michael Lutinsky]

I’m half tempted to take you up on the offer just for the challenge of it, but 
my time isn’t so free either.

Would it be a good idea to post this request on the pygame.org site where 
others could find it? There must be someone who’d be happy to do this.

~ Michael 



> Lenard committed changes that resolve issue 211 for x86 machines. To close 
> issue 211 in Bitbucket, we need to confirm that Lenard's fix doesn't mixed up 
> the color channels when saving PNG and JPEG images on big-endian hardware. I 
> have an iMac G4 (which is big-endian), but I don't have the time & patience 
> to figure out all of the makefile, environment variables, and compiler 
> argument black magic to compile the latest pygame code on it. If someone (in 
> the USA or Canada) wants to do the chore of confirming that issue 211's 
> changes work for big-endian machines too but lacks the necessary hardware to 
> test it, I'll put my iMac G4 in a box and ship it to that person. (No need to 
> send it back.) Just let me know if you want to volunteer.
> 
> Jason
>