Re: LeMaker HiKey960 boards arrived (was: Re: [Reproducible-builds] arm64 reproducible build network)

[Holger Levsen]

On Tue, Jun 13, 2017 at 01:18:27AM +0200, Axel Beckert wrote:
> Long story short: 14 months passed and we didn't get any LeMaker Cello
> boards (http://www.96boards.org/product/cello/). But around pentecost
> we got a mail that we will get LeMaker HiKey960 boards
> (http://www.96boards.org/product/hikey960/) instead and that they
> more or less already got shipped.
> 
> The boards were sponsored by Hewlett Packard Enterprise
> (https://www.hpe.com/) and will be hosted by the students association
> SOSETH (https://sos.ethz.ch/) at ETH Zurich (https://www.ethz.ch/).
 
awesome, thanks to everyone involved!

> But the most severe implication of this switch from a EE to a CE board
> I noticed only at home when I got my hands dirty on one of the boards:
> The Consumer Edition does not have any Ethernet RJ45 port, only wifi.
> :-(
 
eeks

> I don't think we want to rely on wifi, so we likely will also need 8x
> USB 3.0 (A, not C) Gigabit Ethernet adapters, probably around 20€-30€
> each. *sigh*

yup.

> > For those we will need to get SSDs too, so leaving some funds for that
> > would be useful.
> 
> They have an PCIe M.2 slot, but also 32 GB "UFS Flash" on board. The
> current idea (by Holger) is to add a 120GB/128GB M.2 SSD to each board
> for additional storage.

and format those 120gb so that we'll use 90gb of it for /srv and keep the
rest unused for the SSD to wear out…

> I've tried to get Debian installed on one of these HiKey960 boards.

did you manage? :)

> * Flashing a new OS needs to be done over the USB C OTG socket with
>   Android style fastboot/adb methods. :-( As far as I understand it,
>   that means I have to flash an image and can't use the Debian
>   Installer for that. (Please correct me if I'm wrong.)

eeks. I'll leave this for others to shed a light on… 

> * As I don't have an appropriate USB-A to USB-C cable I'll first have
>   to buy one. Afterwards I can see if I can get anything working on
>   these boards.
 
more sigh :)

> So far for today. More maybe tomorrow.

Thanks a lot for all your work so far!


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

LeMaker HiKey960 boards arrived (was: Re: [Reproducible-builds] arm64 reproducible build network)

[Axel Beckert]

Hi,

Holger Levsen wrote on 24 Mar 2016:
> On Wed, Mar 23, 2016 at 06:40:09PM -0700, Vagrant Cascadian wrote:
> > There's about US$1200 left.
> > 
> > To follow through with number of boards in the original proposal, I
> > should probably get at least 3-4 more boards... or should we branch out
> > into an arm64 network with the remaining funds?
> 
> tbm will give us something like 4 cello boards (iirc), but it will take some
> months as they are only scheduled in the 2nd quarter of 2016. These are
> to be hosted at Xtaran.

Long story short: 14 months passed and we didn't get any LeMaker Cello
boards (http://www.96boards.org/product/cello/). But around pentecost
we got a mail that we will get LeMaker HiKey960 boards
(http://www.96boards.org/product/hikey960/) instead and that they
more or less already got shipped.

The boards were sponsored by Hewlett Packard Enterprise
(https://www.hpe.com/) and will be hosted by the students association
SOSETH (https://sos.ethz.ch/) at ETH Zurich (https://www.ethz.ch/).

So in general I'd say: 8 smaller, but more modern boards are better
than no boards at all. :-)

But I nevertheless wasn't really sure how well it will work out for us
with that model swap. But at the time I got informed about the swap,
the boards seemed already being shipped.

The main difference between the Cello and the HiKey/HiKey960 boards is
that the Cello board fulfils the 96boards Enterprise Edition (EE)
specifications while the Hikey/Hikey960 are 96boards Consumer Edition (CE)
boards.

But the most severe implication of this switch from a EE to a CE board
I noticed only at home when I got my hands dirty on one of the boards:
The Consumer Edition does not have any Ethernet RJ45 port, only wifi.
:-(

I don't think we want to rely on wifi, so we likely will also need 8x
USB 3.0 (A, not C) Gigabit Ethernet adapters, probably around 20€-30€
each. *sigh*

> For those we will need to get SSDs too, so leaving some funds for that
> would be useful.

They have an PCIe M.2 slot, but also 32 GB "UFS Flash" on board. The
current idea (by Holger) is to add a 120GB/128GB M.2 SSD to each board
for additional storage.

I've tried to get Debian installed on one of these HiKey960 boards.
The small fact sheet points you to the more or less empty
http://wiki.lemaker.org/HiKey960 -- it only contains a link to
http://wiki.lemaker.org/HiKey960:Differences_between_HiKey960_and_HiKey620

The HiKey620 boards are what was formerly known solely as "HiKey". But
the difference in there don't list all differences.

For example so far I noticed the following things not listed on that
page:

* No more jumpers but DIP switches.
* No more seems to be able to boot from SD card. At least I didn't
  manage it if I follow the instructions for the HiKey620 boards.

Later I found
https://github.com/96boards/documentation/tree/master/ConsumerEdition/HiKey960
and its sub pages.
https://github.com/96boards/documentation/blob/master/ConsumerEdition/HiKey960/GettingStarted/README.md
more or less confirms this:

* Flashing a new OS needs to be done over the USB C OTG socket with
  Android style fastboot/adb methods. :-( As far as I understand it,
  that means I have to flash an image and can't use the Debian
  Installer for that. (Please correct me if I'm wrong.)

* As I don't have an appropriate USB-A to USB-C cable I'll first have
  to buy one. Afterwards I can see if I can get anything working on
  these boards.

  At least the manufacture currently only lists AOSP as available for
  the Hikey960 while you also can get Linaro and "Debian" images for
  the Hikey620.

So far for today. More maybe tomorrow.

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Holger Levsen]

Hi Axel,

On Tue, Dec 06, 2016 at 09:05:32PM +0100, Axel Beckert wrote:
> Not that I want to undermine Martin's efforts, but I wonder if we
> should start by taking some more boards into account for arm64, too:

we actually have arm64 hardware available now, 8 machines with octocores
and 64 gb ram, we're just waiting to use postgresql instead of sqlite so
we can handle all the new jobs…


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Vagrant Cascadian]

On 2016-12-06, Axel Beckert wrote:
> Holger Levsen wrote:
>> On Tue, Dec 06, 2016 at 09:24:50AM -0800, Martin Michlmayr wrote:
>> > I heard from a Linaro contact that LeMaker wasn't able to fix the PCIE
>> > issue and was going to produce the board without, but that update is
>> > from October and there has been nothing since. :(
> […]
>> on the plus side we now got access to some moonshot-arm64 hardware, so
>> at least we'll be testing on arm64 soon. though for diversity reasons,
>> we absolutly still like to get access to LeMaker boards too!
>
> Not that I want to undermine Martin's efforts, but I wonder if we
> should start by taking some more boards into account for arm64, too:
>
> * Raspberry Pi 3 now works also with arm64:
>   https://wiki.debian.org/RaspberryPi3
>   https://people.debian.org/~stapelberg/raspberrypi3/
>   (Serial console only as of now, but that should be ok-ish for a
>   reproducible builds node.)

I just don't think 1GB of ram is worth it. I'd like to decommission the
1GB ram systems we have running in the armhf network...


> * Odroid C2 works fine with arm64, too. Not sure what's needed to get
>   all kernel modifications upstreamed:
>   https://www.armbian.com/odroid-c2/

I've got an odroid-c2 as part of the funded boards for
reproducible-builds, but upstream support for u-boot and kernel are
quite a ways off, last I looked.


> I'm aware of the connectivity requirements, but wrt. Raspberry Pi 3
> and Odroid C2 I wonder about disk space and speed requirements. Would
> e.g. a Class 10 UHS-1 8 GB microSD card suffice?

microSD is surely too slow.


> http://www.pollin.de/shop/dt/OTQ2NzcyOTk-/Computer_Informationstechnik/Speichermedien/microSD_SDHC_Speicherkarten/MicroSDHC_Card_VERBATIM_44004_8_GB.html
>
> Or are the probably faster but also more expensive eMMC devices
> preferred?

eMMC might be fast enough, not sure how easy it is to find a 64-128GB
eMMC, though.


> http://www.pollin.de/shop/dt/NTk0OTgxOTk-/Bauelemente_Bauteile/Entwicklerboards/Odroid/ODROID_C2_eMMC_Modul_8_GB_mit_Linux.html
>
> Or is even some real hard disk or SSD needed?

Many of the boards for the armhf build network use USB-to-SATA adapters
over USB2 or USB3, and a 128GB SSD, although typically leave 20-50%
unpartitioned to leave extra room for wear levelling, since these are
doing almost constant writes 24/7.


I have three pine64+ boards that I'd consider using, and mainline or
near-mainline support is coming along. I'm not sure if the lemaker hikey
will ever be good enough to use, but I have one and keep an eye on
upstream commits.


Although, any of the boards I have access to I might want to use as part
of a "test armhf chroots on arm64 hardware" installation, to get similar
kernel variation to i386 on amd64 hardware (and also not to tax my
bandwidth limits by downloading packages for another architecture).


live well,
  vagrant


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Axel Beckert]

Hi,

Holger Levsen wrote:
> On Tue, Dec 06, 2016 at 09:24:50AM -0800, Martin Michlmayr wrote:
> > I heard from a Linaro contact that LeMaker wasn't able to fix the PCIE
> > issue and was going to produce the board without, but that update is
> > from October and there has been nothing since. :(
[…]
> on the plus side we now got access to some moonshot-arm64 hardware, so
> at least we'll be testing on arm64 soon. though for diversity reasons,
> we absolutly still like to get access to LeMaker boards too!

Not that I want to undermine Martin's efforts, but I wonder if we
should start by taking some more boards into account for arm64, too:

* Raspberry Pi 3 now works also with arm64:
  https://wiki.debian.org/RaspberryPi3
  https://people.debian.org/~stapelberg/raspberrypi3/
  (Serial console only as of now, but that should be ok-ish for a
  reproducible builds node.)

* Odroid C2 works fine with arm64, too. Not sure what's needed to get
  all kernel modifications upstreamed:
  https://www.armbian.com/odroid-c2/

I could imagine providing the hardware and hosting for a few such
devices. (Maybe in addition to the hopefully somewhen arriving LeMaker
Cello boards.)

http://www.pollin.de/shop/dt/ODA1OTgxOTk-/Bauelemente_Bauteile/Entwicklerboards/Odroid/ODROID_C2_Einplatinen_Computer_1_5_GHz_QuadCore_2_GB_RAM_4x_USB.html
http://www.pollin.de/shop/dt/OTQxNzkyOTk-/Bauelemente_Bauteile/Entwicklerboards/Raspberry_Pi/Raspberry_Pi_3_Model_B.html

I'm aware of the connectivity requirements, but wrt. Raspberry Pi 3
and Odroid C2 I wonder about disk space and speed requirements. Would
e.g. a Class 10 UHS-1 8 GB microSD card suffice?

http://www.pollin.de/shop/dt/OTQ2NzcyOTk-/Computer_Informationstechnik/Speichermedien/microSD_SDHC_Speicherkarten/MicroSDHC_Card_VERBATIM_44004_8_GB.html

Or are the probably faster but also more expensive eMMC devices
preferred?

http://www.pollin.de/shop/dt/NTk0OTgxOTk-/Bauelemente_Bauteile/Entwicklerboards/Odroid/ODROID_C2_eMMC_Modul_8_GB_mit_Linux.html

Or is even some real hard disk or SSD needed?

As a power supply e.g. this one might suffice:
http://www.pollin.de/shop/dt/MjUyODQ2OTk-/Stromversorgung/Ladegeraete/USB_Ladegeraete/4_fach_USB_Lader_LOGILINK_PA0096_weiss.html

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Holger Levsen]

Hi Martin,

On Tue, Dec 06, 2016 at 09:24:50AM -0800, Martin Michlmayr wrote:
> I heard from a Linaro contact that LeMaker wasn't able to fix the PCIE
> issue and was going to produce the board without, but that update is
> from October and there has been nothing since. :(

ouch & thanks for keeping us informed & getting back to them…

on the plus side we now got access to some moonshot-arm64 hardware, so
at least we'll be testing on arm64 soon. though for diversity reasons,
we absolutly still like to get access to LeMaker boards too!


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Martin Michlmayr]

* Martin Michlmayr  [2016-09-02 14:59]:
> * Martin Michlmayr  [2016-05-25 11:51]:
> > The LeMaker Cello web site still says "shipping in Q2".  I'll let you
> > know when that changes.
> 
> LeMaker sent out an update a few weeks ago saying that there is some
> PCIE problem they are still debugging before mass production can
> start.

Unfortunaetly, there's still no update.  I tried to contact LeMaker
several times but didn't get a reply.

I heard from a Linaro contact that LeMaker wasn't able to fix the PCIE
issue and was going to produce the board without, but that update is
from October and there has been nothing since. :(

-- 
Martin Michlmayr
http://www.cyrius.com/

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Holger Levsen]

On Fri, Sep 02, 2016 at 02:59:54PM -0700, Martin Michlmayr wrote:
> LeMaker sent out an update a few weeks ago saying that there is some
> PCIE problem they are still debugging before mass production can
> start.

thanks for the update, Martin! Still looking forward to use those
boards! :-)


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Martin Michlmayr]

* Martin Michlmayr  [2016-05-25 11:51]:
> The LeMaker Cello web site still says "shipping in Q2".  I'll let you
> know when that changes.

LeMaker sent out an update a few weeks ago saying that there is some
PCIE problem they are still debugging before mass production can
start.

-- 
Martin Michlmayr
http://www.cyrius.com/

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Holger Levsen]

Hi Martin,

On Wed, May 25, 2016 at 11:51:45AM -0700, Martin Michlmayr wrote:
> Right.  We (HPE) placed an order for some boards and they will be
> shipped to ETH Zurich.  It's 4 Cello boards for reproducible builds
> and some more boards for other use cases (maybe ci.debian.net).

looking much forward to those :)

> The LeMaker Cello web site still says "shipping in Q2".  I'll let you
> know when that changes.

thanks!

> Note that these are bare boards.  We'll also need (at least) RAM and
> SSDs.  I'll check if HPE can provide that.  If not, maybe you can
> request funds from Debian.

thanks for checking with HPE! I'll happily approach the DPL if that wont
work out…


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Martin Michlmayr]

* Holger Levsen  [2016-03-23 22:02]:
> tbm will give us something like 4 cello boards (iirc), but it will take some
> months as they are only scheduled in the 2nd quarter of 2016. These are
> to be hosted at Xtaran.

Right.  We (HPE) placed an order for some boards and they will be
shipped to ETH Zurich.  It's 4 Cello boards for reproducible builds
and some more boards for other use cases (maybe ci.debian.net).

The LeMaker Cello web site still says "shipping in Q2".  I'll let you
know when that changes.

Note that these are bare boards.  We'll also need (at least) RAM and
SSDs.  I'll check if HPE can provide that.  If not, maybe you can
request funds from Debian.

-- 
Martin Michlmayr
http://www.cyrius.com/

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Steven Chamberlain]

Hi,

I'm curious if anyone has tried using a network filesystem in this kind
of setup.  I would think, "diskless" boards sharing a NAS allows for
easier provisioning and probably cheaper storage by centralising it.
Though I don't know how that performs in practice?

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [Reproducible-builds] arm64 reproducible build network

[Holger Levsen]

Hi,

On Wed, Mar 23, 2016 at 06:40:09PM -0700, Vagrant Cascadian wrote:
> There's about US$1200 left.
> 
> To follow through with number of boards in the original proposal, I
> should probably get at least 3-4 more boards... or should we branch out
> into an arm64 network with the remaining funds?

tbm will give us something like 4 cello boards (iirc), but it will take some
months as they are only scheduled in the 2nd quarter of 2016. These are
to be hosted at Xtaran.

For those we will need to get SSDs too, so leaving some funds for that
would be useful.

And then some company also has offered us (privately to me so far) some
moonshot cartriges (which are 8 core, 64gb ram systems, each), but it's
also not clear when we get access to them. (The company will host them
for us.) Once we get them, we can _probably_ build armel, armhf and
arm64 on those, but I might be wrong here. And it might take some time.

If/when this moonshot hardware becomes reality your armhf build network
will still be useful, as we then can keep it for hw diversity.

Until then, I'm not convinced we should invest in arm64 hardware
(+testing), but rather improve the armhf setup. 

long story short: if you think more/different hw is useful in your
setup, go and use the funds for it. It might take some more months til
we can test arm64 and the armhf build network IMO is still too slow.


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

[Reproducible-builds] arm64 reproducible build network

[Vagrant Cascadian]

On 2016-03-06, Holger Levsen wrote:
> On Sonntag, 6. März 2016, Axel Beckert wrote:
>> I now wonder if a bunch of Raspberry Pi 3 -- since they have 64-bit
>> CPUs (IIRC an Allwinner A53T) -- would make us able to check
>> reproducible-builds for Debian arm64, too.
>
> as far as I know, yes. Though I would want to use some other arm64 boards as 
> well, to have hardware variety right from the start.
...
> I think we'd need 10-12 of such boards for a start, not sure if we still have 
> Debian funds to buy those (but I almost think so… Vagrant?) - providing we 
> would find someone interested+able to host these. Would you (Axel) be able 
> too?

There's about US$1200 left.

To follow through with number of boards in the original proposal, I
should probably get at least 3-4 more boards... or should we branch out
into an arm64 network with the remaining funds?

If that's the case, I'd probably need someone else to host the boards,
though I could proably host some initial experimentation just to get the
basic infrastructure set up. I could also experiment with using the
arm64 boards but building armhf packages, which seems like an
interesting middle-ground. Curious what folks think.


Could get a few odroid-c2 for ~$60 with usb-sata adapter, power
supply, and case. Mainline linux for similar boards exists... but not
for odroid-c2 explicitly.

The LeMaker Hikey goes for ~$100+accessories. Limited mainline support
exists. I already have one donated by LeMaker for experimenting with.

The Pine64 boards are currently going for ~$30 for the 2GB ram
variants. Not sure what the timeline on delivery is, though I have a few
coming for experimentation.

Seems like raspberry PI 3 doesn't yet really support arm64, and only has
1GB of ram. I don't think it's worth considering.

Cheap 120GB SSD costs around US$40-50 each.

So, you might be able to get 8-10 boards running with the remaining
funds...


The LeMaker Cello is about ~US$300+accessories, with dual native SATA,
and may come with 8GB ram out of the box, upgradable to 32GB or more:

  http://www.lenovator.com/product/103.html

Two to three LeMaker Cello might outperform a network of 8-10 lower spec
machines, and probably be a lot easier to maintain. Mainline linux
support for similar boards exists... that said, The Cello isn't actually
shipping yet (but soon?)... s...


live well,
  vagrant


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds