Re: [Nix-dev] Multiple ssh substituter hosts

[zimbatm]

Yes that's it. S3 should be useable as binary cache out of the box unless
authentication is required. In that case a proxy is required to transform
S3 signatures into basic or network based authorisation.

Within AWS, the VPC endpoint feature might be useful. I haven't tried it
yet and don't know if it's possible to avoid the signatures that way.
https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/ .

On Mon, 21 Nov 2016, 01:50 Bas van Dijk,  wrote:

> This seems to cover the first part about pushing nars from hydra to s3:
>
>
> https://github.com/NixOS/hydra/pull/118/commits/7c000713370bbb896e127503e8b0276c65eb76ae
>
> Now I just need to configure my private s3 bucket as a binary cache.
> Anyone know how to set that up?
>
> On 21 November 2016 at 02:18, Bas van Dijk  wrote:
>
> That seems like a good idea.
>
> Ideally my hydra service would upload the nars it builds to a private s3
> bucket. Then my machines need to use this private s3 bucket as a binary
> cache.
>
> Is there any documentation on how to do this?
>
> On 20 November 2016 at 22:34, zimbatm  wrote:
>
> It's probably expensive to open all the ssh connections. Depending on your
> use-case you might be better off uploading all the binaries to a single
> destination like S3.
>
> On Sun, 20 Nov 2016 at 20:16 Bas van Dijk  wrote:
>
> I see. Thanks for pointing me to the source.
>
> I also see that supporting more hosts is listed as a TODO:
>
>
> https://github.com/NixOS/nix/blob/master/src/download-via-ssh/download-via-ssh.cc#L18
>
> Hopefully that gets implemented some day!
>
> Cheers,
>
> Bas
>
> On 20 November 2016 at 20:13, zimbatm  wrote:
>
> Hi Bas,
>
> It looks like it's only taking the first host from the list[1]. I suppose
> the intention was to provide fallbacks but it didn't get implemented.
>
> [1]:
> https://github.com/NixOS/nix/blob/master/src/download-via-ssh/download-via-ssh.cc#L114
>
>
> On Sun, 20 Nov 2016 at 15:49 Bas van Dijk  wrote:
>
> Dear all,
>
> Is it possible to have multiple ssh-substituter-hosts, as in:
>
>   ssh-substituter-hosts = nix-ssh@host1 nix-ssh@host2
>
> The documentation[1] is not entirely clear on that.
>
> Bas
>
> [1] http://nixos.org/nix/manual/#ssec-ssh-substituter
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>
>
>
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Multiple ssh substituter hosts

[Bas van Dijk]

This seems to cover the first part about pushing nars from hydra to s3:


https://github.com/NixOS/hydra/pull/118/commits/7c000713370bbb896e127503e8b0276c65eb76ae

Now I just need to configure my private s3 bucket as a binary cache. Anyone
know how to set that up?

On 21 November 2016 at 02:18, Bas van Dijk  wrote:

> That seems like a good idea.
>
> Ideally my hydra service would upload the nars it builds to a private s3
> bucket. Then my machines need to use this private s3 bucket as a binary
> cache.
>
> Is there any documentation on how to do this?
>
> On 20 November 2016 at 22:34, zimbatm  wrote:
>
>> It's probably expensive to open all the ssh connections. Depending on
>> your use-case you might be better off uploading all the binaries to a
>> single destination like S3.
>>
>> On Sun, 20 Nov 2016 at 20:16 Bas van Dijk  wrote:
>>
>> I see. Thanks for pointing me to the source.
>>
>> I also see that supporting more hosts is listed as a TODO:
>>
>>   https://github.com/NixOS/nix/blob/master/src/download-via-ss
>> h/download-via-ssh.cc#L18
>>
>> Hopefully that gets implemented some day!
>>
>> Cheers,
>>
>> Bas
>>
>> On 20 November 2016 at 20:13, zimbatm  wrote:
>>
>> Hi Bas,
>>
>> It looks like it's only taking the first host from the list[1]. I suppose
>> the intention was to provide fallbacks but it didn't get implemented.
>>
>> [1]: https://github.com/NixOS/nix/blob/master/src/download-via-ss
>> h/download-via-ssh.cc#L114
>>
>>
>> On Sun, 20 Nov 2016 at 15:49 Bas van Dijk  wrote:
>>
>> Dear all,
>>
>> Is it possible to have multiple ssh-substituter-hosts, as in:
>>
>>   ssh-substituter-hosts = nix-ssh@host1 nix-ssh@host2
>>
>> The documentation[1] is not entirely clear on that.
>>
>> Bas
>>
>> [1] http://nixos.org/nix/manual/#ssec-ssh-substituter
>> ___
>> nix-dev mailing list
>> nix-dev@lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>
>>
>>
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Multiple ssh substituter hosts

[Bas van Dijk]

That seems like a good idea.

Ideally my hydra service would upload the nars it builds to a private s3
bucket. Then my machines need to use this private s3 bucket as a binary
cache.

Is there any documentation on how to do this?

On 20 November 2016 at 22:34, zimbatm  wrote:

> It's probably expensive to open all the ssh connections. Depending on your
> use-case you might be better off uploading all the binaries to a single
> destination like S3.
>
> On Sun, 20 Nov 2016 at 20:16 Bas van Dijk  wrote:
>
> I see. Thanks for pointing me to the source.
>
> I also see that supporting more hosts is listed as a TODO:
>
>   https://github.com/NixOS/nix/blob/master/src/download-via-
> ssh/download-via-ssh.cc#L18
>
> Hopefully that gets implemented some day!
>
> Cheers,
>
> Bas
>
> On 20 November 2016 at 20:13, zimbatm  wrote:
>
> Hi Bas,
>
> It looks like it's only taking the first host from the list[1]. I suppose
> the intention was to provide fallbacks but it didn't get implemented.
>
> [1]: https://github.com/NixOS/nix/blob/master/src/download-via-
> ssh/download-via-ssh.cc#L114
>
>
> On Sun, 20 Nov 2016 at 15:49 Bas van Dijk  wrote:
>
> Dear all,
>
> Is it possible to have multiple ssh-substituter-hosts, as in:
>
>   ssh-substituter-hosts = nix-ssh@host1 nix-ssh@host2
>
> The documentation[1] is not entirely clear on that.
>
> Bas
>
> [1] http://nixos.org/nix/manual/#ssec-ssh-substituter
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
>
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Multiple ssh substituter hosts

[zimbatm]

It's probably expensive to open all the ssh connections. Depending on your
use-case you might be better off uploading all the binaries to a single
destination like S3.

On Sun, 20 Nov 2016 at 20:16 Bas van Dijk  wrote:

I see. Thanks for pointing me to the source.

I also see that supporting more hosts is listed as a TODO:


https://github.com/NixOS/nix/blob/master/src/download-via-ssh/download-via-ssh.cc#L18

Hopefully that gets implemented some day!

Cheers,

Bas

On 20 November 2016 at 20:13, zimbatm  wrote:

Hi Bas,

It looks like it's only taking the first host from the list[1]. I suppose
the intention was to provide fallbacks but it didn't get implemented.

[1]:
https://github.com/NixOS/nix/blob/master/src/download-via-ssh/download-via-ssh.cc#L114


On Sun, 20 Nov 2016 at 15:49 Bas van Dijk  wrote:

Dear all,

Is it possible to have multiple ssh-substituter-hosts, as in:

  ssh-substituter-hosts = nix-ssh@host1 nix-ssh@host2

The documentation[1] is not entirely clear on that.

Bas

[1] http://nixos.org/nix/manual/#ssec-ssh-substituter
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Multiple ssh substituter hosts

[Bas van Dijk]

I see. Thanks for pointing me to the source.

I also see that supporting more hosts is listed as a TODO:


https://github.com/NixOS/nix/blob/master/src/download-via-ssh/download-via-ssh.cc#L18

Hopefully that gets implemented some day!

Cheers,

Bas

On 20 November 2016 at 20:13, zimbatm  wrote:

> Hi Bas,
>
> It looks like it's only taking the first host from the list[1]. I suppose
> the intention was to provide fallbacks but it didn't get implemented.
>
> [1]: https://github.com/NixOS/nix/blob/master/src/download-via-
> ssh/download-via-ssh.cc#L114
>
>
> On Sun, 20 Nov 2016 at 15:49 Bas van Dijk  wrote:
>
>> Dear all,
>>
>> Is it possible to have multiple ssh-substituter-hosts, as in:
>>
>>   ssh-substituter-hosts = nix-ssh@host1 nix-ssh@host2
>>
>> The documentation[1] is not entirely clear on that.
>>
>> Bas
>>
>> [1] http://nixos.org/nix/manual/#ssec-ssh-substituter
>> ___
>> nix-dev mailing list
>> nix-dev@lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] Multiple ssh substituter hosts

[zimbatm]

Hi Bas,

It looks like it's only taking the first host from the list[1]. I suppose
the intention was to provide fallbacks but it didn't get implemented.

[1]:
https://github.com/NixOS/nix/blob/master/src/download-via-ssh/download-via-ssh.cc#L114


On Sun, 20 Nov 2016 at 15:49 Bas van Dijk  wrote:

> Dear all,
>
> Is it possible to have multiple ssh-substituter-hosts, as in:
>
>   ssh-substituter-hosts = nix-ssh@host1 nix-ssh@host2
>
> The documentation[1] is not entirely clear on that.
>
> Bas
>
> [1] http://nixos.org/nix/manual/#ssec-ssh-substituter
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev