Re: [arch-general] Virtualization accross hardware border

2016-02-09 Thread Patrick Burroughs (Celti) 
On Tue, 9 Feb 2016 09:13:28 +0100
Wolfgang Mader  wrote:
> [snip]
> For my needs, I want to run "usual" software, specifically R, the 
> statistics language. Utlimately, I want to bind several physical
> hosts together to appear as one host on OS level, such that e.g. htop
> would show the total number of cores accross all bound boxes.

What you're describing is a classic clustering environment, like a
Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll
never be as smooth as "a single OS on a single logical host across
multiple machines", but it's as close as you'll get.

~Celti


pgpOSOFzvVcxg.pgp
Description: OpenPGP digital signature

Re: [arch-general] Virtualization accross hardware border

2016-02-09 Thread Wolfgang Mader 



On 02/09/2016 09:23 AM, Patrick Burroughs (Celti) wrote:

On Tue, 9 Feb 2016 09:13:28 +0100
Wolfgang Mader  wrote:

[snip]
For my needs, I want to run "usual" software, specifically R, the
statistics language. Utlimately, I want to bind several physical
hosts together to appear as one host on OS level, such that e.g. htop
would show the total number of cores accross all bound boxes.

What you're describing is a classic clustering environment, like a
Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll
never be as smooth as "a single OS on a single logical host across
multiple machines", but it's as close as you'll get.

~Celti

This sounds like the thing I am after.

Much apprechiated!

Re: [arch-general] Virtualization accross hardware border

2016-02-09 Thread Wolfgang Mader 



On 02/09/2016 12:30 AM, Damian Nowak wrote:

if I understand the offering of Amazons cloud service correctly, there, you can 
install an
OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and 
down just as
you need it.

Well, yes and no. You can scale up resources (e.g. increase RAM) but this 
requires a full
restart of your virtual machine. See:
https://serverfault.com/questions/591533/can-ec2-instances-dynamically-add-ram-while-running

Moreover, adding more and more resources will stop working at some point. 
That's why you'd
be looking to add several independent virtual machine running your application 
(Amazon
EC2), which connect to one big database (another Amazon EC2), and put all of 
those
computing machines in from of a load balancer (Amazon ELB).


While I can to this using e.g. KVM+qemu on a single machine, I want to be able 
to bind

together a bunch of machines such that they appear as a single big machine. Is 
there a way
to do this?

You can achieve it with both solutions, is it Amazon (EC2+ELB), or your own 
dedicated
server(s) where you'll be likely to use KVM for virtualization and HAProxy, 
nginx or other
solutions for load balancing.

 From a technological point of view, both ways are the same.


Thanks for your answer. I still try to wrap my head around the topic, so 
some of my assumptions my not hold. To me it sounds, that the proposed 
approach only works for web-apps running in a http-server, since the 
load balancing is done via HAProxy and nginx.


For my needs, I want to run "usual" software, specifically R, the 
statistics language. Utlimately, I want to bind several physical hosts 
together to appear as one host on OS level, such that e.g. htop would 
show the total number of cores accross all bound boxes.

Re: [arch-general] Virtualization accross hardware border

2016-02-09 Thread Christoph Gysin 
> For my needs, I want to run "usual" software, specifically R, the statistics
> language. Utlimately, I want to bind several physical hosts together to
> appear as one host on OS level, such that e.g. htop would show the total
> number of cores accross all bound boxes.

There are often optimized distributed solutions for specific tasks. So
if your goal is to run R across multiple machines, check out something
like http://www.distributedr.org

Chris
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3

Re: [arch-general] Virtualization accross hardware border

2016-02-08 Thread Alexander Terry 
You might want to look into docker: https://www.docker.com/

On Mon, Feb 8, 2016, 12:59 AM Wolfgang Mader 
wrote:

> Dear list,
>
> if I understand the offering of Amazons cloud service correctly, there,
> you can install an OS, say arch, on a virtualized machine and scale CPU,
> RAM, etc. freely up and down just as you need it. While I can to this
> using e.g. KVM+qemu on a single machine, I want to be able to bind
> together a bunch of machines such that they appear as a single big
> machine. Is there a way to do this? I my question clear enought to be
> answered?
>
> Thanks,
> Wolfgang
>

[arch-general] Virtualization accross hardware border

2016-02-08 Thread Wolfgang Mader 

Dear list,

if I understand the offering of Amazons cloud service correctly, there, 
you can install an OS, say arch, on a virtualized machine and scale CPU, 
RAM, etc. freely up and down just as you need it. While I can to this 
using e.g. KVM+qemu on a single machine, I want to be able to bind 
together a bunch of machines such that they appear as a single big 
machine. Is there a way to do this? I my question clear enought to be 
answered?


Thanks,
Wolfgang

Re: [arch-general] Virtualization accross hardware border

2016-02-08 Thread Damian Nowak 
> if I understand the offering of Amazons cloud service correctly, there, you 
> can install an
> OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and 
> down just as
> you need it. 

Well, yes and no. You can scale up resources (e.g. increase RAM) but this 
requires a full
restart of your virtual machine. See:
https://serverfault.com/questions/591533/can-ec2-instances-dynamically-add-ram-while-running

Moreover, adding more and more resources will stop working at some point. 
That's why you'd
be looking to add several independent virtual machine running your application 
(Amazon
EC2), which connect to one big database (another Amazon EC2), and put all of 
those
computing machines in from of a load balancer (Amazon ELB).

> While I can to this using e.g. KVM+qemu on a single machine, I want to be 
> able to bind
together a bunch of machines such that they appear as a single big machine. Is 
there a way
to do this?

You can achieve it with both solutions, is it Amazon (EC2+ELB), or your own 
dedicated
server(s) where you'll be likely to use KVM for virtualization and HAProxy, 
nginx or other
solutions for load balancing.

>From a technological point of view, both ways are the same.

-- 
Damian Nowak
CEO, Virtkick
www.virtkick.com