Re: [arch-general] Virtualization accross hardware border

[Patrick Burroughs (Celti)]

On Tue, 9 Feb 2016 09:13:28 +0100
Wolfgang Mader  wrote:
> [snip]
> For my needs, I want to run "usual" software, specifically R, the 
> statistics language. Utlimately, I want to bind several physical
> hosts together to appear as one host on OS level, such that e.g. htop
> would show the total number of cores accross all bound boxes.

What you're describing is a classic clustering environment, like a
Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll
never be as smooth as "a single OS on a single logical host across
multiple machines", but it's as close as you'll get.

~Celti


pgpOSOFzvVcxg.pgp
Description: OpenPGP digital signature

Re: [arch-general] Virtualization accross hardware border

[Wolfgang Mader]

On 02/09/2016 09:23 AM, Patrick Burroughs (Celti) wrote:

On Tue, 9 Feb 2016 09:13:28 +0100
Wolfgang Mader  wrote:

[snip]
For my needs, I want to run "usual" software, specifically R, the
statistics language. Utlimately, I want to bind several physical
hosts together to appear as one host on OS level, such that e.g. htop
would show the total number of cores accross all bound boxes.

What you're describing is a classic clustering environment, like a
Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll
never be as smooth as "a single OS on a single logical host across
multiple machines", but it's as close as you'll get.

~Celti

This sounds like the thing I am after.

Much apprechiated!

Re: [arch-general] Alternative init system proposal

[Ivan]

On Mon, 08 Feb 2016, Guus Snijders wrote:

> Op 8 feb. 2016 01:59 schreef "Ivan" :
> >
> > Hello, I have a proposal for Arch Linux developers and by mailing
> > on this list I would also appreciate feedback from non-developers that
> > use Arch Linux.
> > Note: I am not here to hate on the current status, nor
> > to disapprove of current Arch choices.
> >
> > So, to get to the point...
> > I would like to propose development and official support of an
> > alternative init system and service manager. My main preference would be
> > OpenRC + sysvinit. The combination of those two has shown to work
> > surprisingly well even with the current Arch's binary packages which are
> > compiled with systemd support/dependencies.
> 
> I love the idea, even just from a technical pov.
> However... It may be a bit much to ask from the developers to officially
> support another init system.
> 
> That said, how about a archbang-like approach? Archbang is basically
> Archlinux and uses the Arch packages. I could imagine the same sort of
> approach, perhaps offering custom packages and/or add-ons where necessary.
> 
> This approach gives users a simple way to choose (and possibly switch!),
> while giving developers an easier way to experiment with options.  Where
> things work without too much overhead, they could be incorporated into
> Arch, whereas packages with compatibility problems could be confined to
> this specific projects repositories.
> 
> Perhaps I'm being ignorant here and has this approach already been tried.
> If so, my apologies.
> 
> Mvg, Guus Snijders
> 

Definitely not a bad idea, but I would need a team that would help me
maintain it. All Arch Linux packages are compiled with
systemd support, so lots of recompiling and package editing would need
to be done.


signature.asc
Description: PGP signature

Re: [arch-general] Virtualization accross hardware border

[Wolfgang Mader]

On 02/09/2016 12:30 AM, Damian Nowak wrote:

if I understand the offering of Amazons cloud service correctly, there, you can 
install an
OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and 
down just as
you need it.

Well, yes and no. You can scale up resources (e.g. increase RAM) but this 
requires a full
restart of your virtual machine. See:
https://serverfault.com/questions/591533/can-ec2-instances-dynamically-add-ram-while-running

Moreover, adding more and more resources will stop working at some point. 
That's why you'd
be looking to add several independent virtual machine running your application 
(Amazon
EC2), which connect to one big database (another Amazon EC2), and put all of 
those
computing machines in from of a load balancer (Amazon ELB).


While I can to this using e.g. KVM+qemu on a single machine, I want to be able 
to bind

together a bunch of machines such that they appear as a single big machine. Is 
there a way
to do this?

You can achieve it with both solutions, is it Amazon (EC2+ELB), or your own 
dedicated
server(s) where you'll be likely to use KVM for virtualization and HAProxy, 
nginx or other
solutions for load balancing.

 From a technological point of view, both ways are the same.


Thanks for your answer. I still try to wrap my head around the topic, so 
some of my assumptions my not hold. To me it sounds, that the proposed 
approach only works for web-apps running in a http-server, since the 
load balancing is done via HAProxy and nginx.


For my needs, I want to run "usual" software, specifically R, the 
statistics language. Utlimately, I want to bind several physical hosts 
together to appear as one host on OS level, such that e.g. htop would 
show the total number of cores accross all bound boxes.

Re: [arch-general] Alternative init system proposal

[Tobias Hunger]

Hi Ivan,

On Tue, Feb 9, 2016 at 9:17 AM, Ivan  wrote:
>> That said, how about a archbang-like approach?

> Definitely not a bad idea, but I would need a team that would help me
> maintain it. All Arch Linux packages are compiled with
> systemd support, so lots of recompiling and package editing would need
> to be done.

Almost all packages depend on libsystemd only and that is just a
wrapper that allows applications to use systemd if available and do
something else if not. So the amount of work should be manageable, if
that library does not bother your sense of aesthetics.

Best Regards,
Tobias

Re: [arch-general] Alternative init system proposal

[Michał Zegan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

A note about using shell scripts in systemd:
Who said you can't? and I don't talk about systemd's init.d
compatibility that is disabled in arch. Although you have to write
unit files, you can start scripts, so you do not really lose
flexibility. Also systemd's isolation capabilities are superior, there
are some things you currently cannot do from scripts, like
PrivateTmp=yes and stuff.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJWuhNFAAoJEHb1CzgxXKwYiZAQAI5CCNULoGoCJfp+faXV+huN
xOCtuJ1M4AcjJW1TE412HcLn2rB9fo6NEE/IDHbe1HogksKFQX59qVKmKh6Xesq2
V26/4lHrLJPRIX0Tr99tRGeXkz3fofHmCmfErvz2gMbCw1Qq8BVx8fbPGIxi9Vm/
nAn4x0mexFNPPz9l1P27mGPUBjpUvKJtUcvk8BzH2oTskRJiYeZ3kkpEXP2dib6t
bWZMeihYnQ11jynX38tZwZQltjZLuyITBnrT0TZb1sTSnTjLWWEW5ggNgr0ZOtcp
M1Z5ZvgMaVUGXa7rkFo5nR9Jt8HMdi58R380kmU8+TTaaMHMTAKdaWI6hZim9hFG
vNTO0jvBTv4KIOVuwZj2zAbC7MPcSVenBK2WE+M1tJlqtmheqDcj7DE1YY6dos9m
OJ87ZE1iVK9IHSr3pwPqxif2ZZzUNJa2LsD6EnIk/WocfF1/VIee/nphdQepUWjv
A7cTREyUc4sFlqGiDkCfDU3iDldv5m/tQcvTBmtjp2P2PJ4rrpJAcHI7jRna8mlY
jV9fUmUjTPrkmzfEdRHVBLaiFhHPaql4sJ0Htu728Pie9/OsiN3CncpdtD/vy2Gh
aG/8VEMkM3a+3YmjCSXZmSj/bphXcsYSdOcHQdst0cuHnmu5ST0+W0nmFsvI0WcI
v6lnh0nSuyqfedIkZzW8
=QNfn
-END PGP SIGNATURE-

Re: [arch-general] Alternative init system proposal

[Michał Zegan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256


The isolation is not fully cgroup based, also cgroups require/prefer a
single manager, this is going to be enforced in kernel someday, so it
is better for init to do it as it is a parent of everything.
PrivateTmp uses namespaces, so it is a real isolation. same with
PrivateNetwork, ProtectSystem, etc.
I do not say that you cannot do this from script, but you would have
to make cmdline utilities for some of those things, so it is currently
not possible.


W dniu 09.02.2016 o 17:34, Guus Snijders pisze:
> Op 9 feb. 2016 17:27 schreef "Michał Zegan"
> :
>> 
> 
>> A note about using shell scripts in systemd: Who said you can't?
>> and I don't talk about systemd's init.d compatibility that is
>> disabled in arch. Although you have to write unit files, you can
>> start scripts, so you do not really lose flexibility. Also
>> systemd's isolation capabilities are superior, there are some
>> things you currently cannot do from scripts, like PrivateTmp=yes
>> and stuff.
> 
> Isolation is AFAIK based on cgroups, not the easiest subject, but
> certainly not impossible to implement.
> 
> PrivateTmp: Does that more then setting $TEMP to a custom value?
> 
> I'm just being curious here.
> 
> Mvg, Guus Snijders
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJWuhmOAAoJEHb1CzgxXKwYns8P/24XkWt9vC/Sngfmcgkjc77I
IFjshUljV5sVO4oOHvDb4oPQcPIsACc24feN1pvy/MNtWdeMJJg2MyFYmNy9GkYc
z3WqnRr+pFbQZWCCbdtMIvshvJi141DBrPSVoI1T6hfD0wR3ptkHh0n72bcH1HTH
VkkjfhAsz4V7i6G8Bt4vOK89kjPKQJF1HieBPiUNZgNBjbhoq/1Nv5jfCW47Dvgl
TA3gXJlSAshkCGdogL0WqFyzA/78MDQ3x90DfdLITZ7Yk/G0bpNM9Lk6MJbW/y3E
eNnfy8S/D9TcTE5k4ST6DBl3XSLMbr3HlxSUmme+0sfDa4BUz5asTmgMYrdZ5Zfl
DIReoDvgld2BEK2sgKk2BkQPPmnblZ7OwDLYPC8QmWCWBzIESshHWgYs0Ditsq8N
f3Q15Cj6QDALfxYTlk3TasQ2DRul6S8wFwotEktGYO9Gvi9ktWoFhaVGem1hBB6X
7p3kdEzrwOXQvfqOxqblzPQpTu/0FS9LxRwRcNCKqtqgi6MuRcWeVcKw8pBBJeKe
QJudU7pXvjbwovZzPKEfmL2RsJ4Cb+PFR6nnRUUkXjuCfDj69l5LbnnijnLf4U34
ofJYo2MSSlqqjR+MaSUo8DNbZcTmRaVq8TsnUHZHoRbhOPgXKYr4B9HXG3lwSCmF
YoP+zd75A/xB51DnVoHq
=3gQy
-END PGP SIGNATURE-

Re: [arch-general] Alternative init system proposal

[Damjan Georgievski]

On 9 February 2016 at 17:34, Guus Snijders  wrote:
> Op 9 feb. 2016 17:27 schreef "Michał Zegan" :
>>
>
>> A note about using shell scripts in systemd:
>> Who said you can't? and I don't talk about systemd's init.d
>> compatibility that is disabled in arch. Although you have to write
>> unit files, you can start scripts, so you do not really lose
>> flexibility. Also systemd's isolation capabilities are superior, there
>> are some things you currently cannot do from scripts, like
>> PrivateTmp=yes and stuff.
>
> Isolation is AFAIK based on cgroups, not the easiest subject, but certainly
> not impossible to implement.

not impossible, if you reimplement systemd :)

> PrivateTmp: Does that more then setting $TEMP to a custom value?
>
> I'm just being curious here.

yes, it creates a filesystem/mount namespace for the process(es) and mount's a
/tmp/systemd-private-/ directory as /tmp. from the point of view
of the process it will never see
anything else from the outer /tmp

-- 
damjan

Re: [arch-general] Alternative init system proposal

[Guus Snijders]

Op 9 feb. 2016 17:27 schreef "Michał Zegan" :
>

> A note about using shell scripts in systemd:
> Who said you can't? and I don't talk about systemd's init.d
> compatibility that is disabled in arch. Although you have to write
> unit files, you can start scripts, so you do not really lose
> flexibility. Also systemd's isolation capabilities are superior, there
> are some things you currently cannot do from scripts, like
> PrivateTmp=yes and stuff.

Isolation is AFAIK based on cgroups, not the easiest subject, but certainly
not impossible to implement.

PrivateTmp: Does that more then setting $TEMP to a custom value?

I'm just being curious here.

Mvg, Guus Snijders

Re: [arch-general] Alternative init system proposal

[Guus Snijders]

Op 9 feb. 2016 17:52 schreef "Damjan Georgievski" :
>
> On 9 February 2016 at 17:34, Guus Snijders  wrote:
> > Op 9 feb. 2016 17:27 schreef "Michał Zegan" :
> >>
> >
> >> Although you have to write
> >> unit files, you can start scripts, so you do not really lose
> >> flexibility. Also systemd's isolation capabilities are superior, there
> >> are some things you currently cannot do from scripts, like
> >> PrivateTmp=yes and stuff.
> >
> > Isolation is AFAIK based on cgroups, not the easiest subject, but
certainly
> > not impossible to implement.
>
> not impossible, if you reimplement systemd :)

;)

> > PrivateTmp: Does that more then setting $TEMP to a custom value?
> >
> > I'm just being curious here.
>
> yes, it creates a filesystem/mount namespace for the process(es) and
mount's a
> /tmp/systemd-private-/ directory as /tmp. from the point of view
> of the process it will never see
> anything else from the outer /tmp

Ok, that is a nice trick.

Mvg, Guus Snijders

Re: [arch-general] Virtualization accross hardware border

[Christoph Gysin]

> For my needs, I want to run "usual" software, specifically R, the statistics
> language. Utlimately, I want to bind several physical hosts together to
> appear as one host on OS level, such that e.g. htop would show the total
> number of cores accross all bound boxes.

There are often optimized distributed solutions for specific tasks. So
if your goal is to run R across multiple machines, check out something
like http://www.distributedr.org

Chris
-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3

Re: [arch-general] Alternative init system proposal

[Christian Rebischke]

Hello everone,
First of all I want to remind you that systemd is no longer an init system.
Systemd has become to be much more than just starting/stopping some daemons.
You must see systemd with every part. You cannot just strip systemd from
every package, ignore the other parts of systemd and run openRC. This will
not work. This is just one part..

the other part is: I don't think that the Arch Linux developers would be
happy if they would need to maintain an alternative init system like openRC.

Here are some reasons for it:

1. Maintaining:
The maintaining of systemd and another init-system would be a lot of double
work. We would need every package twice. One systemd version and one version
without systemd. Moreover some packages like netctl depend on systemd. You
cannot just repackage netctl without systemd. You would have to change the
codebase of netctl.

Moreover I think that the developer team has other work todo. For example I
would prefer to have full SELinux support this would be for me much more
important as another init system.

2. Arch Linux itself:
When I think about Arch Linux i must think about:
- rolling release
- a very fast release system. Upstream version == package version in the
  official repositories.

What does this mean? It means that I prefer a linux distribution that
supports the newest changes in the linux development. Systemd is one of
thesee changes. Systemd improves a lot of stuff. There is a reason why all
other big distribtions are also moving to systemd. It's the future. All the
shellscript-based init systems are the past. Also, with systemd comes a lot
of new cool stuff like cgroups, systemd-nspawn, networkd, logind and a lot
more. 

I really think that Arch Linux shouldn't be a rock in this flow of
development. We should do it like fedora and support it. We shouldn't help
to tube-fed all other init systems. 

Furthermore there will be (maybe) kdbus in the kernel. Kdbus is at the
moment still systemd only. I am sure there will come more systemd-specific
interfaces for the kernel. Kdbus is just one example.

3. The ISO and Arch Linux installation process
If Arch Linux would support openRC we would have to offer two ISOs. One with
systemd and one with openRC. Also the way of the installation process would
be different. We would need to edit all wikipages. I often hear that the
Arch Linux wiki is one of the best wikis for GNU/Linux. I think this would
change if we would have a second init system. We would have a lot of chaos.
And for what? Only for making 'some' people happy.


This is just my humble opinion,

best regards,

chris

--
Christian Rebischke

Website: www.nullday.de
Twitter: @sh1bumi
Jabber : shib...@jabber.ccc.de
PGP: 0xDFE2060D
Fingerprint: 6DAF 7B80 8F9D F251 3962  D214 61E3 DFE2 060D
--



signature.asc
Description: PGP signature

Re: [arch-general] Alternative init system proposal

[Ralf Mardorf]

On Tue, 9 Feb 2016 17:26:57 +0100, Michał Zegan wrote:
>A note about using shell scripts in systemd:
>Who said you can't? and I don't talk about systemd's init.d
>compatibility that is disabled in arch. Although you have to write
>unit files, you can start scripts, so you do not really lose
>flexibility.

I'm doing this. However, if you remove and mount cards, in my case just
audio related, not network related cards, then even network device
names are not consistent. It's not a serious issue, you just need to
decide to use one or the other workaround, but some users who have
portable scripts, perhaps dislike to check all scripts against
systemd/udev pitfalls.

[rocketmouse@archlinux ~]$ grep enp?s0 /usr/local/sbin/alice-dhcp
  echo ; dhcpcd $(basename $(ls -d /sys/class/net/enp?s0)) ; echo
  echo ; dhcpcd -x $(basename $(ls -d /sys/class/net/enp?s0)) ; echo

Re: [arch-general] X keeps crashing

[SET]

I started to have exactly the same problem on my new HP laptop, complete UI 
and peripheral freeze. By just pulling out the power cord, the system 
instantly becomes responsive again and I can continue work normally. It won't 
appear again until next reboot, always randomly. I always find a vgaarb error 
in everything.log at the very moment of the lockup :

Feb  9 08:03:47 hp2 kernel: [70005.222588] vgaarb: this pci device is not a 
vga device

The above line is also logged without any system freeze.

I wish to know if the trick works for you, please update.

Cheers.