Re: [DRBD-user] multiple pools

[Roland Kammerer]

On Tue, Dec 12, 2017 at 12:09:26PM +0100, Julien Escario wrote:
> And it's really heavy RAM consuming, even for an 'hello world'.

The target here are servers anyways, today RAM isn't an argument. Can
you even buy notebooks that don't have 16GB RAM? And linstor is not
particular RAM hungry anyways.

And that is the server part that is written in Java. The client is still
written in Python, because:
- Code reuse from the old client (the client never was the problem).
- Python starts quite fast.
- argparse/argcomplete are one of the nicest libs for handling
  options/arguments, which is basically all the client does.

So you can still run the client on your Raspi gen 1 ;-)

Regards, rck
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] multiple pools

[Robert Altnoeder]

On 12/12/2017 12:09 PM, Julien Escario wrote:

> Sounds promising !
> I would just have a reserve about using Java as main language : it's always 
> been
> a nightmare to get a working version of JRE. There's a lot of versions and
> implementations depending upon the running OS.
There should be a precompiled or even a packaged JRE available on most
platforms, and any standards-compliant JRE will work. We are currently
running our tests on different Linux distributions on amd64 running JRE
7 and JRE 8, on Mac OS X running JRE 8 (controller only) and on Ubuntu
ppc64 running JRE 8, and the behavior is identical on all platforms.
> And it's really heavy RAM consuming, even for an 'hello world'.
The Java virtual machine has a base RAM consumption that obviously uses
lots of memory for just 'hello world', but that does not mean that the
application's data structures will necessarily use more RAM than the
same application written in another language.
And we are talking about a software that is running a complete
implementation of an SQL database in its address space, so I don't think
the base RAM consumption of the Java virtual machine will matter much.
> From my point of view, it doesn't really seems to be a wise choice. A modern
> language like Go, Python, Ruby, etc ... could have been far more future-proof.
I disagree, because the main problems of the current implementation of
drbdmanage were reliability and performance.

While most of the reliability problems were caused by the behavior of
the external environment, like LVM utilities, dm kernel driver, udev,
D-Bus, etc., maintaining the Python project had also become a nightmare,
mostly due to the fact that Python is a dynamically typed programming
language.
We had code breakage between different minor versions of Python because
the input or output type of functions in the standard library changed
subtly and without notice, and it only ever failed during runtime - even
though we performed static checking on the source code (using pylint).
Fixing one thing quite commonly broke two other things. Everyone on the
team had become frustrated enough with Python that noone wanted to work
with it anymore.

I suggested the use of Java because it is one of the strictest
programming languages that is in widespread use. It is strongly
statically typed, it forces you to catch most exceptions (except for
runtime exceptions), so you always know what could go wrong, it does not
hide bugs by performing potentially dangerous conversions between data
types automatically (e.g., any condition _must_ be of type boolean, you
can't just use a non-null reference, a non-empty list or a non-zero
integer to represent a value of 'true'), so it prevents lots of classes
of implementation errors (often really just typos or unexpected
behavior) that frequently broke our previous Python implementation of
drbdmanage. The same classes of implementation errors break projects in
other programming languages on a daily basis - especially those written
in dymanically typed languages like Python, Ruby, Perl, etc.
In this regard, Java does exactly what we want - our primary goal for
the project is improved reliability.

Performance-wise, multithreading with Python is more or less useless
(because inefficient), and Python is generally slower than Java and uses
just as much memory.

Also, regarding the modernness of programming languages - although
modernness is hard to measure anyways, Java is, in every respect, a more
modern programming language than Python, both technically and regarding
the date of its invention.

Go was also considered as a possible candidate, but it is more of a
replacement for procedural programming languages like C than it is a
replacement for any fully object-oriented programming language, and the
linstor project turned out to make extensive use of interfaces,
inheritance, polymorphism and structured exception handling, all of
which Go lacks.
Also, Go tends to use large amounts of RAM, possibly even more than the
Java VM, and Java had similar performance as Go in most tests, sometimes
even outperforming Go (both were typically 5 to 10 percent slower than
equivalent C++ code).

Actually, the final decision was between Go and Java.

Now factor in the extensive standard library and excellent documentation
of it that Java comes with, strong IDE support, and the fact that
finding programmers who have experience with the language is quite easy
(which was actually the final reason to decide in favor of Java and
against Go), because Java is in widespread use and has been around for a
while, and you have a winner.

Looking at the state of the project as it is now, I still believe it was
the right choice. I doubt that we could have developed an equivalent
solution in the same amount of time and in the same quality with any of
the other programming languages.

> Best regards,
> Julien Escario

best regards,
-- 
Robert Altnoeder
+43 1 817 82 92 0
robert.altnoe...@linbit.com

LINBIT | 

Re: [DRBD-user] multiple pools

[Julien Escario]

Le 12/12/2017 à 11:54, Robert Altnoeder a écrit :
> On 12/12/2017 11:10 AM, Julien Escario wrote:
> 
>> Hello,
>> May we have a pointer to linstor informations ? I can't find any info on this
>> software by googling 5 min.
>>
>> Best regards,
>> Julien Escario
> That's because no information about the project has been publicly
> released so far.
> 
> A very concise overview is:
> - It is a completely new design and implementation meant as a
> replacement for the existing drbdmanage
> - It's a two-component system comprising a controller and a satellite
> component
> - All communication is through TCP/IP (no control volume, no D-Bus),
> plain or encrypted (SSL/TLS)
> - It supports multiple storage pools
> - It does not keep persistent information on DRBD's state
> - Instead, it tracks DRBD state changes and makes decisions based on
> what state the external environment is in
> - The configuration is kept in an embedded SQL database
> - It's a parallelized system (multiple nodes can run multiple actions
> concurrently)
> - It has very extensive logging and error reporting to make tracking
> problems as easy as possible
> - It has multiuser-security (different strength levels can be configured
> as required)
> - The controller and satellite components are implemented in Java
>   (currently Java 7 compatible, with plans to move to Java 8 in the future)
> - The first command-line client for it is still written in Python
> - It's currently still in a very early stage (an experimental version
> for LINBIT-internal tests will be ready within a few days)
> - There are currently three developers working full-time on it, with a
> fourth one joining in 2018

Sounds promising !
I would just have a reserve about using Java as main language : it's always been
a nightmare to get a working version of JRE. There's a lot of versions and
implementations depending upon the running OS.

And it's really heavy RAM consuming, even for an 'hello world'.

>From my point of view, it doesn't really seems to be a wise choice. A modern
language like Go, Python, Ruby, etc ... could have been far more future-proof.

Best regards,
Julien Escario
<>___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] multiple pools

[Robert Altnoeder]

On 12/12/2017 11:10 AM, Julien Escario wrote:

> Hello,
> May we have a pointer to linstor informations ? I can't find any info on this
> software by googling 5 min.
>
> Best regards,
> Julien Escario
That's because no information about the project has been publicly
released so far.

A very concise overview is:
- It is a completely new design and implementation meant as a
replacement for the existing drbdmanage
- It's a two-component system comprising a controller and a satellite
component
- All communication is through TCP/IP (no control volume, no D-Bus),
plain or encrypted (SSL/TLS)
- It supports multiple storage pools
- It does not keep persistent information on DRBD's state
- Instead, it tracks DRBD state changes and makes decisions based on
what state the external environment is in
- The configuration is kept in an embedded SQL database
- It's a parallelized system (multiple nodes can run multiple actions
concurrently)
- It has very extensive logging and error reporting to make tracking
problems as easy as possible
- It has multiuser-security (different strength levels can be configured
as required)
- The controller and satellite components are implemented in Java
  (currently Java 7 compatible, with plans to move to Java 8 in the future)
- The first command-line client for it is still written in Python
- It's currently still in a very early stage (an experimental version
for LINBIT-internal tests will be ready within a few days)
- There are currently three developers working full-time on it, with a
fourth one joining in 2018

best regards,
-- 
Robert Altnoeder
+43 1 817 82 92 0 
robert.altnoe...@linbit.com 

LIN BIT  | Keeping
The Digital World Running
DRBD - Corosync - Pacemaker
f  /  t
 /  in
 /  g+


DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] multiple pools

[Julien Escario]

Le 12/12/2017 à 10:06, Roland Kammerer a écrit :
> On Mon, Dec 11, 2017 at 08:24:26PM +, Henning Svane wrote:
>> Hi
>> Is it possible in version 9 to have multiple pools.
>>
>> I would like to make a configuration with 2 pools.
> 
> You have to be a bit more specific on that...
> 
> - For manual configuration you put in the res files whatever you like.
>   One backing device from pool A for resA, one from B for resB.
> - drbdmanage does not support multiple pools.
> - linstor (the project that will eventually replace drbdmanage) will
>   support that.

Hello,
May we have a pointer to linstor informations ? I can't find any info on this
software by googling 5 min.

Best regards,
Julien Escario
<>___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] multiple pools

[Roland Kammerer]

On Mon, Dec 11, 2017 at 08:24:26PM +, Henning Svane wrote:
> Hi
> Is it possible in version 9 to have multiple pools.
> 
> I would like to make a configuration with 2 pools.

You have to be a bit more specific on that...

- For manual configuration you put in the res files whatever you like.
  One backing device from pool A for resA, one from B for resB.
- drbdmanage does not support multiple pools.
- linstor (the project that will eventually replace drbdmanage) will
  support that.

Regards, rck
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user