Hi,
Reasonable question, but I propose to keep 2.2 as a release version
for this urgent release, since we never released X.Y.Z version before.
Let's discuss this at separate thread.
Could you please start it?
On Fri, Sep 1, 2017 at 4:49 AM, 李玉珏@163 <[email protected]
<mailto:[email protected]>> wrote:
Since it's an urgent bugfix version, why can't the version number
be defined as 2.1.1 or 2.1.4?
After all, functionality has not increased and documents do not
require big changes.
If the version number is 2.1 series, there is no need to publish a
new version of the document.
在 2017/8/28 下午10:19, Anton Vinogradov 写道:
Igniters,
Seems 2.2 is a urgent bugfix release, so it should be based on
2.1,
In this case all other issues with fixVersion = 2.2 should be
moved to 2.3.
Currently, I see 835 issues with fixVersion = 2.2
Seems we should have only 4 issues with fixVersion = 2.2:
- Change default max memory size from 80% to 20% -
https://issues.apache.org/jira/browse/IGNITE-6182
<https://issues.apache.org/jira/browse/IGNITE-6182>
- Detecting low memory on ignite node startup -
https://issues.apache.org/jira/browse/IGNITE-6003
<https://issues.apache.org/jira/browse/IGNITE-6003>
- Backport of improvements to checkpoint algorithm -
https://issues.apache.org/jira/browse/IGNITE-??
<https://issues.apache.org/jira/browse/IGNITE-??>??
- ML profile is missing in 2.1 binary release -
https://issues.apache.org/jira/browse/IGNITE-6193
<https://issues.apache.org/jira/browse/IGNITE-6193>
Please correct me in case I've missed something.
Ivan,
Let's include to the release tickets with optimizations of
checkpointing
algorithm:
https://issues.apache.org/jira/browse/IGNITE-6178
<https://issues.apache.org/jira/browse/IGNITE-6178>
https://issues.apache.org/jira/browse/IGNITE-6033
<https://issues.apache.org/jira/browse/IGNITE-6033>
https://issues.apache.org/jira/browse/IGNITE-5961
<https://issues.apache.org/jira/browse/IGNITE-5961>
This will help users who are experiencing problems with
slow checkpoints.
Also, let's include
https://issues.apache.org/jira/browse/IGNITE-6183
<https://issues.apache.org/jira/browse/IGNITE-6183> to
soften "Ignite node crashed in the middle of checkpoint"
message as per
discussion <
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-close-G-stop-name-true-Change-flag-cancel-to-false-td20473.html
<http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-close-G-stop-name-true-Change-flag-cancel-to-false-td20473.html>
.
All of these issues should have fixVersion 2.3.
Please create backport issue and link it to all necessary issues.
P.s. I've created branch ignite-2.2, currently it equals to
ignite-2.1.
On Mon, Aug 28, 2017 at 4:05 PM, Anton Vinogradov
<[email protected] <mailto:[email protected]>>
wrote:
Denis,
BTW, who is considered to be the release manager of
this release?
I'll do it.
On Mon, Aug 28, 2017 at 3:54 PM, Seliverstov Igor
<[email protected] <mailto:[email protected]>>
wrote:
Ok, the check happens at the node start time or on
NODE_JOIN event
in general it looks like:
1) calculate expected used memory = heap max + system
cache max + all
custom policies max + default policy size and put it
into a node attribute
2) get total physycal memory, calculate expected safe
to be used memory
amount (leave 4 gb min or 20% of available memory for OS)
3) if expected used memory + expected used memory of
other nodes on the
host > than safe to be used memory amount, start
calculating suggestions
4) Each ignite instance needs at least 512mb heap +
40mb system cache +
100mb default polycy, if available memory is less we
cannot suggest
anything reasonable, print warning, stop calculation.
5) check heap size (shouldn't exceed 30% of available
memory (total_memory
- reserved for OS memory) * 30% for all JVMs, if it
exeedes, suggest just
calculated value or 512MB minimal)
6) check if system cache size changed, suggest default
value if it's so
7) in case 100 mb * policies count < available memory,
suggest using
default policy with max size equals to remaining
memory (available - heap
-
system cache)
8) calculate new size for each memory policy ( it's
user defined size *
(remaining / (all_policies_size * nodes_cnt)); in
proportion to
remaining memory, devided by nodes number on the host
or 100 mb minimal)
9) print suggestions
2017-08-28 15:10 GMT+03:00 Dmitriy Setrakyan
<[email protected] <mailto:[email protected]>>:
Igor, can you please describe the algorithm with
all the thresholds?
On Mon, Aug 28, 2017 at 4:56 AM, Seliverstov Igor
<[email protected] <mailto:[email protected]>
wrote:
The suggestion here is based on initial
settings, and it's so because
there
is no other nodes on the host in the example.
The algorithm tries to preserve the original
ratio of memory policies
keeping numbers reasonable (for example after
some thresshold it will
suggest not to use several memory policies if
there is not enough of
memory
for all of them) and taking into consideration
nodes count on the
host,
each jvm heap, needed memory for OS, etc
2017-08-28 14:38 GMT+03:00 Dmitriy Setrakyan
<[email protected]
<mailto:[email protected]>>:
Looks good, but why in the example
provided are we suggesting 8GB? 2
nodes
with 8GB will completely exhaust the
available memory. I would
suggest
6
or
7GB.
Also, why 100MB for default policy.
Anything under 1GB seems too
small.
Can you please comment?
D.
On Mon, Aug 28, 2017 at 3:31 AM,
Seliverstov Igor <
[email protected] <mailto:[email protected]>>
wrote:
One more example of possible warning:
-----------------------------------------------------
Excessive memory usage by Ignite node
process (performance may
drop)
[requested=44613MB, available=15942MB].
Please tune the folowing settings as
suggested:
MemoryPolicyConfiguration.initialSize
for bigPlc: 8102MB
MemoryPolicyConfiguration.maxSize
for bigPlc: 8102MB
MemoryPolicyConfiguration.initialSize
for dfltPlc: 100MB
MemoryPolicyConfiguration.maxSize
for dfltPlc: 100MB
Current settings:
Java Heap maxSize: 3543MB
Java Heap initSize: 250MB
MemoryPolicyConfiguration.initialSize
for bigPlc: 256MB
MemoryPolicyConfiguration.maxSize
for bigPlc: 40960MB
MemoryPolicyConfiguration.initialSize
for dfltPlc: 10MB
MemoryPolicyConfiguration.maxSize
for dfltPlc: 10MB
The overall expected memory usage
by all Ignite nodes on the
host:
44613MB
-----------------------------------------------------
Your thoughts?
2017-08-28 5:06 GMT+03:00 Denis Magda
<[email protected]
<mailto:[email protected]>>:
Guys,
ML lib profile is missing in 2.1
release! That must be fixed and
rolled
out in this emergency release:
https://issues.apache.org/jira/browse/IGNITE-6193
<https://issues.apache.org/jira/browse/IGNITE-6193>
<
https://issues.apache.org/jira/browse/IGNITE-6193
<https://issues.apache.org/jira/browse/IGNITE-6193>>
Oleg, Yuri, please step in and
handle the issue.
BTW, who is considered to be the
release manager of this
release?
—
Denis
On Aug 25, 2017, at 2:29 PM,
Dmitriy Setrakyan <
[email protected]
<mailto:[email protected]>>
wrote:
I like the format proposed by
Denis, very clear.
However, I also do not
understand why a user should
change the
size
of
some
system cache. How would a user
ever know what value to put
there?
This
value should be configured by
Ignite automatically.
D.
On Fri, Aug 25, 2017 at 2:24
PM, Denis Magda <
[email protected] <mailto:[email protected]>>
wrote:
Igor,
Let me suggest this format.
---------------------------------------------
Excessive memory usage by
Ignite node process
(performance
may
drop)
[requested=29251MB,
available=15942MB]
Please tune the following
settings:
[MemoryConfiguration.defaultMemoryPolicySize
= suggested
value]
MemoryConfiguration.systemCacheMaxSize
= suggested value
[MemoryPolicyConfiguration.maxSize
for {policy_name_1} =
suggested
value]
[MemoryPolicyConfiguration.maxSize
for {policy_name_2} =
suggested
value]
Current settings:
[DefaultMemoryPolicySize
= value]
[{policy_name_1} size =
value]
[{policy_name_1} size =
value]
SystemCacheInitialSize
= value
SystemCacheMaxSize = value
Java Heap Init Size = value
Java Heap Max Size = value
The overall memory usage
by all Ignite nodes on the
host:
value
-------------------------------------------
Records in […] are
optional. If custom memory
policy is not
set
or
the
default memory policy is
overridden the output will
miss
some of
the
rows.
As for systemCacheMaxSize,
it should be show ONLY if the
parameter
was
set
explicitly by user code.
Otherwise, the platform
should be
wise
enough
to
instantiate it properly
depending on the host
memory usage.
—
Denis
On Aug 25, 2017, at
1:49 PM, Seliverstov
Igor <
[email protected]
<mailto:[email protected]>>
wrote:
The message without
logging layout:
Not enough memory for
current process
[required=29251MB,
available=15942MB].
Please change
MemoryConfiguration.systemCacheMaxSize
and
MemoryConfiguration.defaultMemoryPolicySize
to decrease
memory
allocated
for each node.
Current settings:
HeapInit=250MB
HeapMax=3543MB
DefaultMemoryPolicySize=12753MB
SystemCacheInitialSize=40MB
SystemCacheMaxSize=100MB
Other ignite instanses
on the server require:
12853MB
I think it's make
sense to describe what
these numbers are
consist
of.
We simple say which
parameters have an
impact on how much
memory
the
instance needs and
their (parameters)
actual values.
Also we notice that
more than one Ignite
instance are ran on
the
server
or
workstation and it
also consumes memory.
25 авг. 2017 г. 21:30
пользователь "Dmitriy
Setrakyan" <
[email protected]
<mailto:[email protected]>>
написал:
Igor, what is this
flood of WARN
messaging coming
after the
text?
Are
we
really going to
print this whole
thing out?
On Fri, Aug 25,
2017 at 9:49 AM,
Seliverstov Igor <
[email protected]
<mailto:[email protected]>
wrote:
This message
appears on
topology
change in case the
available
memory
is
exceeded
2017-08-25
19:47
GMT+03:00
Seliverstov Igor <
[email protected]
<mailto:[email protected]>
:
An example
of current
impl:
[2017-08-25
19:44:37,740][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
[2017-08-25
19:44:37,740][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
Not enough
memory
for
current
process
[required=29251MB,
available=15942MB].
[2017-08-25
19:44:37,740][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
Please
change
MemoryConfiguration.systemCacheMaxSize
and
MemoryConfiguration.
defaultMemoryPolicySize
to
decrease
memory
allocated
for each node.
[2017-08-25
19:44:37,740][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
[2017-08-25
19:44:37,740][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
Current
settings:
[2017-08-25
19:44:37,740][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
HeapInit=250MB
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
HeapMax=3543MB
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
DefaultMemoryPolicySize=
12753MB
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
SystemCacheInitialSize=40MB
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
SystemCacheMaxSize=100MB
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
Other
ignite
instanses
on
the server
require:
12853MB
[2017-08-25
19:44:37,741][WARN
][disco-event-worker-#29%
internal.
GridHomePathSelfTest0%][GridDiscoveryManager]
2017-08-25
17:40
GMT+03:00
Sergey
Kozlov <
[email protected] <mailto:[email protected]>
:
I
suppose
we
should
not
forget
JVM
heap
size and
suggest to
reduce
both
options
On
Fri,
Aug
25,
2017
at
5:24
PM,
Dmitriy
Setrakyan
<
[email protected]
<mailto:[email protected]>
wrote:
Igor,
I
would
change
the
message.
How
about
this:
Required
RAM
size
is
larger
than
total
physical
memory
available
for
OS.
Please
change
MemoryConfiguration.WhichProperty
and
MemoryPolicyConfiguration.WhichProperty
to
decrease
memory
allocated
for
each
node.
Also,
can
we
calculate
what
the
memory
size
allocated
for
each
node
should
be?
In
that
case
we
should
suggest
it.
D.
On
Fri,
Aug
25,
2017
at
7:20
AM,
Seliverstov
Igor <
[email protected]
<mailto:[email protected]>
wrote:
What
do
you,
guys
think
about
next
warning?
[2017-08-25
17:17:04,718][INFO
][test-runner-#1%internal.GridHomePathSelfTest%][
GridHomePat
hSelfTest0]
System
cache's
MemoryPolicy
size
is
configured
to
40
MB.
Use
MemoryConfiguration.systemCacheMemorySize
property
to
change
the
setting.
[2017-08-25
17:17:04,718][WARN
][test-runner-#1%internal.GridHomePathSelfTest%][
GridHomePat
hSelfTest0]
Required
RAM
size
is
larger
than
total
physical
memory
available
for
OS.
Check
your
configuration
to
avoid
swap
partition
usage.
Use
MemoryConfiguration
and
MemoryPolicyConfiguration
to
change
the
settings.
Physical
memory
[required=16397MB,
available=15942MB]
[2017-08-25
17:17:04,726][WARN
][test-runner-#1%internal.GridHomePathSelfTest%][
GridHomePat
hSelfTest0]
Peer
class
loading
is
enabled
(disable
it
in
production
for
performance
and
deployment
consistency
reasons)
[2017-08-25
17:17:04,726][INFO
][test-runner-#1%internal.GridHomePathSelfTest%][
GridHomePat
hSelfTest0]
Configured
caches
[in
'sysMemPlc'
memoryPolicy:
['ignite-sys-cache']]
[2017-08-25
17:17:04,731][INFO
][test-runner-#1%internal.GridHomePathSelfTest%][
GridHomePat
hSelfTest0]
3-rd
party
licenses
can
be
found
at:
/home/gvvinblade/projects/igni
te/incubator-ignite/libs/
licenses
2017-08-25
13:26
GMT+03:00
Yakov
Zhdanov
<
[email protected]
<mailto:[email protected]>
:
Agree,
let's
release
new
version
including
tickets
mentioned
by
Denis
and
Ivan.
--Yakov
--
Sergey
Kozlov
GridGain
Systems
www.gridgain.com
<http://www.gridgain.com>
