[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anthony Baker updated GEODE-8859:
-
Labels: pull-request-available unreleased (was: pull-request-available)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: Hale Bales
>Priority: Major
> Labels: pull-request-available, unreleased
> Fix For: 1.15.0
>
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen Nichols updated GEODE-8859:
Labels: pull-request-available (was: pull-request-available
release-blocker)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: Hale Bales
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Smith updated GEODE-8859:
-
Labels: pull-request-available release-blocker (was: blocks-1.14.0
pull-request-available release-blocker)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: Hale Bales
>Priority: Major
> Labels: pull-request-available, release-blocker
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nabarun Nag updated GEODE-8859:
---
Fix Version/s: (was: 1.15.0)
(was: 1.14.0)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: John Hutchison
>Priority: Major
> Labels: blocks-1.14.0, pull-request-available, release-blocker
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen Nichols updated GEODE-8859:
Fix Version/s: 1.15.0
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: John Hutchison
>Priority: Major
> Labels: blocks-1.14.0, pull-request-available, release-blocker
> Fix For: 1.15.0
>
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated GEODE-8859:
--
Labels: blocks-1.14.0 pull-request-available release-blocker (was:
blocks-1.14.0 release-blocker)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: Raymond Ingles
>Priority: Major
> Labels: blocks-1.14.0, pull-request-available, release-blocker
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen Nichols updated GEODE-8859:
Labels: blocks-1.14.0 release-blocker (was: release-blocker)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Assignee: Raymond Ingles
>Priority: Major
> Labels: blocks-1.14.0, release-blocker
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jens Deppe updated GEODE-8859:
--
Affects Version/s: 1.14.0
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Affects Versions: 1.14.0
>Reporter: Jens Deppe
>Priority: Major
> Labels: release-blocker
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jens Deppe updated GEODE-8859:
--
Labels: release-blocker (was: )
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Reporter: Jens Deppe
>Priority: Major
> Labels: release-blocker
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Darrel Schneider updated GEODE-8859:
Issue Type: Bug (was: Improvement)
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Bug
> Components: redis, statistics
>Reporter: Jens Deppe
>Priority: Major
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GEODE-8859) Redis data structures may not accurately reflect their size in Geode stats
[
https://issues.apache.org/jira/browse/GEODE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Darrel Schneider updated GEODE-8859:
Component/s: redis
> Redis data structures may not accurately reflect their size in Geode stats
> --
>
> Key: GEODE-8859
> URL: https://issues.apache.org/jira/browse/GEODE-8859
> Project: Geode
> Issue Type: Improvement
> Components: redis, statistics
>Reporter: Jens Deppe
>Priority: Major
>
> Here is a comment from Darrel regarding this issue. For some background, the
> Redis structures implement {{Delta}}.
>
> {quote}I was playing around with RedisInsight and was able to get most the
> the overview dashboard and the data browser working with geode redis. But I
> found a problem with how we are using geode that causes the geode stats that
> track how much data is stored in a partitioned region to be wrong and the
> bucket sizes used for rebalancing are also wrong. Basically when we do create
> ops on the region the stats track it okay. But when we do updates then geode
> always thinks that nothing (size wise) changed. So for example I created a
> string by doing a redis “set” command. I saw the size of the string accounted
> for in dataStoreBytesInUse. But then I kept doing redis “append” commands on
> that key and the dataStoreBytesInUse did not change at all. I think the
> problem is in how we are updating the data structure in place instead of
> getting a copy, modifying it, and then putting the copy into the region.
> Avoiding this copy gives us MUCH better performance but it messes up geode
> when it is trying to calculate the memory increase or decrease. It is
> possible that this is only an issue on the primary and that the secondary
> sizing may be correct. If so that could lead to other problems because for a
> given bucket our primary size would be different than the secondary. The
> bucket sizes are used when you do a rebalance but basically we can have a
> bunch of memory that is “untracked” so we might see the JVM heaps unbalanced
> but geode will think the buckets are balanced. I’m not sure what we should do
> about this.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
