[jira] [Commented] (GEODE-10215) WAN replication not working after re-creating the partitioned region
[ https://issues.apache.org/jira/browse/GEODE-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518097#comment-17518097 ] Jakov Varenina commented on GEODE-10215: Hi [~agingade] , I have tried to troubleshoot this issue, but unfortunately, I haven't found the root cause yet. I could use some help on this, and if you are interested in helping, you can find the test case that reproduces the issue in linked PR. I have found the following so far: When you configure a parallel gateway-sender on a partitioned region, the parallel gateway-sender queue is created as a shadow collocated region of the primary data partition region. Because of that, the set of primary buckets IDs of the data region is the same as the primary bucket IDs of the queue region. The aforementioned is also valid for the secondary buckets. When you alter the region to remove the gateway-sender, you only remove the connection between the region and the gateway-sender. Still, the parallel gateway-sender queue region remains on servers. Recreating the region again with the same gateway sender results in new data buckets distributed differently across the servers. After this bucket advisor is not working correctly and causing some events not to be dispatched to the remote site. Lets go deeper in the logs: Distribution of primary buckets first time region is created: {code:java} [vm6] Jale buckets: [1, 7, 11, 15, 17, 20, 27, 29, 34, 38, 42, 46, 48, 53, 59, 60, 67, 70, 72, 76, 80, 85, 91, 92, 99, 103, 104, 108] [vm7] Jale buckets: [3, 5, 8, 12, 19, 21, 24, 30, 32, 39, 40, 47, 49, 54, 56, 61, 66, 71, 75, 79, 81, 84, 90, 94, 98, 101, 107, 111] [vm10] Jale buckets:[0, 4, 9, 13, 16, 22, 26, 28, 35, 36, 43, 44, 51, 55, 57, 63, 65, 68, 73, 78, 82, 86, 89, 93, 96, 102, 105, 109, 112] [vm11] Jale buckets:[2, 6, 10, 14, 18, 23, 25, 31, 33, 37, 41, 45, 50, 52, 58, 62, 64, 69, 74, 77, 83, 87, 88, 95, 97, 100, 106, 110]{code} Distribution of primary buckets after region is recreated: {code:java} [vm6] Jale buckets: [0, 4, 8, 14, 17, 20, 25, 27, 34, 36, 40, 46, 51, 54, 56, 63, 66, 71, 72, 77, 82, 85, 91, 95, 97, 103, 107, 108] [vm7] Jale buckets: [2, 5, 7, 11, 15, 21, 26, 28, 31, 38, 42, 44, 47, 50, 52, 57, 60, 67, 70, 74, 79, 83, 87, 90, 93, 98, 101, 105, 110] [vm10] Jale buckets: [3, 10, 13, 16, 22, 23, 30, 32, 37, 39, 43, 48, 55, 58, 62, 64, 68, 73, 78, 81, 84, 88, 92, 99, 100, 106, 109, 112] [vm11] Jale buckets: [1, 6, 9, 12, 18, 19, 24, 29, 33, 35, 41, 45, 49, 53, 59, 61, 65, 69, 75, 76, 80, 86, 89, 94, 96, 102, 104, 111]{code} In above logs it can be seen that buckets are distributed differently after region is re-created. Events (keys) that did not replicate to remote site: {code:java} [vm9] Jale result: [500, 501, 502, 503, 504, 506, 508, 513, 514, 515, 517, 518, 527, 528, 529, 539, 541, 544, 545, 547, 548, 549, 551, 556, 557, 558, 559, 563, 565, 566, 567, 568, 569, 572, 573, 574, 575, 576, 577, 579, 580, 588, 589, 594, 595, 597, 600, 602, 603, 605, 609, 613, 614, 615, 616, 617, 619, 621, 626, 627, 628, 630, 631, 640, 641, 642, 652, 654, 657, 658, 660, 661, 662, 664, 669, 670, 671, 672, 676, 678, 679, 680, 681, 682, 685, 686, 687, 688, 689, 690, 692, 693, 701, 702, 707, 708, 710, 713, 715, 716, 718, 722, 726, 727, 728, 729, 730, 732, 734, 739, 740, 741, 743, 744, 753, 754, 755, 765, 767, 770, 771, 773, 774, 775, 777, 782, 783, 784, 785, 789, 791, 792, 793, 794, 795, 798, 799, 800, 801, 802, 803, 805, 806, 814, 815, 820, 821, 823, 826, 828, 829, 831, 835, 839, 840, 841, 842, 843, 845, 847, 852, 853, 854, 856, 857, 866, 867, 868, 878, 880, 883, 884, 886, 887, 888, 890, 895, 896, 897, 898, 902, 904, 905, 906, 907, 908, 911, 912, 913, 914, 915, 916, 918, 919, 927, 928, 933, 934, 936, 939, 941, 942, 944, 948, 952, 953, 954, 955, 956, 958, 960, 965, 966, 967, 969, 970, 979, 980, 981, 991, 993, 996, 997, 999, 1000, 1001, 1003, 1008, 1009, 1010, 1011, 1015, 1017, 1018, 1019, 1020, 1021, 1024, 1025, 1026, 1027, 1028, 1029, 1031, 1032, 1040, 1041, 1046, 1047, 1049, 1052, 1054, 1055, 1057, 1061, 1065, 1066, 1067, 1068, 1069, 1071, 1073, 1078, 1079, 1080, 1082, 1083, 1092, 1093, 1094, 1104, 1106, 1109, 1110, 1112, 1113, 1114, 1116, 1121, 1122, 1123, 1124, 1128, 1130, 1131, 1132, 1133, 1134, 1137, 1138, 1139, 1140, 1141, 1142, 1144, 1145, 1153, 1154, 1159, 1160, 1162, 1165, 1167, 1168, 1170, 1174, 1178, 1179, 1180, 1181, 1182, 1184, 1186, 1191, 1192, 1193, 1195, 1196, 1205, 1206, 1207, 1211, 1217, 1219, 1222, 1223, 1225, 1226, 1227, 1229, 1231, 1234, 1235, 1236, 1237, 1241, 1243, 1244, 1245, 1246, 1247, 1250, 1251, 1252, 1253, 1254, 1255, 1257, 1258, 1264, 1266, 1267, 1272, 1273, 1275, 1278, 1280, 1281, 1283, 1287, 1291, 1292, 1293, 1294, 1295, 1297, 1299, 1304, 1305, 1306, 1308, 1309, 1318, 1319, 1320, 1330, 1332, 1335, 1336, 1338, 1339, 1340, 1342, 1343, 1347, 1348, 1349, 1350, 1353, 1354, 1356, 1357, 1358,
[jira] [Comment Edited] (GEODE-10215) WAN replication not working after re-creating the partitioned region
[ https://issues.apache.org/jira/browse/GEODE-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518097#comment-17518097 ] Jakov Varenina edited comment on GEODE-10215 at 4/6/22 12:50 PM: - Hi [~agingade] , I have tried to troubleshoot this issue, but unfortunately, I haven't found the root cause yet. I could use some help on this, and if you are interested in helping, you can find the test case that reproduces the issue in linked PR. I have found the following so far: When you configure a parallel gateway-sender on a partitioned region, the parallel gateway-sender queue is created as a shadow collocated region of the primary data partition region. Because of that, the set of primary buckets IDs of the data region is the same as the primary bucket IDs of the queue region. The aforementioned is also valid for the secondary buckets. When you alter the region to remove the gateway-sender, you only remove the connection between the region and the gateway-sender. Still, the parallel gateway-sender queue region remains on servers. Recreating the region again with the same gateway sender results in new data buckets distributed differently across the servers and this affects also queue region buckets since they are tightly coupled. After this bucket advisor for queue region is not working correctly and causing some events not to be dispatched to the remote site. Lets go deeper in the logs: Distribution of primary buckets first time region is created: {code:java} [vm6] Jale buckets: [1, 7, 11, 15, 17, 20, 27, 29, 34, 38, 42, 46, 48, 53, 59, 60, 67, 70, 72, 76, 80, 85, 91, 92, 99, 103, 104, 108] [vm7] Jale buckets: [3, 5, 8, 12, 19, 21, 24, 30, 32, 39, 40, 47, 49, 54, 56, 61, 66, 71, 75, 79, 81, 84, 90, 94, 98, 101, 107, 111] [vm10] Jale buckets:[0, 4, 9, 13, 16, 22, 26, 28, 35, 36, 43, 44, 51, 55, 57, 63, 65, 68, 73, 78, 82, 86, 89, 93, 96, 102, 105, 109, 112] [vm11] Jale buckets:[2, 6, 10, 14, 18, 23, 25, 31, 33, 37, 41, 45, 50, 52, 58, 62, 64, 69, 74, 77, 83, 87, 88, 95, 97, 100, 106, 110]{code} Distribution of primary buckets after region is recreated: {code:java} [vm6] Jale buckets: [0, 4, 8, 14, 17, 20, 25, 27, 34, 36, 40, 46, 51, 54, 56, 63, 66, 71, 72, 77, 82, 85, 91, 95, 97, 103, 107, 108] [vm7] Jale buckets: [2, 5, 7, 11, 15, 21, 26, 28, 31, 38, 42, 44, 47, 50, 52, 57, 60, 67, 70, 74, 79, 83, 87, 90, 93, 98, 101, 105, 110] [vm10] Jale buckets: [3, 10, 13, 16, 22, 23, 30, 32, 37, 39, 43, 48, 55, 58, 62, 64, 68, 73, 78, 81, 84, 88, 92, 99, 100, 106, 109, 112] [vm11] Jale buckets: [1, 6, 9, 12, 18, 19, 24, 29, 33, 35, 41, 45, 49, 53, 59, 61, 65, 69, 75, 76, 80, 86, 89, 94, 96, 102, 104, 111]{code} In above logs it can be seen that buckets are distributed differently after region is re-created. Events (keys) that did not replicate to remote site: {code:java} [vm9] Jale result: [500, 501, 502, 503, 504, 506, 508, 513, 514, 515, 517, 518, 527, 528, 529, 539, 541, 544, 545, 547, 548, 549, 551, 556, 557, 558, 559, 563, 565, 566, 567, 568, 569, 572, 573, 574, 575, 576, 577, 579, 580, 588, 589, 594, 595, 597, 600, 602, 603, 605, 609, 613, 614, 615, 616, 617, 619, 621, 626, 627, 628, 630, 631, 640, 641, 642, 652, 654, 657, 658, 660, 661, 662, 664, 669, 670, 671, 672, 676, 678, 679, 680, 681, 682, 685, 686, 687, 688, 689, 690, 692, 693, 701, 702, 707, 708, 710, 713, 715, 716, 718, 722, 726, 727, 728, 729, 730, 732, 734, 739, 740, 741, 743, 744, 753, 754, 755, 765, 767, 770, 771, 773, 774, 775, 777, 782, 783, 784, 785, 789, 791, 792, 793, 794, 795, 798, 799, 800, 801, 802, 803, 805, 806, 814, 815, 820, 821, 823, 826, 828, 829, 831, 835, 839, 840, 841, 842, 843, 845, 847, 852, 853, 854, 856, 857, 866, 867, 868, 878, 880, 883, 884, 886, 887, 888, 890, 895, 896, 897, 898, 902, 904, 905, 906, 907, 908, 911, 912, 913, 914, 915, 916, 918, 919, 927, 928, 933, 934, 936, 939, 941, 942, 944, 948, 952, 953, 954, 955, 956, 958, 960, 965, 966, 967, 969, 970, 979, 980, 981, 991, 993, 996, 997, 999, 1000, 1001, 1003, 1008, 1009, 1010, 1011, 1015, 1017, 1018, 1019, 1020, 1021, 1024, 1025, 1026, 1027, 1028, 1029, 1031, 1032, 1040, 1041, 1046, 1047, 1049, 1052, 1054, 1055, 1057, 1061, 1065, 1066, 1067, 1068, 1069, 1071, 1073, 1078, 1079, 1080, 1082, 1083, 1092, 1093, 1094, 1104, 1106, 1109, 1110, 1112, 1113, 1114, 1116, 1121, 1122, 1123, 1124, 1128, 1130, 1131, 1132, 1133, 1134, 1137, 1138, 1139, 1140, 1141, 1142, 1144, 1145, 1153, 1154, 1159, 1160, 1162, 1165, 1167, 1168, 1170, 1174, 1178, 1179, 1180, 1181, 1182, 1184, 1186, 1191, 1192, 1193, 1195, 1196, 1205, 1206, 1207, 1211, 1217, 1219, 1222, 1223, 1225, 1226, 1227, 1229, 1231, 1234, 1235, 1236, 1237, 1241, 1243, 1244, 1245, 1246, 1247, 1250, 1251, 1252, 1253, 1254, 1255, 1257, 1258, 1264, 1266, 1267, 1272, 1273, 1275, 1278, 1280, 1281, 1283, 1287, 1291, 1292, 1293, 1294, 1295, 1297, 1299, 1304, 1305, 1306, 1308,
[jira] [Commented] (GEODE-10160) SizeableByteArrayList does not update the sizeInBytes for some non-overriden methods
[ https://issues.apache.org/jira/browse/GEODE-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518158#comment-17518158 ] ASF subversion and git services commented on GEODE-10160: - Commit 4cae7ac07641780663f648ef533cdb9a4b978aac in geode's branch refs/heads/develop from Steve Sienkowski [ https://gitbox.apache.org/repos/asf?p=geode.git;h=4cae7ac076 ] GEODE-10160: fixes SizeableByteArrayList sizing (#7519) * Update SizeableByteArrayList with overrides for LinkedList's set() and add() methods to ensure memory overhead is updated appropriately. This helps to correct sizing issues in other RedisList methods such as LINSERT and LTRM. Co-authored-by: Jens Deppe > SizeableByteArrayList does not update the sizeInBytes for some non-overriden > methods > > > Key: GEODE-10160 > URL: https://issues.apache.org/jira/browse/GEODE-10160 > Project: Geode > Issue Type: Bug > Components: redis >Affects Versions: 1.15.0 >Reporter: Hale Bales >Assignee: Steve Sienkowski >Priority: Major > Labels: pull-request-available, redis-triage > > Some List methods aren't overriden in SizeableByteArrayList which means that > the memoryOverhead doesn't get updated. add(int index, byte[] element), > clear(), and set(int index, byte[] newElement) all need to update the size > and don't. > This can be accomplished by adding overrides to SizeableByteArrayList: > {code:java} > @Override > public byte[] set(int index, byte[] newElement) { > byte[] replacedElement = super.set(index, newElement); > memberOverhead -= calculateByteArrayOverhead(replacedElement); > memberOverhead += calculateByteArrayOverhead(newElement); > return replacedElement; > } > @Override > public void add(int index, byte[] element) { > memberOverhead += calculateByteArrayOverhead(element); > super.add(index, element); > } > {code} > The test for set could look something like: > {code:java} > @Test > public void sizeInBytesGetsUpdatedAccurately_whenDoingSets() { > SizeableByteArrayList list = new SizeableByteArrayList(); > byte[] element = "element".getBytes(StandardCharsets.UTF_8); > list.addFirst(element); > long initialSize = list.getSizeInBytes(); > assertThat(initialSize).isEqualTo(sizer.sizeof(list)); > list.set(0, "some really big new element > name".getBytes(StandardCharsets.UTF_8)); > assertThat(list.getSizeInBytes()).isEqualTo(sizer.sizeof(list)); > list.set(0, element); > assertThat(list.getSizeInBytes()).isEqualTo(initialSize); > } > {code} > We need more tests than just this one. add(int, byte[]) needs to be tested as > well. Any method on SizeableByteArrayList that modify the data should have a > test that the memoryOverhead gets updated correctly. > Clear itself isn't a problem - just set the memberOverhead to 0 - but the > issue is that for the version of LTRIM in > [PR#7403|https://github.com/apache/geode/pull/7403], we clear a sublist of > SizeableByteArrayList. This means that the overridden clear method does not > get called and the LinkedList implementation of clear does not call any other > methods that we can override. There needs to be some new approach to LTRIM > that doesn't use sublists. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-7016) CI failure: ServerStartupRedundancyRecoveryNotificationTest > startupReportsOnlineOnlyAfterRedundancyRestored FAILED
[ https://issues.apache.org/jira/browse/GEODE-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518185#comment-17518185 ] Geode Integration commented on GEODE-7016: -- Seen on support/1.12 in [acceptance-test-openjdk11 #51|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main/jobs/acceptance-test-openjdk11/builds/51]. > CI failure: ServerStartupRedundancyRecoveryNotificationTest > > startupReportsOnlineOnlyAfterRedundancyRestored FAILED > > > Key: GEODE-7016 > URL: https://issues.apache.org/jira/browse/GEODE-7016 > Project: Geode > Issue Type: Bug > Components: gfsh >Affects Versions: 1.10.0 >Reporter: Anilkumar Gingade >Assignee: Kirk Lund >Priority: Major > Labels: pull-request-available > Attachments: acceptancetestfiles-OpenJDK11-1.14.0-build.0628 (1).tgz, > acceptancetestfiles-OpenJDK11-1.14.0-build.0628 (2).tgz > > > {noformat} > org.apache.geode.launchers.ServerStartupRedundancyRecoveryNotificationTest > > startupReportsOnlineOnlyAfterRedundancyRestored FAILED > org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at > org.apache.geode.test.junit.rules.gfsh.GfshExecution.awaitTermination(GfshExecution.java:125) > at > org.apache.geode.test.junit.rules.gfsh.GfshRule.execute(GfshRule.java:125) > at > org.apache.geode.test.junit.rules.gfsh.GfshRule.execute(GfshRule.java:112) > at > org.apache.geode.launchers.ServerStartupRedundancyRecoveryNotificationTest.startupReportsOnlineOnlyAfterRedundancyRestored(ServerStartupRedundancyRecoveryNotificationTest.java:142) > org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at > org.apache.geode.test.junit.rules.gfsh.GfshExecution.awaitTermination(GfshExecution.java:125) > at > org.apache.geode.test.junit.rules.gfsh.GfshRule.execute(GfshRule.java:125) > at > org.apache.geode.test.junit.rules.gfsh.GfshRule.execute(GfshRule.java:112) > at > org.apache.geode.launchers.ServerStartupRedundancyRecoveryNotificationTest.stopAllMembers(ServerStartupRedundancyRecoveryNotificationTest.java:128) > {noformat} > https://concourse.gemfire-ci.info/teams/main/pipelines/gemfire-develop-main/jobs/AcceptanceTestOpenJDK8/builds/797 > Test report artifacts from this job are available at: > gs://gemfire-test-artifacts/builds/gemfire-develop-main/9.9.0-build.0258/test-artifacts/1564078711/acceptancetestfiles-OpenJDK8-9.9.0-build.0258.tgz -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10121) Transactional Redis commands are not actually transactional
[ https://issues.apache.org/jira/browse/GEODE-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518207#comment-17518207 ] ASF subversion and git services commented on GEODE-10121: - Commit 141d43dc67452af4940c24203719e6193be42300 in geode's branch refs/heads/develop from Bala Kaza Venkata [ https://gitbox.apache.org/repos/asf?p=geode.git;h=141d43dc67 ] GEODE-10121: Fix transactional redis commands (#7513) * GEODE-10121: Fix transactional redis commands MSET, SMOVE commands are transactional but they have not been working as transactional. This commit will fix these commands to behave that way. Authored-by: Bala Kaza Venkata > Transactional Redis commands are not actually transactional > --- > > Key: GEODE-10121 > URL: https://issues.apache.org/jira/browse/GEODE-10121 > Project: Geode > Issue Type: Bug > Components: redis >Affects Versions: 1.15.0 >Reporter: Donal Evans >Assignee: Bala Tripura Sundari Kaza Venkata >Priority: Major > Labels: pull-request-available, redis-triage > > The following test (if added to MSetDUnitTest) is intended to show that MSET > is transactional in nature by having the final key to be set throw an > exception, which should roll back the previous set values: > {code:java} > public static final String THROWING_REDIS_STRING_EXCEPTION = "to be > ignored"; > > @Test > public void mset_isTransactional() { > IgnoredException.addIgnoredException(THROWING_REDIS_STRING_EXCEPTION); > String hashTag = "{" + clusterStartUp.getKeyOnServer("tag", 1) + "}"; > String key1 = hashTag + "key1"; > String value1 = "value1"; > jedis.set(key1, value1); > String key2 = hashTag + "key2"; > String value2 = "value2"; > jedis.set(key2, value2); > String throwingRedisStringKey = hashTag + "ThrowingRedisString"; > String throwingStringValue = "ThrowingRedisStringValue"; > // Put a test version of RedisString directly into the region that throws > if the multi-argument set() is called on it > clusterStartUp.getMember(1).invoke(() -> { > RedisKey throwingKey = new > RedisKey(throwingRedisStringKey.getBytes(StandardCharsets.UTF_8)); > ThrowingRedisString throwingRedisString = new ThrowingRedisString(); > > throwingRedisString.set(throwingStringValue.getBytes(StandardCharsets.UTF_8)); > > ClusterStartupRule.getCache().getRegion(DEFAULT_REDIS_REGION_NAME).put(throwingKey, > throwingRedisString); > }); > String newValue = "should_not_be_set"; > assertThatThrownBy(() -> jedis.mset(key1, newValue, key2, newValue, > throwingRedisStringKey, newValue)).hasMessage(SERVER_ERROR_MESSAGE); > > assertThat(jedis.get(key1)).isEqualTo(value1); > assertThat(jedis.get(key2)).isEqualTo(value2); > > assertThat(jedis.get(throwingRedisStringKey)).isEqualTo(throwingStringValue); > IgnoredException.removeAllExpectedExceptions(); > } > static class ThrowingRedisString extends RedisString { > @Override > public void set(Region region, RedisKey key, byte[] > newValue, > SetOptions options) { > throw new RuntimeException(THROWING_REDIS_STRING_EXCEPTION); > } > }{code} > This test fails due to {{key1}} having its value set to {{newValue}} despite > the fact that MSET is supposed to be atomic. > Given the similar implementations each command uses, it is very likely that > MSETNX and SMOVE also show the same behaviour. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (GEODE-10121) Transactional Redis commands are not actually transactional
[ https://issues.apache.org/jira/browse/GEODE-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bala Tripura Sundari Kaza Venkata resolved GEODE-10121. --- Fix Version/s: 1.15.0 Resolution: Fixed > Transactional Redis commands are not actually transactional > --- > > Key: GEODE-10121 > URL: https://issues.apache.org/jira/browse/GEODE-10121 > Project: Geode > Issue Type: Bug > Components: redis >Affects Versions: 1.15.0 >Reporter: Donal Evans >Assignee: Bala Tripura Sundari Kaza Venkata >Priority: Major > Labels: pull-request-available, redis-triage > Fix For: 1.15.0 > > > The following test (if added to MSetDUnitTest) is intended to show that MSET > is transactional in nature by having the final key to be set throw an > exception, which should roll back the previous set values: > {code:java} > public static final String THROWING_REDIS_STRING_EXCEPTION = "to be > ignored"; > > @Test > public void mset_isTransactional() { > IgnoredException.addIgnoredException(THROWING_REDIS_STRING_EXCEPTION); > String hashTag = "{" + clusterStartUp.getKeyOnServer("tag", 1) + "}"; > String key1 = hashTag + "key1"; > String value1 = "value1"; > jedis.set(key1, value1); > String key2 = hashTag + "key2"; > String value2 = "value2"; > jedis.set(key2, value2); > String throwingRedisStringKey = hashTag + "ThrowingRedisString"; > String throwingStringValue = "ThrowingRedisStringValue"; > // Put a test version of RedisString directly into the region that throws > if the multi-argument set() is called on it > clusterStartUp.getMember(1).invoke(() -> { > RedisKey throwingKey = new > RedisKey(throwingRedisStringKey.getBytes(StandardCharsets.UTF_8)); > ThrowingRedisString throwingRedisString = new ThrowingRedisString(); > > throwingRedisString.set(throwingStringValue.getBytes(StandardCharsets.UTF_8)); > > ClusterStartupRule.getCache().getRegion(DEFAULT_REDIS_REGION_NAME).put(throwingKey, > throwingRedisString); > }); > String newValue = "should_not_be_set"; > assertThatThrownBy(() -> jedis.mset(key1, newValue, key2, newValue, > throwingRedisStringKey, newValue)).hasMessage(SERVER_ERROR_MESSAGE); > > assertThat(jedis.get(key1)).isEqualTo(value1); > assertThat(jedis.get(key2)).isEqualTo(value2); > > assertThat(jedis.get(throwingRedisStringKey)).isEqualTo(throwingStringValue); > IgnoredException.removeAllExpectedExceptions(); > } > static class ThrowingRedisString extends RedisString { > @Override > public void set(Region region, RedisKey key, byte[] > newValue, > SetOptions options) { > throw new RuntimeException(THROWING_REDIS_STRING_EXCEPTION); > } > }{code} > This test fails due to {{key1}} having its value set to {{newValue}} despite > the fact that MSET is supposed to be atomic. > Given the similar implementations each command uses, it is very likely that > MSETNX and SMOVE also show the same behaviour. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Smith reopened GEODE-10126: --- Assignee: Dan Smith (was: Jens Deppe) Re-opening this to fix the benchmarks as well, which need to use the new system properties. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518265#comment-17518265 ] ASF GitHub Bot commented on GEODE-10126: upthewaterspout opened a new pull request, #166: URL: https://github.com/apache/geode-benchmarks/pull/166 Use system properties to configure the redis server, instead of the gemfire properties, which have now been removed. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518268#comment-17518268 ] Geode Integration commented on GEODE-10126: --- Seen in [benchmark-radish #277|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/277]. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518278#comment-17518278 ] ASF GitHub Bot commented on GEODE-10126: upthewaterspout merged PR #166: URL: https://github.com/apache/geode-benchmarks/pull/166 > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518279#comment-17518279 ] ASF subversion and git services commented on GEODE-10126: - Commit 0892daf69939fc5c508856f787690dd61c40 in geode-benchmarks's branch refs/heads/develop from Dan Smith [ https://gitbox.apache.org/repos/asf?p=geode-benchmarks.git;h=0892daf6 ] GEODE-10126: Use system properties to configure redis Use system properties to configure the redis server, instead of the gemfire properties, which have now been removed. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-9704) When durable clients recovers, it sends "ready for event" signal before register for interest, this might cause problem for caching_proxy regions
[ https://issues.apache.org/jira/browse/GEODE-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518280#comment-17518280 ] ASF subversion and git services commented on GEODE-9704: Commit 30bd1cef01b555c84e970c548cfb0e55f06fbf1c in geode's branch refs/heads/develop from mhansonp [ https://gitbox.apache.org/repos/asf?p=geode.git;h=30bd1cef01 ] GEODE-9704: Ensure that register interest is called before ready for events (#7442) - RegisterInterestOps need to happen before ReadyForEventsOp is sent These changes make sure that happens. - Added an InterestResultPolicyCheck Authored-by: Barry Oglesby Un-ignored a test that will reproduce the issue periodically during a number of runs. It is a flaky test without the core fix. Authored-by: Jinmei Liao > When durable clients recovers, it sends "ready for event" signal before > register for interest, this might cause problem for caching_proxy regions > - > > Key: GEODE-9704 > URL: https://issues.apache.org/jira/browse/GEODE-9704 > Project: Geode > Issue Type: Bug > Components: regions >Affects Versions: 1.15.0 >Reporter: Jinmei Liao >Assignee: Mark Hanson >Priority: Major > Labels: GeodeOperationAPI, blocks-1.15.1, pull-request-available > > This is the old Geode behavior, but may or may not be the correct behavior. > When durable clients recovers, there is a queueTimer thread that runs > `QueueManagerImp.recoverPrimary` method, it > * makes new connection to server > - sends readyForEvents (which will cause the server to start sending the > queued events) > - recovers interest > - clears the region of keys of interest > - re-registers interest > It sends readyForEvents before it clears region of keys of interest, if > server sends some events of those keys in between, it will clear them, thus > it seems to the user that the client region doesn't have those keys. > > Run geode-core distributedTest > AuthExpirationDUnitTest.registeredInterest_slowReAuth_policyKeys_durableClient(), > change the InterestResultPolicy to NONE, you would see the test would fail > occasionally, Adding sleep code in QueueManagerImp.recoverPrimary between > `createNewPrimary` and `recoverInterest` would make the test fail more > consistently. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Smith resolved GEODE-10126. --- Resolution: Fixed Benchmarks have been updated to use system properties. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518289#comment-17518289 ] Geode Integration commented on GEODE-10126: --- Seen in [benchmark-radish #277|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/277]. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518293#comment-17518293 ] Geode Integration commented on GEODE-10126: --- Seen in [benchmark-radish #274|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/274]. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518292#comment-17518292 ] Geode Integration commented on GEODE-10126: --- Seen in [benchmark-radish #275|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/275]. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10126) Refactor Configuration To Use System Properties
[ https://issues.apache.org/jira/browse/GEODE-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518291#comment-17518291 ] Geode Integration commented on GEODE-10126: --- Seen in [benchmark-radish #276|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/benchmark-radish/builds/276]. > Refactor Configuration To Use System Properties > --- > > Key: GEODE-10126 > URL: https://issues.apache.org/jira/browse/GEODE-10126 > Project: Geode > Issue Type: Improvement > Components: redis >Reporter: Wayne >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > The properties currently being used by the Redis module are defined in Geode > core. These properties need to be removed from Geode core and refactored to > system properties. > > Validators must also be added for the system properties to ensure that users > provide correct values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (GEODE-10218) Launch Cluster with Attachability Flag
Michael Martell created GEODE-10218: --- Summary: Launch Cluster with Attachability Flag Key: GEODE-10218 URL: https://issues.apache.org/jira/browse/GEODE-10218 Project: Geode Issue Type: Test Components: native client Reporter: Michael Martell To assist with protocol analysis it is very helpful to be able to step through the server code in the Intellij debugger while running native client test code. However, to be allowed to set breakpoints in the server code, the server must be started with the special flag below: ```--J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT``` This ticket is to add support for the above flag in the native client test frameworks. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (GEODE-10218) Launch Cluster with Attachability Flag
[ https://issues.apache.org/jira/browse/GEODE-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Martell updated GEODE-10218: Description: To assist with protocol analysis it is very helpful to be able to step through the server code in the Intellij debugger while running native client test code. However, to be allowed to set breakpoints in the server code, the server must be started with the special flag below: --J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT This ticket is to add support for the above flag in the native client test frameworks. was: To assist with protocol analysis it is very helpful to be able to step through the server code in the Intellij debugger while running native client test code. However, to be allowed to set breakpoints in the server code, the server must be started with the special flag below: ```--J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT``` This ticket is to add support for the above flag in the native client test frameworks. > Launch Cluster with Attachability Flag > -- > > Key: GEODE-10218 > URL: https://issues.apache.org/jira/browse/GEODE-10218 > Project: Geode > Issue Type: Test > Components: native client >Reporter: Michael Martell >Assignee: Michael Martell >Priority: Minor > > To assist with protocol analysis it is very helpful to be able to step > through the server code in the Intellij debugger while running native client > test code. However, to be allowed to set breakpoints in the server code, the > server must be started with the special flag below: > > --J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT > This ticket is to add support for the above flag in the native client test > frameworks. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (GEODE-10218) Launch Cluster with Attachability Flag
[ https://issues.apache.org/jira/browse/GEODE-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Martell reassigned GEODE-10218: --- Assignee: Michael Martell Priority: Minor (was: Major) > Launch Cluster with Attachability Flag > -- > > Key: GEODE-10218 > URL: https://issues.apache.org/jira/browse/GEODE-10218 > Project: Geode > Issue Type: Test > Components: native client >Reporter: Michael Martell >Assignee: Michael Martell >Priority: Minor > > To assist with protocol analysis it is very helpful to be able to step > through the server code in the Intellij debugger while running native client > test code. However, to be allowed to set breakpoints in the server code, the > server must be started with the special flag below: > ```--J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT``` > This ticket is to add support for the above flag in the native client test > frameworks. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (GEODE-10160) SizeableByteArrayList does not update the sizeInBytes for some non-overriden methods
[ https://issues.apache.org/jira/browse/GEODE-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jens Deppe resolved GEODE-10160. Fix Version/s: 1.15.0 Resolution: Fixed > SizeableByteArrayList does not update the sizeInBytes for some non-overriden > methods > > > Key: GEODE-10160 > URL: https://issues.apache.org/jira/browse/GEODE-10160 > Project: Geode > Issue Type: Bug > Components: redis >Affects Versions: 1.15.0 >Reporter: Hale Bales >Assignee: Steve Sienkowski >Priority: Major > Labels: pull-request-available, redis-triage > Fix For: 1.15.0 > > > Some List methods aren't overriden in SizeableByteArrayList which means that > the memoryOverhead doesn't get updated. add(int index, byte[] element), > clear(), and set(int index, byte[] newElement) all need to update the size > and don't. > This can be accomplished by adding overrides to SizeableByteArrayList: > {code:java} > @Override > public byte[] set(int index, byte[] newElement) { > byte[] replacedElement = super.set(index, newElement); > memberOverhead -= calculateByteArrayOverhead(replacedElement); > memberOverhead += calculateByteArrayOverhead(newElement); > return replacedElement; > } > @Override > public void add(int index, byte[] element) { > memberOverhead += calculateByteArrayOverhead(element); > super.add(index, element); > } > {code} > The test for set could look something like: > {code:java} > @Test > public void sizeInBytesGetsUpdatedAccurately_whenDoingSets() { > SizeableByteArrayList list = new SizeableByteArrayList(); > byte[] element = "element".getBytes(StandardCharsets.UTF_8); > list.addFirst(element); > long initialSize = list.getSizeInBytes(); > assertThat(initialSize).isEqualTo(sizer.sizeof(list)); > list.set(0, "some really big new element > name".getBytes(StandardCharsets.UTF_8)); > assertThat(list.getSizeInBytes()).isEqualTo(sizer.sizeof(list)); > list.set(0, element); > assertThat(list.getSizeInBytes()).isEqualTo(initialSize); > } > {code} > We need more tests than just this one. add(int, byte[]) needs to be tested as > well. Any method on SizeableByteArrayList that modify the data should have a > test that the memoryOverhead gets updated correctly. > Clear itself isn't a problem - just set the memberOverhead to 0 - but the > issue is that for the version of LTRIM in > [PR#7403|https://github.com/apache/geode/pull/7403], we clear a sublist of > SizeableByteArrayList. This means that the overridden clear method does not > get called and the LinkedList implementation of clear does not call any other > methods that we can override. There needs to be some new approach to LTRIM > that doesn't use sublists. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (GEODE-10218) Launch Cluster with Attachability Flag
[ https://issues.apache.org/jira/browse/GEODE-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated GEODE-10218: --- Labels: pull-request-available (was: ) > Launch Cluster with Attachability Flag > -- > > Key: GEODE-10218 > URL: https://issues.apache.org/jira/browse/GEODE-10218 > Project: Geode > Issue Type: Test > Components: native client >Reporter: Michael Martell >Assignee: Michael Martell >Priority: Minor > Labels: pull-request-available > > To assist with protocol analysis it is very helpful to be able to step > through the server code in the Intellij debugger while running native client > test code. However, to be allowed to set breakpoints in the server code, the > server must be started with the special flag below: > > --J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT > This ticket is to add support for the above flag in the native client test > frameworks. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10218) Launch Cluster with Attachability Flag
[ https://issues.apache.org/jira/browse/GEODE-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518302#comment-17518302 ] ASF GitHub Bot commented on GEODE-10218: mmartell opened a new pull request, #954: URL: https://github.com/apache/geode-native/pull/954 To assist with protocol analysis it is very helpful to be able to step through the server code in the Intellij debugger while running native client test code. However, to be allowed to set breakpoints in the server code, the server must be started with the special flag below: ``` --J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT``` This ticket is to add support for the above flag in the native client test frameworks. > Launch Cluster with Attachability Flag > -- > > Key: GEODE-10218 > URL: https://issues.apache.org/jira/browse/GEODE-10218 > Project: Geode > Issue Type: Test > Components: native client >Reporter: Michael Martell >Assignee: Michael Martell >Priority: Minor > > To assist with protocol analysis it is very helpful to be able to step > through the server code in the Intellij debugger while running native client > test code. However, to be allowed to set breakpoints in the server code, the > server must be started with the special flag below: > > --J=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:FREEPORT > This ticket is to add support for the above flag in the native client test > frameworks. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (GEODE-10219) CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and support/1.13
Joris Melchior created GEODE-10219: -- Summary: CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and support/1.13 Key: GEODE-10219 URL: https://issues.apache.org/jira/browse/GEODE-10219 Project: Geode Issue Type: Bug Components: gfsh Affects Versions: 1.12.10, 1.13.9 Reporter: Joris Melchior Repeated failure of `acceptance-test-openjdk11` with tests being unable to properly shut down members as part of the tests. See [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main/jobs/acceptance-test-openjdk11/builds/51] See [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-13-main/jobs/acceptance-test-openjdk11/builds/42] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (GEODE-10219) CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and support/1.13
[ https://issues.apache.org/jira/browse/GEODE-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Murmann updated GEODE-10219: -- Labels: needsTriage (was: ) > CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and > support/1.13 > --- > > Key: GEODE-10219 > URL: https://issues.apache.org/jira/browse/GEODE-10219 > Project: Geode > Issue Type: Bug > Components: gfsh >Affects Versions: 1.12.10, 1.13.9 >Reporter: Joris Melchior >Priority: Major > Labels: needsTriage > > Repeated failure of `acceptance-test-openjdk11` with tests being unable to > properly shut down members as part of the tests. > See > [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main/jobs/acceptance-test-openjdk11/builds/51] > See > [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-13-main/jobs/acceptance-test-openjdk11/builds/42] > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10219) CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and support/1.13
[ https://issues.apache.org/jira/browse/GEODE-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518342#comment-17518342 ] Geode Integration commented on GEODE-10219: --- Seen on support/1.12 in [acceptance-test-openjdk11 #51|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main/jobs/acceptance-test-openjdk11/builds/51]. > CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and > support/1.13 > --- > > Key: GEODE-10219 > URL: https://issues.apache.org/jira/browse/GEODE-10219 > Project: Geode > Issue Type: Bug > Components: gfsh >Affects Versions: 1.12.10, 1.13.9 >Reporter: Joris Melchior >Priority: Major > Labels: needsTriage > > Repeated failure of `acceptance-test-openjdk11` with tests being unable to > properly shut down members as part of the tests. > See > [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main/jobs/acceptance-test-openjdk11/builds/51] > See > [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-13-main/jobs/acceptance-test-openjdk11/builds/42] > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (GEODE-10219) CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and support/1.13
[ https://issues.apache.org/jira/browse/GEODE-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk Lund reassigned GEODE-10219: - Assignee: Kirk Lund > CI Failure: geode-assembly:acceptanceTest failing for support/1.12 and > support/1.13 > --- > > Key: GEODE-10219 > URL: https://issues.apache.org/jira/browse/GEODE-10219 > Project: Geode > Issue Type: Bug > Components: gfsh >Affects Versions: 1.12.10, 1.13.9 >Reporter: Joris Melchior >Assignee: Kirk Lund >Priority: Major > Labels: needsTriage > > Repeated failure of `acceptance-test-openjdk11` with tests being unable to > properly shut down members as part of the tests. > See > [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main/jobs/acceptance-test-openjdk11/builds/51] > See > [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-13-main/jobs/acceptance-test-openjdk11/builds/42] > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-9991) SSL protocol and cipher preferences are ignored when endpoint verification is enabled.
[ https://issues.apache.org/jira/browse/GEODE-9991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518352#comment-17518352 ] ASF subversion and git services commented on GEODE-9991: Commit 75ea5f7f6f8f6cbd751463dfaab6342762858c58 in geode's branch refs/heads/develop from Jacob Barrett [ https://gitbox.apache.org/repos/asf?p=geode.git;h=75ea5f7f6f ] GEODE-9991: Refactor for consistency and add tests. (#7533) * Combine common configuration into method for consistency. * Adds tests for new extracted methods. > SSL protocol and cipher preferences are ignored when endpoint verification is > enabled. > -- > > Key: GEODE-9991 > URL: https://issues.apache.org/jira/browse/GEODE-9991 > Project: Geode > Issue Type: Bug > Components: core, security >Affects Versions: 1.12.8, 1.12.9, 1.13.7, 1.13.8, 1.14.3, 1.14.4, 1.15.0 >Reporter: Jacob Barrett >Assignee: Jacob Barrett >Priority: Major > Labels: blocks-1.15.0, pull-request-available, ssl > Fix For: 1.15.0 > > > When SSL endpoint verification is enabled the configuration for protocols and > ciphers reverts to the {{SSLContext}}'s client mode defaults. This can result > in difficulty upgrade the JDK when the newer JDK may use different defaults > for client and server mode SSL. > Oracle JDK 1.8.0_u261 and OpenJDK 1.8.0_u272 replaced the SSL implementation > with a back port from Java 11. This changed the default server protocols from > {{[SSLv2Hello, TLSv1, TLSv1.1, TLSv1.2]}} to {{[TLSv1.3,TLSv1.2,SSLv2Hello]}} > and client to {{[TLSv1.3,TLSv1.2]}}. With this bug the the server protocols > get reset to the client protocols dropping support for the {{SSLv2Hello}} > protocol, which is the first priority protocol by default in the old JDK. > The result is a failure to handshake with the following exception: > {{javax.net.ssl.SSLHandshakeException: SSLv2Hello is not enabled}} > To reproduce you need to have endpoint validation enabled on your SSL > configuration. Set your protocols to `any`. Start 1st locator with JDK older > than 1.8.0_u261. Start 2nd locator with JDK at least as new as JDK > 1.8.0_u272. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10046) bump dependencies in 1.16
[ https://issues.apache.org/jira/browse/GEODE-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518431#comment-17518431 ] ASF subversion and git services commented on GEODE-10046: - Commit f110b9be3547474e74bc16cf2f24337b90e8fae2 in geode's branch refs/heads/develop from Owen Nichols [ https://gitbox.apache.org/repos/asf?p=geode.git;h=f110b9be35 ] GEODE-10046: Bump 3rd-party dependency versions (#7557) Geode endeavors to regularly update 3rd-party dependencies to increase shelf life, security and reliability of releases. Dependency bumps in this batch: * Bump classgraph from 4.8.141 to 4.8.143 * Bump jetty from 9.4.45.v20220203 to 9.4.46.v20220331 * Bump jna from 5.10.0 to 5.11.0 * Bump junit-pioneer from 1.6.1 to 1.6.2 * Bump lettuce-core from 6.1.6.RELEASE to 6.1.8.RELEASE * Bump maven-artifact from 3.8.1 to 3.8.5 * Bump micrometer-core from 1.8.3 to 1.8.4 * Bump nebula.lint from 17.6.1 to 17.7.0 * Bump netty from 4.1.74.Final to 4.1.75.Final * Bump rat from 0.7.0 to 0.7.1 * Bump shiro-core from 1.8.0 to 1.9.0 * Bump spotless from 6.2.2 to 6.4.1 * Bump spring-boot-starter-web from 2.6.5 to 2.6.6 * Bump swagger-annotations from 1.6.2 to 1.6.6 * Bump tomcat from 9.0.59 to 9.0.62 > bump dependencies in 1.16 > - > > Key: GEODE-10046 > URL: https://issues.apache.org/jira/browse/GEODE-10046 > Project: Geode > Issue Type: Improvement > Components: build >Reporter: Owen Nichols >Assignee: Owen Nichols >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > until support/1.16 is cut, periodically check for and switch to latest > version of 3rd-party dependencies. this will extend the shelf-life of > eventual Geode 1.16 release and hopefully reduce bugs and cve exposure, or at > least give a smaller delta if there is later a cve found that we need to > patch for -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (GEODE-9759) User Guide: gfsh command pages - problem with double-hyphens
[ https://issues.apache.org/jira/browse/GEODE-9759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated GEODE-9759: -- Labels: pull-request-available (was: ) > User Guide: gfsh command pages - problem with double-hyphens > > > Key: GEODE-9759 > URL: https://issues.apache.org/jira/browse/GEODE-9759 > Project: Geode > Issue Type: Bug > Components: docs >Affects Versions: 1.14.0 >Reporter: Dave Barnes >Assignee: Dave Barnes >Priority: Major > Labels: pull-request-available > > In the tables of gfsh command options, double-hyphen prefixes are often (but > not always) collapsed to a single hyphen. > This is a format issue with dozens (possibly hundreds?) of occurrences. > See, for example, the `alter` command page, > http://geode.apache.org/docs/guide/114/tools_modules/gfsh/command-pages/alter.html. > Note the difference between the entry for > `--entry-idle-time-expiration-action` and the following entry > `-entry-time-to-live-expiration`. The tricky bit is that the source code in > both cases uses the same HTML construct: `\-\-entry-time-to-live-expiration`, but the results > differ. > One likely remedy is to replace the hyphen pair with the code for two > non-breaking hyphens: `‑‑`. > Behavior may differ depending on whether the entry occurs in an HTML table, a > nested HTML table, or a Markdown table. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10127) Incorrect locator hostname used in remote locator connections
[ https://issues.apache.org/jira/browse/GEODE-10127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518445#comment-17518445 ] ASF subversion and git services commented on GEODE-10127: - Commit 44293ffa54ac85b3fd7681af93aff4107d955fa9 in geode's branch refs/heads/develop from Jacob Barrett [ https://gitbox.apache.org/repos/asf?p=geode.git;h=44293ffa54 ] GEODE-10127: Corrects NullPointerException in hashCode(). (#7559) When unmarshalling an instance with a host name that can't be resolved a NullPointerException is thrown in hashCode(). Use Objects.hash() to resolve hash code to avoid NullPointerException. > Incorrect locator hostname used in remote locator connections > - > > Key: GEODE-10127 > URL: https://issues.apache.org/jira/browse/GEODE-10127 > Project: Geode > Issue Type: Bug > Components: wan >Affects Versions: 1.15.0 >Reporter: Jacob Barrett >Assignee: Jacob Barrett >Priority: Major > Labels: blocks-1.15.0, pull-request-available > Fix For: 1.15.0 > > > When locators in distributed system (DS) B as for locators in DS A they are > sent the local host name and IP address of the locators and not that of the > {{hostname-for-clients}} or {{bind-address}} properties. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10214) Improve performance of JvmSizeUtils.roundUpSize
[ https://issues.apache.org/jira/browse/GEODE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518464#comment-17518464 ] ASF subversion and git services commented on GEODE-10214: - Commit 982f846fb5d8b61e617c7d0e1a484ea0664229ab in geode's branch refs/heads/develop from Jens Deppe [ https://gitbox.apache.org/repos/asf?p=geode.git;h=982f846fb5 ] GEODE-10214: Improve speed of JvmSizeUtils.roundUpSize (#7548) This small change improves the speed of this method by about 30% (as per basic JMH benchmarking). benchamrk1 is the original and benchmark2 the change. ``` Benchmark Mode Cnt Score Error Units PerformanceSample.benchmark1 avgt5 3480.815 ± 387.131 us/op PerformanceSample.benchmark2 avgt5 2418.937 ± 586.209 us/op ``` > Improve performance of JvmSizeUtils.roundUpSize > --- > > Key: GEODE-10214 > URL: https://issues.apache.org/jira/browse/GEODE-10214 > Project: Geode > Issue Type: Improvement > Components: core >Reporter: Jens Deppe >Assignee: Jens Deppe >Priority: Major > Labels: pull-request-available > > This small method can still be optimized. It is used extensively (albeit > indirectly) in much of the geode-for-redis modules' data structure size > calculations. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (GEODE-10214) Improve performance of JvmSizeUtils.roundUpSize
[ https://issues.apache.org/jira/browse/GEODE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jens Deppe resolved GEODE-10214. Fix Version/s: 1.15.0 Resolution: Fixed > Improve performance of JvmSizeUtils.roundUpSize > --- > > Key: GEODE-10214 > URL: https://issues.apache.org/jira/browse/GEODE-10214 > Project: Geode > Issue Type: Improvement > Components: core >Reporter: Jens Deppe >Assignee: Jens Deppe >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > This small method can still be optimized. It is used extensively (albeit > indirectly) in much of the geode-for-redis modules' data structure size > calculations. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (GEODE-10045) Upgrade to Micrometer 2.0
[ https://issues.apache.org/jira/browse/GEODE-10045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Blum updated GEODE-10045: -- Priority: Major (was: Blocker) > Upgrade to Micrometer 2.0 > - > > Key: GEODE-10045 > URL: https://issues.apache.org/jira/browse/GEODE-10045 > Project: Geode > Issue Type: Improvement > Components: core >Affects Versions: 1.14.3 >Reporter: John Blum >Priority: Major > Labels: Micrometer > > As part of the ongoing story and themes in Spring Framework 6, Spring Data > 3.0, Spring Boot 3.0 and the rest of the Spring portfolio, 1 of the new and > major topics is Observability (Metrics and Monitoring, with Tracing, and so > on). > As part of that effort, Micrometer is undergoing a major revision change from > 1.x to 2.0. Many of their APIs have changed, and as a result, Apache Geode > no longer runs (or even will build) with Micrometer 2.0. > Either Micrometer should be upgraded to 2.0 (most likely in the next major > version of Geode, i.e. 2.0) or Micrometer should be an optional dependency, > perhaps only enabled when Micrometer is on the classpath. > Still a cleaner separation is needed if [Spring] Apache Geode users require > and use a newer version of Micrometer (e.g. 2.0) and Apache Geode remains on > Micrometer 1.x (currently > [1.6.3|https://github.com/apache/geode/blob/rel/v1.14.3/boms/geode-all-bom/src/test/resources/expected-pom.xml#L226-L231] > in Apache Geode {{1.14.3}}). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10122) With TLSv1.3 and GCM-based cipher (the default), P2P Messaging Fails When Encrypted Data Limit is Reached
[ https://issues.apache.org/jira/browse/GEODE-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518561#comment-17518561 ] ASF subversion and git services commented on GEODE-10122: - Commit d2535394a82ac5faf10f004f4e3c15f756f7b177 in geode's branch refs/heads/develop from Bill Burcham [ https://gitbox.apache.org/repos/asf?p=geode.git;h=d2535394a8 ] GEODE-10122: P2P Messaging Handles TLS KeyUpdate Message (#7449) * Key expiration works for TLSv1.3 and GCM-based ciphers * TLS KeyUpdate messages are processed correctly > With TLSv1.3 and GCM-based cipher (the default), P2P Messaging Fails When > Encrypted Data Limit is Reached > - > > Key: GEODE-10122 > URL: https://issues.apache.org/jira/browse/GEODE-10122 > Project: Geode > Issue Type: Bug > Components: messaging >Affects Versions: 1.13.7, 1.14.3, 1.15.0 >Reporter: Bill Burcham >Assignee: Bill Burcham >Priority: Major > Labels: blocks-1.15.0, pull-request-available, ssl > Attachments: patch-P2PMessagingConcurrencyDUnitTest.txt > > > TLSv1.3 introduced [1] the ability to set per-algorithm limits on symmetric > key usage lifetimes. Once a certain number of bytes have been encrypted, a > KeyUpdate post-handshake message [2] is sent. > With default settings, on Liberica JDK 11, Geode's P2P framework will > negotiate TLSv1.3 with the TLS_AES_256_GCM_SHA384 cipher suite. Geode P2P > messaging will eventually fail, with a "Tag mismatch!" IOException in shared > ordered receivers, after a session has been in heavy use for days. > We have not see this failure on TLSv1.2. > The implementation of TLSv1.3 in the Java runtime provides a security > property [3] to configure the encrypted data limit. The attached patch to > P2PMessagingConcurrencyDUnitTest configures the limit large enough that the > test makes it through the (P2P) TLS handshake but small enough so that the > "Tag mismatch!" exception is encountered less than a minute later. > The bug is caused by Geode’s NioSslEngine class’ ignorance of the > “rehandshaking” phase of the TLS protocol [4]: > Creation - ready to be configured. > Initial handshaking - perform authentication and negotiate communication > parameters. > Application data - ready for application exchange. > *Rehandshaking* - renegotiate communications parameters/authentication; > handshaking data may be mixed with application data. > Closure - ready to shut down connection. > Geode's tcp.Connection and NioSslEngine classes (particularly wrap() and > unwrap()), as they are currently implemented, fail to fully attend to the > handshake status from javax.net.ssl.SSLEngine. As a result these Geode > classes fail to respond to the KeyUpdate message, resulting in the "Tag > mismatch!" IOException. > When that exception is encountered, the Connection is destroyed and a new one > created in its place. But users of the old Connection, waiting for > acknowledgements, will never receive them. This can result in cluster-wide > hangs. > [1] [https://datatracker.ietf.org/doc/html/rfc8446#section-5.5] > [2] > [https://www.ibm.com/docs/en/sdk-java-technology/8?topic=handshake-post-messages] > > [3] > [https://docs.oracle.com/en/java/javase/11/security/java-secure-socket-extension-jsse-reference-guide.html#GUID-B970ADD6-1E9F-4C18-A26E-0679B50CC946] > [4] [https://www.ibm.com/docs/en/sdk-java-technology/7.1?topic=sslengine-] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (GEODE-10122) With TLSv1.3 and GCM-based cipher (the default), P2P Messaging Fails When Encrypted Data Limit is Reached
[ https://issues.apache.org/jira/browse/GEODE-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Burcham resolved GEODE-10122. -- Fix Version/s: 1.15.0 Resolution: Fixed > With TLSv1.3 and GCM-based cipher (the default), P2P Messaging Fails When > Encrypted Data Limit is Reached > - > > Key: GEODE-10122 > URL: https://issues.apache.org/jira/browse/GEODE-10122 > Project: Geode > Issue Type: Bug > Components: messaging >Affects Versions: 1.13.7, 1.14.3, 1.15.0 >Reporter: Bill Burcham >Assignee: Bill Burcham >Priority: Major > Labels: blocks-1.15.0, pull-request-available, ssl > Fix For: 1.15.0 > > Attachments: patch-P2PMessagingConcurrencyDUnitTest.txt > > > TLSv1.3 introduced [1] the ability to set per-algorithm limits on symmetric > key usage lifetimes. Once a certain number of bytes have been encrypted, a > KeyUpdate post-handshake message [2] is sent. > With default settings, on Liberica JDK 11, Geode's P2P framework will > negotiate TLSv1.3 with the TLS_AES_256_GCM_SHA384 cipher suite. Geode P2P > messaging will eventually fail, with a "Tag mismatch!" IOException in shared > ordered receivers, after a session has been in heavy use for days. > We have not see this failure on TLSv1.2. > The implementation of TLSv1.3 in the Java runtime provides a security > property [3] to configure the encrypted data limit. The attached patch to > P2PMessagingConcurrencyDUnitTest configures the limit large enough that the > test makes it through the (P2P) TLS handshake but small enough so that the > "Tag mismatch!" exception is encountered less than a minute later. > The bug is caused by Geode’s NioSslEngine class’ ignorance of the > “rehandshaking” phase of the TLS protocol [4]: > Creation - ready to be configured. > Initial handshaking - perform authentication and negotiate communication > parameters. > Application data - ready for application exchange. > *Rehandshaking* - renegotiate communications parameters/authentication; > handshaking data may be mixed with application data. > Closure - ready to shut down connection. > Geode's tcp.Connection and NioSslEngine classes (particularly wrap() and > unwrap()), as they are currently implemented, fail to fully attend to the > handshake status from javax.net.ssl.SSLEngine. As a result these Geode > classes fail to respond to the KeyUpdate message, resulting in the "Tag > mismatch!" IOException. > When that exception is encountered, the Connection is destroyed and a new one > created in its place. But users of the old Connection, waiting for > acknowledgements, will never receive them. This can result in cluster-wide > hangs. > [1] [https://datatracker.ietf.org/doc/html/rfc8446#section-5.5] > [2] > [https://www.ibm.com/docs/en/sdk-java-technology/8?topic=handshake-post-messages] > > [3] > [https://docs.oracle.com/en/java/javase/11/security/java-secure-socket-extension-jsse-reference-guide.html#GUID-B970ADD6-1E9F-4C18-A26E-0679B50CC946] > [4] [https://www.ibm.com/docs/en/sdk-java-technology/7.1?topic=sslengine-] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (GEODE-10088) Support ServerGroups in new test frameworks
[ https://issues.apache.org/jira/browse/GEODE-10088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518575#comment-17518575 ] ASF GitHub Bot commented on GEODE-10088: mmartell commented on PR #941: URL: https://github.com/apache/geode-native/pull/941#issuecomment-1091060612 > Please add at least one trivial integration test that exercises this code. Thanks for adding this, it will be helpful to improve our test coverage! Modififed the existing StartServerStringsTest (in new test framework) to validate that .withGroups() adds the --groups arg to start server command. Also, we already have tests that exercise the --groups functionality on the client side: testXmlCacheCreationWithPools > Support ServerGroups in new test frameworks > --- > > Key: GEODE-10088 > URL: https://issues.apache.org/jira/browse/GEODE-10088 > Project: Geode > Issue Type: Test > Components: native client >Reporter: Michael Martell >Priority: Minor > Labels: pull-request-available > > The new integration test framewoks for .NET and C++ do not support server > groups. Adding this support will help in creating a protocol document, as > several messages support groups. -- This message was sent by Atlassian Jira (v8.20.1#820001)