[jira] [Created] (CASSANDRA-14275) Cassandra Driver should send identification information to Server
Dinesh Joshi created CASSANDRA-14275: Summary: Cassandra Driver should send identification information to Server Key: CASSANDRA-14275 URL: https://issues.apache.org/jira/browse/CASSANDRA-14275 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Dinesh Joshi Assignee: Dinesh Joshi Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x Currently there doesn't seem to be any way to readily identify the driver that clients are using to connect to Cassandra. Add the capability of identifying the driver through metadata information much like how HTTP Clients identify themselves through User-Agent HTTP header. This is useful for debugging in large deployments where clients tend to use different drivers, wrappers and language bindings to connect to Cassandra. This can help surface issues as well as help detect clients which are using older or unsupported drivers. The identification information should be a string that unambiguously identifies the driver. It should include information such as the name of the driver, it's version, CQL version, Platform (Linux, macOS, Windows, etc.) and architecture (x86, x86_64). We should surface this information in `nodetool clientstats` command. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch
[ https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378100#comment-16378100 ] Simon Zhou commented on CASSANDRA-14252: For nodes in the same remote data center, if we don't have score for one node because there is no read response yet and we set an artificially low score 0 for it, does it mean this node will be picked with higher probability than other nodes that have scores? > Use zero as default score in DynamicEndpointSnitch > -- > > Key: CASSANDRA-14252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14252 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Major > Fix For: 4.0, 3.0.17, 3.11.3 > > > The problem I want to solve is that I found in our deployment, one slow but > alive data node can slow down the whole cluster, even caused timeout of our > requests. > We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the > DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node > latency is too high. > I added some debug log, and figured out that in a lot of cases, the score > from remote data node was not populated, so the fallback to > sortByProximityWithScore never happened. That's why a single slow data node, > can cause huge problems to the whole cluster. > In this jira, I'd like to use zero as default score, so that we will get a > chance to try remote data node, if local one is slow. > I tested it in our test cluster, it improved the client latency in single > slow data node case significantly. > I flag this as a Bug, because it caused problems to our use cases multiple > times. > logs === > _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [0.0]_ > _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch
[ https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372232#comment-16372232 ] Dikang Gu edited comment on CASSANDRA-14252 at 2/27/18 5:19 AM: |[3.0 | https://github.com/DikangGu/cassandra/commit/163ad12c7c84cfd44932b814ce61c7a0145e9bcd]|[3.11 | https://github.com/DikangGu/cassandra/commit/980a7c36d410448b3714111b541193b68c952d34]|[trunk | https://github.com/DikangGu/cassandra/commit/fd00c51321f6252294b066a758bfaaa5ba810ea9]|[unit test | https://circleci.com/gh/DikangGu/cassandra/20]| was (Author: dikanggu): |[3.0 | https://github.com/DikangGu/cassandra/commit/163ad12c7c84cfd44932b814ce61c7a0145e9bcd]|[3.11 | https://github.com/DikangGu/cassandra/commit/980a7c36d410448b3714111b541193b68c952d34]|[trunk | https://github.com/DikangGu/cassandra/commit/fd00c51321f6252294b066a758bfaaa5ba810ea9]|[unit test | https://circleci.com/gh/DikangGu/cassandra/14]| > Use zero as default score in DynamicEndpointSnitch > -- > > Key: CASSANDRA-14252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14252 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Major > Fix For: 4.0, 3.0.17, 3.11.3 > > > The problem I want to solve is that I found in our deployment, one slow but > alive data node can slow down the whole cluster, even caused timeout of our > requests. > We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the > DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node > latency is too high. > I added some debug log, and figured out that in a lot of cases, the score > from remote data node was not populated, so the fallback to > sortByProximityWithScore never happened. That's why a single slow data node, > can cause huge problems to the whole cluster. > In this jira, I'd like to use zero as default score, so that we will get a > chance to try remote data node, if local one is slow. > I tested it in our test cluster, it improved the client latency in single > slow data node case significantly. > I flag this as a Bug, because it caused problems to our use cases multiple > times. > logs === > _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [0.0]_ > _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12151) Audit logging for database activity
[ https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377534#comment-16377534 ] Vinay Chella edited comment on CASSANDRA-12151 at 2/27/18 2:22 AM: --- Hi [~eanujwa] [~jasobrown], I’m excited to see the design document and it looks good to us! Netflix had a similar requirement recently for our internal 2.1 clusters and we implemented a simple version (no query categories, etc…) for sox auditing. As your design is very close to what we implemented, just a few differently named classes for the most part, can we work together on the trunk [patchset|https://github.com/vinaykumarchella/cassandra/pull/2] to add the missing components from your design? Alternatively, we could take an incremental approach, review what we have on the trunk branch of the simple version and get it committed and then add in some of the more advanced features next. I believe this patch follows the design goals that you put together. Please review and let me know if you have any questions or concerns about the first iteration. If folks are interested in the 3.x/2.x branches I can put those up on my github as well. [~jhb] {quote}I just have one question, do you think enabling/updating/disabling audit require a node restart? {quote} The posted patch allows online auditlog enable/disable via JMX. [~jjordan] {quote}You should take a look at the infrastructure added in CASSANDRA-13983 for query logging {quote} Yes, we looked and that certainly looks interesting, perhaps this design allows us to use it as another implementation of {{IAuditLogger}}? was (Author: vinaykumarcse): Hi [~eanujwa] [~jasobrown], I’m excited to see the design document and it looks good to us! Netflix had a similar requirement recently for our internal 2.1 clusters and we implemented a simple version (no query categories, etc…) for sox auditing. As your design is very close to what we implemented, just a few differently named classes for the most part, can we work together on the trunk [patchset|https://github.com/vinaykumarchella/cassandra/pull/2] to add the missing components from your design? Alternatively, we could take an incremental approach, review what we have on the trunk branch of the simple version and get it committed and then add in some of the more advanced features next. I believe this patch follows the design goals that you put together. Please review and let me know if you have any questions or concerns about the first iteration. If folks are interested in the 3.x/2.x branches I can put those up on my github as well. [~jhb] {quote}I just have one question, do you think enabling/updating/disabling audit require a node restart? {quote} The posted patch allows online auditlog enable/disable or filter updates via JMX. [~jjordan] {quote}You should take a look at the infrastructure added in CASSANDRA-13983 for query logging {quote} Yes, we looked and that certainly looks interesting, perhaps this design allows us to use it as another implementation of {{IAuditLogger}}? > Audit logging for database activity > --- > > Key: CASSANDRA-12151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12151 > Project: Cassandra > Issue Type: New Feature >Reporter: stefan setyadi >Assignee: Anuj Wadehra >Priority: Major > Fix For: 4.x > > Attachments: 12151.txt, > DesignProposal_AuditingFeature_ApacheCassandra_v1.docx > > > we would like a way to enable cassandra to log database activity being done > on our server. > It should show username, remote address, timestamp, action type, keyspace, > column family, and the query statement. > it should also be able to log connection attempt and changes to the > user/roles. > I was thinking of making a new keyspace and insert an entry for every > activity that occurs. > Then It would be possible to query for specific activity or a query targeting > a specific keyspace and column family. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch
[ https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377885#comment-16377885 ] Dikang Gu commented on CASSANDRA-14252: --- [~szhou], 1, We will need to populate the scores for all replicas first, which should be finished within seconds. After that, a local and healthy node will always have lower score than remotes nodes, then we can stick to local node, and only switch to remote node if local node is bad, right? 2. Yeah, we back-ported that patch and disabled the IO factor for severity already. > Use zero as default score in DynamicEndpointSnitch > -- > > Key: CASSANDRA-14252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14252 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Major > Fix For: 4.0, 3.0.17, 3.11.3 > > > The problem I want to solve is that I found in our deployment, one slow but > alive data node can slow down the whole cluster, even caused timeout of our > requests. > We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the > DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node > latency is too high. > I added some debug log, and figured out that in a lot of cases, the score > from remote data node was not populated, so the fallback to > sortByProximityWithScore never happened. That's why a single slow data node, > can cause huge problems to the whole cluster. > In this jira, I'd like to use zero as default score, so that we will get a > chance to try remote data node, if local one is slow. > I tested it in our test cluster, it improved the client latency in single > slow data node case significantly. > I flag this as a Bug, because it caused problems to our use cases multiple > times. > logs === > _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [0.0]_ > _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-11075) Consider making SASI the default index implementation
[ https://issues.apache.org/jira/browse/CASSANDRA-11075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mck updated CASSANDRA-11075: Component/s: sasi > Consider making SASI the default index implementation > - > > Key: CASSANDRA-11075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11075 > Project: Cassandra > Issue Type: Improvement > Components: sasi >Reporter: Sylvain Lebresne >Assignee: Pavel Yaskevich >Priority: Major > Labels: sasi > > We now have 2 secondary index implementation in tree: the old native ones and > SASI. Moving forward, that feels like one too much to maintain, especially > since it seems that SASI is an overall better implementation. > So we should gather enough data to decide if SASI is indeed always better (or > at least sufficiently better than we're convinced no-one would want to stick > with the native implementation), and if that's the case, we should consider > making it the default (and ultimately get rid of the current implementation). > So first, we should at least: > # double check that SASI handles all cases that the native implementation > handles. A good start would probably be run all our dtest and utests on a > version where SASI is hard-coded as default. > # compare the performance of SASI and native indexes. In particular our > native indexes, in all their weaknesses, have the advantage of not doing a > read-before-write. Haven't looked at SASI much so I don't know if it's the > case but anyway, we need numbers on both reads and writes. > Once we have that, if we do decide to make SASI the default, then we need to > figure out what is the upgrade path (and whether we add extra syntax for SASI > specific options). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14055) Index redistribution breaks SASI index
[ https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mck updated CASSANDRA-14055: Labels: patch sasi (was: patch) > Index redistribution breaks SASI index > -- > > Key: CASSANDRA-14055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14055 > Project: Cassandra > Issue Type: Bug > Components: sasi >Reporter: Ludovic Boutros >Assignee: Ludovic Boutros >Priority: Major > Labels: patch, sasi > Fix For: 3.11.x, 4.x > > Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, > CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, > CASSANDRA-14055.patch > > > During index redistribution process, a new view is created. > During this creation, old indexes should be released. > But, new indexes are "attached" to the same SSTable as the old indexes. > This leads to the deletion of the last SASI index file and breaks the index. > The issue is in this function : > [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10765) add RangeIterator interface and QueryPlan for SI
[ https://issues.apache.org/jira/browse/CASSANDRA-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mck updated CASSANDRA-10765: Labels: 2i sasi (was: ) > add RangeIterator interface and QueryPlan for SI > > > Key: CASSANDRA-10765 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10765 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Major > Labels: 2i, sasi > Fix For: 4.x > > Attachments: server-load.png > > > Currently built-in indexes have only one way of handling > intersections/unions: pick the highest selectivity predicate and filter on > other index expressions. This is not always the most efficient approach. > Dynamic query planning based on the different index characteristics would be > more optimal. Query Plan should be able to choose how to do intersections, > unions based on the metadata provided by indexes (returned by RangeIterator) > and RangeIterator would became a base for cross index interactions and should > have information such as min/max token, estimate number of wrapped tokens etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-11182) Enable SASI index for collections
[ https://issues.apache.org/jira/browse/CASSANDRA-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mck updated CASSANDRA-11182: Labels: sasi (was: ) > Enable SASI index for collections > - > > Key: CASSANDRA-11182 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11182 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: DOAN DuyHai >Priority: Minor > Labels: sasi > > This is a follow up ticket for post Cassandra 3.4 SASI integration. > Right now it is possible with standard Cassandra 2nd index to: > 1. index list and set elements ( {{WHERE list CONTAINS xxx}}) > 2. index map keys ( {{WHERE map CONTAINS KEYS 'abc'}} ) > 3. index map entries ( {{WHERE map\['key'\]=value}}) > It would be nice to enable these features in SASI too. > With regard to tokenizing, we might want to allow wildcards ({{%}}) with the > CONTAINS syntax as well as with index map entries. Ex: > * {{WHERE list CONTAINS 'John%'}} > * {{WHERE map CONTAINS KEY '%an%'}} > * {{WHERE map\['key'\] LIKE '%val%'}} > /cc [~xedin] [~rustyrazorblade] [~jkrupan] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377767#comment-16377767 ] Venkata Harikrishna Nukala commented on CASSANDRA-13853: [~pree] Please go ahead. Took my step back. > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Harikrishna Nukala updated CASSANDRA-13853: --- Attachment: (was: 13853-trunk.txt) > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Harikrishna Nukala updated CASSANDRA-13853: --- Comment: was deleted (was: [~pree] My apologies if it is against the Cassandra community guidelines. I am also a newbie. The last update I saw was on last year. I thought this ticket is stale, so picked it up. I don't mind taking a step back.) > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Harikrishna Nukala updated CASSANDRA-13853: --- Comment: was deleted (was: [~rustyrazorblade] [~pree] Hi, I made a patch (attached) for the required changes. Hope you don't mind. With these changes, "nodetool describecluster" output looks like this: {noformat} Cluster Information: Name: test [^13853-trunk.txt] Live nodes: 6 Joining nodes: [127.0.0.6, 127.0.0.5, 127.0.0.4, 127.0.0.3, 127.0.0.2, 127.0.0.1] Moving nodes: [] Leaving nodes: [] Keyspaces: system_traces RF: 2 system RF: 1 system_distributed RF: 3 system_schema RF: 1 system_auth RF: 1 Release versions: 4.0.0: [127.0.0.6, 127.0.0.5, 127.0.0.4, 127.0.0.3, 127.0.0.2, 127.0.0.1] Datacenters: dc1 Up# 3 Down# 0 Unknown# 0 dc2 Up# 3 Down# 0 Unknown# 0 Snitch: org.apache.cassandra.locator.PropertyFileSnitch DynamicEndPointSnitch: enabled Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: cc4dbfad-1e8f-386d-9a79-de80def27f33: [127.0.0.6, 127.0.0.5, 127.0.0.4, 127.0.0.3, 127.0.0.2, 127.0.0.1] {noformat} Let me know what you think.) > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377754#comment-16377754 ] Preetika Tyagi commented on CASSANDRA-13853: The last update was on 28th December (last year) and I clearly mentioned that I need to run it through some process before pushing it here. Since I'm new to that as well, I ended up taking more time. Ideally, it is best to ask in comments to confirm that the same work is not repeated. At this point, I will leave it to up to [~rustyrazorblade] since he is the reporter of this bug. > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377729#comment-16377729 ] Venkata Harikrishna Nukala edited comment on CASSANDRA-13853 at 2/26/18 10:47 PM: -- [~pree] My apologies if it is against the Cassandra community guidelines. I am also a newbie. The last update I saw was on last year. I thought this ticket is stale, so picked it up. I don't mind taking a step back. was (Author: hari_nv): [~pree] My apologies if it is against the Cassandra community guidelines. I am also a newbie. That last update I saw was on last year. I thought this ticket is stale, so picked it up. I don't mind taking a step back. > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377729#comment-16377729 ] Venkata Harikrishna Nukala commented on CASSANDRA-13853: [~pree] My apologies if it is against the Cassandra community guidelines. I am also a newbie. That last update I saw was on last year. I thought this ticket is stale, so picked it up. I don't mind taking a step back. > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377717#comment-16377717 ] Preetika Tyagi edited comment on CASSANDRA-13853 at 2/26/18 10:39 PM: -- I am still new to the Cassandra community but is it okay to work on a ticket even when it is assigned to someone else and that person is working on it already? was (Author: pree): I am still new to the Cassandra community but is it okay to work on a ticket even when it is assigned to someone else and that person is working on it already? > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377717#comment-16377717 ] Preetika Tyagi commented on CASSANDRA-13853: I am still new to the Cassandra community but is it okay to work on a ticket even when it is assigned to someone else and that person is working on it already? > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14269) The Data Modeling section of the web site is empty
[ https://issues.apache.org/jira/browse/CASSANDRA-14269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377714#comment-16377714 ] Michael Mior commented on CASSANDRA-14269: -- A few resources that someone wanting to tackle this may want to look at: https://www.datastax.com/dev/blog/basic-rules-of-cassandra-data-modeling https://www.ebayinc.com/stories/blogs/tech/cassandra-data-modeling-best-practices-part-1/ https://www.ebayinc.com/stories/blogs/tech/cassandra-data-modeling-best-practices-part-2/ > The Data Modeling section of the web site is empty > -- > > Key: CASSANDRA-14269 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14269 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Kenneth Brotman >Priority: Major > > [http://cassandra.apache.org/doc/latest/data_modeling/index.html] > is empty. Please submit content. Myself or someone else will take it from > there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14273) The Bulk Loading web page on the web site is empty
[ https://issues.apache.org/jira/browse/CASSANDRA-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Brotman updated CASSANDRA-14273: Summary: The Bulk Loading web page on the web site is empty (was: The Bulk Loading web page on the web site is emplty) > The Bulk Loading web page on the web site is empty > -- > > Key: CASSANDRA-14273 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14273 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Kenneth Brotman >Priority: Major > > [http://cassandra.apache.org/doc/latest/operating/bulk_loading.html] > is empty. Please contribute content. Myself or someone else will take it > from there. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14274) The troubleshooting section of the web site is empty
Kenneth Brotman created CASSANDRA-14274: --- Summary: The troubleshooting section of the web site is empty Key: CASSANDRA-14274 URL: https://issues.apache.org/jira/browse/CASSANDRA-14274 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/troubleshooting/index.html] is empty. Please contribute content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14273) The Bulk Loading web page on the web site is emplty
Kenneth Brotman created CASSANDRA-14273: --- Summary: The Bulk Loading web page on the web site is emplty Key: CASSANDRA-14273 URL: https://issues.apache.org/jira/browse/CASSANDRA-14273 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/operating/bulk_loading.html] is empty. Please contribute content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14272) The Backups web page on the web site is empty
Kenneth Brotman created CASSANDRA-14272: --- Summary: The Backups web page on the web site is empty Key: CASSANDRA-14272 URL: https://issues.apache.org/jira/browse/CASSANDRA-14272 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/operating/backups.html] is empty. Please contribute content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14271) The Hints web page in the web site is empty
Kenneth Brotman created CASSANDRA-14271: --- Summary: The Hints web page in the web site is empty Key: CASSANDRA-14271 URL: https://issues.apache.org/jira/browse/CASSANDRA-14271 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/operating/hints.html] is empty. Please contribute content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14270) The Read repair web page is empty
Kenneth Brotman created CASSANDRA-14270: --- Summary: The Read repair web page is empty Key: CASSANDRA-14270 URL: https://issues.apache.org/jira/browse/CASSANDRA-14270 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/operating/read_repair.html] is empty. Please contribute content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14269) The Data Modeling section of the web site is empty
Kenneth Brotman created CASSANDRA-14269: --- Summary: The Data Modeling section of the web site is empty Key: CASSANDRA-14269 URL: https://issues.apache.org/jira/browse/CASSANDRA-14269 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/data_modeling/index.html] is empty. Please submit content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14268) The Architecture:Guarantees web page is empty
Kenneth Brotman created CASSANDRA-14268: --- Summary: The Architecture:Guarantees web page is empty Key: CASSANDRA-14268 URL: https://issues.apache.org/jira/browse/CASSANDRA-14268 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/architecture/guarantees.html] Please submit content and myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14267) The Dynamo web page on the Apache Cassandra site is missing content
Kenneth Brotman created CASSANDRA-14267: --- Summary: The Dynamo web page on the Apache Cassandra site is missing content Key: CASSANDRA-14267 URL: https://issues.apache.org/jira/browse/CASSANDRA-14267 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman Content for the following topics is needed for [http://cassandra.apache.org/doc/latest/architecture/dynamo.html] Gossip Failure Detection Token Ring/Ranges Please post content. Myself or someone else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14266) The Architecture Overview web page on the Apache Cassandra site is empty
Kenneth Brotman created CASSANDRA-14266: --- Summary: The Architecture Overview web page on the Apache Cassandra site is empty Key: CASSANDRA-14266 URL: https://issues.apache.org/jira/browse/CASSANDRA-14266 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman [http://cassandra.apache.org/doc/latest/architecture/overview.html] Please contribute content. Myself or some else will take it from there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14241) Apache dtests not passing after pytest/python 3
[ https://issues.apache.org/jira/browse/CASSANDRA-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-14241: Resolution: Fixed Status: Resolved (was: Patch Available) and committed, thanks! > Apache dtests not passing after pytest/python 3 > --- > > Key: CASSANDRA-14241 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14241 > Project: Cassandra > Issue Type: Task > Components: Testing >Reporter: Ariel Weisberg >Assignee: Marcus Eriksson >Priority: Major > > Apache dtests are still not running correctly yet with pytest. Most of the > tests are running and passing but a solid chunk are still failing and these > are tests that don't fail in CircleCI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra-dtest git commit: Dont call decode
Repository: cassandra-dtest Updated Branches: refs/heads/master 3d0d0f45f -> 7fd89e146 Dont call decode Patch by marcuse; reviewed by Michael Shuler for CASSANDRA-14241 Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/7fd89e14 Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/7fd89e14 Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/7fd89e14 Branch: refs/heads/master Commit: 7fd89e14653ab196c5cb5cddf862ae060608056a Parents: 3d0d0f4 Author: Marcus Eriksson Authored: Fri Feb 23 08:32:42 2018 -0800 Committer: Marcus Eriksson Committed: Mon Feb 26 13:50:12 2018 -0800 -- bootstrap_test.py | 2 +- meta_tests/assertion_test.py| 2 +- offline_tools_test.py | 20 ++-- repair_tests/incremental_repair_test.py | 12 ++-- tools/assertions.py | 2 +- 5 files changed, 19 insertions(+), 19 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7fd89e14/bootstrap_test.py -- diff --git a/bootstrap_test.py b/bootstrap_test.py index 22dddcd..661aef9 100644 --- a/bootstrap_test.py +++ b/bootstrap_test.py @@ -335,7 +335,7 @@ class TestBootstrap(Tester): stdout, stderr, _ = node3.stress(['read', 'n=1k', 'no-warmup', '-schema', 'replication(factor=2)', '-rate', 'threads=8']) if stdout is not None: -assert "FAILURE" not in stdout.decode("utf-8") +assert "FAILURE" not in stdout @since('2.2') def test_bootstrap_with_reset_bootstrap_state(self): http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7fd89e14/meta_tests/assertion_test.py -- diff --git a/meta_tests/assertion_test.py b/meta_tests/assertion_test.py index 2c3fe09..f5d8b51 100644 --- a/meta_tests/assertion_test.py +++ b/meta_tests/assertion_test.py @@ -8,7 +8,7 @@ from tools.assertions import (assert_all, assert_almost_equal, assert_exception, assert_invalid, assert_length_equal, assert_none, assert_one, assert_row_count, assert_stderr_clean, assert_unauthorized, assert_unavailable) - +import pytest class TestAssertStderrClean(TestCase): http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/7fd89e14/offline_tools_test.py -- diff --git a/offline_tools_test.py b/offline_tools_test.py index 9e3dc73..fce1f5f 100644 --- a/offline_tools_test.py +++ b/offline_tools_test.py @@ -54,7 +54,7 @@ class TestOfflineTools(Tester): output, error, rc = node1.run_sstablelevelreset("keyspace1", "standard1") self._check_stderr_error(error) -assert re.search("Found no sstables, did you give the correct keyspace", output.decode("utf-8")) +assert re.search("Found no sstables, did you give the correct keyspace", output) assert rc == 0, str(rc) # test by writing small amount of data and flushing (all sstables should be level 0) @@ -67,8 +67,8 @@ class TestOfflineTools(Tester): cluster.stop(gently=False) output, error, rc = node1.run_sstablelevelreset("keyspace1", "standard1") -self._check_stderr_error(error.decode("utf-8")) -assert re.search("since it is already on level 0", output.decode("utf-8")) +self._check_stderr_error(error) +assert re.search("since it is already on level 0", output) assert rc == 0, str(rc) # test by loading large amount data so we have multiple levels and checking all levels are 0 at end @@ -82,7 +82,7 @@ class TestOfflineTools(Tester): initial_levels = self.get_levels(node1.run_sstablemetadata(keyspace="keyspace1", column_families=["standard1"])) _, error, rc = node1.run_sstablelevelreset("keyspace1", "standard1") final_levels = self.get_levels(node1.run_sstablemetadata(keyspace="keyspace1", column_families=["standard1"])) -self._check_stderr_error(error.decode("utf-8")) +self._check_stderr_error(error) assert rc == 0, str(rc) logger.debug(initial_levels) @@ -96,7 +96,7 @@ class TestOfflineTools(Tester): def get_levels(self, data): (out, err, rc) = data -return list(map(int, re.findall("SSTable Level: ([0-9])", out.decode("utf-8" +return list(map(int, re.findall("SSTable Level: ([0-9])", out))) def wait_for_compactions(self, node): pattern = re.compile("pending tasks: 0") @@ -139,7 +139,7 @@ class TestOfflineTools(Tester): try: output, error, _ = nod
cassandra-builds git commit: Move apache dtests to running in docker
Repository: cassandra-builds Updated Branches: refs/heads/master 27bc20645 -> 2c1842cef Move apache dtests to running in docker use dumb-init to get signal handling to work (https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html) Patch by marcuse; reviewed by Michael Shuler for CASSANDRA-14241 Project: http://git-wip-us.apache.org/repos/asf/cassandra-builds/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra-builds/commit/2c1842ce Tree: http://git-wip-us.apache.org/repos/asf/cassandra-builds/tree/2c1842ce Diff: http://git-wip-us.apache.org/repos/asf/cassandra-builds/diff/2c1842ce Branch: refs/heads/master Commit: 2c1842cef7ce08327c95114df635fab1f81b8bc8 Parents: 27bc206 Author: Marcus Eriksson Authored: Fri Feb 23 09:48:08 2018 -0800 Committer: Marcus Eriksson Committed: Mon Feb 26 13:47:54 2018 -0800 -- build-scripts/cassandra-dtest-pytest.sh | 8 ++-- docker/jenkins/dtest.sh | 4 +- docker/jenkins/jenkinscommand.sh | 15 --- jenkins-dsl/cassandra_job_dsl_seed.groovy | 62 ++ 4 files changed, 20 insertions(+), 69 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/2c1842ce/build-scripts/cassandra-dtest-pytest.sh -- diff --git a/build-scripts/cassandra-dtest-pytest.sh b/build-scripts/cassandra-dtest-pytest.sh index 993bcb3..7eae11e 100755 --- a/build-scripts/cassandra-dtest-pytest.sh +++ b/build-scripts/cassandra-dtest-pytest.sh @@ -52,13 +52,13 @@ cd cassandra-dtest/ rm -r upgrade_tests/ # TEMP: remove upgrade_tests - we have no dual JDK installation set +e # disable immediate exit from this point if [ "${DTEST_TARGET}" = "dtest" ]; then -pytest --log-level="INFO" --use-vnodes --num-tokens=32 --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --skip-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt +pytest -vv --log-level="INFO" --use-vnodes --num-tokens=32 --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --skip-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt elif [ "${DTEST_TARGET}" = "dtest-novnode" ]; then -pytest --log-level="INFO" --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --skip-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt +pytest -vv --log-level="INFO" --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --skip-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt elif [ "${DTEST_TARGET}" = "dtest-offheap" ]; then -pytest --log-level="INFO" --use-vnodes --num-tokens=32 --use-off-heap-memtables --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --skip-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt +pytest -vv --log-level="INFO" --use-vnodes --num-tokens=32 --use-off-heap-memtables --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --skip-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt elif [ "${DTEST_TARGET}" = "dtest-large" ]; then -pytest --log-level="INFO" --use-vnodes --num-tokens=32 --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --force-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt +pytest -vv --log-level="INFO" --use-vnodes --num-tokens=32 --junit-xml=nosetests.xml -s --cassandra-dir=$CASSANDRA_DIR --force-resource-intensive-tests 2>&1 | tee -a ${WORKSPACE}/test_stdout.txt else echo "Unknown dtest target: ${DTEST_TARGET}" exit 1 http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/2c1842ce/docker/jenkins/dtest.sh -- diff --git a/docker/jenkins/dtest.sh b/docker/jenkins/dtest.sh index 87dd77f..0d8cdb9 100644 --- a/docker/jenkins/dtest.sh +++ b/docker/jenkins/dtest.sh @@ -3,7 +3,9 @@ export WORKSPACE=/home/cassandra/cassandra export LANG=en_US.UTF-8 export PYTHONIOENCODING=utf-8 export PYTHONUNBUFFERED=true +echo "dtest.sh: running: git clone --depth=1 --branch=$BRANCH https://github.com/$REPO/cassandra.git"; git clone --depth=1 --branch=$BRANCH https://github.com/$REPO/cassandra.git cd cassandra +echo git clone --branch=$DTEST_BRANCH $DTEST_REPO git clone --branch=$DTEST_BRANCH $DTEST_REPO -../cassandra-builds/build-scripts/cassandra-dtest-pytest.sh +../cassandra-builds/build-scripts/cassandra-dtest-pytest.sh $1 http://git-wip-us.apache.org/repos/asf/cassandra-builds/blob/2c1842ce/docker/jenkins/jenkinscommand.sh -- diff --git a/docker/jenkins/jenkinscommand.sh b/docker/jenkins/jenkinscommand.sh index f218ff9..5e4741d 100644 --- a/docker/jenkins/jenkinscommand.sh +++ b/docker/jenkins/jenkinscommand.sh @@ -1,13 +1,18 @@ #!/bin/sh +DOCKER_IM
[jira] [Commented] (CASSANDRA-14260) Refactor pair to avoid boxing longs/ints
[ https://issues.apache.org/jira/browse/CASSANDRA-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377629#comment-16377629 ] Blake Eggleston commented on CASSANDRA-14260: - {quote} I would prefer moving out of using {{Pair}} all over the place and use specific classes instead {quote} Agreed, that sounds like the more correct way to go here > Refactor pair to avoid boxing longs/ints > > > Key: CASSANDRA-14260 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14260 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > We uses Pair all over the place, and in many cases either/both of X and > Y are primitives (ints, longs), and we end up boxing them into Integers and > Longs. We should have specialized versions that take primitives. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14265) Add explanation of vNodes to online documentation
[ https://issues.apache.org/jira/browse/CASSANDRA-14265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Brotman updated CASSANDRA-14265: Description: A lot of inquiries on the mailing list about how vNodes work and how to set configuration properly. We should add an explanation to the documentation. (was: A lot of inquires on the mailing list about how vNodes work and how to set configuration properly. We should add and explanation to the documentation.) > Add explanation of vNodes to online documentation > - > > Key: CASSANDRA-14265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14265 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Kenneth Brotman >Priority: Major > > A lot of inquiries on the mailing list about how vNodes work and how to set > configuration properly. We should add an explanation to the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14265) Add explanation of vNodes to online documentation
[ https://issues.apache.org/jira/browse/CASSANDRA-14265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Brotman updated CASSANDRA-14265: Summary: Add explanation of vNodes to online documentation (was: Add explaination of vNodes to online documentation) > Add explanation of vNodes to online documentation > - > > Key: CASSANDRA-14265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14265 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Kenneth Brotman >Priority: Major > > A lot of inquires on the mailing list about how vNodes work and how to set > configuration properly. We should add and explanation to the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14263) The home page of Cassandra is mobile friendly but the link to the third parties is not
[ https://issues.apache.org/jira/browse/CASSANDRA-14263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377608#comment-16377608 ] Michael Mior commented on CASSANDRA-14263: -- For me some of the logos get cut off but otherwise, the page is quite readable. It might be nice to fix, but I don't think it makes a big difference. (This is assuming others aren't seeing worse issues in other mobile browsers.) > The home page of Cassandra is mobile friendly but the link to the third > parties is not > -- > > Key: CASSANDRA-14263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14263 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Kenneth Brotman >Priority: Major > > Near the bottom the home page of the Apache Cassandra web site: > [http://cassandra.apache.org/] > is a link to third parties who professionally support Cassandra. > The home page of Cassandra is mobile friendly but the link to the third > parties is not: > [https://wiki.apache.org/cassandra/ThirdPartySupport] > Any suggestions? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377591#comment-16377591 ] Venkata Harikrishna Nukala commented on CASSANDRA-13853: [~rustyrazorblade] [~pree] Hi, I made a patch (attached) for the required changes. Hope you don't mind. With these changes, "nodetool describecluster" output looks like this: {noformat} Cluster Information: Name: test [^13853-trunk.txt] Live nodes: 6 Joining nodes: [127.0.0.6, 127.0.0.5, 127.0.0.4, 127.0.0.3, 127.0.0.2, 127.0.0.1] Moving nodes: [] Leaving nodes: [] Keyspaces: system_traces RF: 2 system RF: 1 system_distributed RF: 3 system_schema RF: 1 system_auth RF: 1 Release versions: 4.0.0: [127.0.0.6, 127.0.0.5, 127.0.0.4, 127.0.0.3, 127.0.0.2, 127.0.0.1] Datacenters: dc1 Up# 3 Down# 0 Unknown# 0 dc2 Up# 3 Down# 0 Unknown# 0 Snitch: org.apache.cassandra.locator.PropertyFileSnitch DynamicEndPointSnitch: enabled Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: cc4dbfad-1e8f-386d-9a79-de80def27f33: [127.0.0.6, 127.0.0.5, 127.0.0.4, 127.0.0.3, 127.0.0.2, 127.0.0.1] {noformat} Let me know what you think. > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13853) nodetool describecluster should be more informative
[ https://issues.apache.org/jira/browse/CASSANDRA-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Harikrishna Nukala updated CASSANDRA-13853: --- Attachment: 13853-trunk.txt > nodetool describecluster should be more informative > --- > > Key: CASSANDRA-13853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13853 > Project: Cassandra > Issue Type: Improvement > Components: Observability, Tools >Reporter: Jon Haddad >Assignee: Preetika Tyagi >Priority: Minor > Labels: lhf > Attachments: 13853-trunk.txt > > > Additional information we should be displaying: > * Total node count > * List of datacenters, RF, with number of nodes per dc, how many are down, > * Version(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14265) Add explaination of vNodes to online documentation
Kenneth Brotman created CASSANDRA-14265: --- Summary: Add explaination of vNodes to online documentation Key: CASSANDRA-14265 URL: https://issues.apache.org/jira/browse/CASSANDRA-14265 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman A lot of inquires on the mailing list about how vNodes work and how to set configuration properly. We should add and explanation to the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14252) Use zero as default score in DynamicEndpointSnitch
[ https://issues.apache.org/jira/browse/CASSANDRA-14252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377571#comment-16377571 ] Simon Zhou commented on CASSANDRA-14252: This is an interesting change but I'm not sure it fixes all problems. The code that you changed was introduced in CASSANDRA-13074, which also claims to fix slow node issue, by totally ignoring nodes that we don't have a score, no matter it's a node in local or remote data center. Now with your fix, we still give these (remote) nodes a try by assigning an artificially low score. However, isn't 0 the lowest score that could result in these slow/unresponsive remote nodes being picked up before other remote nodes that have normal scores (such as 1.0)? Btw, badness_threshold=0.1 may be too conservative. We also disabled IO factor when calculating the scores through -Dcassandra.ignore_dynamic_snitch_severity=true. See CASSANDRA-11738 for details. > Use zero as default score in DynamicEndpointSnitch > -- > > Key: CASSANDRA-14252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14252 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Major > Fix For: 4.0, 3.0.17, 3.11.3 > > > The problem I want to solve is that I found in our deployment, one slow but > alive data node can slow down the whole cluster, even caused timeout of our > requests. > We are using DynamicEndpointSnitch, with badness_threshold 0.1. I expect the > DynamicEndpointSnitch switch to sortByProximityWithScore, if local data node > latency is too high. > I added some debug log, and figured out that in a lot of cases, the score > from remote data node was not populated, so the fallback to > sortByProximityWithScore never happened. That's why a single slow data node, > can cause huge problems to the whole cluster. > In this jira, I'd like to use zero as default score, so that we will get a > chance to try remote data node, if local one is slow. > I tested it in our test cluster, it improved the client latency in single > slow data node case significantly. > I flag this as a Bug, because it caused problems to our use cases multiple > times. > logs === > _2018-02-21_23:08:57.54145 WARN 23:08:57 [RPC-Thread:978]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.54319 WARN 23:08:57 [RPC-Thread:967]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [0.0]_ > _2018-02-21_23:08:57.55111 WARN 23:08:57 [RPC-Thread:453]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > _2018-02-21_23:08:57.55687 WARN 23:08:57 [RPC-Thread:753]: > sortByProximityWithBadness: after sorting by proximity, addresses order > change to [ip1, ip2], with scores [1.0]_ > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity
[ https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377534#comment-16377534 ] Vinay Chella commented on CASSANDRA-12151: -- Hi [~eanujwa] [~jasobrown], I’m excited to see the design document and it looks good to us! Netflix had a similar requirement recently for our internal 2.1 clusters and we implemented a simple version (no query categories, etc…) for sox auditing. As your design is very close to what we implemented, just a few differently named classes for the most part, can we work together on the trunk [patchset|https://github.com/vinaykumarchella/cassandra/pull/2] to add the missing components from your design? Alternatively, we could take an incremental approach, review what we have on the trunk branch of the simple version and get it committed and then add in some of the more advanced features next. I believe this patch follows the design goals that you put together. Please review and let me know if you have any questions or concerns about the first iteration. If folks are interested in the 3.x/2.x branches I can put those up on my github as well. [~jhb] {quote}I just have one question, do you think enabling/updating/disabling audit require a node restart? {quote} The posted patch allows online auditlog enable/disable or filter updates via JMX. [~jjordan] {quote}You should take a look at the infrastructure added in CASSANDRA-13983 for query logging {quote} Yes, we looked and that certainly looks interesting, perhaps this design allows us to use it as another implementation of {{IAuditLogger}}? > Audit logging for database activity > --- > > Key: CASSANDRA-12151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12151 > Project: Cassandra > Issue Type: New Feature >Reporter: stefan setyadi >Assignee: Anuj Wadehra >Priority: Major > Fix For: 4.x > > Attachments: 12151.txt, > DesignProposal_AuditingFeature_ApacheCassandra_v1.docx > > > we would like a way to enable cassandra to log database activity being done > on our server. > It should show username, remote address, timestamp, action type, keyspace, > column family, and the query statement. > it should also be able to log connection attempt and changes to the > user/roles. > I was thinking of making a new keyspace and insert an entry for every > activity that occurs. > Then It would be possible to query for specific activity or a query targeting > a specific keyspace and column family. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14264) Quick Tour Document for dev's that want to get oriented on the code efficiently.
Kenneth Brotman created CASSANDRA-14264: --- Summary: Quick Tour Document for dev's that want to get oriented on the code efficiently. Key: CASSANDRA-14264 URL: https://issues.apache.org/jira/browse/CASSANDRA-14264 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman Create a Quick Tour Document for dev's that want to get oriented on the code efficiently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13016) log messages should include human readable sizes
[ https://issues.apache.org/jira/browse/CASSANDRA-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377447#comment-16377447 ] Jon Haddad commented on CASSANDRA-13016: Hi [~sumantsahney], thanks for volunteering. I've assigned it over to you. It's probably going to be more manageable to work on a section of the codebase at a time and do incremental patches. Feel free to create subtasks. > log messages should include human readable sizes > > > Key: CASSANDRA-13016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13016 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Jon Haddad >Assignee: Sumant Sahney >Priority: Major > Labels: lhf > > displaying bytes by itself is difficult to read when going through log > messages. we should add a human readable version in parens (10MB) after > displaying bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13016) log messages should include human readable sizes
[ https://issues.apache.org/jira/browse/CASSANDRA-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad reassigned CASSANDRA-13016: -- Assignee: Sumant Sahney (was: Jon Haddad) > log messages should include human readable sizes > > > Key: CASSANDRA-13016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13016 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Jon Haddad >Assignee: Sumant Sahney >Priority: Major > Labels: lhf > > displaying bytes by itself is difficult to read when going through log > messages. we should add a human readable version in parens (10MB) after > displaying bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13016) log messages should include human readable sizes
[ https://issues.apache.org/jira/browse/CASSANDRA-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad reassigned CASSANDRA-13016: -- Assignee: Jon Haddad > log messages should include human readable sizes > > > Key: CASSANDRA-13016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13016 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Jon Haddad >Assignee: Jon Haddad >Priority: Major > Labels: lhf > > displaying bytes by itself is difficult to read when going through log > messages. we should add a human readable version in parens (10MB) after > displaying bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13016) log messages should include human readable sizes
[ https://issues.apache.org/jira/browse/CASSANDRA-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377441#comment-16377441 ] Sumant Sahney commented on CASSANDRA-13016: --- Hi, I am new to open source and would like to contribute. Can I take up this request? Thanks, Sumant > log messages should include human readable sizes > > > Key: CASSANDRA-13016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13016 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Jon Haddad >Priority: Major > Labels: lhf > > displaying bytes by itself is difficult to read when going through log > messages. we should add a human readable version in parens (10MB) after > displaying bytes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14263) The home page of Cassandra is mobile friendly but the link to the third parties is not
Kenneth Brotman created CASSANDRA-14263: --- Summary: The home page of Cassandra is mobile friendly but the link to the third parties is not Key: CASSANDRA-14263 URL: https://issues.apache.org/jira/browse/CASSANDRA-14263 Project: Cassandra Issue Type: Improvement Components: Documentation and Website Reporter: Kenneth Brotman Near the bottom the home page of the Apache Cassandra web site: [http://cassandra.apache.org/] is a link to third parties who professionally support Cassandra. The home page of Cassandra is mobile friendly but the link to the third parties is not: [https://wiki.apache.org/cassandra/ThirdPartySupport] Any suggestions? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14261) Compaction Profiling Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377386#comment-16377386 ] Joshua McKenzie commented on CASSANDRA-14261: - bq. but these looked simple and sufficiently isolated not to warrant a full run in perf. Right. Was thinking more something along the lines of jmh microbenchmarks or something that might be usable in the future as well. What you have here is pretty non-controversial, but performance optimization code-changes w/out perf testing is something we have a pretty long history on this project of having bad hygiene around. > Compaction Profiling Improvements > - > > Key: CASSANDRA-14261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14261 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > There's some low hanging fruit in some laptop compaction runs, such as > creating a ton of the same object unnecessarily and hashing cell names > repeatedly to see if a column is dropped even when we should know that the > table has no dropped columns. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-9605) Error message when creating a KS replicated to a non-existent DC
[ https://issues.apache.org/jira/browse/CASSANDRA-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan resolved CASSANDRA-9605. Resolution: Duplicate Fix Version/s: (was: 4.x) 4.0 > Error message when creating a KS replicated to a non-existent DC > > > Key: CASSANDRA-9605 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9605 > Project: Cassandra > Issue Type: New Feature >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 4.0 > > > One of the most common mistakes I see with beginners is they mix up the data > center configuration when using network topology strategy because they are > copying from some tutorial or sample code or have recently changed snitches. > This should not be legal: > {code} > create KEYSPACE test1 WITH replication = {'class': > 'NetworkTopologyStrategy', 'doesnotexist': 1} ; > cqlsh> desc KEYSPACE test1 ; > CREATE KEYSPACE test1 WITH replication = {'class': 'NetworkTopologyStrategy', > 'doesnotexist': '1'} AND durable_writes = true; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14261) Compaction Profiling Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377368#comment-16377368 ] Jeff Jirsa commented on CASSANDRA-14261: I don't, because I did them on a laptop on a plane over the holidays, and it's decidedly synthetic non-prod tests, but these looked simple and sufficiently isolated not to warrant a full run in perf. Here's an estimate from memory though: [this|https://github.com/jeffjirsa/cassandra/commit/b2b2d765c089c5be609d65f04611b2800ffa70b8] was based on seeing that function in the stack about 5% of the time in compaction, and it goes from 5% to ~0.001% with that trivial patch. [this|https://github.com/jeffjirsa/cassandra/commit/dc8070eaa5ec52e8be46358777fe42d9944f5f30] was based on seeing ~130MB of allocations (about 3.12% of the TLAB allocation for a 9 second span). And [this|https://github.com/jeffjirsa/cassandra/commit/391846e4d0cfd8c8076c3e6050fb0b13496e24ed] I expect to never show up in profiles except under very high contention, which I have little desire to manually test, but it should be fairly obvious to most people that it's both safe and necessary. > Compaction Profiling Improvements > - > > Key: CASSANDRA-14261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14261 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > There's some low hanging fruit in some laptop compaction runs, such as > creating a ton of the same object unnecessarily and hashing cell names > repeatedly to see if a column is dropped even when we should know that the > table has no dropped columns. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13993) Add optional startup delay to wait until peers are ready
[ https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377336#comment-16377336 ] Joseph Lynch commented on CASSANDRA-13993: -- {quote}Further, I intentionally wanted this feature to "just work out of the box", without requiring extra configuration (for local vs each dc, and so on). {quote} [~jasobrown], I completely agree, and I believe there is a difference in "percent UP" from "count of DOWN" from a usability perspective, in particular "percent UP" is harder (or impossible) for users of the database to set properly (it will do what they want) or consistently (they leave it to the default or if they change it they use one setting everywhere), and the best default I can think of is 100%. Compare this to a "count DOWN" which is more likely to be a constant 1 or 2. Consider a user who has two multi-region clusters, one that has 12 nodes and one with 120 nodes. Seventy percent is an ok default for the first cluster, but a very bad one in the second and in either case you still have no guarantee that you will not see latency or errors even if you put the timeout at 2 days, and reflecting on it I think {{(percent_up, timeout) = (100%, 10-30s)}} would be the only default that gives users what they expect (restarting their database does not lead to errors). That aggressive setting would have clients doing local CLs waiting on all remote replicas, however, which other than preventing hint replay is a bit wasteful. On the other hand, in both clusters a {{block_for_peers_local_dc=1}} default setting is quite reasonable. The way that my patch implemented the three options it works out of the box for all deployments (vnodes, no vnodes, large clusters, small clusters, etc) whereas percent up only works well if the user _changes_ the default percentage to 100% or is not using vnodes. {quote}I'm reticent to tie this new behavior to one of those values as the use cases are different; meaning, if you change the value for one semantic meaning, you alter the other. {quote} Ok, that makes sense. {quote}This is a fair point, and I'd be open to bumping up the default threshold. However, remember that behavior exists already in cassandra (it's what you buy in to when using vnodes); this patch helps to alleviate the unavailables/timeouts, not eliminate nor accentuate them. {quote} I agree, this is a great step forward, but with a small change I think this strategy could practically eliminate the unavailables/timeouts. If I implemented the functionality with unit tests in a separate Jira would you consider reviewing it or do you think the slight additional complexity is not worth it? Even separating percentage up by local/remote datacenters would be a big step forward I think, and if we went with counts I could reduce the number of settings to 2 or 1 instead of 3 to give the advanced users less control if you think that would be less confusing for newer users. > Add optional startup delay to wait until peers are ready > > > Key: CASSANDRA-13993 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13993 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Jason Brown >Assignee: Jason Brown >Priority: Minor > Fix For: 4.0 > > > When bouncing a node in a large cluster, is can take a while to recognize the > rest of the cluster as available. This is especially true if using TLS on > internode messaging connections. The bouncing node (and any clients connected > to it) may see a series of Unavailable or Timeout exceptions until the node > is 'warmed up' as connecting to the rest of the cluster is asynchronous from > the rest of the startup process. > There are two aspects that drive a node's ability to successfully communicate > with a peer after a bounce: > - marking the peer as 'alive' (state that is held in gossip). This affects > the unavailable exceptions > - having both open outbound and inbound connections open and ready to each > peer. This affects timeouts. > Details of each of these mechanisms are described in the comments below. > This ticket proposes adding a mechanism, optional and configurable, to delay > opening the client native protocol port until some percentage of the peers in > the cluster is marked alive and connected to/from. Thus while we potentially > slow down startup (delay opening the client port), we alleviate the chance > that queries made by clients don't hit transient unavailable/timeout > exceptions. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail
[jira] [Assigned] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi reassigned CASSANDRA-2848: --- Assignee: Dinesh Joshi (was: Geoffrey Yu) > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Goffinet >Assignee: Dinesh Joshi >Priority: Minor > Fix For: 3.11.x > > Attachments: 2848-trunk-v2.txt, 2848-trunk.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14241) Apache dtests not passing after pytest/python 3
[ https://issues.apache.org/jira/browse/CASSANDRA-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377299#comment-16377299 ] Ariel Weisberg commented on CASSANDRA-14241: Thank you Marcus! > Apache dtests not passing after pytest/python 3 > --- > > Key: CASSANDRA-14241 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14241 > Project: Cassandra > Issue Type: Task > Components: Testing >Reporter: Ariel Weisberg >Assignee: Marcus Eriksson >Priority: Major > > Apache dtests are still not running correctly yet with pytest. Most of the > tests are running and passing but a solid chunk are still failing and these > are tests that don't fail in CircleCI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-5836) Seed nodes should be able to bootstrap without manual intervention
[ https://issues.apache.org/jira/browse/CASSANDRA-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377289#comment-16377289 ] Jeff Jirsa commented on CASSANDRA-5836: --- Still causing pain, re-opening to discuss valid options. MAY END UP CLOSING AGAIN if no good option is found. > Seed nodes should be able to bootstrap without manual intervention > -- > > Key: CASSANDRA-5836 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5836 > Project: Cassandra > Issue Type: Bug >Reporter: Bill Hathaway >Priority: Minor > > The current logic doesn't allow a seed node to be bootstrapped. If a user > wants to bootstrap a node configured as a seed (for example to replace a seed > node via replace_token), they first need to remove the node's own IP from the > seed list, and then start the bootstrap process. This seems like an > unnecessary step since a node never uses itself as a seed. > I think it would be a better experience if the logic was changed to allow a > seed node to bootstrap without manual intervention when there are other seed > nodes up in a ring. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Reopened] (CASSANDRA-5836) Seed nodes should be able to bootstrap without manual intervention
[ https://issues.apache.org/jira/browse/CASSANDRA-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa reopened CASSANDRA-5836: --- > Seed nodes should be able to bootstrap without manual intervention > -- > > Key: CASSANDRA-5836 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5836 > Project: Cassandra > Issue Type: Bug >Reporter: Bill Hathaway >Priority: Minor > > The current logic doesn't allow a seed node to be bootstrapped. If a user > wants to bootstrap a node configured as a seed (for example to replace a seed > node via replace_token), they first need to remove the node's own IP from the > seed list, and then start the bootstrap process. This seems like an > unnecessary step since a node never uses itself as a seed. > I think it would be a better experience if the logic was changed to allow a > seed node to bootstrap without manual intervention when there are other seed > nodes up in a ring. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14241) Apache dtests not passing after pytest/python 3
[ https://issues.apache.org/jira/browse/CASSANDRA-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377265#comment-16377265 ] Michael Shuler commented on CASSANDRA-14241: +1 on the PR - thanks a bunch for working on this, Marcus. > Apache dtests not passing after pytest/python 3 > --- > > Key: CASSANDRA-14241 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14241 > Project: Cassandra > Issue Type: Task > Components: Testing >Reporter: Ariel Weisberg >Assignee: Marcus Eriksson >Priority: Major > > Apache dtests are still not running correctly yet with pytest. Most of the > tests are running and passing but a solid chunk are still failing and these > are tests that don't fail in CircleCI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity
[ https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377259#comment-16377259 ] Jeremiah Jordan commented on CASSANDRA-12151: - You should take a look at the infrastructure added in CASSANDRA-13983 for query logging > Audit logging for database activity > --- > > Key: CASSANDRA-12151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12151 > Project: Cassandra > Issue Type: New Feature >Reporter: stefan setyadi >Assignee: Anuj Wadehra >Priority: Major > Fix For: 4.x > > Attachments: 12151.txt, > DesignProposal_AuditingFeature_ApacheCassandra_v1.docx > > > we would like a way to enable cassandra to log database activity being done > on our server. > It should show username, remote address, timestamp, action type, keyspace, > column family, and the query statement. > it should also be able to log connection attempt and changes to the > user/roles. > I was thinking of making a new keyspace and insert an entry for every > activity that occurs. > Then It would be possible to query for specific activity or a query targeting > a specific keyspace and column family. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity
[ https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377175#comment-16377175 ] Michael Shuler commented on CASSANDRA-12151: Title: {{s/3.x/4.x/}} > Audit logging for database activity > --- > > Key: CASSANDRA-12151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12151 > Project: Cassandra > Issue Type: New Feature >Reporter: stefan setyadi >Assignee: Anuj Wadehra >Priority: Major > Fix For: 4.x > > Attachments: 12151.txt, > DesignProposal_AuditingFeature_ApacheCassandra_v1.docx > > > we would like a way to enable cassandra to log database activity being done > on our server. > It should show username, remote address, timestamp, action type, keyspace, > column family, and the query statement. > it should also be able to log connection attempt and changes to the > user/roles. > I was thinking of making a new keyspace and insert an entry for every > activity that occurs. > Then It would be possible to query for specific activity or a query targeting > a specific keyspace and column family. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14241) Apache dtests not passing after pytest/python 3
[ https://issues.apache.org/jira/browse/CASSANDRA-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-14241: Reviewer: Michael Shuler Status: Patch Available (was: Open) this allows us to run the dtests using the same docker image as we use in circleci: https://github.com/apache/cassandra-builds/pull/6 running with [this|https://github.com/krummas/cassandra-dtest/commit/a339542bb81a1db760544d47901ddac340dcdb36] dtest fix we got a clean run: https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest-docker/7/ > Apache dtests not passing after pytest/python 3 > --- > > Key: CASSANDRA-14241 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14241 > Project: Cassandra > Issue Type: Task > Components: Testing >Reporter: Ariel Weisberg >Assignee: Marcus Eriksson >Priority: Major > > Apache dtests are still not running correctly yet with pytest. Most of the > tests are running and passing but a solid chunk are still failing and these > are tests that don't fail in CircleCI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13993) Add optional startup delay to wait until peers are ready
[ https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-13993: Resolution: Fixed Fix Version/s: (was: 4.x) 4.0 Status: Resolved (was: Patch Available) committed as sha {{b86801e95a58c5f1a9c779b21fa57136e0225d61}}. thanks, all > Add optional startup delay to wait until peers are ready > > > Key: CASSANDRA-13993 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13993 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Jason Brown >Assignee: Jason Brown >Priority: Minor > Fix For: 4.0 > > > When bouncing a node in a large cluster, is can take a while to recognize the > rest of the cluster as available. This is especially true if using TLS on > internode messaging connections. The bouncing node (and any clients connected > to it) may see a series of Unavailable or Timeout exceptions until the node > is 'warmed up' as connecting to the rest of the cluster is asynchronous from > the rest of the startup process. > There are two aspects that drive a node's ability to successfully communicate > with a peer after a bounce: > - marking the peer as 'alive' (state that is held in gossip). This affects > the unavailable exceptions > - having both open outbound and inbound connections open and ready to each > peer. This affects timeouts. > Details of each of these mechanisms are described in the comments below. > This ticket proposes adding a mechanism, optional and configurable, to delay > opening the client native protocol port until some percentage of the peers in > the cluster is marked alive and connected to/from. Thus while we potentially > slow down startup (delay opening the client port), we alleviate the chance > that queries made by clients don't hit transient unavailable/timeout > exceptions. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Add optional startup delay to wait until peers are ready
Repository: cassandra Updated Branches: refs/heads/trunk 1b82de8c9 -> b86801e95 Add optional startup delay to wait until peers are ready patch by jasobrown; reviewed by Ariel Weisberg for CASSANDRA-13993 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b86801e9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b86801e9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b86801e9 Branch: refs/heads/trunk Commit: b86801e95a58c5f1a9c779b21fa57136e0225d61 Parents: 1b82de8 Author: Jason Brown Authored: Mon Feb 26 06:38:33 2018 -0800 Committer: Jason Brown Committed: Mon Feb 26 06:40:18 2018 -0800 -- .circleci/config.yml| 12 +- CHANGES.txt | 1 + .../org/apache/cassandra/config/Config.java | 5 + .../cassandra/config/DatabaseDescriptor.java| 10 ++ .../org/apache/cassandra/net/MessageOut.java| 29 +++- .../apache/cassandra/net/MessagingService.java | 16 +- .../org/apache/cassandra/net/PingMessage.java | 82 + .../apache/cassandra/net/PingVerbHandler.java | 31 .../org/apache/cassandra/net/PongMessage.java | 50 ++ .../net/StartupClusterConnectivityChecker.java | 171 +++ .../net/async/OutboundConnectionIdentifier.java | 34 +++- .../net/async/OutboundMessagingConnection.java | 2 +- .../net/async/OutboundMessagingPool.java| 33 ++-- .../cassandra/service/CassandraDaemon.java | 10 ++ .../cassandra/service/EchoVerbHandler.java | 6 +- .../cassandra/service/StorageService.java | 3 + .../StartupClusterConnectivityCheckerTest.java | 129 ++ .../apache/cassandra/service/RemoveTest.java| 2 +- 18 files changed, 587 insertions(+), 39 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b86801e9/.circleci/config.yml -- diff --git a/.circleci/config.yml b/.circleci/config.yml index f881b70..13bc11d 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -58,16 +58,16 @@ with_dtest_jobs_only: &with_dtest_jobs_only - build # Set env_settings, env_vars, and workflows/build_and_run_tests based on environment env_settings: &env_settings -<<: *default_env_settings -#<<: *high_capacity_env_settings +#<<: *default_env_settings +<<: *high_capacity_env_settings env_vars: &env_vars -<<: *resource_constrained_env_vars -#<<: *high_capacity_env_vars +#<<: *resource_constrained_env_vars +<<: *high_capacity_env_vars workflows: version: 2 -build_and_run_tests: *default_jobs +#build_and_run_tests: *default_jobs #build_and_run_tests: *with_dtest_jobs_only -#build_and_run_tests: *with_dtest_jobs +build_and_run_tests: *with_dtest_jobs docker_image: &docker_image kjellman/cassandra-test:0.4.3 version: 2 jobs: http://git-wip-us.apache.org/repos/asf/cassandra/blob/b86801e9/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 40b18ae..9e7a599 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0 + * Add optional startup delay to wait until peers are ready (CASSANDRA-13993) * Add a few options to nodetool verify (CASSANDRA-14201) * CVE-2017-5929 Security vulnerability and redefine default log rotation policy (CASSANDRA-14183) * Use JVM default SSL validation algorithm instead of custom default (CASSANDRA-13259) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b86801e9/src/java/org/apache/cassandra/config/Config.java -- diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index 875751b..ad91a9b 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -373,6 +373,11 @@ public class Config public String full_query_log_dir = null; +// parameters to adjust how much to delay startup until a certain amount of the cluster is connect to and marked alive +public int block_for_peers_percentage = 70; +public int block_for_peers_timeout_in_secs = 10; + + /** * @deprecated migrate to {@link DatabaseDescriptor#isClientInitialized()} */ http://git-wip-us.apache.org/repos/asf/cassandra/blob/b86801e9/src/java/org/apache/cassandra/config/DatabaseDescriptor.java -- diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java index ccb0a30..2e772c5 100644 --- a/src/java/org/apache/cassandra/config/D
[jira] [Commented] (CASSANDRA-13993) Add optional startup delay to wait until peers are ready
[ https://issues.apache.org/jira/browse/CASSANDRA-13993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376945#comment-16376945 ] Jason Brown commented on CASSANDRA-13993: - [~jolynch] I understand what you are saying. I think the difference between a "percent UP" and a "count of DOWN nodes" isn't that much, so either one is probably fine. Further, I intentionally wanted this feature to "just work out of the box", without requiring extra configuration (for local vs each dc, and so on). bq. relying on the timeout for large clusters (although it would be awesome if this timeout re-used or defaulted to an existing timeout relevant to gossip convergence such as BROADCAST_INTERVAL or RING_DELAY). I'm reticent to tie this new behavior to one of those values as the use cases are different; meaning, if you change the value for one semantic meaning, you alter the other. bq. especially with vnode=256 clusters where any 2 nodes down in different racks essentially guarantees an unavailable error for some intersecting token range. This is a fair point, and I'd be open to bumping up the default threshold. However, remember that behavior exists already in cassandra (it's what you buy in to when using vnodes); this patch helps to alleviate the unavailables/timeouts, not eliminate nor accentuate them. > Add optional startup delay to wait until peers are ready > > > Key: CASSANDRA-13993 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13993 > Project: Cassandra > Issue Type: Improvement > Components: Lifecycle >Reporter: Jason Brown >Assignee: Jason Brown >Priority: Minor > Fix For: 4.x > > > When bouncing a node in a large cluster, is can take a while to recognize the > rest of the cluster as available. This is especially true if using TLS on > internode messaging connections. The bouncing node (and any clients connected > to it) may see a series of Unavailable or Timeout exceptions until the node > is 'warmed up' as connecting to the rest of the cluster is asynchronous from > the rest of the startup process. > There are two aspects that drive a node's ability to successfully communicate > with a peer after a bounce: > - marking the peer as 'alive' (state that is held in gossip). This affects > the unavailable exceptions > - having both open outbound and inbound connections open and ready to each > peer. This affects timeouts. > Details of each of these mechanisms are described in the comments below. > This ticket proposes adding a mechanism, optional and configurable, to delay > opening the client native protocol port until some percentage of the peers in > the cluster is marked alive and connected to/from. Thus while we potentially > slow down startup (delay opening the client port), we alleviate the chance > that queries made by clients don't hit transient unavailable/timeout > exceptions. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14261) Compaction Profiling Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376897#comment-16376897 ] Joshua McKenzie commented on CASSANDRA-14261: - You have some perf runs to go with those correctness runs to indicate the impact of the change? > Compaction Profiling Improvements > - > > Key: CASSANDRA-14261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14261 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > There's some low hanging fruit in some laptop compaction runs, such as > creating a ton of the same object unnecessarily and hashing cell names > repeatedly to see if a column is dropped even when we should know that the > table has no dropped columns. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14262) View update sent multiple times during range movement
Nadav Har'El created CASSANDRA-14262: Summary: View update sent multiple times during range movement Key: CASSANDRA-14262 URL: https://issues.apache.org/jira/browse/CASSANDRA-14262 Project: Cassandra Issue Type: Improvement Components: Materialized Views Reporter: Nadav Har'El This issue is about updating a base table with materialized views while token-ranges are being moved, i.e., while a node is being added or removed from the cluster (this is a long process because the data needs to be streamed to its new owning node). During this process, each view-mutation we want to write to a view table may have an additional "pending node" (or several of them) - another node (or nodes) which will hold this view mutation, and we need to send the view mutations to these new nodes too. This code existed until CASSANDRA-13069, when it was accidentally removed, and returned in CASSANDRA-14251. However, the current code, in mutateMV(), has each of the RF (e.g., 3) base replicas send the view mutation to the the same pending node. This is of course redundant, and reduces write throughput while the streaming is performed. I suggested (based on an idea by [~shlomi_livne]) that it may be enough for only the single node which will be paired (when the range movement completes) with the pending node to send it the update. [~pauloricardomg] replied (see [https://lists.apache.org/thread.html/12c78582a3f709ca33a45e5fa6121148b1b1ad9c9b290d1a21e4409b@%3Cdev.cassandra.apache.org%3E] ) that it appears that such an optimization would work in the common case of single movements but will not work in rarer more complex cases (I did not fully understand the details, check out the above link for the details). I believe there's another problem with the current code, which is of correctness: If any view replica ends up with two different view rows for the same partition key, such a mistake cannot currently be fixed (see CASSANDRA-10346). But if we have different base replicas with two different values (a consistency an ordinary base repair could fix, if we ran it) and both of them send their update to the same pending view replica, this view replica will now have two rows, one of them wrong (and cannot currently be repaired). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity
[ https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376850#comment-16376850 ] Jacques-Henri Berthemet commented on CASSANDRA-12151: - Proposed design looks good to me. I think using a file is better than a table as a rogue user may just truncate/drop/overwrite the table. I just have one question, do you think enabling/updating/disabling audit require a node restart? > Audit logging for database activity > --- > > Key: CASSANDRA-12151 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12151 > Project: Cassandra > Issue Type: New Feature >Reporter: stefan setyadi >Assignee: Anuj Wadehra >Priority: Major > Fix For: 4.x > > Attachments: 12151.txt, > DesignProposal_AuditingFeature_ApacheCassandra_v1.docx > > > we would like a way to enable cassandra to log database activity being done > on our server. > It should show username, remote address, timestamp, action type, keyspace, > column family, and the query statement. > it should also be able to log connection attempt and changes to the > user/roles. > I was thinking of making a new keyspace and insert an entry for every > activity that occurs. > Then It would be possible to query for specific activity or a query targeting > a specific keyspace and column family. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12941) Backport CASSANDRA-9967
[ https://issues.apache.org/jira/browse/CASSANDRA-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-12941: Resolution: Won't Fix Fix Version/s: (was: 3.0.x) Status: Resolved (was: Patch Available) +1 for WON'T FIX > Backport CASSANDRA-9967 > --- > > Key: CASSANDRA-12941 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12941 > Project: Cassandra > Issue Type: Improvement > Components: Coordination, Materialized Views, Observability >Reporter: Haijun Cao >Priority: Trivial > Attachments: 12941-3.0.txt > > > Backport CASSANDRA-9967 > Materialized view is available for use in 3.0.x, it would be nice to check > view built status by issuing one CQL query against system_distributed table, > hence back port CASSANDRA-9967 to 3.0.x. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14260) Refactor pair to avoid boxing longs/ints
[ https://issues.apache.org/jira/browse/CASSANDRA-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376627#comment-16376627 ] Sylvain Lebresne commented on CASSANDRA-14260: -- For what it's worth, I would prefer moving out of using {{Pair}} all over the place and use specific classes instead. Just to illustrate, I'm talking of adding a {{SizeEstimate}} (or other name) with 2 fields {{partitionCount}} and {{meanSize}} for instance instead of the pair used for size estimates. I reckon this technically add a bit more lines of code over reusing a {{Pair}} class, but I feel it largely pay for itself in term of code clarity and error-proneness. If you get a {{LongLongPair}} from a method, you constantly have to double check which field is which, which I personally find annoying and is easy to mess up. Anyway, that remark obviously apply to all usage of {{Pair}} while I understand you intent here is just to avoid boxing, but in the off-change that there is agreement on the remarks above, I figured this may be a good occasion to start cleaning this up as well rather than add more "pair" classes. > Refactor pair to avoid boxing longs/ints > > > Key: CASSANDRA-14260 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14260 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Minor > Fix For: 4.x > > > We uses Pair all over the place, and in many cases either/both of X and > Y are primitives (ints, longs), and we end up boxing them into Integers and > Longs. We should have specialized versions that take primitives. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14247) SASI tokenizer for simple delimiter based entries
[ https://issues.apache.org/jira/browse/CASSANDRA-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376253#comment-16376253 ] mck edited comment on CASSANDRA-14247 at 2/26/18 9:29 AM: -- [~mkjellman], have forced pushed the branch again. (let me know if you want to be adding checkpoint commits rather than overwriting the existing commit.) This adds the test file {{test/resources/tokenization/world_cities_a.csv}}, and a unit test to match. The other unit test methods have been updated to use different delimiters as appropriate for the existing test data files. Example {{cqlsh}} corridor testing… {code:java} create table test ( one text, two int, three text, PRIMARY KEY (one,two) ); # insert a new row, with the contents of test/resources/tokenization/world_cities_a.csv going into column 'three'. create CUSTOM INDEX on test (three) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = { 'analyzer_class': 'org.apache.cassandra.index.sasi.analyzer.DelimiterAnalyzer', 'delimiter': ',', 'mode': 'prefix', 'analyzed': 'true'}; select one,two from test where three LIKE 'azzazl' ALLOW FILTERING; {code} Aside: this tokenizer raises the need for a "exact" mode. Querying a csv inside a column like this is one example where the user may never require wildcarding LIKE clause (using %) and an 'exact' mode would be significantly more performant and use less disk. (btw I'm suspecting that {{is_literal: false}} would have the same impact as an 'exact' mode…) was (Author: michaelsembwever): [~mkjellman], have forced pushed the branch again. (let me know if you want to be adding checkpoint commits rather than overwriting the existing commit.) This adds the test file {{test/resources/tokenization/world_cities_a.csv}}, and a unit test to match. The other unit test methods have been updated to use different delimiters as appropriate for the existing test data files. Example corridor testing… {code:java} create table test ( one text, two int, three text, PRIMARY KEY (one,two) ); # insert a new row, with the contents of test/resources/tokenization/world_cities_a.csv going into column 'three'. create CUSTOM INDEX on test (three) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = { 'analyzer_class': 'org.apache.cassandra.index.sasi.analyzer.DelimiterAnalyzer', 'delimiter': ',', 'mode': 'prefix', 'analyzed': 'true'}; select one,two from test where three LIKE 'azzazl' ALLOW FILTERING; {code} Aside: this tokenizer raises the need for a "exact" mode. Querying a csv inside a column like this is one example where the user may never require wildcarding LIKE clause (using %) and an 'exact' mode would be significantly more performant and use less disk. (btw I'm suspecting that {{is_literal: false}} would have the same impact as an 'exact' mode…) > SASI tokenizer for simple delimiter based entries > - > > Key: CASSANDRA-14247 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14247 > Project: Cassandra > Issue Type: Improvement > Components: sasi >Reporter: mck >Assignee: mck >Priority: Major > Fix For: 4.0, 3.11.x > > > Currently SASI offers only two tokenizer options: > - NonTokenizerAnalyser > - StandardAnalyzer > The latter is built upon Snowball, powerful for human languages but overkill > for simple tokenization. > A simple tokenizer is proposed here. The need for this arose as a workaround > of CASSANDRA-11182, and to avoid the disk usage explosion when having to > resort to {{CONTAINS}}. See https://github.com/openzipkin/zipkin/issues/1861 > Example use of this would be: > {code} > CREATE CUSTOM INDEX span_annotation_query_idx > ON zipkin2.span (annotation_query) USING > 'org.apache.cassandra.index.sasi.SASIIndex' > WITH OPTIONS = { > 'analyzer_class': > 'org.apache.cassandra.index.sasi.analyzer.DelimiterAnalyzer', > 'delimiter': '░', > 'case_sensitive': 'true', > 'mode': 'prefix', > 'analyzed': 'true'}; > {code} > Original credit for this work goes to https://github.com/zuochangan -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14245) SELECT JSON prints null on empty strings
[ https://issues.apache.org/jira/browse/CASSANDRA-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer reassigned CASSANDRA-14245: -- Assignee: Benjamin Lerer > SELECT JSON prints null on empty strings > > > Key: CASSANDRA-14245 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14245 > Project: Cassandra > Issue Type: Bug > Components: CQL > Environment: Cassandra 3.11.2, Ubuntu 16.04 LTS > >Reporter: Norbert Schultz >Assignee: Benjamin Lerer >Priority: Major > > SELECT JSON reports an empty string as null. > > Example: > {code:java} > cqlsh:unittest> create table test(id INT, name TEXT, PRIMARY KEY(id)); > cqlsh:unittest> insert into test (id, name) VALUES (1, 'Foo'); > cqlsh:unittest> insert into test (id, name) VALUES (2, ''); > cqlsh:unittest> insert into test (id, name) VALUES (3, null); > cqlsh:unittest> select * from test; > id | name > +-- > 1 | Foo > 2 | > 3 | null > (3 rows) > cqlsh:unittest> select JSON * from test; > [json] > -- > {"id": 1, "name": "Foo"} > {"id": 2, "name": null} > {"id": 3, "name": null} > (3 rows){code} > > This even happens, if the string is part of the Primary Key, which makes the > generated string not insertable. > > {code:java} > cqlsh:unittest> create table test2 (id INT, name TEXT, age INT, PRIMARY > KEY(id, name)); > cqlsh:unittest> insert into test2 (id, name, age) VALUES (1, '', 42); > cqlsh:unittest> select JSON * from test2; > [json] > > {"id": 1, "name": null, "age": 42} > (1 rows) > cqlsh:unittest> insert into test2 JSON '{"id": 1, "name": null, "age": 42}'; > InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid > null value in condition for column name"{code} > > On an older version of Cassandra (3.0.8) does not have this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org