[jira] [Created] (CASSANDRA-15366) Backup and Restore
Kamal created CASSANDRA-15366: - Summary: Backup and Restore Key: CASSANDRA-15366 URL: https://issues.apache.org/jira/browse/CASSANDRA-15366 Project: Cassandra Issue Type: Bug Reporter: Kamal Hi Team, We are testing the backup and restore of CASSANDRA in a cluster and found that while restoring the backup from the second node to the new node (or previous node) then complete data is not restored. Please enlight us on it. Regards, Kamal -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955732#comment-16955732
]
Vinay Chella commented on CASSANDRA-15076:
--
{quote}One option could be to add a new flag {{-i, --ignore}}. When used, the
auditlogviewer tool would silently skip unexpected records, try to recover and
continue on next record. That way users could at least try to read out records
from the queue even when some unexpected records are present. What do you think?
{quote}
I like this option. +1.
{quote}
On the topic of saving disk space and improving efficiency - we ran some tests
with shorter field-names, or even fieldless queues. That makes a significant
difference. But that's probably for another ticket.
{quote}
Please create a ticket so that we can follow up on this as well.
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Tool/fql
>Reporter: Per Otterström
>Assignee: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15081) LegacyLayout does not have same behavior as 2.x when handling unknown column names
[
https://issues.apache.org/jira/browse/CASSANDRA-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955264#comment-16955264
]
Michael Semb Wever edited comment on CASSANDRA-15081 at 10/20/19 7:41 PM:
--
||branch||circleci||asf jenkins testall||asf jenkins dtests||
|[cassandra-3.11_15081|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/cassandra-3.11_15081]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_15081]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/62//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/62/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/700//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/700]|
can we write a unit or dtest for this?
was (Author: michaelsembwever):
||branch||circleci||asf jenkins testall||asf jenkins dtests||
|[cassandra-3.11_15081|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/cassandra-3.11_15081]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_15081]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/62//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/62/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/697//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/697]|
can we write a unit or dtest for this?
> LegacyLayout does not have same behavior as 2.x when handling unknown column
> names
> --
>
> Key: CASSANDRA-15081
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15081
> Project: Cassandra
> Issue Type: Bug
>Reporter: Cameron Zemek
>Priority: High
> Labels: patch
> Attachments: 15081.patch
>
>
> Due to a bug I haven't been able to reproduce the production cluster had
> unknown column names. To replicate the issue for this test I did the
> following:
> {noformat}
> $ ccm create -v 2.1.19 -n 1 -s bug
> $ cat > schema.cql << 'EOF'
> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test.unknowntest (id int primary key, payload text, "paylo!d"
> text);
> EOF
> $ ccm node1 cqlsh -f schema.cql
> $ export CASSANDRA_INCLUDE=~/.ccm/bug/node1/bin/cassandra.in.sh
> $ cat > bug.json << 'EOF'
> [
> {"key": "1",
> "cells": [["","",1554432501209207],
> ["paylo!d","hello world",1554432501209207],
> ["payload","hello world",1554432501209207]]}
> ]
> EOF
> $ ~/.ccm/repository/2.1.19/tools/bin/json2sstable -K test -c unknowntest
> ~/bug.json
> ~/.ccm/bug/node1/data0/test/unknowntest-/test-unknowntest-ka-1-Data.db{noformat}
> Then test the behavior of unknown columns in 2.1:
> {noformat}
> $ ccm stop
> $ ccm create -v 2.1.19 -n 1 -s bug2_1_19
> $ cat > schema2.cql << 'EOF'
> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test.unknowntest (id int primary key, payload text);
> EOF
> $ ccm node1 cqlsh -f schema2.cql
> $ ccm stop
> $ cp ~/.ccm/bug/node1/data0/test/unknowntest-/test-unknowntest-ka-1-*
> ~/.ccm/bug2_1_19/node1/data0/test/unknowntest-/
> $ ccm start
> $ ccm node1 cqlsh
> Connected to bug2_1_19 at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.19 | CQL spec 3.2.1 | Native protocol v3]
> Use HELP for help.
> cqlsh> select * from test.unknowntest where id = 1;
> id | payload
> +-
> 1 | hello world
> (1 rows){noformat}
> Compared to 3.11.4 which did the following:
> {noformat}
> $ ccm stop
> $ ccm create -v 3.11.4 -n 1 -s bug3_11_4
> $ ccm node1 cqlsh -f schema2.cql
> $ ccm stop
> $ cp ~/.ccm/bug/node1/data0/test/unknowntest-/test-unknowntest-ka-1-*
> ~/.ccm/bug3_11_4/node1/data0/test/unknowntest-/
> $ ccm start
> $ ccm node1 cqlsh
> Connected to bug3_11_4 at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> cqlsh> select * from test.unknowntest where id = 1;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures"
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1,
> 'consistency': 'ONE'}
> {noformat}
> In the logs this resulted in an IllegalStateException from LegacyLayout line
> 1127
> The expected behavior would be to ignore the column and return results the
> same as in 2.1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CASSANDRA-14013) Data loss in snapshots keyspace after service restart
[
https://issues.apache.org/jira/browse/CASSANDRA-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955186#comment-16955186
]
Michael Semb Wever edited comment on CASSANDRA-14013 at 10/20/19 7:39 PM:
--
||branch||circleci||asf jenkins testall||asf jenkins dtests||
|[cassandra-3.0_14013|https://github.com/apache/cassandra/compare/trunk...vincewhite:14013-30]|[circleci|https://circleci.com/gh/vincewhite/workflows/cassandra/tree/14013-30]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/59//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/59/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/694//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/694]|
|[cassandra-3.11_14013|https://github.com/apache/cassandra/compare/trunk...vincewhite:14013-test]|[circleci|https://circleci.com/gh/vincewhite/workflows/cassandra/tree/14013-test]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/60//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/60/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/695//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/695]|
|[trunk_14013|https://github.com/apache/cassandra/compare/trunk...vincewhite:14013-trunk]|[circleci|https://circleci.com/gh/vincewhite/workflows/cassandra/tree/14013-trunk]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/61//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/61/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/699//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/699]|
was (Author: michaelsembwever):
||branch||circleci||asf jenkins testall||asf jenkins dtests||
|[cassandra-3.0_14013|https://github.com/apache/cassandra/compare/trunk...vincewhite:14013-30]|[circleci|https://circleci.com/gh/vincewhite/workflows/cassandra/tree/14013-30]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/59//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/59/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/694//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/694]|
|[cassandra-3.11_14013|https://github.com/apache/cassandra/compare/trunk...vincewhite:14013-test]|[circleci|https://circleci.com/gh/vincewhite/workflows/cassandra/tree/14013-test]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/60//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/60/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/695//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/695]|
|[trunk_14013|https://github.com/apache/cassandra/compare/trunk...vincewhite:14013-trunk]|[circleci|https://circleci.com/gh/vincewhite/workflows/cassandra/tree/14013-trunk]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/61//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-testall/61/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/696//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/696]|
> Data loss in snapshots keyspace after service restart
> -
>
> Key: CASSANDRA-14013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14013
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Gregor Uhlenheuer
>Assignee: Vincent White
>Priority: Normal
>
> I am posting this bug in hope to discover the stupid mistake I am doing
> because I can't imagine a reasonable answer for the behavior I see right now
> :-)
> In short words, I do observe data loss in a keyspace called *snapshots* after
> restarting the Cassandra service. Say I do have 1000 records in a table
> called *snapshots.test_idx* then after restart the table has less entries or
> is even empty.
> My kind of "mysterious" observation is that it happens only in a keyspace
> called *snapshots*...
> h3. Steps to reproduce
> These steps to reproduce show the described behavior in "most" attempts (not
> every single time though).
> {code}
> # create keyspace
> CREATE KEYSPACE snapshots WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> # create table
[jira] [Commented] (CASSANDRA-15362) Add a resumable commit log reader for CDC consumption
[
https://issues.apache.org/jira/browse/CASSANDRA-15362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1694#comment-1694
]
Josh McKenzie commented on CASSANDRA-15362:
---
My brain's been chewing on something related to this in the background; I think
I'd like to at least prototype an approach that doesn't cache any state or
break out a new Resumable reader API and instead always assumes RAR recreation
and scan before read with a signature such as
{noformat}
/**
* @return offset of last successfully read SyncSegment for subsequent resuming
*/
long CommitLogReader.startReadAtSyncSegmentOffset(long startOffset, [other
args])
{noformat}
This would require significantly less logic than the current patch and simplify
{{SegmentIterator.computeNext}} as it'd only have to cache the last read good
SyncSegment start and then return that on read completion as per the signature
above. Would also get rid of the "have to allocate a Resumable reader object"
on every read, etc.
Just read the {{CommitLogDescriptor}} for any read (just like now) and then
{{RAR.seek}} on a resumed read w/a provided offset and you're off to the races.
Not going to have time to dig into that soon but felt it worth commenting on
(mostly to get it out of my head). It wasn't until near the end of working on
the patch that I realized we had the constraint of the
{{CompressedChunkReader.maxCompressedLength}} inside the RAR's Rebufferer that
would constrain us from re-using that Segmenter on resuming, which lead to the
{{ResumableCommitLogReader.rebufferData}} and .readToExhaustion sentinel in the
compressed case. Definitely felt that just re-initializing the RAR was better
than poking any holes in or further extending the Rebufferer hierarchy, but
that decision of course means that in the compressed case, the current approach
basically consists functionally of the above _far_ simpler approach + some ease
of use or friendliness on the API, and I'm not convinced it's a big enough UX
delta to justify the iterator complexity.
> Add a resumable commit log reader for CDC consumption
> -
>
> Key: CASSANDRA-15362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15362
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Change Data Capture
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
>
> [Link to wip
> branch|https://github.com/apache/cassandra/compare/trunk...josh-mckenzie:resumable_clreader]
> This ticket builds upon and supersedes CASSANDRA-15196.
> One of the initial shortcomings of CDC after its initial implementation is
> that, upon update of the related CDC index file if parsing an active segment,
> a user needs to re-scan files (specifically compressed or encrypted) as
> they're written and discard duplicate mutations. The primary push of this
> patch is to introduce a new class, the
> [ResumableCommitLogReader|https://github.com/josh-mckenzie/cassandra/blob/resumable_clreader/src/java/org/apache/cassandra/db/commitlog/ResumableCommitLogReader.java#L35-L53]
> that greatly simplifies and optimizes repeated reads from a commitlog file
> being actively written underneath you.
> The primary API consists of:
> {noformat}
> public void readPartial(int readLimit) throws IOException;
> public void readToCompletion() throws IOException
> public void close(); // is AutoCloseable
> {noformat}
> An example of the usage of this can be seen in the changed standard read
> implementation within the
> [CommitLogReader|https://github.com/josh-mckenzie/cassandra/blob/resumable_clreader/src/java/org/apache/cassandra/db/commitlog/CommitLogReader.java#L129-L134]
> The simple "read straight through" case:
> {noformat}
> try(ResumableCommitLogReader resumableReader = new
> ResumableCommitLogReader(file, handler, minPosition, mutationLimit,
> tolerateTruncation))
> {
> resumableReader.readToCompletion();
> }
> {noformat}
> The more complex "resume reading" case:
> {noformat}
> try(ResumableCommitLogReader resumableReader = new
> ResumableCommitLogReader(file, handler, minPosition, mutationLimit,
> tolerateTruncation))
> {
> while(!parseCDCCompletionStatus(cdcIndexFile))
> {
> Thread.sleep(1);
> rr.readPartial(parseCDCOffset(cdcIndexFile));
> }
> }
> {noformat}
> A brief enumeration of some of the most obvious pros and cons of this current
> design and implementation:
> Pros:
> * Very simple external user interface
> * All the complexity of resuming, RAR seeking, RAR re-creation hidden from
> end-users
> * Minimal risk to existing CL reading code (isPartial() path very separate
> from default in computeNext())
> Cons:
> * Creation of a resumable reader object required on each new segment file
> read (trivially fixable if
