[jira] [Commented] (CASSANDRA-8049) Explicitly examine current C* state on startup to detect incompatibilities before upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14529481#comment-14529481 ] Tyler Hobbs commented on CASSANDRA-8049: The test results look much better: http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-8049-follow-up-dtest/3/testReport/ Explicitly examine current C* state on startup to detect incompatibilities before upgrade - Key: CASSANDRA-8049 URL: https://issues.apache.org/jira/browse/CASSANDRA-8049 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Sam Tunnicliffe Fix For: 3.0 Attachments: 8049-follow-up.txt, 8049-v2.txt, 8049.txt Unfortunately, we cannot rely on users reading, and following, NEWS.txt before upgrading. People don't read, or ignore it, and sometimes have issues as the result (see CASSANDRA-8047, for example, and I know of several cases like that one). We should add an explicit compatibility check on startup, before we modify anything, or write out sstables with the new format. We should fail and complain loudly if we detect a skipped upgrade step. We should also snapshot the schema tables before attempting any conversions (since it's not uncommon to make schema modifications as part of the upgrade). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8049) Explicitly examine current C* state on startup to detect incompatibilities before upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14528905#comment-14528905
]
Tyler Hobbs commented on CASSANDRA-8049:
Snapshot manifest files ({{manifest.json}}) are triggering the unreadable
sstables check, which is causing a lot of dtest failures (e.g. [snapshot
tests|http://cassci.datastax.com/job/trunk_dtest/62/testReport/junit/snapshot_test/TestArchiveCommitlog/dont_test_archive_commitlog_4/]).
It seems like that check should ignore files without a {{.db}} extension,
correct?
Explicitly examine current C* state on startup to detect incompatibilities
before upgrade
-
Key: CASSANDRA-8049
URL: https://issues.apache.org/jira/browse/CASSANDRA-8049
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Sam Tunnicliffe
Fix For: 3.0
Attachments: 8049-v2.txt, 8049.txt
Unfortunately, we cannot rely on users reading, and following, NEWS.txt
before upgrading. People don't read, or ignore it, and sometimes have issues
as the result (see CASSANDRA-8047, for example, and I know of several cases
like that one).
We should add an explicit compatibility check on startup, before we modify
anything, or write out sstables with the new format. We should fail and
complain loudly if we detect a skipped upgrade step.
We should also snapshot the schema tables before attempting any conversions
(since it's not uncommon to make schema modifications as part of the upgrade).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8049) Explicitly examine current C* state on startup to detect incompatibilities before upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513859#comment-14513859 ] Sam Tunnicliffe commented on CASSANDRA-8049: Fair enough, this is a good opportunity to clean up some of that existing stuff. I've pushed another commit to the branch addressing all of the above comments. Explicitly examine current C* state on startup to detect incompatibilities before upgrade - Key: CASSANDRA-8049 URL: https://issues.apache.org/jira/browse/CASSANDRA-8049 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Sam Tunnicliffe Fix For: 3.0 Attachments: 8049-v2.txt, 8049.txt Unfortunately, we cannot rely on users reading, and following, NEWS.txt before upgrading. People don't read, or ignore it, and sometimes have issues as the result (see CASSANDRA-8047, for example, and I know of several cases like that one). We should add an explicit compatibility check on startup, before we modify anything, or write out sstables with the new format. We should fail and complain loudly if we detect a skipped upgrade step. We should also snapshot the schema tables before attempting any conversions (since it's not uncommon to make schema modifications as part of the upgrade). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8049) Explicitly examine current C* state on startup to detect incompatibilities before upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14508166#comment-14508166
]
Aleksey Yeschenko commented on CASSANDRA-8049:
--
- initializing JMX arguably shouldn't be part of a startup check. Errors and
warnings should be, if we don't mind some logic duplication of the port
checking for nulls (I don't)
- the {{info}} logging in {{inspectJvmOptions}} for heap size and JVM
vendor/version should probably also remain in {{CassandraDaemon}}, and the
32bit VM logger.info should really be a {{warn}}.
- {{checkJnaInitialization}} shouldn't be calling {{CLibrary.tryMlockall()}}.
Catching {{LastErrorException}} there is also kinda pointless -
{{CLibrary.tryMlockall()}} never ever rethrows it (and provides more than
sufficient logging itself)
- {{initSigarLibrary}} if/else logic is redundant (and I know it's a
pre-existing condition). A single-line {{new
SigarLibrary().warnIfRunningInDegradedMode()}} should be sufficient, b/c the
method itself has an else branch on {{initialized}} that logs 'failed to
initialize' if so.
- {{checkCacheServiceInitialization}} can go away entirely. I don't see a way
it could happen, it's basically dead code (and has been so forevah)
- snapshotting system keyspace on upgrades should be done, but probably not as
part of {{checkSystemKeyspaceState}} or the startup check framework. Can we
make it a separate method in {{CassandraDaemon}} or, better,
{{SystemKeyspace}}, instead?
Nits:
- {{StartupCheck}} javadocs are incorrect, not reflecting the fact that we only
throw the exception now and not return any values
- {{StartupException}} should probably be a checked one here, given that it can
only be caught in one place and *is* being caught there
Most of the issues here are ones in the pre-existing code, but this ticket is
an opportunity to fix them anyway, since they are related. Everything else LGTM.
Explicitly examine current C* state on startup to detect incompatibilities
before upgrade
-
Key: CASSANDRA-8049
URL: https://issues.apache.org/jira/browse/CASSANDRA-8049
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Sam Tunnicliffe
Fix For: 3.0
Attachments: 8049-v2.txt, 8049.txt
Unfortunately, we cannot rely on users reading, and following, NEWS.txt
before upgrading. People don't read, or ignore it, and sometimes have issues
as the result (see CASSANDRA-8047, for example, and I know of several cases
like that one).
We should add an explicit compatibility check on startup, before we modify
anything, or write out sstables with the new format. We should fail and
complain loudly if we detect a skipped upgrade step.
We should also snapshot the schema tables before attempting any conversions
(since it's not uncommon to make schema modifications as part of the upgrade).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8049) Explicitly examine current C* state on startup to detect incompatibilities before upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14506840#comment-14506840 ] Sam Tunnicliffe commented on CASSANDRA-8049: I've pushed a rebased version [here|https://github.com/beobal/cassandra/tree/8049]. I've also relaxed the check on previous and current installed versions. If a previous install is detected we still snapshot the tables in the system keyspace, but I've removed the check that aborts startup if the previous version was too old as hardcoding the oldest permitted version was just too brittle burdensome. Explicitly examine current C* state on startup to detect incompatibilities before upgrade - Key: CASSANDRA-8049 URL: https://issues.apache.org/jira/browse/CASSANDRA-8049 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Sam Tunnicliffe Fix For: 3.0 Attachments: 8049-v2.txt, 8049.txt Unfortunately, we cannot rely on users reading, and following, NEWS.txt before upgrading. People don't read, or ignore it, and sometimes have issues as the result (see CASSANDRA-8047, for example, and I know of several cases like that one). We should add an explicit compatibility check on startup, before we modify anything, or write out sstables with the new format. We should fail and complain loudly if we detect a skipped upgrade step. We should also snapshot the schema tables before attempting any conversions (since it's not uncommon to make schema modifications as part of the upgrade). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8049) Explicitly examine current C* state on startup to detect incompatibilities before upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169821#comment-14169821 ] Jeremy Hanna commented on CASSANDRA-8049: - Would be great to see this as there was CASSANDRA-6598 which made upgrading sstables in 1.2 not include index sstables. That made it so if a user didn't upgrade sstables with a 1.2.14+ version of C* before going to 2.0, 2.0 would migrate some system data and then fail to startup. It was seen in a few different cases and painful to go back. It is documented, but something to fail fast would be very welcome. Explicitly examine current C* state on startup to detect incompatibilities before upgrade - Key: CASSANDRA-8049 URL: https://issues.apache.org/jira/browse/CASSANDRA-8049 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Fix For: 3.0 Unfortunately, we cannot rely on users reading, and following, NEWS.txt before upgrading. People don't read, or ignore it, and sometimes have issues as the result (see CASSANDRA-8047, for example, and I know of several cases like that one). We should add an explicit compatibility check on startup, before we modify anything, or write out sstables with the new format. We should fail and complain loudly if we detect a skipped upgrade step. We should also snapshot the schema tables before attempting any conversions (since it's not uncommon to make schema modifications as part of the upgrade). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
