[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842718#comment-17842718
]
Brandon Williams commented on CASSANDRA-19401:
--
I see. I'm not crazy about sleeping but I don't see a better path currently.
+1
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842715#comment-17842715
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
[~brandon.williams] if you look into testImportInvalidateCache, when that sleep
is not added here (1), the very next assert
{code}
assertEquals(10, CacheService.instance.rowCache.size());
{code}
fails because CacheService.instance.rowCache.size() might see 20.
(1)
https://github.com/apache/cassandra/blob/f8929d62ee565b612df00cd9ee8825918d0e359e/test/unit/org/apache/cassandra/db/ImportTest.java#L550
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842693#comment-17842693
]
Brandon Williams commented on CASSANDRA-19401:
--
bq. added 2s sleep also seem to improve the non-flakiness of the other test
across the branches.
Can you explain what you mean here? Do we know why 2s is needed?
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840373#comment-17840373
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
[~brandon.williams]
4.0
https://app.circleci.com/pipelines/github/instaclustr/cassandra/4223/workflows/a82d0483-a0df-44ed-8127-088b303c78ba/jobs/225432/steps
4.1
https://app.circleci.com/pipelines/github/instaclustr/cassandra/4224/workflows/eae7a5e2-89dd-46cd-aaca-1e4250d0fa8b/jobs/225531/steps
5.0 j11
https://app.circleci.com/pipelines/github/instaclustr/cassandra/4226/workflows/9805ec75-fd02-4c5a-8996-fa5bce71e8c2/jobs/225728/steps
5.0 j17
https://app.circleci.com/pipelines/github/instaclustr/cassandra/4226/workflows/9805ec75-fd02-4c5a-8996-fa5bce71e8c2/jobs/225727/steps
trunk
https://app.circleci.com/pipelines/github/instaclustr/cassandra/4240/workflows/b931bc9e-23f8-4b3c-b3b3-86de9029bed6/jobs/227089
not increasing number of tokens from 5 to 15 tends to fail on this:
{code}
INFO [main] 2024-04-23 10:50:01,832 Verifying
BigTableReader(path='/tmp/importtest6316811178601946178/cql_test_keyspace/table_09/nb-1-big-Data.db')
(29.151KiB)
INFO [main] 2024-04-23 10:50:01,832 Deserializing sstable metadata for
BigTableReader(path='/tmp/importtest6316811178601946178/cql_test_keyspace/table_09/nb-1-big-Data.db')
WARN [main] 2024-04-23 10:50:01,835 Key DecoratedKey(-9202596579742833778,
0301) is not contained in the given ranges
java.lang.RuntimeException: Key DecoratedKey(-9202596579742833778, 0301) is
not contained in the given ranges
at
org.apache.cassandra.db.compaction.Verifier$RangeOwnHelper.validate(Verifier.java:381)
at org.apache.cassandra.db.compaction.Verifier.verify(Verifier.java:197)
at
org.apache.cassandra.db.SSTableImporter.verifySSTableForImport(SSTableImporter.java:347)
at
org.apache.cassandra.db.SSTableImporter.importNewSSTables(SSTableImporter.java:96)
at
org.apache.cassandra.db.ImportTest.testImportOutOfRange(ImportTest.java:429)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
{code}
All branches are prepared, once merged, we can close CASSANDRA-19572.
added 2s sleep also seem to improve the non-flakiness of the other test across
the branches.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-dat
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838772#comment-17838772
]
Brandon Williams commented on CASSANDRA-19401:
--
No, we have to fix those tests first.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838771#comment-17838771
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
OK so given CASSANDRA-19572 I think we can merge this. Moving to ready to
commit, correct me if not desirable.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838756#comment-17838756
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
We need to postpone the merging of this patch until we resolve the flakiness of
ImportTest across all branches.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838627#comment-17838627
]
Brandon Williams commented on CASSANDRA-19401:
--
The flakiness in the ImportTests seems to be caused by this patch, as without
it 5.0 can pass 5k times
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1586/workflows/2f172253-80a8-4c5f-b299-174067f0c4c6/jobs/84768/tests]
and
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1589/workflows/868187b9-32e5-49ce-98a4-db4b9c48d148/jobs/85046/tests].
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838623#comment-17838623
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
[CASSANDRA-19401-4.0|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19401-4.0]
{noformat}
java8_pre-commit_tests
✓ j8_build 4m 54s
✓ j8_cqlsh-dtests-py2-no-vnodes6m 22s
✓ j8_cqlsh-dtests-py2-with-vnodes 8m 30s
✓ j8_cqlsh_dtests_py3 6m 22s
✓ j8_cqlsh_dtests_py311 9m 1s
✓ j8_cqlsh_dtests_py311_vnode 6m 20s
✓ j8_cqlsh_dtests_py38 8m 37s
✓ j8_cqlsh_dtests_py38_vnode7m 2s
✓ j8_cqlsh_dtests_py3_vnode5m 44s
✓ j8_cqlshlib_tests8m 13s
✓ j8_dtests 32m 11s
✓ j8_dtests_vnode 37m 10s
✓ j8_jvm_dtests 12m 46s
✓ j8_unit_tests_repeat 8m 12s
✓ j11_unit_tests_repeat7m 42s
✓ j11_dtests_vnode 35m 3s
✓ j11_dtests 31m 14s
✓ j11_cqlsh_dtests_py3_vnode 5m 29s
✓ j11_cqlsh_dtests_py38_vnode 5m 51s
✓ j11_cqlsh_dtests_py385m 26s
✓ j11_cqlsh_dtests_py311_vnode 5m 43s
✓ j11_cqlsh_dtests_py311 5m 56s
✓ j11_cqlsh_dtests_py3 6m 3s
✓ j11_cqlsh-dtests-py2-with-vnodes 5m 47s
✓ j11_cqlsh-dtests-py2-no-vnodes 5m 51s
✕ j8_unit_tests 10m 58s
org.apache.cassandra.cql3.MemtableSizeTest testTruncationReleasesLogSpace
✕ j8_utests_system_keyspace_directory 10m 52s
org.apache.cassandra.index.sasi.SASICQLTest testPagingWithClustering
org.apache.cassandra.cql3.MemtableSizeTest testTruncationReleasesLogSpace
✕ j8_utests_system_keyspace_directory_repeat 8m 56s
org.apache.cassandra.db.ImportTest testImportInvalidateCache
✕ j11_unit_tests 8m 13s
org.apache.cassandra.net.ConnectionTest testMessageDeliveryOnReconnect
org.apache.cassandra.cql3.MemtableSizeTest testTruncationReleasesLogSpace
{noformat}
[java8_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4197/workflows/fa55d5f0-4ed8-4125-bb9e-f83a22446050]
org.apache.cassandra.db.ImportTest testImportInvalidateCache seem to be a flaky
one, similar case as for 5.0 build
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nod
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838501#comment-17838501
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
[CASSANDRA-19401-trunk|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19401-trunk]
{noformat}
java11_pre-commit_tests
✓ j11_build8m 35s
✓ j11_cqlsh_dtests_py311 10m 43s
✓ j11_cqlsh_dtests_py311_vnode 8m 33s
✓ j11_cqlsh_dtests_py388m 37s
✓ j11_cqlsh_dtests_py38_vnode 9m 16s
✓ j11_cqlshlib_cython_tests 12m 31s
✓ j11_cqlshlib_tests 10m 9s
✓ j11_jvm_dtests_latest_vnode 22m 51s
✓ j11_unit_tests 19m 22s
✓ j11_unit_tests_repeat 10m 45s
✓ j11_utests_latest17m 4s
✓ j11_utests_latest_repeat 9m 50s
✓ j11_utests_oa_repeat10m 37s
✓ j11_utests_system_keyspace_directory_repeat 10m 12s
✓ j17_cqlsh_dtests_py311 7m 19s
✓ j17_cqlsh_dtests_py311_vnode 7m 34s
✓ j17_cqlsh_dtests_py386m 55s
✓ j17_cqlsh_dtests_py38_vnode 7m 38s
✓ j17_cqlshlib_cython_tests7m 54s
✓ j17_cqlshlib_tests 6m 16s
✓ j17_unit_tests 13m 45s
✓ j17_unit_tests_repeat 6m 7s
✓ j17_utests_latest16m 6s
✓ j17_utests_latest_repeat 6m 42s
✓ j17_utests_oa 17m 23s
✓ j17_utests_oa_repeat 6m 6s
✕ j11_dtests 38m 56s
gossip_test.TestGossip test_assassinate_valid_node
✕ j11_dtests_latest 37m 52s
configuration_test.TestConfiguration test_change_durable_writes
gossip_test.TestGossip test_assassinate_valid_node
✕ j11_dtests_vnode37m 21s
gossip_test.TestGossip test_assassinate_valid_node
✕ j11_jvm_dtests 39m 56s
org.apache.cassandra.distributed.test.log.FetchLogFromPeersTest
testCMSCatchupTest TIMEOUTED
✕ j11_simulator_dtests36m 28s
org.apache.cassandra.simulator.test.HarrySimulatorTest test
org.apache.cassandra.simulator.test.ShortPaxosSimulationTest
simulationTest
✕ j11_utests_oa 19m 44s
org.apache.cassandra.net.ConnectionTest testTimeout
✕ j11_utests_system_keyspace_directory 21m 5s
org.apache.cassandra.tools.TopPartitionsTest
testServiceTopPartitionsSingleTable
✕ j17_dtests 37m 14s
gossip_test.TestGossip test_assassinate_valid_node
✕ j17_dtests_latest 36m 41s
configuration_test.TestConfiguration test_change_durable_writes
gossip_test.TestGossip test_assassinate_valid_node
✕ j17_dtests_vnode 37m 1s
gossip_test.TestGossip test_assassinate_valid_node
✕ j17_jvm_dtests 27m 32s
org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest
testOptionalMtlsModeDoNotAllowNonSSLConnections TIMEOUTED
org.apache.cassandra.distributed.test.log.FetchLogFromPeersTest
testCMSCatchupTest
org.apache.cassandra.distributed.test.tcm.SplitBrainTest
testSplitBrainStartup TIMEOUTED
✕ j17_jvm_dtests_latest_vnode 22m 24s
junit.framework.TestSuite
org.apache.cassandra.fuzz.harry.integration.model.InJVMTokenAwareExecutorTest
TIMEOUTED
java11_separate_tests
{noformat}
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4192/workflows/09240a96-74db-4549-bb0d-7ad9a6776b09]
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the ke
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838244#comment-17838244
]
Brandon Williams commented on CASSANDRA-19401:
--
bq. ImportTest.testImportCorruptWithoutValidationWithCopying is just flaky and
we discovered it here.
If you are saying that failure is unrelated to this ticket, I don't think
that's accurate:
https://app.circleci.com/pipelines/github/driftx/cassandra/1584/workflows/7797df46-e331-46de-ba5e-885814d8f9a8
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838237#comment-17838237
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
for reviewers: I did it in such a way that only nodetool import will behave
like it does not skip when the dir structure does not match. The listing is
done elsewhere too and I have not touched that. It behaves as it was.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838225#comment-17838225
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
5.0 looks fine too. cqlsh_dtests failed on some circle / github networking
issue.
ImportTest.testImportCorruptWithoutValidationWithCopying is just flaky and we
discovered it here.
[CASSANDRA-19401-5.0|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19401-5.0]
{noformat}
java17_pre-commit_tests
✓ j17_build3m 55s
✓ j17_cqlsh_dtests_py311 6m 18s
✓ j17_cqlsh_dtests_py311_vnode 6m 6s
✓ j17_cqlsh_dtests_py38 6m 9s
✓ j17_cqlshlib_cython_tests7m 47s
✓ j17_cqlshlib_tests 6m 34s
✓ j17_dtests 33m 27s
✓ j17_dtests_latest 32m 34s
✓ j17_dtests_vnode33m 32s
✓ j17_jvm_dtests 17m 57s
✓ j17_jvm_dtests_latest_vnode 16m 3s
✓ j17_unit_tests 16m 12s
✓ j17_utests_latest 14m 15s
✓ j17_utests_latest_repeat 6m 0s
✓ j17_utests_oa 14m 36s
✓ j17_utests_oa_repeat 6m 3s
✕ j17_cqlsh_dtests_py38_vnode 6m 31s
✕ j17_unit_tests_repeat 12m 43s
org.apache.cassandra.db.ImportTest
testImportCorruptWithoutValidationWithCopying
org.apache.cassandra.db.ImportTest
testImportCorruptWithoutValidationWithCopying
{noformat}
[java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4189/workflows/06a9fd1f-a7c6-4be4-bb45-ad09f1bc6e79]
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838135#comment-17838135
]
Norbert Schultz commented on CASSANDRA-19401:
-
Hi [~smiklosovic] I tested your patch it it works for the nodetool import use
case :) Thanks alot!
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838115#comment-17838115
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
4.1 build seems reasonabley fine
[CASSANDRA-19401-4.1|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19401-4.1]
{noformat}
java11_pre-commit_tests
✓ j11_build1m 28s
✓ j11_cqlsh_dtests_py3 5m 24s
✓ j11_cqlsh_dtests_py311 5m 25s
✓ j11_cqlsh_dtests_py311_vnode 6m 14s
✓ j11_cqlsh_dtests_py385m 47s
✓ j11_cqlsh_dtests_py38_vnode 5m 57s
✓ j11_cqlsh_dtests_py3_vnode 5m 33s
✓ j11_cqlshlib_cython_tests6m 55s
✓ j11_cqlshlib_tests6m 9s
✓ j11_dtests 33m 37s
✓ j11_jvm_dtests 19m 2s
✓ j11_jvm_dtests_vnode11m 38s
✓ j11_unit_tests_repeat 8m 4s
✕ j11_dtests_vnode33m 39s
rebuild_test.TestRebuild test_simple_rebuild
✕ j11_unit_tests 7m 34s
org.apache.cassandra.cql3.MemtableSizeTest testSize[skiplist]
{noformat}
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4187/workflows/9164b340-3472-4ba7-a1ae-6ab5cbf695cb]
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838054#comment-17838054
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
[~nob13] this one is more complete patch
[https://github.com/apache/cassandra/pull/3259]
would you verify that? I will create patches for other branches as well.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837992#comment-17837992
]
Norbert Schultz commented on CASSANDRA-19401:
-
Hi [~smiklosovic] are there any updates on this?
We can workaround on this issue, now that we know, but maybe the rest of the
community is happy if it's not happening anymore.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818379#comment-17818379
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
The problem here is that we construct a Descriptor and its inner logic will
still look into the directory structure on the disk from which it derives a
keyspace and a table. Then it is used upon import, but since a keyspace / table
from descriptor is not same as what was specified on the command line, it will
skip it as it thinks it is for a different keyspace / table. All I am doing is
that whatever Descriptor resolves, we will set there keyspace / table from the
command line so it will not skip that.
If we say that a user does not need to have SSTables in a directory structure
where parent is table and parent of parent is keyspace and it will load all
what is in the directory, then I do not see what we have that extra check (I
removed) for.
I will run a full build to see what else it breaks if anything and let you know.
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818375#comment-17818375
]
Norbert Schultz commented on CASSANDRA-19401:
-
[~smiklosovic] Hi Stefan, I tested your PR and the nodetool import is working
there :)
I cannot say to much about the PR itself. From my perspective It looks as if
removing the desired check could have other implications.
Thanks alot for your rapid response!
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Norbert Schultz
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19401) Nodetool import expects directory structure
[
https://issues.apache.org/jira/browse/CASSANDRA-19401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817632#comment-17817632
]
Stefan Miklosovic commented on CASSANDRA-19401:
---
[~nob13] this fixes your issue https://github.com/apache/cassandra/pull/3106
I have not provided any format builds and it might error something else but
with this patch, it just "nodetool import ks tb /tmp/mysstables" fine into ks
tb
> Nodetool import expects directory structure
> ---
>
> Key: CASSANDRA-19401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19401
> Project: Cassandra
> Issue Type: Bug
>Reporter: Norbert Schultz
>Priority: Normal
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> According to the
> [documentation|https://cassandra.apache.org/doc/4.1/cassandra/operating/bulk_loading.html]
> the nodetool import should not rely on the folder structure of the imported
> sst files:
> {quote}
> Because the keyspace and table are specified on the command line for nodetool
> import, there is not the same requirement as with sstableloader, to have the
> SSTables in a specific directory path. When importing snapshots or
> incremental backups with nodetool import, the SSTables don’t need to be
> copied to another directory.
> {quote}
> However when importing old cassandra snapshots, we figured out, that sstables
> still need to be in a directory called like $KEYSPACE/$TABLENAME files, even
> when keyspace and table name are already present as parameters for the
> nodetool import call.
> Call we used:
> {code}
> nodetool import --copy-data mykeyspace mytable /full_path_to/test1
> {code}
> Log:
> {code}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,565
> SSTableImporter.java:72 - Loading new SSTables for mykeyspace/mytable:
> Options{srcPaths='[/full_path_to/test1]', resetLevel=true,
> clearRepaired=true, verifySSTables=true, verifyTokens=true,
> invalidateCaches=true, extendedVerify=false, copyData= true}
> INFO [RMI TCP Connection(21)-127.0.0.1] 2024-02-15 10:41:06,566
> SSTableImporter.java:173 - No new SSTables were found for mykeyspace/mytable
> {code}
> However, when we move the sstables (.db-Files) to
> {{alternative/mykeyspace/mytable}}
> and import with
> {code}
> nodetool import --copy-data mykeyspace mytable
> /fullpath/alternative/mykeyspace/mytable
> {code}
> the import works
> {code}
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:177 - Loading new SSTables and building secondary
> indexes for mykeyspace/mytable:
> [BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-2-big-Data.db'),
>
> BigTableReader(path='/mnt/ramdisk/cassandra4/data/mykeyspace/mytable-561a12d0cbe611eead78fbfd293cee40/me-1-big-Data.db')]
> INFO [RMI TCP Connection(23)-127.0.0.1] 2024-02-15 10:43:36,093
> SSTableImporter.java:190 - Done loading load new SSTables for
> mykeyspace/mytable
> {code}
> We experienced this in Cassandra 4.1.3 on Java 11 (Linux)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
