[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17025541#comment-17025541
]
Brandon Williams commented on CASSANDRA-4763:
-
Upon further inspection, it looks like CASSANDRA-13884 superseded this.
> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Fix For: 4.0
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the
> path of the directoy of sstables it is loading. This isn't really documented
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a
> snapshot the file names will have the keyspace which is slightly better but
> people manually creating their own sstables might not format them the same.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133937#comment-16133937
]
Eduard Tudenhoefner commented on CASSANDRA-4763:
Here's a first draft
Branch: https://github.com/nastra/cassandra/tree/CASSANDRA-4763-40
Test: https://circleci.com/gh/nastra/cassandra/14
New parameters:
{code}
-k,--keyspace Name of the keyspace
-tb,--table Name of the table
{code}
Example:
{code}
$ bin/sstableloader -d localhost /Users/nastra/Development/sstable_data/ -k
keyspace1 -tb standard1
Established connection to initial hosts
Opening sstables and calculating sections to stream
Streaming relevant part of
/Users/nastra/Development/sstable_data/na-1-big-Data.db to [localhost/127.0.0.1]
progress: [localhost/127.0.0.1]0:1/1 100% total: 100% 0.740KiB/s (avg:
0.740KiB/s)
progress: [localhost/127.0.0.1]0:1/1 100% total: 100% 0.000KiB/s (avg:
0.736KiB/s)
Summary statistics:
Connections per host: 1
Total files transferred : 1
Total bytes transferred : 2.197KiB
Total duration : 2981 ms
Average transfer rate : 0.736KiB/s
Peak transfer rate : 0.740KiB/s
{code}
If either keyspace or table isn't provided, then we're using the old way aka
extracting the info from the directory path.
> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Eduard Tudenhoefner
>Priority: Minor
> Fix For: 4.0
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the
> path of the directoy of sstables it is loading. This isn't really documented
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a
> snapshot the file names will have the keyspace which is slightly better but
> people manually creating their own sstables might not format them the same.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133019#comment-16133019
]
Eduard Tudenhoefner commented on CASSANDRA-4763:
It seems that even with the old format where the sstable would contain
keyspace/table names, *sstableloader* would still try to get the name from the
path
{code}
$ ll
/var/lib/cassandra/data/keyspace1/standard1-4752be9083af11e7a8f5270d6df438fd/snapshots/1503017601804
total 80
drwxr-xr-x 11 root wheel 374B Aug 17 17:53 .
drwxr-xr-x 3 root wheel 102B Aug 17 17:53 ..
-rw-r--r-- 2 root wheel 8B Aug 17 17:53 keyspace1-standard1-ka-1-CRC.db
-rw-r--r-- 2 root wheel 2.7K Aug 17 17:53 keyspace1-standard1-ka-1-Data.db
-rw-r--r-- 2 root wheel10B Aug 17 17:53
keyspace1-standard1-ka-1-Digest.sha1
-rw-r--r-- 2 root wheel24B Aug 17 17:53
keyspace1-standard1-ka-1-Filter.db
-rw-r--r-- 2 root wheel 240B Aug 17 17:53 keyspace1-standard1-ka-1-Index.db
-rw-r--r-- 2 root wheel 4.4K Aug 17 17:53
keyspace1-standard1-ka-1-Statistics.db
-rw-r--r-- 2 root wheel 130B Aug 17 17:53
keyspace1-standard1-ka-1-Summary.db
-rw-r--r-- 2 root wheel79B Aug 17 17:53 keyspace1-standard1-ka-1-TOC.txt
-rw-r--r-- 1 root wheel47B Aug 17 17:53 manifest.json
{code}
{code}
$ sstableloader -d localhost
/var/lib/cassandra/data/keyspace1/standard1-4752be9083af11e7a8f5270d6df438fd/snapshots/1503017601804/
Could not retrieve endpoint ranges:
InvalidRequestException(why:No such keyspace: snapshots)
java.lang.RuntimeException: Could not retrieve endpoint ranges:
at
org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:342)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:156)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:109)
Caused by: InvalidRequestException(why:No such keyspace: snapshots)
at
org.apache.cassandra.thrift.Cassandra$describe_ring_result$describe_ring_resultStandardScheme.read(Cassandra.java:35320)
at
org.apache.cassandra.thrift.Cassandra$describe_ring_result$describe_ring_resultStandardScheme.read(Cassandra.java:35287)
at
org.apache.cassandra.thrift.Cassandra$describe_ring_result.read(Cassandra.java:35229)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_describe_ring(Cassandra.java:1292)
at
org.apache.cassandra.thrift.Cassandra$Client.describe_ring(Cassandra.java:1279)
at
org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:307)
... 2 more
{code}
> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Eduard Tudenhoefner
>Priority: Minor
> Fix For: 2.1.x
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the
> path of the directoy of sstables it is loading. This isn't really documented
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a
> snapshot the file names will have the keyspace which is slightly better but
> people manually creating their own sstables might not format them the same.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197521#comment-15197521
]
Yuki Morishita commented on CASSANDRA-4763:
---
After CASSANDRA-6962, SSTable file does not have keyspace and table name
anymore.
So inferring keyspace and table name from file name is not possible.
But there maybe a use case that users want to load SSTables to different
keyspace and table(of course, this may fail because of incompatibility).
Then, specifying keyspace and table name through command line option maybe
useful.
> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 2.1.x
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the
> path of the directoy of sstables it is loading. This isn't really documented
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a
> snapshot the file names will have the keyspace which is slightly better but
> people manually creating their own sstables might not format them the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197498#comment-15197498
]
Joshua McKenzie commented on CASSANDRA-4763:
[~yukim]: is this still relevant?
> SSTableLoader shouldn't get keyspace from path
> --
>
> Key: CASSANDRA-4763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Affects Versions: 1.2.0 beta 1
>Reporter: Nick Bailey
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 2.1.x
>
>
> SSTableLoader currently gets the keyspace it is going to load to from the
> path of the directoy of sstables it is loading. This isn't really documented
> (or I didn't see it), but also isn't really a good way of doing it in general.
> {noformat}
> this.keyspace = directory.getParentFile().getName();
> {noformat}
> We should probably just let users pass the name in. If you are loading a
> snapshot the file names will have the keyspace which is slightly better but
> people manually creating their own sstables might not format them the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470088#comment-13470088
]
Sylvain Lebresne commented on CASSANDRA-4763:
-
bq. we require sstable files to be named according to the convention cassandra
is using?
We do, and so there is no need for the user to provide the keyspace name as
argument to the loader. That being said, we can lift the limitation that
sstables currently must be in a directory named after the keyspace (I think we
all agree here, just wanted to clarify).
SSTableLoader shouldn't get keyspace from path
--
Key: CASSANDRA-4763
URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.0 beta 1
Reporter: Nick Bailey
Priority: Minor
SSTableLoader currently gets the keyspace it is going to load to from the
path of the directoy of sstables it is loading. This isn't really documented
(or I didn't see it), but also isn't really a good way of doing it in general.
{noformat}
this.keyspace = directory.getParentFile().getName();
{noformat}
We should probably just let users pass the name in. If you are loading a
snapshot the file names will have the keyspace which is slightly better but
people manually creating their own sstables might not format them the same.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469518#comment-13469518
]
Sylvain Lebresne commented on CASSANDRA-4763:
-
Just to add context, this was done initially because it was pretty hard to do
otherwise due to the hard-coded way sstables names were parsed. This was
changed in 1.1 however, so we should be able to do that easily now.
SSTableLoader shouldn't get keyspace from path
--
Key: CASSANDRA-4763
URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.0 beta 1
Reporter: Nick Bailey
Priority: Minor
SSTableLoader currently gets the keyspace it is going to load to from the
path of the directoy of sstables it is loading. This isn't really documented
(or I didn't see it), but also isn't really a good way of doing it in general.
{noformat}
this.keyspace = directory.getParentFile().getName();
{noformat}
We should probably just let users pass the name in. If you are loading a
snapshot the file names will have the keyspace which is slightly better but
people manually creating their own sstables might not format them the same.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4763) SSTableLoader shouldn't get keyspace from path
[
https://issues.apache.org/jira/browse/CASSANDRA-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469571#comment-13469571
]
Nick Bailey commented on CASSANDRA-4763:
I didn't notice before but it looks like we require sstable files to be named
according to the convention cassandra is using? If that is true, then yeah
getting the keyspace name from the actual sstable file name is probably the
right approach here.
SSTableLoader shouldn't get keyspace from path
--
Key: CASSANDRA-4763
URL: https://issues.apache.org/jira/browse/CASSANDRA-4763
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 1.2.0 beta 1
Reporter: Nick Bailey
Priority: Minor
SSTableLoader currently gets the keyspace it is going to load to from the
path of the directoy of sstables it is loading. This isn't really documented
(or I didn't see it), but also isn't really a good way of doing it in general.
{noformat}
this.keyspace = directory.getParentFile().getName();
{noformat}
We should probably just let users pass the name in. If you are loading a
snapshot the file names will have the keyspace which is slightly better but
people manually creating their own sstables might not format them the same.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
