[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101855#comment-15101855 ] Sylvain Lebresne commented on CASSANDRA-7555: - This was more that work had already be done on this and it felt like finishing this was reasonably low effort (and would avoid prior work to go to waste). But this is certainly not a priority as far as I can tell so not trying to be pushy otherwise. > Support copy and link for commitlog archiving without forking the jvm > - > > Key: CASSANDRA-7555 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Nick Bailey >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 3.x > > > Right now for commitlog archiving the user specifies a command to run and c* > forks the jvm to run that command. The most common operations will be either > copy or link (hard or soft). Since we can do all of these operations without > forking the jvm, which is very expensive, we should have special cases for > those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101765#comment-15101765 ] Joshua McKenzie commented on CASSANDRA-7555: I think "put on a back-burner" is a more political way to phrase it. :) Given the limited scope of use-cases this affects and the uncategorized (at least thus far) performance differential of a native vs. forked solution, this hasn't been high on my priority list. I can adjust if there's a strong case to be made otherwise. > Support copy and link for commitlog archiving without forking the jvm > - > > Key: CASSANDRA-7555 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Nick Bailey >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 3.x > > > Right now for commitlog archiving the user specifies a command to run and c* > forks the jvm to run that command. The most common operations will be either > copy or link (hard or soft). Since we can do all of these operations without > forking the jvm, which is very expensive, we should have special cases for > those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102420#comment-15102420 ] Joshua McKenzie commented on CASSANDRA-7555: Gotcha. It's at the top of my "work on this when other things are blocked and/or driving me crazy". CLA doesn't change that much so it shouldn't be too much work to rebase to current, hence I was willing to let it slide longer than I usually would. I'll take a look when I come back in Feb. > Support copy and link for commitlog archiving without forking the jvm > - > > Key: CASSANDRA-7555 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Nick Bailey >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 3.x > > > Right now for commitlog archiving the user specifies a command to run and c* > forks the jvm to run that command. The most common operations will be either > copy or link (hard or soft). Since we can do all of these operations without > forking the jvm, which is very expensive, we should have special cases for > those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15098151#comment-15098151 ] Sylvain Lebresne commented on CASSANDRA-7555: - [~JoshuaMcKenzie] Pinging you since you seem to have dropped the ball on this. > Support copy and link for commitlog archiving without forking the jvm > - > > Key: CASSANDRA-7555 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Nick Bailey >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 3.x > > > Right now for commitlog archiving the user specifies a command to run and c* > forks the jvm to run that command. The most common operations will be either > copy or link (hard or soft). Since we can do all of these operations without > forking the jvm, which is very expensive, we should have special cases for > those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[
https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633426#comment-14633426
]
Branimir Lambov commented on CASSANDRA-7555:
I think having both a flag and destination directory is a bit too much
configuration for this. Why not get rid of the flag and perform the operation
when the directory is specified? Perhaps renaming to copy_to / hard_link_to
to make it sound more like a directive?
Is there a reason not to [put the logic of the command into the
type|http://allenlsy.com/NOTES-of-Effective-Java-5/#strategy-design-pattern-using-enum]
to avoid the switch and repetition in {{maybeArchive}}? The initial
configuration could also avoid some repetition if you use an enum instance
field. (Feel free to ignore if you don't want to waste time on this.)
{{FILE_COPY_BUFFER_SIZE}} is not really a buffer size as a buffer of that size
is not allocated (neither by us or the OS). To avoid surprises if someone tries
to reuse the constant, I'd rename it to CHUNK or something similar. Do we need
both stream and channel in {{FileUtils.copyFile}}?
Support copy and link for commitlog archiving without forking the jvm
-
Key: CASSANDRA-7555
URL: https://issues.apache.org/jira/browse/CASSANDRA-7555
Project: Cassandra
Issue Type: Improvement
Reporter: Nick Bailey
Assignee: Joshua McKenzie
Priority: Minor
Fix For: 3.x
Right now for commitlog archiving the user specifies a command to run and c*
forks the jvm to run that command. The most common operations will be either
copy or link (hard or soft). Since we can do all of these operations without
forking the jvm, which is very expensive, we should have special cases for
those.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631641#comment-14631641 ] Joshua McKenzie commented on CASSANDRA-7555: Rebased and updated branch [here|https://github.com/apache/cassandra/compare/trunk...josh-mckenzie:7555]. Soft-linking removed. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 3.x Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492463#comment-14492463 ] Branimir Lambov commented on CASSANDRA-7555: Is this ready for review? If it targets 2.1, I think the hard and soft link options should be removed. I am not sure that the soft link makes sense for 3.0 either as the commit log will delete the file if archiving was successful. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 2.1.5 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492497#comment-14492497 ] Joshua McKenzie commented on CASSANDRA-7555: Cancelled patch and targeted 3.0. I'll remove the soft-linking; I agree that it doesn't make a lot of sense in retrospect. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 3.0 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347166#comment-14347166 ] Nick Bailey commented on CASSANDRA-7555: Oh nice. Maybe we just split this into two tickets then. One for copy now and one for link that depends on 8771. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 2.1.4 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347168#comment-14347168 ] Joshua McKenzie commented on CASSANDRA-7555: Since this is an optimization only and there's pretty strong momentum behind CASSANDRA-8771, I'd be fine targeting this change for 3.0 and flag the ticket as dependent on 8771. Good catch though [~nickmbailey]. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 2.1.4 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347163#comment-14347163 ] Jonathan Ellis commented on CASSANDRA-7555: --- That's not set in stone, though. CASSANDRA-8771 Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 2.1.4 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347152#comment-14347152 ] Nick Bailey commented on CASSANDRA-7555: So it seems that C* recycles commitlog segments rather than creating new ones. So I'm not sure either of the link options are actually viable, perhaps just the copy option. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 2.1.4 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7555) Support copy and link for commitlog archiving without forking the jvm
[ https://issues.apache.org/jira/browse/CASSANDRA-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14297354#comment-14297354 ] Joshua McKenzie commented on CASSANDRA-7555: Feature branch [available here|https://github.com/josh-mckenzie/cassandra/compare/7555]. Added some new Windows-specific handling for createDirectory in FileUtils, a copyFile method I pulled from commons-io, and some unit tests. The only change that escapes the scope of the new additions is that I switched to using FileUtils.createDirectory in the constructor for the CommitLogArchiver so errors creating the directories will bubble up as FSWriteErrors now instead of RuntimeErrors. I believe that will be acceptable but figured I'd point it out. Support copy and link for commitlog archiving without forking the jvm - Key: CASSANDRA-7555 URL: https://issues.apache.org/jira/browse/CASSANDRA-7555 Project: Cassandra Issue Type: Improvement Reporter: Nick Bailey Assignee: Joshua McKenzie Priority: Minor Fix For: 2.1.3 Right now for commitlog archiving the user specifies a command to run and c* forks the jvm to run that command. The most common operations will be either copy or link (hard or soft). Since we can do all of these operations without forking the jvm, which is very expensive, we should have special cases for those. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
