[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903595#comment-13903595 ] Brandon Williams commented on CASSANDRA-3991: - done. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.0.1 > > Attachments: > 0001-CASSANDRA-3991-refactor-init-script-to-use-start-sto.patch > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[
https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903539#comment-13903539
]
Cyril Scetbon commented on CASSANDRA-3991:
--
you should remove JVSC from init file as it's not used anymore :
{code}
[cyril:~/src/git/cassandra] trunk(110)+ 2 ± grep JSVC debian/init
JSVC=/usr/bin/jsvc
{code}
> Investigate importance of jsvc in debian packages
> -
>
> Key: CASSANDRA-3991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3991
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Brandon Williams
>Assignee: Eric Evans
>Priority: Minor
> Fix For: 2.0.1
>
> Attachments:
> 0001-CASSANDRA-3991-refactor-init-script-to-use-start-sto.patch
>
>
> jsvc seems to be buggy at best. For instance, if you set a small heap like
> 128M it seems to completely ignore this and use as much memory as it wants.
> I don't know what this is buying us over launching /usr/bin/cassandra
> directly like the redhat scripts do, but I've seen multiple complaints about
> its memory usage.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13745242#comment-13745242 ] Brandon Williams commented on CASSANDRA-3991: - nit: we can remove the jsvc dep too. Otherwise +1 > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > Attachments: > 0001-CASSANDRA-3991-refactor-init-script-to-use-start-sto.patch > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13741338#comment-13741338 ] Jonathan Ellis commented on CASSANDRA-3991: --- WFM. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > Attachments: > 0001-CASSANDRA-3991-refactor-init-script-to-use-start-sto.patch > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[
https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13741310#comment-13741310
]
Eric Evans commented on CASSANDRA-3991:
---
bc. Isn't there some jsvc-specific hook code in our java that isn't necessary
w/ this change?
I don't think so. {{CassandraDaemon}} conforms to the required interface
(without actually implementing a Java one, see
http://commons.apache.org/proper/commons-daemon/jsvc.html#Directly), and I
don't see commons-daemon anywhere among our dependencies.
You could probably eliminate the init(String[] args) and destroy() methods, but
it seems cheap to leave them in case someone ever wants to use it themselves.
> Investigate importance of jsvc in debian packages
> -
>
> Key: CASSANDRA-3991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3991
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Brandon Williams
>Assignee: Eric Evans
>Priority: Minor
> Fix For: 2.1
>
> Attachments:
> 0001-CASSANDRA-3991-refactor-init-script-to-use-start-sto.patch
>
>
> jsvc seems to be buggy at best. For instance, if you set a small heap like
> 128M it seems to completely ignore this and use as much memory as it wants.
> I don't know what this is buying us over launching /usr/bin/cassandra
> directly like the redhat scripts do, but I've seen multiple complaints about
> its memory usage.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13741175#comment-13741175 ] Jonathan Ellis commented on CASSANDRA-3991: --- Isn't there some jsvc-specific hook code in our java that isn't necessary w/ this change? > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > Attachments: > 0001-CASSANDRA-3991-refactor-init-script-to-use-start-sto.patch > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[
https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13741160#comment-13741160
]
Eric Evans commented on CASSANDRA-3991:
---
{quote}
The issue I described originally can be repro'd 100% of the time. I've seen
this burning people on irc who install from the debian package (thus getting
jsvc) and wanting to use a tiny jvm for development.
{quote}
After all this time, I'm not sure I tested that. If it struck me then the way
it does now, I might put it off. Certainly, {{jsvc}} can't be the only
obstacle to running under such low memory requirements.
I _have_ spent a fair amount of time trying to reproduced the runaway
utilization and memory leak complaints, to no avail.
> Investigate importance of jsvc in debian packages
> -
>
> Key: CASSANDRA-3991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3991
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Brandon Williams
>Assignee: Eric Evans
>Priority: Minor
> Fix For: 2.1
>
>
> jsvc seems to be buggy at best. For instance, if you set a small heap like
> 128M it seems to completely ignore this and use as much memory as it wants.
> I don't know what this is buying us over launching /usr/bin/cassandra
> directly like the redhat scripts do, but I've seen multiple complaints about
> its memory usage.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13738656#comment-13738656 ] Arya Goudarzi commented on CASSANDRA-3991: -- Back in November 2012, we had a 12 node cluster of m1.xlarge EC2 instances. We had about 30 ColumnFamilies on the cluster. Each node had 8Gb heap and we didn't change the defaults. We had a workload of about 3K writes and 1.5k reads per second on the cluster as a result of running some batch jobs. The heap just got filled up all the way to the point that GC was not happening any more. At this time gossip was still running and announcing but all operations to the node were timing out. Top showed JSVC is using 60Gb of virtual memory, and I supposed that was mmaped files. I suppose if you stress C* on a debian node you can reproduce this easily, although I never got a chance to try it independently. We use Ubuntu 10.4 LTS. At the time this was happening we had C* 1.1.x. Now we have 1.2.5. I ended up replacing JSVC with start-stop-daemon. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13738350#comment-13738350 ] Brandon Williams commented on CASSANDRA-3991: - The issue I described originally can be repro'd 100% of the time. I've seen this burning people on irc who install from the debian package (thus getting jsvc) and wanting to use a tiny jvm for development. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736059#comment-13736059 ] Eric Evans commented on CASSANDRA-3991: --- I probably should have updated this ticket with as much, but I have tried to reproduce this a number of times with an eye toward debugging and fixing `jsvc`, with no luck. If someone could give me some concrete steps to reproduce, I would be more than happy to look into this. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736045#comment-13736045 ] Jonathan Ellis commented on CASSANDRA-3991: --- Can anyone actually show us how to reproduce a problem with jsvc? If not, let's put this ticket out of its misery. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 2.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493729#comment-13493729 ] Arya Goudarzi commented on CASSANDRA-3991: -- +1 on this issue. I was not monitoring the inodes but JSVC definitely was eating 68Gb of memory and using a lot of CPU. Starting cassandra without it solved the problem. It was really bad as for a week Cassandra was being blamed but really that was not the issue. For sake of Cassandra reputation, this deserves a higher priority. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 1.3 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419526#comment-13419526 ] Taras Ovsyankin commented on CASSANDRA-3991: We ran into a similar issue with 1.1.2 - nearly 100K open unix sockets got the cluster hosed. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 1.2 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396174#comment-13396174 ] Viktor Kuzmin commented on CASSANDRA-3991: -- Just to confirm: my problem is not related to jsvc at all. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 1.2 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396164#comment-13396164 ] Brandon Williams commented on CASSANDRA-3991: - Viktor mentioned on irc that his problem is actually not related to jsvc. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Eric Evans >Priority: Minor > Fix For: 1.2 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292103#comment-13292103 ] Viktor Kuzmin commented on CASSANDRA-3991: -- Os: Debian Squeeze (minimal setup, latest packages at the moment of post) After upgrading from 1.1.0 to 1.1.1 I've faced the problem with error: "Too many open files" After investigation it appears that it was jsvc who spawns lot's of unix sockets FD's for communication between forked processes. There is no more such problem after moving to start-stop-daemon. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 1.2 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287564#comment-13287564 ] Brandon Williams commented on CASSANDRA-3991: - So, I think where we left on irc last was "we'll fix jsvc." Any progress on that front? > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 1.2 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231165#comment-13231165 ] Eric Evans commented on CASSANDRA-3991: --- bq. Certainly OOMing in a loop, with heap dumps enabled, is not a thing we really ought to be doing. FWIW, jsvc isn't restarting for an OOM, it requires the JVM itself to crash (something I don't think happens very frequently these days). > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228203#comment-13228203 ] Brandon Williams commented on CASSANDRA-3991: - bq. But if the main goal is to restart after a crash I don't think that is a main goal. Certainly OOMing in a loop, with heap dumps enabled, is not a thing we really ought to be doing. Thanks for the JSW pointer, though! > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228201#comment-13228201 ] Drew Kutcharian commented on CASSANDRA-3991: Alternatively there is Java Service Wrapper http://wrapper.tanukisoftware.com/doc/english/download.jsp which is a lot better than jsvc but it's GPLv2. But if the main goal is to restart after a crash, Java natively supports it via -XX:OnError=... command line option. http://www.oracle.com/technetwork/java/javase/clopts-139448.html#gbmum > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223937#comment-13223937 ] Brandon Williams commented on CASSANDRA-3991: - bq. If jsvc is buggy (is this memory thing the only problem?) I've also heard of output.log getting huge, as you noted on irc we can logrotate that problem away. bq. Try to properly daemonize entirely from shell (I tried doing this with bin/cassandra FWIW, I don't think it's practical) Out of curiosity, what was the problem? I'm doing this on a machine right now (by removing jsvc from the init) and it hasn't been a problem (though I'll admit I'm running it as root, heh) bq. Looking at the source, jsvc seems pretty simple. I might be willing to take a crack at bug-fixing in the weeks to come assuming a) I knew how to reproduce the issue(s), and b) everyone doesn't already have their hearts set on #4. I don't see anything pressing enough to jump straight to 4 yet, but if that changes it's the easiest option to implement so I'm willing to wait and try to do things the Right Way. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[
https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223935#comment-13223935
]
Eric Evans commented on CASSANDRA-3991:
---
Java's "platform neutrality" makes it impossible to properly daemonize, jsvc is
meant as a work-around. It disassociates from the controlling terminal,
becomes the session leader, double-forks, sets the PWD to "/", sets the umask
to 0, closes file descriptors, etc, etc.
As Sylvain mentions, it's also supposed to restart when the JVM (not the app)
crashes.
bq. I guess the main point of JSVC is this? "Jsvc allows the application (e.g.
Tomcat) to perform some privileged operations as root (e.g. bind to a port <
1024), and then switch identity to a non-privileged user." We don't even use
that...
We might be opening root-owned files at start-up (we used to anyway).
If jsvc is buggy (is this memory thing the only problem?) the options would
seem to be:
# File a bug report with Debian, commons-daemon, or both
# Fix the jsvc bug(s)
# Try to properly daemonize entirely from shell (I tried doing this with
{{bin/cassandra}} FWIW, I don't think it's practical)
# Rip out jsvc and call it Good Enough(tm)
Looking at the source, jsvc seems pretty simple. I might be willing to take a
crack at bug-fixing in the weeks to come assuming a) I knew how to reproduce
the issue(s), and b) everyone doesn't already have their hearts set on #4.
> Investigate importance of jsvc in debian packages
> -
>
> Key: CASSANDRA-3991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3991
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Brandon Williams
>Assignee: Brandon Williams
> Fix For: 1.1.1
>
>
> jsvc seems to be buggy at best. For instance, if you set a small heap like
> 128M it seems to completely ignore this and use as much memory as it wants.
> I don't know what this is buying us over launching /usr/bin/cassandra
> directly like the redhat scripts do, but I've seen multiple complaints about
> its memory usage.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221765#comment-13221765 ] Brandon Williams commented on CASSANDRA-3991: - bq. Are we sure it actually restart the process? I could be wrong but last time I checked, jsvc was only restarting a crashed process if the exit code was 123 (or something like that) I know from experience if you set the heap to a small value like 64M and use jsvc on a machine with a small amount of memory (like 256M) it will a) use more memory than the system has, b) get killed by the oom killer, and c) restart the cycle again. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221591#comment-13221591 ] Jonathan Ellis commented on CASSANDRA-3991: --- I guess the main point of JSVC is this? "Jsvc allows the application (e.g. Tomcat) to perform some privileged operations as root (e.g. bind to a port < 1024), and then switch identity to a non-privileged user." We don't even use that... > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221562#comment-13221562 ] Sylvain Lebresne commented on CASSANDRA-3991: - Are we sure it actually restart the process? I could be wrong but last time I checked, jsvc was only restarting a crashed process if the exit code was 123 (or something like that) and I'm not sure we do that. Besides, restarting the process on crash by default in the debian package is a bad idea imho. Anyway, not sure this is really buying us anything so I'm good removing it too. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3991) Investigate importance of jsvc in debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221252#comment-13221252 ] Brandon Williams commented on CASSANDRA-3991: - One thing jsvc does is restart a crashed process -- but there are other, better ways of accomplishing this. > Investigate importance of jsvc in debian packages > - > > Key: CASSANDRA-3991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3991 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 1.1.1 > > > jsvc seems to be buggy at best. For instance, if you set a small heap like > 128M it seems to completely ignore this and use as much memory as it wants. > I don't know what this is buying us over launching /usr/bin/cassandra > directly like the redhat scripts do, but I've seen multiple complaints about > its memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
