[jruby-dev] [jira] (JRUBY-7182) 1.7.3 crashes with my multithreaded benchmark
Mike Perham created JRUBY-7182 1.7.3 crashes with my multithreaded benchmark Issue Type: Bug Assignee: Thomas E Enebo Created: 30/Jun/13 2:22 PM Description: I'm trying to benchmark running 20,000 empty Sidekiq jobs, to understand the threading overhead for each of 1.9.3, 2.0.0 and JRuby. Both versions of MRI finish the benchmark without a problem, JRuby crashes after about 15 sec. Exception in thread "JRubyWorker-24" org.jruby.exceptions.RaiseException: (DeadTaskError) cannot resume a dead task (dead fiber called) at Celluloid::TaskFiber.deliver(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/tasks/task_fiber.rb:25) at Celluloid::TaskFiber.deliver(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/tasks/task_fiber.rb:21) at Celluloid::Task.resume(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/tasks.rb:69) at Celluloid::Response.dispatch(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/responses.rb:11) at Celluloid::Actor.handle_message(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/actor.rb:331) at RUBY.run(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/actor.rb:174) at RUBY.initialize(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/actor.rb:157) at RUBY.initialize(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/thread_handle.rb:13) at org.jruby.RubyProc.call(org/jruby/RubyProc.java:249) at RUBY.create(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/internal_pool.rb:59) Exception in thread "JRubyWorker-19" org.jruby.exceptions.RaiseException: (DeadTaskError) cannot resume a dead task (dead fiber called) at Celluloid::TaskFiber.deliver(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/tasks/task_fiber.rb:25) at Celluloid::TaskFiber.deliver(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/tasks/task_fiber.rb:21) at Celluloid::Task.resume(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/tasks.rb:69) at Celluloid::Response.dispatch(/Users/mperham/.rbenv/versions/jruby-1.7.3/lib/ruby/gems/shared/gems/celluloid-0.14.1/lib/celluloid/responses.rb:11) To reproduce: 1. Install redis. 2. redis-cli flushall 3. git clone git://github.com/mperham/sidekiq 4. cd sidekiq/myapp 5. bundle 6. bundle exec ruby bench.rb (this just loads 20k jobs into redis) 7. bundle exec sidekiq With 2.0.0, you'll see a ton of job start/done log output and takes 50 sec on my MBA. Environment: JRuby 1.7.3, OSX 10.8 Project: JRuby Priority: Major Reporter: Mike Perham This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
[jruby-dev] Re: Latest on maven build
Ok, things are slowly falling together, so he's another update. New working: * tzdata stuff (waiting on confirmation from Hiro) Known remaining: * tests passing (they're closer but embed tests and a couple main tests still fail) * OSGi bnd bundle stuff from jar-jruby-dist into maven (I tried using Apache Felix plugin but could not get it to work right. Maybe this doesn't need to happen during normal package...only in maven artifacts we actually push?) * Scrubbing of build.xml to continue removing things mvn is doing for us. Is there any way to get tests to not run during package, but still work for "test" goal (i.e. hardcoding skipTests is NOT a solution)? It's a serious impediment to have to run tests every time we package up a new jar locally. - Charlie On Sat, Jun 29, 2013 at 2:01 PM, Charles Oliver Nutter wrote: > Done: > >> * From old jar building: install spi services stuff under meta-inf >> * From old test runs: compile test files. I'm hoping these can be run >> as part of the maven build and we won't actually need to add them to >> the rake test runs (i.e. these can be our smoke tests during >> packaging) > > I added in surefire test configs so tests are running, but they don't > pass yet (environmental oddities mostly). > > Good news... jar-jruby ant target is now empty! jar-jruby-dist > piggybacks off it but is not building correctly (just the osgi bnd > stuff). > > Known items remaining to do: > > * get tests passing ;-) >> * Eliminate build_lib artifacts. I've started removing references to >> build_lib but have not removed most jars yet. >> * dist confirmation: need to confirm dist artifacts are building properly >> * cext as its own submodule: I think we should move cext to its own >> ext/ module like openssl and readline rather than move out of the repo >> entirely (for now) >> * psych as its own module: this is going to be merged into upstream >> psych gem soon so we need to start building it as a separate jar > > - Charlie - To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Re: [jruby-dev] Welcome christian to JRuby Core :-)
Great news! Congrats, Kristian! Cheers, Rodrigo. Em 29-06-2013 13:16, Charles Oliver Nutter escreveu: Tom and I have agreed to add christian as a JRuby committer, for help with maven in the past and help with the current reorganization. Welcome to the team (officially)! - Charlie - To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email - To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Re: [jruby-dev] Pre-branch (1.7) reorg of codebase
just resend since it bounced. -christian On Sun, Jun 30, 2013 at 11:31 AM, kristian wrote: > > On Sat, Jun 29, 2013 at 5:21 PM, Charles Oliver Nutter < > head...@headius.com> wrote: > >> This seems pretty good, but is there a reason we need all the >> jruby-blah poms to be in their own directories? Could there be a >> single maven dir with pom-complete, pom-core, pom-stdlib etc? >> >> > the modules section is such a list of directory-names and each directory > needs a pom.xml to be module. the problem is only the pom - to create and > attach any jar to pom is no problem but with jruby-core, jruby-complete, > etc we a tailored pom for those jars. > > > >> One goal of this reorg is to clean up the repo root a bit, and moving >> those to top-level seems like a step backward. >> > > we can keep the current maven project with all its subdirectories no need > to them into basedir. > > but I also realized one more thing: IDE support > and for this we really need to make the main jruby artifact as straight > forward as possible. this means move the main sources into its own module > speak directory, I think the IDE support is really important - starting > with myself as eclipse user. > > I also wondered when all the dependencies are maven artifacts then there > is no difference between jruby.jar and jruby-core.jar ? not much I guess. > maybe we can call the main artifact jruby-core then and make ONE more > directory to the basedir: jruby-core ? > > - christian > >
Re: [jruby-dev] Latest on maven build
I think the joda-timezones module christian set up should be doing what you need. I just commented on your issue. Let us know if it's not doing what you need it to. - Charlie On Sat, Jun 29, 2013 at 11:11 PM, Hirotsugu Asari wrote: > > On Jun 29, 2013, at 2:01 PM, Charles Oliver Nutter > wrote: > >> Done: >> >>> * From old jar building: install spi services stuff under meta-inf >>> * From old test runs: compile test files. I'm hoping these can be run >>> as part of the maven build and we won't actually need to add them to >>> the rake test runs (i.e. these can be our smoke tests during >>> packaging) >> >> I added in surefire test configs so tests are running, but they don't >> pass yet (environmental oddities mostly). >> >> Good news... jar-jruby ant target is now empty! jar-jruby-dist >> piggybacks off it but is not building correctly (just the osgi bnd >> stuff). >> >> Known items remaining to do: >> >> * get tests passing ;-) >>> * Eliminate build_lib artifacts. I've started removing references to >>> build_lib but have not removed most jars yet. >>> * dist confirmation: need to confirm dist artifacts are building properly >>> * cext as its own submodule: I think we should move cext to its own >>> ext/ module like openssl and readline rather than move out of the repo >>> entirely (for now) >>> * psych as its own module: this is going to be merged into upstream >>> psych gem soon so we need to start building it as a separate jar > > > Since it is near and dear to my heart (and I whined about it on Twitter), I > raise the issue here: tzdata ant targets are completely forgotten. > > The generated Constants.java does not overwrite @tzdata.version@ and the ant > targets needed to recompile joda-time-*.jar to include the latest tzdata > release do not have equivalents in the maven lifecycle. > > I briefly considered learning maven, but I gave up (at least for now). > > Hiro > - > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > - To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Re: [jruby-dev] Latest on maven build
I did the tzdata before and removed it from the main pom.xml since it was not the right place to do. just added the tzdata stuff to jruby master as an artifact. and reverted douglas build.xml revert. - christian On Sun, Jun 30, 2013 at 4:11 AM, Hirotsugu Asari wrote: > > On Jun 29, 2013, at 2:01 PM, Charles Oliver Nutter > wrote: > > > Done: > > > >> * From old jar building: install spi services stuff under meta-inf > >> * From old test runs: compile test files. I'm hoping these can be run > >> as part of the maven build and we won't actually need to add them to > >> the rake test runs (i.e. these can be our smoke tests during > >> packaging) > > > > I added in surefire test configs so tests are running, but they don't > > pass yet (environmental oddities mostly). > > > > Good news... jar-jruby ant target is now empty! jar-jruby-dist > > piggybacks off it but is not building correctly (just the osgi bnd > > stuff). > > > > Known items remaining to do: > > > > * get tests passing ;-) > >> * Eliminate build_lib artifacts. I've started removing references to > >> build_lib but have not removed most jars yet. > >> * dist confirmation: need to confirm dist artifacts are building > properly > >> * cext as its own submodule: I think we should move cext to its own > >> ext/ module like openssl and readline rather than move out of the repo > >> entirely (for now) > >> * psych as its own module: this is going to be merged into upstream > >> psych gem soon so we need to start building it as a separate jar > > > Since it is near and dear to my heart (and I whined about it on Twitter), > I raise the issue here: tzdata ant targets are completely forgotten. > > The generated Constants.java does not overwrite @tzdata.version@ and the > ant targets needed to recompile joda-time-*.jar to include the latest > tzdata release do not have equivalents in the maven lifecycle. > > I briefly considered learning maven, but I gave up (at least for now). > > Hiro > - > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > >
Re: [jruby-dev] Pre-branch (1.7) reorg of codebase
On Sat, Jun 29, 2013 at 5:21 PM, Charles Oliver Nutter wrote: > This seems pretty good, but is there a reason we need all the > jruby-blah poms to be in their own directories? Could there be a > single maven dir with pom-complete, pom-core, pom-stdlib etc? > > the modules section is such a list of directory-names and each directory needs a pom.xml to be module. the problem is only the pom - to create and attach any jar to pom is no problem but with jruby-core, jruby-complete, etc we a tailored pom for those jars. > One goal of this reorg is to clean up the repo root a bit, and moving > those to top-level seems like a step backward. > we can keep the current maven project with all its subdirectories no need to them into basedir. but I also realized one more thing: IDE support and for this we really need to make the main jruby artifact as straight forward as possible. this means move the main sources into its own module speak directory, I think the IDE support is really important - starting with myself as eclipse user. I also wondered when all the dependencies are maven artifacts then there is no difference between jruby.jar and jruby-core.jar ? not much I guess. maybe we can call the main artifact jruby-core then and make ONE more directory to the basedir: jruby-core ? - christian
Re: [jruby-dev] Welcome christian to JRuby Core :-)
wow - comes as a surprise. if I look at my jruby projects (jruby-maven-plugins, jbundler, nexus-ruby-support, etc) they are all related to maven :) I am happy to help out here and appreciate the more direct access to do so regards, christian (kristian) On Sat, Jun 29, 2013 at 5:16 PM, Charles Oliver Nutter wrote: > Tom and I have agreed to add christian as a JRuby committer, for help > with maven in the past and help with the current reorganization. > > Welcome to the team (officially)! > > - Charlie > > - > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > >
