Re: XS Leap second questions
On Wed, 18 Jan 2006, Chase Venters wrote: Unfortunately, I've been plagued by an oddity in the way DateTime handles leap seconds in face of the Olson timezone database for some time. This shows up as test failures: t/04epoch...NOK 12# Failed test (t/04epoch.t at line 40) # got: '997120978' # expected: '997121000' t/04epoch...NOK 14# Failed test (t/04epoch.t at line 43) # got: '2' # expected: '3' t/04epoch...NOK 20# Failed test (t/04epoch.t at line 81) # got: '1049160580' # expected: '1049160602' t/04epoch...NOK 22# Failed test (t/04epoch.t at line 84) # got: '29' # expected: '30' t/04epoch...NOK 24# Failed test (t/04epoch.t at line 90) # got: '1049167780' # expected: '1049167802' t/04epoch...ok 32/32# Looks like you failed 5 tests of 32. t/04epoch...dubious This happens when /etc/localtime is linked to a timezone in /usr/share/zoneinfo/right as opposed to /usr/share/zoneinfo. The problem What is /usr/share/zoneinfo/right and how does it differ from /usr/share/zoneinfo? Also, it's been clear to me that using DateTime heavily is a good way to quickly kill performance. Pages that took 20ms to render jump to 500ms when I try to do something simple like apply a recurrence set to a month. Recurrences are not simple at all, and I'm hardly surprised that using them things makes page rendering much slower. But without a description of what you're trying to achieve, and how you are doing it, I can't offer any suggestions. NOTE TO FUTURE DEVELOPERS: Attempting to implement add_duration in XS actually seemed to slow date math operations down. Sad but true. Curious - what was the strategy? What do you assume to be the bottleneck? C has *got* to be faster at math than Perl, so I have a feeling the above remark is specific to a particular implementation approach. Actually, I think it has to do with the cost of passing data across the Perl - C boundary. From what I've heard, this is somewhat costly, so the C code has to be enough faster to really make a difference. That said, I tried that optimization a _long_ time ago, so it might be possible to try again. Along those lines, are there any known outstanding areas where DateTime could use some optimization? I'm handy with C, very handy with Perl and somewhat capable with XS... I'd like to help. Daisuke Maki tried making all the DT::TZ modules use XS, and in fact went so far as to turn them into C-based data structures that were primarily accessed via XS. Strangely, this was also slower and more memory-intensive than the Perl version. I think there's a branch for this in the DateTime CVS repo. I'd love if other folks could take a look at it and see if there is some way to make it faster and smaller. It seems like it should be possible, so maybe there's some obvious fix that Daisuke and I missed. Other than that, I don't know that the DateTime.pm module has any obvious problem spots, but I haven't profiled it either, so I'm pretty much talking out my rump. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: XS Leap second questions
On Thursday 19 January 2006 03:05, Dave Rolsky wrote: On Wed, 18 Jan 2006, Chase Venters wrote: Unfortunately, I've been plagued by an oddity in the way DateTime handles leap seconds in face of the Olson timezone database for some time. This shows up as test failures: t/04epoch...NOK 12# Failed test (t/04epoch.t at line 40) # got: '997120978' # expected: '997121000' t/04epoch...NOK 14# Failed test (t/04epoch.t at line 43) # got: '2' # expected: '3' t/04epoch...NOK 20# Failed test (t/04epoch.t at line 81) # got: '1049160580' # expected: '1049160602' t/04epoch...NOK 22# Failed test (t/04epoch.t at line 84) # got: '29' # expected: '30' t/04epoch...NOK 24# Failed test (t/04epoch.t at line 90) # got: '1049167780' # expected: '1049167802' t/04epoch...ok 32/32# Looks like you failed 5 tests of 32. t/04epoch...dubious This happens when /etc/localtime is linked to a timezone in /usr/share/zoneinfo/right as opposed to /usr/share/zoneinfo. The problem What is /usr/share/zoneinfo/right and how does it differ from /usr/share/zoneinfo? http://www.thedjbway.org/clockspeed/leapsecs.html has a bit of information... basically (my understanding) it's a timezone with proper leap second accounting. I have this sneaking suspicion that DateTime doesn't understand this and is hence doing the calculation when it's already been done. (But then again, I could be talking out of my ass - I haven't spent too much time in the internals yet) Also, it's been clear to me that using DateTime heavily is a good way to quickly kill performance. Pages that took 20ms to render jump to 500ms when I try to do something simple like apply a recurrence set to a month. Recurrences are not simple at all, and I'm hardly surprised that using them things makes page rendering much slower. But without a description of what you're trying to achieve, and how you are doing it, I can't offer any suggestions. Well, DateTime::Event::Recurrence is what I'm relying on, and it's pure perl. But I think in my course of working with DateTime it's unfortunately often felt a bit slow :( NOTE TO FUTURE DEVELOPERS: Attempting to implement add_duration in XS actually seemed to slow date math operations down. Sad but true. Curious - what was the strategy? What do you assume to be the bottleneck? C has *got* to be faster at math than Perl, so I have a feeling the above remark is specific to a particular implementation approach. Actually, I think it has to do with the cost of passing data across the Perl - C boundary. From what I've heard, this is somewhat costly, so the C code has to be enough faster to really make a difference. That said, I tried that optimization a _long_ time ago, so it might be possible to try again. Yeah, I figured it might have something to do with that. What are your thoughts about a DateTime implementation that was *mostly* XS? It would be a lot of work (though it would certainly be made easier because you've already done the hard part of figuring out the logic, which means all a guy like me has to do is reproduce your logic in C). I think by keeping most of the computation in C, you'd certainly get gains well beyond the XS overhead. Along those lines, are there any known outstanding areas where DateTime could use some optimization? I'm handy with C, very handy with Perl and somewhat capable with XS... I'd like to help. Daisuke Maki tried making all the DT::TZ modules use XS, and in fact went so far as to turn them into C-based data structures that were primarily accessed via XS. Strangely, this was also slower and more memory-intensive than the Perl version. Odd. It might have something to do with what we're talking about above. I think there's a branch for this in the DateTime CVS repo. I'd love if other folks could take a look at it and see if there is some way to make it faster and smaller. It seems like it should be possible, so maybe there's some obvious fix that Daisuke and I missed. Other than that, I don't know that the DateTime.pm module has any obvious problem spots, but I haven't profiled it either, so I'm pretty much talking out my rump. Welp, I'm unfortunately quite preoccupied with a project at the moment, but I consider DateTime to be a vital piece of my CPAN arsenal for web / tel app development, so I'm more than willing to put in time when I can. Thanks for your response! -dave Cheers, Chase
Re: XS Leap second questions
Daisuke Maki tried making all the DT::TZ modules use XS, and in fact went so far as to turn them into C-based data structures that were primarily accessed via XS. Strangely, this was also slower and more memory-intensive than the Perl version. My next guess was that this had to do with how Perl allocates memory for each namespace. So I was thinking I could try to make a single object that could represent all the timezones, but I was kind of lost momentum while designing that. sad. Perhaps somebody can think of a way to do something like this? my $tz = DateTime::TimeZone-new( name = ... aliases = ... other_required_fields = ... ); This would mean that we would be creating a single .xs file that has a hash lookup of timezone names to C structs, and we would be initializing the object with that data. --d
Re: XS Leap second questions
On Thursday 19 January 2006 07:08, Flavio S. Glock wrote: 2006/1/19, Chase Venters [EMAIL PROTECTED]: Also, it's been clear to me that using DateTime heavily is a good way to quickly kill performance. Pages that took 20ms to render jump to 500ms when I try to do something simple like apply a recurrence set to a month. This is the command sequence I use in order to profile recurrences: cd ~/cvs/modules/DateTime-Event-ICal perl -d:DProf -Ilib t/04rfc2445.t dprofpp I think that inlining the parameter validation (the calls to Params::Validate::_validate) in a few places in DateTime.pm should help performance a bit. Yes - Indeed. I've noticed this as well. By localizing Params::Validate::NO_VALIDATION to true, I save 100ms on the aforementioned operation (which still takes ~400ms without it, unfortunately). - Flavio S. Glock Cheers, Chase
Re: XS Leap second questions
On Thursday 19 January 2006 03:57, Daisuke Maki wrote: Perhaps somebody can think of a way to do something like this? my $tz = DateTime::TimeZone-new( name = ... aliases = ... other_required_fields = ... ); This would mean that we would be creating a single .xs file that has a hash lookup of timezone names to C structs, and we would be initializing the object with that data. This makes me curious about something else... is it possible for different XS modules to become aware of their existence and then use, say, the exchange of a function pointer or vtable to establish cross-communication, sans perl overhead? This could allow for some remarkable optimizations across the board, I think, with the advantage that it would be easy to fall back on Pure Perl wherever necessary. --d Cheers, Chase
