Re: DateTime performance

[Ashley Pond V]

I love and use DateTime for for 10s of millions of records at once I
would be choosing Date::Calc instead and dealing with any necessary
futzy bits manually.

On Thu, May 3, 2012 at 2:53 AM, Rick Measham r...@measham.id.au wrote:
 In the spirit of TIMTOWTDI, there's my DateTime::LazyInit module which I 
 wrote for this sort of case. It only inflates to a full DateTime object when 
 you call methods that aren't simple.

 http://search.cpan.org/~rickm/DateTime-LazyInit-1.0200/lib/DateTime/LazyInit.pm

 Caveat: I haven't tested it against any recent DateTime releases.

 Cheers!
 Rick Measham
 

 On 02/05/2012, at 8:29, Philipp K. Janert jan...@ieee.org wrote:


 Question:

 When using DateTime for a large number of
 instances, it becomes a serious performance
 drag.

 A typical application for me involves things like
 log files: I use DateTime to translate the timestamps
 in these files into a canonical format, and then get
 information such as day-of-week or time-of-day
 from DateTime.

 However, when working through a files with a few
 tens of millions of records, DateTime turns into a
 REAL drag on performance.

 Is this expected behavior? And are there access
 patterns that I can use to mitigate this effect?
 (I tried to supply a time_zone explicitly, but that
 does not seem to improve things significantly.)

 Best,

        Ph.

 --
 Message  protected for iSite by MailGuard: e-mail anti-virus, anti-spam and 
 content filtering.http://www.mailguard.com.au
 Click here to report this message as spam:
 https://login.mailguard.com.au/report/1EEXMobD68/14EZiTvCo3I3sbAw7UgxdE/0

 --
 Message  protected for iSite by MailGuard: e-mail anti-virus, anti-spam and 
 content filtering.http://www.mailguard.com.au

Re: Potential DateTime DOS Attack

[Ashley Pond V]

Schwern++.

On Wed, Dec 16, 2009 at 4:16 PM, Michael G Schwern schw...@pobox.com wrote:
 I am, quite frankly, appalled at the response I've gotten to this report.

 No, this is not something the user should be guarding against.  No,
 documenting it does not make it go away.  No, you shouldn't put an arbitrary
 upper bound in.  No, I should not have been using UTC.  That is all
 accepting low-quality.  We don't do that in Perl.

 This isn't just an annoyance, its a wide spread security hole triggered by
 totally innocent use.  Its like finding out my doorbell will cause my house
 to explode if somebody buzzes too long.  I do not want to program in an
 environment where the assumption is that everything is dangerous.  This
 isn't even a problem one should imagine encountering.

 The reaction I was expecting was more along the lines of oh fuck, that's
 really bad, let's figure out how to fix this.  Yes, its ok to report a
 problem without a ready solution.  Instead, I'm told its documented and to
 cough up a patch.

 This is not a simple problem to solve.  I'm not even sure what the efficient
 DST algorithm is, though I'm looking for it.  I've looked into the
 DateTime::TimeZone code before and this is not going to be a simple patch.
 I'm willing to try and fix it, but not if the DateTime folks, the folks who
 know the code, are reacting with something between lethargy and hostility.
  It going to be hard enough without dragging everyone along.

 Give me something to work with here.  Some insight into what and why
 DateTime is doing what its doing.  Is there a reason that DST info has to be
 generated linearly?  Would it be difficult to hold off on generating time
 zone info until its needed?  Are there instructions somewhere for dealing
 with the Olsen database?  SOME sort of discussion about how to solve the
 problem rather than the ways to paper over it.