Re: Must DateTime load all Locales?
On Sat, 9 Aug 2014, David E. Wheeler wrote: What could speed things up would be to pre-generate the data structures that are built by calling -register on everything in the Catalog module. It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no? Or rather than building this all up via repeated calls to _register, the Catalog code could just contain the fully realized structure. That all said, DT::Locale is woefully crufty and out of date. I haven't kept up to date with the CLDR data for years because the approach that DT::Locale uses to generate individual locales makes it incredibly hard to deal with data format changes in CLDR. Ah. So some locale results will probably be wrong now, eh? Oh yes. Nick Patch had mentioned that he was interested in working on proper CLDR support for Perl (which is more than just DateTime info), and I was hoping he'd take all this over ;) Is CLDR::Number part of that, perhaps? http://search.cpan.org/dist/CLDR-Number/ Probably. But CLDR is a huge beast. I really think we need a Perl interface to libicu, although there'd be a serious mismatch between how it handles dates (POSIX style with no leap seconds) and DateTime. So just having the interface would just be a first step. -dave /* http://VegGuide.org http://blog.urth.org Your guide to all that's veg House Absolute(ly Pointless) */
Re: Must DateTime load all Locales?
On Aug 11, 2014, at 10:01 AM, Dave Rolsky auta...@urth.org wrote: It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no? Or rather than building this all up via repeated calls to _register, the Catalog code could just contain the fully realized structure. How hard would that be to do? D signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Must DateTime load all Locales?
On Mon, 11 Aug 2014, David E. Wheeler wrote: On Aug 11, 2014, at 10:01 AM, Dave Rolsky auta...@urth.org wrote: It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no? Or rather than building this all up via repeated calls to _register, the Catalog code could just contain the fully realized structure. How hard would that be to do? Hard and not hard ;) The Catalog is currently generated as part of the code that generates each individual locale. Changing that part of the code generator isn't too hard. The trick is you have to use the exact version of the CLDR data that this generator can parse, which is CLDR 1.7.1. You can probably get it from their historical archives or repo. -ave /* http://VegGuide.org http://blog.urth.org Your guide to all that's veg House Absolute(ly Pointless) */
Re: Must DateTime load all Locales?
On Aug 7, 2014, at 11:23 AM, Dave Rolsky auta...@urth.org wrote: I think the overhead comes in the DateTime::Locale::_register() function, which is called once for each locale returned by DateTime::Locale::Catalog. I would not think any locales would be loaded until they were needed. And you would be correct. The individual locale classes are not loaded until you call DateTime::Locale-load I see. So _register does not load them, but is still the startup bottleneck. What on earth is it doing? What could speed things up would be to pre-generate the data structures that are built by calling -register on everything in the Catalog module. It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no? That all said, DT::Locale is woefully crufty and out of date. I haven't kept up to date with the CLDR data for years because the approach that DT::Locale uses to generate individual locales makes it incredibly hard to deal with data format changes in CLDR. Ah. So some locale results will probably be wrong now, eh? Nick Patch had mentioned that he was interested in working on proper CLDR support for Perl (which is more than just DateTime info), and I was hoping he'd take all this over ;) Is CLDR::Number part of that, perhaps? http://search.cpan.org/dist/CLDR-Number/ I have updated Sqitch to defer loading DateTime at all until it is needed. Alas, it is often needed. Sure would be nice to cut down on this startup time in general. Best, David signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Must DateTime load all Locales?
On Tue, 5 Aug 2014, David E. Wheeler wrote: On Aug 5, 2014, at 4:18 PM, Olivier Mengué olivier.men...@gmail.com wrote: Here is below the output (latest releases of DateTime, DateTime::Locale, DateTime::TimeZone on perl 5.20.0). DateTime::Locale::Catalog is the module that contains all the locales names. David, is it that module that you worry about or have you found something worse? I think the overhead comes in the DateTime::Locale::_register() function, which is called once for each locale returned by DateTime::Locale::Catalog. I would not think any locales would be loaded until they were needed. And you would be correct. The individual locale classes are not loaded until you call DateTime::Locale-load What could speed things up would be to pre-generate the data structures that are built by calling -register on everything in the Catalog module. That all said, DT::Locale is woefully crufty and out of date. I haven't kept up to date with the CLDR data for years because the approach that DT::Locale uses to generate individual locales makes it incredibly hard to deal with data format changes in CLDR. Nick Patch had mentioned that he was interested in working on proper CLDR support for Perl (which is more than just DateTime info), and I was hoping he'd take all this over ;) -dave /* http://VegGuide.org http://blog.urth.org Your guide to all that's veg House Absolute(ly Pointless) */
Re: Must DateTime load all Locales?
On 08/06/2014 12:34 AM, David E. Wheeler wrote: Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. Best, David I guess it's not enough for you, but T:M's strftime does support locales, it seems.
Re: Must DateTime load all Locales?
On Aug 6, 2014, at 2:21 AM, Kaare Rasmussen ka...@jasonic.dk wrote: Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. Best, David I guess it's not enough for you, but T:M's strftime does support locales, it seems. It does? The docs all say that it uses the C locale. Which is a good default, but unlocalizable, unless I’m missing something. Best, David signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Must DateTime load all Locales?
Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. Best, David I guess it's not enough for you, but T:M's strftime does support locales, it seems. It does? The docs all say that it uses the C locale. Which is a good default, but unlocalizable, unless I’m missing something. Yeah, guess I was sleepy when reading the docs. I also guess it's a future development point, but it won't help you now.
Re: Must DateTime load all Locales?
On 2014-08-05, 3:34 PM, David E. Wheeler wrote: On Aug 5, 2014, at 3:09 PM, Darren Duncan dar...@darrenduncan.net wrote: Coincidentally, see this blog post I saw yesterday: http://blogs.perl.org/users/chansen/2014/08/timemoment-vs-datetime.html Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. By the way, my pointing to the blog post wasn't so much to specifically recommend Time::Moment as a solution for you, as that it had information comparing resource usage of a number of temporal data solutions, and in particular reinforced your comment of DateTime using a lot of resources, relative to most others. -- Darren Duncan
Re: Must DateTime load all Locales?
Coincidentally, see this blog post I saw yesterday: http://blogs.perl.org/users/chansen/2014/08/timemoment-vs-datetime.html This has a lot to say about performance issues. -- Darren Duncan
Re: Must DateTime load all Locales?
On Aug 5, 2014, at 3:09 PM, Darren Duncan dar...@darrenduncan.net wrote: Coincidentally, see this blog post I saw yesterday: http://blogs.perl.org/users/chansen/2014/08/timemoment-vs-datetime.html Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. Best, David signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Must DateTime load all Locales?
Here is another way to profile DateTime for the modules it loads at compile time: perl -d:TraceUse -MDateTime -c -e0 Here is below the output (latest releases of DateTime, DateTime::Locale, DateTime::TimeZone on perl 5.20.0). DateTime::Locale::Catalog is the module that contains all the locales names. David, is it that module that you worry about or have you found something worse? Modules used from -e: 1. DateTime 1.10, -e line 0 [main] 2.strict 1.08, DateTime.pm line 5 3.warnings 1.23, DateTime.pm line 6 4.warnings::register 1.03, DateTime.pm line 7 5.Carp 1.3301, DateTime.pm line 9 6. Exporter 5.70, Carp.pm line 99 7.DateTime::Duration 1.10, DateTime.pm line 10 8. DateTime::Helpers 1.10, DateTime/Duration.pm line 8 9.Scalar::Util 1.38, DateTime/Helpers.pm line 6 10. List::Util 1.38, Scalar/Util.pm line 11 11.XSLoader 0.17, List/Util.pm line 21 12. Params::Validate 1.10, DateTime/Duration.pm line 9 13.Module::Implementation 0.07, Params/Validate.pm line 9 14. Module::Runtime 0.014, Module/Implementation.pm line 12 23.Params::Validate::XS 1.10, Module/Runtime.pm line 317 43.Class::Load::XS 0.08, Module/Runtime.pm line 317 15. Try::Tiny 0.22, Module/Implementation.pm line 13 16.Sub::Name 0.05, Try/Tiny.pm line 18 (eval 4) 17. base 2.22, Sub/Name.pm line 49 18.vars 1.03, base.pm line 4 19.DynaLoader 1.25, base.pm line 99 20. Config 5.02, DynaLoader.pm line 22 52.DateTime::Locale::en, base.pm line 99 53.DateTime::Locale::root, base.pm line 99 21.constant 1.31, Try/Tiny.pm line 144 [Try::Tiny::ScopeGuard] 22.Params::Validate::Constants 1.10, Params/Validate.pm line 10 24. overload 1.22, DateTime/Duration.pm line 18 25.overloading 0.02, overload.pm line 83 26.DateTime::Locale 0.45, DateTime.pm line 12 27. DateTime::Locale::Base, DateTime/Locale.pm line 10 28.List::MoreUtils 0.33, DateTime/Locale/Base.pm line 8 29. DateTime::Locale::Catalog, DateTime/Locale.pm line 11 30.utf8 1.13, DateTime/Locale/Catalog.pm line 19 51. DateTime::Locale::en_US, DateTime/Locale.pm line 280 (eval 21) 31.DateTime::TimeZone 1.72, DateTime.pm line 13 32. DateTime::TimeZone::Catalog 1.72, DateTime/TimeZone.pm line 10 33. DateTime::TimeZone::Floating 1.72, DateTime/TimeZone.pm line 11 34.parent 0.228, DateTime/TimeZone/Floating.pm line 6 35. Class::Singleton 1.4, parent.pm line 20 36. DateTime::TimeZone::OffsetOnly 1.72, parent.pm line 20 37.DateTime::TimeZone::UTC 1.72, DateTime/TimeZone/OffsetOnly.pm line 8 38. DateTime::TimeZone::Local 1.72, DateTime/TimeZone.pm line 12 39.Class::Load 0.21, DateTime/TimeZone/Local.pm line 6 40. Data::OptList 0.109, Class/Load.pm line 10 41.Params::Util 1.07, Data/OptList.pm line 10 42.Sub::Install 0.927, Data/OptList.pm line 11 44.File::Spec 3.47, DateTime/TimeZone/Local.pm line 8 45. File::Spec::Unix 3.47, File/Spec.pm line 22 46.POSIX 1.38_03, DateTime.pm line 16 47. Fcntl 1.11, POSIX.pm line 17 48. Tie::Hash 1.05, POSIX.pm line 422 [POSIX::SigRt] 49.integer 1.01, DateTime.pm line 756 50.DateTime::Infinite 1.10, DateTime.pm line 68 Possible proxies: 3 base.pm line 99, sub base::import 2 Module/Runtime.pm line 317, sub Module::Runtime::require_module 2 parent.pm line 20, sub parent::import 2014-08-05 23:06 GMT+02:00 David E. Wheeler da...@justatheory.com: DateTimers, While profiling Sqitch, I found that the biggest suck on its time (now that I have removed Moose and Mouse in favor of Moo) is DateTime. The reason? It loads all of the locales in its BEGIN block. Run this to see for yourself: perl -d:NYTProf -e 'use DateTime;' nytprofhtml --open Here's the topline: Top 15 Subroutines CallsP FExclusive Time Inclusive TimeSubroutine 4661 115.6ms 24.4ms DateTime::Locale::_register 1 11 11.2ms 119ms main::BEGIN@1 4682 28.65ms 8.65ms Params::Validate::XS::validate (xsub) 1 11 8.29ms8.50ms DateTime::Locale::BEGIN@11 99 65.23ms 6.44ms base::import (recurses: max depth 2, inclusive time 1.79ms) 1 11 3.65ms 7.98ms DateTime::BEGIN@16 11 13.53ms 4.54msDateTime::TimeZone::Local:: BEGIN@8 1 11 3.22ms21.6ms DateTime::BEGIN@13 11 12.99ms 3.08ms DateTime::TimeZone::BEGIN@8 1 11 2.97ms3.19ms DateTime::BEGIN@9 11 12.91ms 4.26ms DateTime::Locale::BEGIN@10 1 11 2.74ms6.58ms DateTime::Locale:: add_aliases 2424 182.73ms 2.97ms Exporter::import 1 11 2.53ms2.87ms Try::Tiny::ScopeGuard::BEGIN@144 11 12.38ms 3.00ms Data::OptList::BEGIN@10 Does DateTime *really* need to always load all 466 locales on startup? Couldn't it load them on
Re: Must DateTime load all Locales?
On Aug 5, 2014, at 4:18 PM, Olivier Mengué olivier.men...@gmail.com wrote: Here is below the output (latest releases of DateTime, DateTime::Locale, DateTime::TimeZone on perl 5.20.0). DateTime::Locale::Catalog is the module that contains all the locales names. David, is it that module that you worry about or have you found something worse? I think the overhead comes in the DateTime::Locale::_register() function, which is called once for each locale returned by DateTime::Locale::Catalog. I would not think any locales would be loaded until they were needed. Best, David signature.asc Description: Message signed with OpenPGP using GPGMail
