Re: dayofyear is not great when going into a new year
Den 2021-01-09 skrev Michael F. Stemper : > > A week is like a piece of string. It has two ends. > The control line of the main sheet traveler on my boat is spliced into an endless loop. http://hem.bredband.net/b262106/pages/controls/index.html I am glad work weeks are not like that :-) /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On Sun, Jan 10, 2021 at 1:31 AM Michael F. Stemper wrote: > > On 09/01/2021 01.51, Christian Gollwitzer wrote: > > Am 05.01.21 um 23:56 schrieb Eli the Bearded: > >> Elijah > >> -- > >> also finds "week starts on Monday" to be oddball about ISO-8601 > >> > > > > In Europe, the week starts on Monday - hence, Saturday and Sunday are > > the last days of the week or the "weekend". Starting on Sunday is weird > > for us, because then the weekend is split into the first and last day of > > the week (?) > > A week is like a piece of string. It has two ends. > And a long weekend is like asking "how long is a piece of string"? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On 09/01/2021 01.51, Christian Gollwitzer wrote: Am 05.01.21 um 23:56 schrieb Eli the Bearded: Elijah -- also finds "week starts on Monday" to be oddball about ISO-8601 In Europe, the week starts on Monday - hence, Saturday and Sunday are the last days of the week or the "weekend". Starting on Sunday is weird for us, because then the weekend is split into the first and last day of the week (?) A week is like a piece of string. It has two ends. -- Michael F. Stemper Deuteronomy 24:17 -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
Am 05.01.21 um 23:56 schrieb Eli the Bearded: Elijah -- also finds "week starts on Monday" to be oddball about ISO-8601 In Europe, the week starts on Monday - hence, Saturday and Sunday are the last days of the week or the "weekend". Starting on Sunday is weird for us, because then the weekend is split into the first and last day of the week (?) - even if, historically, the week was the time from Sunday to Sunday. Christian -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On 9/01/21 11:17 am, Martin Schöön wrote: "regardless of what you have been told, recreational use of mathematics is harmless" I hope that is true for recreational programming as well :-) Mostly harmless, but it can be addictive! -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
Den 2021-01-05 skrev Stefan Ram : > Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?= writes: >>I have had some Python fun with COVID-19 data. I have done >>some curve fitting and to make that easier I have transformed >>date to day of year. Come end of 2020 and beginning of 2021 >>and this idea falls on its face. > > import datetime > > continuous_day_of_the_year = \ > ( datetime.date.today() - datetime.date( 2020, 1, 1 )).days > > Thanks guys, you got me on the right track. After some further meandering I did something close to what Stefan suggest above. I added a column to my Pandas data frame and populated it with content of date column - datetime(2020, 1, 1) "regardless of what you have been told, recreational use of mathematics is harmless" I hope that is true for recreational programming as well :-) /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On 1/5/21 8:02 PM, Mats Wichmann wrote:
> On 1/5/21 4:04 PM, Chris Angelico wrote:
>> On Wed, Jan 6, 2021 at 10:01 AM Eli the Bearded
>> <*@eli.users.panix.com> wrote:
>>>
>>> In comp.lang.python, Chris Angelico wrote:
There are multiple definitions for "day of year", depending on how you
want to handle certain oddities. The simplest is to identify Jan 1st
as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
libraries will define the year as starting with the week that contains
the Thursday, or something, and then will define days of year
accordingly.
>>>
>>> That sounds like some weird off-shoot of the ISO-8601 calendar. That
>>> document primarily concerns itself with weeks. Week 1 of a year is the
>>> first week with a Thursday in it. The last week of a year will be
>>> either
>>> 52 or 53, and you can have things like days in January belonging to the
>>> week of the previous year.
>>
>> The "weird off-shoot" part is probably a result of me misremembering
>> things, so don't read too much into the details :) I just remember
>> coming across something that numbered days within a year in a way that
>> was consistent with the way that it numbered weeks, which would indeed
>> have been based on the ISO 8601 week numbering. Not 100% sure of the
>> exact details.
>
> "workweeks" has always been fun, ISO standard or not, there's been a
> variation for ages since people don't seem to always follow ISO for
> that. I spent over a decade at a place that lived and died by their
> WorkWeek references ("due WW22" or the like would appear in every
> status report ever written, and there were zillions of those) - and it
> didn't agree with ISO on whether WW1 was the week that contained Jan 1
> or whether it was the week that followed the previous year's last
> workweek. After all, those few days can't actually belong to two
> different workweeks, now can they? :)
>
> (that was not a good memory you guys brought back :) )
I'm reminded of a video made describing 'ISO Weeks' that was put out on
Dec 30, 2019, which was the first day of the ISO week bsed 2020, and he
ended it with a comment that if 2019 wasn't going so well, you could
always just use ISO to get an early start on 2020.
Apparently a lot of people were getting this recommended a number of
months later, and people were commenting how that line just was not
aging well. And we couldn't use that te get out of 2020, as 2020 is a
long year, and ISO 2021 didn't start until Jan 4th.
--
Richard Damon
--
https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
In comp.lang.python, Mats Wichmann wrote:
> "workweeks" has always been fun, ISO standard or not, there's been a
> variation for ages since people don't seem to always follow ISO for
> that. I spent over a decade at a place that lived and died by their
> WorkWeek references ("due WW22" or the like would appear in every status
> report ever written, and there were zillions of those) - and it didn't
> agree with ISO on whether WW1 was the week that contained Jan 1 or
> whether it was the week that followed the previous year's last workweek.
> After all, those few days can't actually belong to two different
> workweeks, now can they? :)
I think the ISO standard was to try to unify a bunch of inconsistent
locally defined things like that. In Gnu date(1), there are THREE
different, and sometimes the same and sometimes not, week of year
codes:
%U week number of year, with Sunday as first day of week (00..53)
%V ISO week number, with Monday as first day of week (01..53)
%W week number of year, with Monday as first day of week (00..53)
I don't think that is an exhaustive list of methods used, either.
(excuse the vi command ugliness; % is special to : commands in vi)
:r! date +"U: \%U; V: \%V; W: \%W"
U: 01; V: 01; W: 01
Today they all match. But not always.
:r! date --date="Jan 02 2005" +"U: \%U; V: \%V; W: \%W"
U: 01; V: 53; W: 00
> (that was not a good memory you guys brought back :) )
Oh what a tangled web we weave, when we first begin to [measure time].
Elijah
--
they all disagree for Jan 02 2022, too, but slightly differently
--
https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On 1/5/21 4:04 PM, Chris Angelico wrote:
On Wed, Jan 6, 2021 at 10:01 AM Eli the Bearded <*@eli.users.panix.com> wrote:
In comp.lang.python, Chris Angelico wrote:
There are multiple definitions for "day of year", depending on how you
want to handle certain oddities. The simplest is to identify Jan 1st
as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
libraries will define the year as starting with the week that contains
the Thursday, or something, and then will define days of year
accordingly.
That sounds like some weird off-shoot of the ISO-8601 calendar. That
document primarily concerns itself with weeks. Week 1 of a year is the
first week with a Thursday in it. The last week of a year will be either
52 or 53, and you can have things like days in January belonging to the
week of the previous year.
The "weird off-shoot" part is probably a result of me misremembering
things, so don't read too much into the details :) I just remember
coming across something that numbered days within a year in a way that
was consistent with the way that it numbered weeks, which would indeed
have been based on the ISO 8601 week numbering. Not 100% sure of the
exact details.
"workweeks" has always been fun, ISO standard or not, there's been a
variation for ages since people don't seem to always follow ISO for
that. I spent over a decade at a place that lived and died by their
WorkWeek references ("due WW22" or the like would appear in every status
report ever written, and there were zillions of those) - and it didn't
agree with ISO on whether WW1 was the week that contained Jan 1 or
whether it was the week that followed the previous year's last workweek.
After all, those few days can't actually belong to two different
workweeks, now can they? :)
(that was not a good memory you guys brought back :) )
--
https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On Wed, Jan 6, 2021 at 10:01 AM Eli the Bearded <*@eli.users.panix.com> wrote: > > In comp.lang.python, Chris Angelico wrote: > > There are multiple definitions for "day of year", depending on how you > > want to handle certain oddities. The simplest is to identify Jan 1st > > as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some > > libraries will define the year as starting with the week that contains > > the Thursday, or something, and then will define days of year > > accordingly. > > That sounds like some weird off-shoot of the ISO-8601 calendar. That > document primarily concerns itself with weeks. Week 1 of a year is the > first week with a Thursday in it. The last week of a year will be either > 52 or 53, and you can have things like days in January belonging to the > week of the previous year. The "weird off-shoot" part is probably a result of me misremembering things, so don't read too much into the details :) I just remember coming across something that numbered days within a year in a way that was consistent with the way that it numbered weeks, which would indeed have been based on the ISO 8601 week numbering. Not 100% sure of the exact details. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
In comp.lang.python, Chris Angelico wrote: > There are multiple definitions for "day of year", depending on how you > want to handle certain oddities. The simplest is to identify Jan 1st > as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some > libraries will define the year as starting with the week that contains > the Thursday, or something, and then will define days of year > accordingly. That sounds like some weird off-shoot of the ISO-8601 calendar. That document primarily concerns itself with weeks. Week 1 of a year is the first week with a Thursday in it. The last week of a year will be either 52 or 53, and you can have things like days in January belonging to the week of the previous year. Wikipedia gives examples: https://en.wikipedia.org/wiki/ISO_week_date If you are operating on that, then it might indeed make sense to number the days from -W1-1. I can't say I've ever encountered that. Since W1-1 is always a Monday on the ISO calendar, it would have the neat property that you could always turn day of year into day of week with a mod operation. It would have the quirk that years are either 364 or 371 days, neither of which most people would answer when asked "How many days are there in a year?" I've only used ISO dates for by-week graphs, because they have the nice property of "all weeks are seven days", so you don't get oddball weeks screwing up your plots. > If you want an easy way to graph day-by-day data and the exact day > numbers are irrelevant, what I'd recommend is: Convert the date into > Unix time, divide by 86400, floor it. That'll give you a Julian-style > date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end > of a year, it'll just keep on incrementing. That would get you past > the 2020/2021 boundary pretty smoothly. That works well. The previous suggestion using January 1st 2020 as an epoch start is also good. Elijah -- also finds "week starts on Monday" to be oddball about ISO-8601 -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On Wed, Jan 6, 2021 at 9:51 AM Michael F. Stemper wrote: > > On 05/01/2021 15.27, Chris Angelico wrote: > > On Wed, Jan 6, 2021 at 8:02 AM Martin Schöön > > wrote: > > >> I have had some Python fun with COVID-19 data. I have done > >> some curve fitting and to make that easier I have transformed > >> date to day of year. Come end of 2020 and beginning of 2021 > >> and this idea falls on its face. > > > There are multiple definitions for "day of year", depending on how you > > want to handle certain oddities. The simplest is to identify Jan 1st > > as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some > > libraries will define the year as starting with the week that contains > > the Thursday, or something, and then will define days of year > > accordingly. > > > > If you want an easy way to graph day-by-day data and the exact day > > numbers are irrelevant, what I'd recommend is: Convert the date into > > Unix time, divide by 86400, floor it. That'll give you a Julian-style > > date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end > > of a year, it'll just keep on incrementing. That would get you past > > the 2020/2021 boundary pretty smoothly. > > Possibly better than rolling your own is using standard stuff: > > >>> from datetime import date,datetime > >>> datetime.toordinal(date(2020,1,1)) > 737425 > >>> datetime.toordinal(date(2021,1,5)) > 737795 > >>> datetime.toordinal(date(2021,1,5)) - datetime.toordinal(date(2020,1,1)) > 370 > >>> > Oh yes. In fact, not "possibly", but "definitely" :) ALWAYS use a library for date manipulation. But you have to know what you're doing before you can figure out how to call on the library. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On 05/01/2021 15.27, Chris Angelico wrote: On Wed, Jan 6, 2021 at 8:02 AM Martin Schöön wrote: I have had some Python fun with COVID-19 data. I have done some curve fitting and to make that easier I have transformed date to day of year. Come end of 2020 and beginning of 2021 and this idea falls on its face. There are multiple definitions for "day of year", depending on how you want to handle certain oddities. The simplest is to identify Jan 1st as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some libraries will define the year as starting with the week that contains the Thursday, or something, and then will define days of year accordingly. If you want an easy way to graph day-by-day data and the exact day numbers are irrelevant, what I'd recommend is: Convert the date into Unix time, divide by 86400, floor it. That'll give you a Julian-style date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end of a year, it'll just keep on incrementing. That would get you past the 2020/2021 boundary pretty smoothly. Possibly better than rolling your own is using standard stuff: >>> from datetime import date,datetime >>> datetime.toordinal(date(2020,1,1)) 737425 >>> datetime.toordinal(date(2021,1,5)) 737795 >>> datetime.toordinal(date(2021,1,5)) - datetime.toordinal(date(2020,1,1)) 370 >>> -- Michael F. Stemper This post contains greater than 95% post-consumer bytes by weight. -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On 1/6/21 9:55 AM, Martin Schöön wrote: > Hello, > > I have had some Python fun with COVID-19 data. I have done > some curve fitting and to make that easier I have transformed > date to day of year. Come end of 2020 and beginning of 2021 > and this idea falls on its face. > > There must be a better way of doing this. > > I am using Pandas for reading and manipulating data coming > from a csv file. Scipy, numpy and matplotlib are used > for the curve fitting and plotting. An important question is how the day/date is presented in these graphs. The Julian date format used to be more common in Data Processing. Dates are expressed as yy-ddd. Thus, prepending "20" or "21" to existing day-of-year numbers, would separate and sequence correctly. (NB carefully consider str cf int) Another solution is to use a Unix Timestamp for each day. In both cases, arithmetic comparison-logic is possible, and library routines exist to format various reporting-formats. -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: dayofyear is not great when going into a new year
On Wed, Jan 6, 2021 at 8:02 AM Martin Schöön wrote: > > Hello, > > I have had some Python fun with COVID-19 data. I have done > some curve fitting and to make that easier I have transformed > date to day of year. Come end of 2020 and beginning of 2021 > and this idea falls on its face. > There are multiple definitions for "day of year", depending on how you want to handle certain oddities. The simplest is to identify Jan 1st as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some libraries will define the year as starting with the week that contains the Thursday, or something, and then will define days of year accordingly. If you want an easy way to graph day-by-day data and the exact day numbers are irrelevant, what I'd recommend is: Convert the date into Unix time, divide by 86400, floor it. That'll give you a Julian-style date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end of a year, it'll just keep on incrementing. That would get you past the 2020/2021 boundary pretty smoothly. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
dayofyear is not great when going into a new year
Hello, I have had some Python fun with COVID-19 data. I have done some curve fitting and to make that easier I have transformed date to day of year. Come end of 2020 and beginning of 2021 and this idea falls on its face. There must be a better way of doing this. I am using Pandas for reading and manipulating data coming from a csv file. Scipy, numpy and matplotlib are used for the curve fitting and plotting. TIA, /Martin -- https://mail.python.org/mailman/listinfo/python-list
