Re: dayofyear is not great when going into a new year

2021-01-10 Thread Martin Schöön
Den 2021-01-09 skrev Michael F. Stemper :
>
> A week is like a piece of string. It has two ends.
>
The control line of the main sheet traveler on my boat is spliced into
an endless loop.
http://hem.bredband.net/b262106/pages/controls/index.html

I am glad work weeks are not like that :-)

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-09 Thread Chris Angelico
On Sun, Jan 10, 2021 at 1:31 AM Michael F. Stemper  wrote:
>
> On 09/01/2021 01.51, Christian Gollwitzer wrote:
> > Am 05.01.21 um 23:56 schrieb Eli the Bearded:
> >> Elijah
> >> --
> >> also finds "week starts on Monday" to be oddball about ISO-8601
> >>
> >
> > In Europe, the week starts on Monday - hence, Saturday and Sunday are
> > the last days of the week or the "weekend". Starting on Sunday is weird
> > for us, because then the weekend is split into the first and last day of
> > the week (?)
>
> A week is like a piece of string. It has two ends.
>

And a long weekend is like asking "how long is a piece of string"?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-09 Thread Michael F. Stemper

On 09/01/2021 01.51, Christian Gollwitzer wrote:

Am 05.01.21 um 23:56 schrieb Eli the Bearded:

Elijah
--
also finds "week starts on Monday" to be oddball about ISO-8601



In Europe, the week starts on Monday - hence, Saturday and Sunday are 
the last days of the week or the "weekend". Starting on Sunday is weird 
for us, because then the weekend is split into the first and last day of 
the week (?)


A week is like a piece of string. It has two ends.


--
Michael F. Stemper
Deuteronomy 24:17
--
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-08 Thread Christian Gollwitzer

Am 05.01.21 um 23:56 schrieb Eli the Bearded:

Elijah
--
also finds "week starts on Monday" to be oddball about ISO-8601



In Europe, the week starts on Monday - hence, Saturday and Sunday are 
the last days of the week or the "weekend". Starting on Sunday is weird 
for us, because then the weekend is split into the first and last day of 
the week (?) - even if, historically, the week was the time from Sunday 
to Sunday.



Christian
--
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-08 Thread Greg Ewing

On 9/01/21 11:17 am, Martin Schöön wrote:

"regardless of what you have been told, recreational use of
mathematics is harmless"

I hope that is true for recreational programming as well :-)


Mostly harmless, but it can be addictive!

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-08 Thread Martin Schöön
Den 2021-01-05 skrev Stefan Ram :
> Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=  writes:
>>I have had some Python fun with COVID-19 data. I have done
>>some curve fitting and to make that easier I have transformed
>>date to day of year. Come end of 2020 and beginning of 2021
>>and this idea falls on its face.
>
> import datetime
>
> continuous_day_of_the_year = \
> ( datetime.date.today() - datetime.date( 2020, 1, 1 )).days
>
>
Thanks guys, you got me on the right track. After some further
meandering I did something close to what Stefan suggest above.
I added a column to my Pandas data frame and populated it with

content of date column - datetime(2020, 1, 1)

"regardless of what you have been told, recreational use of
mathematics is harmless"

I hope that is true for recreational programming as well :-)

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Richard Damon
On 1/5/21 8:02 PM, Mats Wichmann wrote:
> On 1/5/21 4:04 PM, Chris Angelico wrote:
>> On Wed, Jan 6, 2021 at 10:01 AM Eli the Bearded
>> <*@eli.users.panix.com> wrote:
>>>
>>> In comp.lang.python, Chris Angelico   wrote:
 There are multiple definitions for "day of year", depending on how you
 want to handle certain oddities. The simplest is to identify Jan 1st
 as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
 libraries will define the year as starting with the week that contains
 the Thursday, or something, and then will define days of year
 accordingly.
>>>
>>> That sounds like some weird off-shoot of the ISO-8601 calendar. That
>>> document primarily concerns itself with weeks. Week 1 of a year is the
>>> first week with a Thursday in it. The last week of a year will be
>>> either
>>> 52 or 53, and you can have things like days in January belonging to the
>>> week of the previous year.
>>
>> The "weird off-shoot" part is probably a result of me misremembering
>> things, so don't read too much into the details :) I just remember
>> coming across something that numbered days within a year in a way that
>> was consistent with the way that it numbered weeks, which would indeed
>> have been based on the ISO 8601 week numbering. Not 100% sure of the
>> exact details.
>
> "workweeks" has always been fun, ISO standard or not, there's been a
> variation for ages since people don't seem to always follow ISO for
> that.  I spent over a decade at a place that lived and died by their
> WorkWeek references ("due WW22" or the like would appear in every
> status report ever written, and there were zillions of those) - and it
> didn't agree with ISO on whether WW1 was the week that contained Jan 1
> or whether it was the week that followed the previous year's last
> workweek. After all, those few days can't actually belong to two
> different workweeks, now can they?  :)
>
> (that was not a good memory you guys brought back :) )

I'm reminded of a video made describing 'ISO Weeks' that was put out on
Dec 30, 2019, which was the first day of the ISO week bsed 2020, and he
ended it with a comment that if 2019 wasn't going so well, you could
always just use ISO to get an early start on 2020.

Apparently a lot of people were getting this recommended a number of
months later, and people were commenting how that line just was not
aging well. And we couldn't use that te get out of 2020, as 2020 is a
long year, and ISO 2021 didn't start until Jan 4th.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Eli the Bearded
In comp.lang.python, Mats Wichmann   wrote:
> "workweeks" has always been fun, ISO standard or not, there's been a 
> variation for ages since people don't seem to always follow ISO for 
> that.  I spent over a decade at a place that lived and died by their 
> WorkWeek references ("due WW22" or the like would appear in every status 
> report ever written, and there were zillions of those) - and it didn't 
> agree with ISO on whether WW1 was the week that contained Jan 1 or 
> whether it was the week that followed the previous year's last workweek. 
> After all, those few days can't actually belong to two different 
> workweeks, now can they?  :)

I think the ISO standard was to try to unify a bunch of inconsistent
locally defined things like that. In Gnu date(1), there are THREE
different, and sometimes the same and sometimes not, week of year
codes:

   %U week number of year, with Sunday as first day of week (00..53)

   %V ISO week number, with Monday as first day of week (01..53)

   %W week number of year, with Monday as first day of week (00..53)

I don't think that is an exhaustive list of methods used, either.

(excuse the vi command ugliness; % is special to : commands in vi)

:r! date +"U: \%U; V: \%V; W: \%W"
U: 01; V: 01; W: 01

Today they all match. But not always.

:r! date --date="Jan 02 2005" +"U: \%U; V: \%V; W: \%W"
U: 01; V: 53; W: 00

> (that was not a good memory you guys brought back :) )

Oh what a tangled web we weave, when we first begin to [measure time].

Elijah
--
they all disagree for Jan 02 2022, too, but slightly differently
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Mats Wichmann

On 1/5/21 4:04 PM, Chris Angelico wrote:

On Wed, Jan 6, 2021 at 10:01 AM Eli the Bearded <*@eli.users.panix.com> wrote:


In comp.lang.python, Chris Angelico   wrote:

There are multiple definitions for "day of year", depending on how you
want to handle certain oddities. The simplest is to identify Jan 1st
as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
libraries will define the year as starting with the week that contains
the Thursday, or something, and then will define days of year
accordingly.


That sounds like some weird off-shoot of the ISO-8601 calendar. That
document primarily concerns itself with weeks. Week 1 of a year is the
first week with a Thursday in it. The last week of a year will be either
52 or 53, and you can have things like days in January belonging to the
week of the previous year.


The "weird off-shoot" part is probably a result of me misremembering
things, so don't read too much into the details :) I just remember
coming across something that numbered days within a year in a way that
was consistent with the way that it numbered weeks, which would indeed
have been based on the ISO 8601 week numbering. Not 100% sure of the
exact details.


"workweeks" has always been fun, ISO standard or not, there's been a 
variation for ages since people don't seem to always follow ISO for 
that.  I spent over a decade at a place that lived and died by their 
WorkWeek references ("due WW22" or the like would appear in every status 
report ever written, and there were zillions of those) - and it didn't 
agree with ISO on whether WW1 was the week that contained Jan 1 or 
whether it was the week that followed the previous year's last workweek. 
After all, those few days can't actually belong to two different 
workweeks, now can they?  :)


(that was not a good memory you guys brought back :) )
--
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Chris Angelico
On Wed, Jan 6, 2021 at 10:01 AM Eli the Bearded <*@eli.users.panix.com> wrote:
>
> In comp.lang.python, Chris Angelico   wrote:
> > There are multiple definitions for "day of year", depending on how you
> > want to handle certain oddities. The simplest is to identify Jan 1st
> > as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
> > libraries will define the year as starting with the week that contains
> > the Thursday, or something, and then will define days of year
> > accordingly.
>
> That sounds like some weird off-shoot of the ISO-8601 calendar. That
> document primarily concerns itself with weeks. Week 1 of a year is the
> first week with a Thursday in it. The last week of a year will be either
> 52 or 53, and you can have things like days in January belonging to the
> week of the previous year.

The "weird off-shoot" part is probably a result of me misremembering
things, so don't read too much into the details :) I just remember
coming across something that numbered days within a year in a way that
was consistent with the way that it numbered weeks, which would indeed
have been based on the ISO 8601 week numbering. Not 100% sure of the
exact details.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Eli the Bearded
In comp.lang.python, Chris Angelico   wrote:
> There are multiple definitions for "day of year", depending on how you
> want to handle certain oddities. The simplest is to identify Jan 1st
> as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
> libraries will define the year as starting with the week that contains
> the Thursday, or something, and then will define days of year
> accordingly.

That sounds like some weird off-shoot of the ISO-8601 calendar. That
document primarily concerns itself with weeks. Week 1 of a year is the
first week with a Thursday in it. The last week of a year will be either
52 or 53, and you can have things like days in January belonging to the
week of the previous year. Wikipedia gives examples:

https://en.wikipedia.org/wiki/ISO_week_date

If you are operating on that, then it might indeed make sense to number
the days from -W1-1. I can't say I've ever encountered that. Since
W1-1 is always a Monday on the ISO calendar, it would have the neat
property that you could always turn day of year into day of week with a
mod operation. It would have the quirk that years are either 364 or 371
days, neither of which most people would answer when asked "How many
days are there in a year?"

I've only used ISO dates for by-week graphs, because they have the nice
property of "all weeks are seven days", so you don't get oddball weeks
screwing up your plots.

> If you want an easy way to graph day-by-day data and the exact day
> numbers are irrelevant, what I'd recommend is: Convert the date into
> Unix time, divide by 86400, floor it. That'll give you a Julian-style
> date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end
> of a year, it'll just keep on incrementing. That would get you past
> the 2020/2021 boundary pretty smoothly.

That works well. The previous suggestion using January 1st 2020 as an
epoch start is also good.

Elijah
--
also finds "week starts on Monday" to be oddball about ISO-8601
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Chris Angelico
On Wed, Jan 6, 2021 at 9:51 AM Michael F. Stemper  wrote:
>
> On 05/01/2021 15.27, Chris Angelico wrote:
> > On Wed, Jan 6, 2021 at 8:02 AM Martin Schöön  
> > wrote:
>
> >> I have had some Python fun with COVID-19 data. I have done
> >> some curve fitting and to make that easier I have transformed
> >> date to day of year. Come end of 2020 and beginning of 2021
> >> and this idea falls on its face.
>
> > There are multiple definitions for "day of year", depending on how you
> > want to handle certain oddities. The simplest is to identify Jan 1st
> > as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
> > libraries will define the year as starting with the week that contains
> > the Thursday, or something, and then will define days of year
> > accordingly.
> >
> > If you want an easy way to graph day-by-day data and the exact day
> > numbers are irrelevant, what I'd recommend is: Convert the date into
> > Unix time, divide by 86400, floor it. That'll give you a Julian-style
> > date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end
> > of a year, it'll just keep on incrementing. That would get you past
> > the 2020/2021 boundary pretty smoothly.
>
> Possibly better than rolling your own is using standard stuff:
>
>  >>> from datetime import date,datetime
>  >>> datetime.toordinal(date(2020,1,1))
> 737425
>  >>> datetime.toordinal(date(2021,1,5))
> 737795
>  >>> datetime.toordinal(date(2021,1,5)) - datetime.toordinal(date(2020,1,1))
> 370
>  >>>
>

Oh yes. In fact, not "possibly", but "definitely" :) ALWAYS use a
library for date manipulation. But you have to know what you're doing
before you can figure out how to call on the library.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Michael F. Stemper

On 05/01/2021 15.27, Chris Angelico wrote:

On Wed, Jan 6, 2021 at 8:02 AM Martin Schöön  wrote:



I have had some Python fun with COVID-19 data. I have done
some curve fitting and to make that easier I have transformed
date to day of year. Come end of 2020 and beginning of 2021
and this idea falls on its face.



There are multiple definitions for "day of year", depending on how you
want to handle certain oddities. The simplest is to identify Jan 1st
as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
libraries will define the year as starting with the week that contains
the Thursday, or something, and then will define days of year
accordingly.

If you want an easy way to graph day-by-day data and the exact day
numbers are irrelevant, what I'd recommend is: Convert the date into
Unix time, divide by 86400, floor it. That'll give you a Julian-style
date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end
of a year, it'll just keep on incrementing. That would get you past
the 2020/2021 boundary pretty smoothly.


Possibly better than rolling your own is using standard stuff:

>>> from datetime import date,datetime
>>> datetime.toordinal(date(2020,1,1))
737425
>>> datetime.toordinal(date(2021,1,5))
737795
>>> datetime.toordinal(date(2021,1,5)) - datetime.toordinal(date(2020,1,1))
370
>>>

--
Michael F. Stemper
This post contains greater than 95% post-consumer bytes by weight.
--
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread dn via Python-list
On 1/6/21 9:55 AM, Martin Schöön wrote:
> Hello,
> 
> I have had some Python fun with COVID-19 data. I have done
> some curve fitting and to make that easier I have transformed
> date to day of year. Come end of 2020 and beginning of 2021
> and this idea falls on its face.
> 
> There must be a better way of doing this.
> 
> I am using Pandas for reading and manipulating data coming
> from a csv file. Scipy, numpy and matplotlib are used
> for the curve fitting and plotting.


An important question is how the day/date is presented in these graphs.

The Julian date format used to be more common in Data Processing. Dates
are expressed as yy-ddd. Thus, prepending "20" or "21" to existing
day-of-year numbers, would separate and sequence correctly.
(NB carefully consider str cf int)

Another solution is to use a Unix Timestamp for each day.

In both cases, arithmetic comparison-logic is possible, and library
routines exist to format various reporting-formats.
-- 
Regards =dn
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: dayofyear is not great when going into a new year

2021-01-05 Thread Chris Angelico
On Wed, Jan 6, 2021 at 8:02 AM Martin Schöön  wrote:
>
> Hello,
>
> I have had some Python fun with COVID-19 data. I have done
> some curve fitting and to make that easier I have transformed
> date to day of year. Come end of 2020 and beginning of 2021
> and this idea falls on its face.
>

There are multiple definitions for "day of year", depending on how you
want to handle certain oddities. The simplest is to identify Jan 1st
as 1, Jan 2nd as 2, etc, to Dec 31st as either 365 or 366; but some
libraries will define the year as starting with the week that contains
the Thursday, or something, and then will define days of year
accordingly.

If you want an easy way to graph day-by-day data and the exact day
numbers are irrelevant, what I'd recommend is: Convert the date into
Unix time, divide by 86400, floor it. That'll give you a Julian-style
date number where Jan 1st 1970 is 0, Jan 2nd is 1, etc, and at the end
of a year, it'll just keep on incrementing. That would get you past
the 2020/2021 boundary pretty smoothly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


dayofyear is not great when going into a new year

2021-01-05 Thread Martin Schöön
Hello,

I have had some Python fun with COVID-19 data. I have done
some curve fitting and to make that easier I have transformed
date to day of year. Come end of 2020 and beginning of 2021
and this idea falls on its face.

There must be a better way of doing this.

I am using Pandas for reading and manipulating data coming
from a csv file. Scipy, numpy and matplotlib are used
for the curve fitting and plotting.

TIA,

/Martin
-- 
https://mail.python.org/mailman/listinfo/python-list