[
https://issues.apache.org/jira/browse/SPARK-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369006#comment-15369006
]
Dilip Biswal commented on SPARK-16423:
--------------------------------------
[~yhuai] [~smilegator]
Just wanted to quickly share the information i have found so far.
- WeekOfYear
- Checked the mysql, hive behaviour. Spark seems to be consistent with the
mysql and hive. They both assume the first day of week to be Monday and
more than 3 days per week.
http://www.techonthenet.com/mysql/functions/weekofyear.php
- Postgres - week
The number of the ISO 8601 week-numbering week of the year. By definition,
ISO weeks start on Mondays and the first week of a year contains January 4
of that year.
In other words, the first Thursday of a year is in week 1 of that year.
https://www.postgresql.org/docs/current/static/functions-datetime.html
(function : week)
- SQL Server
They have a way to set a registry variable to influence the first day of
the week.
SET DATEFIRST { number | @number_var }
When this is set, the DATEPART function considers the setting while
calculating
day of week. When this is not set, they also seem to follow ISO which again
assumes
Monday to be start of the week.
https://msdn.microsoft.com/en-us/library/ms186724.aspx
- Oracle
In case of oracle, the day of the week is controlled by session specific
NLS_TERRITORY
setting.
https://community.oracle.com/thread/2207756?tstart=0
- DB2
Have two flavors of WEEK function. One for ISO (Monday start) and other one
for non ISO
(Sunday start).
http://www.ibm.com/developerworks/data/library/techarticle/0211yip/0211yip3.html
Given this, it seems like more systems follow Monday to be first day of week
semantics and
i am wondering if we should change this ?
Also, is there a co-relation between fromUnixTime and WeekOfYear. fromUnixTime
returns
the user supplied time in seconds in string after applying the date format. In
my understanding
it respects the system locale settings.
> Inconsistent settings on the first day of a week
> ------------------------------------------------
>
> Key: SPARK-16423
> URL: https://issues.apache.org/jira/browse/SPARK-16423
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Yin Huai
>
> For the function {{WeekOfYear}}, we explicitly set the first day of the week
> to {{Calendar.MONDAY}}. However, {{FromUnixTime}} does not explicitly set it.
> So, we are using the default first day of the week based on the locale
> setting (see
> https://docs.oracle.com/javase/8/docs/api/java/util/Calendar.html#setFirstDayOfWeek-int-).
>
> Let's do a survey on what other databases do and make the setting consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
