[jira] [Commented] (SPARK-28471) Formatting dates with negative years

2019-08-04 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899633#comment-16899633 ] Maxim Gekk commented on SPARK-28471: [~yumwang] There are 2 equal methods to output negative years

[jira] [Commented] (SPARK-28017) Enhance EXTRACT/DATE_TRUNC

2019-08-02 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898780#comment-16898780 ] Maxim Gekk commented on SPARK-28017: I am working on DATE_TRUNC now. > Enhance EXTRACT/DATE_TRUNC >

[jira] [Created] (SPARK-28596) Use Java 8 time API in date_trunc

2019-08-01 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-28596: -- Summary: Use Java 8 time API in date_trunc Key: SPARK-28596 URL: https://issues.apache.org/jira/browse/SPARK-28596 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-28471) Formatting dates with negative years

2019-07-22 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889981#comment-16889981 ] Maxim Gekk commented on SPARK-28471: Here is my explanation of difference in years between Spark's

[jira] [Created] (SPARK-28471) Formatting dates with negative years

2019-07-22 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-28471: -- Summary: Formatting dates with negative years Key: SPARK-28471 URL: https://issues.apache.org/jira/browse/SPARK-28471 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-28459) Date/Time Functions: make_timestamp

2019-07-20 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889559#comment-16889559 ] Maxim Gekk commented on SPARK-28459: [~shivuson...@gmail.com] I have already had a draft changes for

[jira] [Commented] (SPARK-28432) Date/Time Functions: make_date/make_timestamp

2019-07-18 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888324#comment-16888324 ] Maxim Gekk commented on SPARK-28432: I am working on this > Date/Time Functions:

[jira] [Commented] (SPARK-28427) Support more Postgres JSON functions

2019-07-18 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888258#comment-16888258 ] Maxim Gekk commented on SPARK-28427: Probably, we can switch the flag spark.sql.legacy.sizeOfNull,

[jira] [Commented] (SPARK-28382) Array Functions: unnest

2019-07-18 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888248#comment-16888248 ] Maxim Gekk commented on SPARK-28382: Is it just explode()? > Array Functions: unnest >

[jira] [Created] (SPARK-28416) Use java.time API in timestampAddInterval

2019-07-16 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-28416: -- Summary: Use java.time API in timestampAddInterval Key: SPARK-28416 URL: https://issues.apache.org/jira/browse/SPARK-28416 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28408) Restrict tested values of DateType, TimestampType and CalendarIntervalType

2019-07-15 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-28408: -- Summary: Restrict tested values of DateType, TimestampType and CalendarIntervalType Key: SPARK-28408 URL: https://issues.apache.org/jira/browse/SPARK-28408 Project:

[jira] [Created] (SPARK-28389) Use Java 8 API to add months to dates

2019-07-14 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-28389: -- Summary: Use Java 8 API to add months to dates Key: SPARK-28389 URL: https://issues.apache.org/jira/browse/SPARK-28389 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27609) from_json expects values of options dictionary to be

2019-07-11 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882885#comment-16882885 ] Maxim Gekk commented on SPARK-27609: [~zmjjmz] Please, let us know if you are working on this. If

[jira] [Commented] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't have quotes in data

2019-07-11 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882879#comment-16882879 ] Maxim Gekk commented on SPARK-28338: This is the expected behavior, actually. Missed column value or

[jira] [Commented] (SPARK-25169) Multiple DataFrames cannot write to the same folder concurrently

2019-07-10 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882058#comment-16882058 ] Maxim Gekk commented on SPARK-25169: If you need to write to the same folder concurrently, you can

[jira] [Commented] (SPARK-28015) Invalid date formats should throw an exception

2019-07-10 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881884#comment-16881884 ] Maxim Gekk commented on SPARK-28015: I don't think the stringToDate() function should throw an

[jira] [Updated] (SPARK-27790) Support SQL INTERVAL types

2019-05-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27790: --- Description: SQL standard defines 2 interval types: # year-month interval contains a YEAR field or

[jira] [Updated] (SPARK-27790) Support SQL INTERVAL types

2019-05-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27790: --- Description: SQL standard defines 2 interval types: # year-month interval contains a YEAR field or

[jira] [Commented] (SPARK-27790) Support SQL INTERVAL types

2019-05-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844760#comment-16844760 ] Maxim Gekk commented on SPARK-27790: I am working on it. > Support SQL INTERVAL types >

[jira] [Created] (SPARK-27793) Support SQL day-time INTERVAL type

2019-05-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27793: -- Summary: Support SQL day-time INTERVAL type Key: SPARK-27793 URL: https://issues.apache.org/jira/browse/SPARK-27793 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-27791) Support SQL year-month INTERVAL type

2019-05-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27791: --- Description: The INTERVAL type must conform to SQL year-month INTERVAL type, has 2 logical types:

[jira] [Created] (SPARK-27791) Support SQL year-month INTERVAL type

2019-05-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27791: -- Summary: Support SQL year-month INTERVAL type Key: SPARK-27791 URL: https://issues.apache.org/jira/browse/SPARK-27791 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-27790) Support SQL INTERVAL types

2019-05-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27790: -- Summary: Support SQL INTERVAL types Key: SPARK-27790 URL: https://issues.apache.org/jira/browse/SPARK-27790 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-27450) Timestamp cast fails when the ISO8601 string omits minutes, seconds or milliseconds

2019-05-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16842311#comment-16842311 ] Maxim Gekk commented on SPARK-27450: > IMHO the proposed solution does not address a datetime value

[jira] [Commented] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834214#comment-16834214 ] Maxim Gekk commented on SPARK-27638: [~pengbo] Are you going to propose a PR for that? If not, I can

[jira] [Comment Edited] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833957#comment-16833957 ] Maxim Gekk edited comment on SPARK-27638 at 5/6/19 4:10 PM: It works with

[jira] [Commented] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833957#comment-16833957 ] Maxim Gekk commented on SPARK-27638: It works with explicit to_date: {code:scala} scala> val ds =

[jira] [Comment Edited] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833949#comment-16833949 ] Maxim Gekk edited comment on SPARK-27638 at 5/6/19 3:57 PM: [~srowen] The

[jira] [Commented] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833949#comment-16833949 ] Maxim Gekk commented on SPARK-27638: [~srowen] The date literal should be casted to the date type by 

[jira] [Commented] (SPARK-27450) Timestamp cast fails when the ISO8601 string omits minutes, seconds or milliseconds

2019-05-04 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833132#comment-16833132 ] Maxim Gekk commented on SPARK-27450: The cast functions supports limited number of timestamp

[jira] [Created] (SPARK-27535) Date and timestamp JSON benchmarks

2019-04-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27535: -- Summary: Date and timestamp JSON benchmarks Key: SPARK-27535 URL: https://issues.apache.org/jira/browse/SPARK-27535 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-27533) Date and timestamp CSV benchmarks

2019-04-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27533: --- Summary: Date and timestamp CSV benchmarks (was: Date/timestamps CSV benchmarks) > Date and

[jira] [Created] (SPARK-27533) CSV benchmarks date/timestamp ops

2019-04-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27533: -- Summary: CSV benchmarks date/timestamp ops Key: SPARK-27533 URL: https://issues.apache.org/jira/browse/SPARK-27533 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-27533) Date/timestamps CSV benchmarks

2019-04-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27533: --- Summary: Date/timestamps CSV benchmarks (was: CSV benchmarks date/timestamp ops ) >

[jira] [Created] (SPARK-27528) Use Parquet logical type TIMESTAMP_MICROS by default

2019-04-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27528: -- Summary: Use Parquet logical type TIMESTAMP_MICROS by default Key: SPARK-27528 URL: https://issues.apache.org/jira/browse/SPARK-27528 Project: Spark Issue Type:

[jira] [Created] (SPARK-27527) Improve description of Timestamp and Date types

2019-04-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27527: -- Summary: Improve description of Timestamp and Date types Key: SPARK-27527 URL: https://issues.apache.org/jira/browse/SPARK-27527 Project: Spark Issue Type:

[jira] [Created] (SPARK-27522) Test migration from INT96 to TIMESTAMP_MICROS in parquet

2019-04-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27522: -- Summary: Test migration from INT96 to TIMESTAMP_MICROS in parquet Key: SPARK-27522 URL: https://issues.apache.org/jira/browse/SPARK-27522 Project: Spark Issue

[jira] [Created] (SPARK-27438) Increase precision of to_timestamp

2019-04-10 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27438: -- Summary: Increase precision of to_timestamp Key: SPARK-27438 URL: https://issues.apache.org/jira/browse/SPARK-27438 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27423) Cast DATE to/from TIMESTAMP according to SQL standard

2019-04-09 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27423: -- Summary: Cast DATE to/from TIMESTAMP according to SQL standard Key: SPARK-27423 URL: https://issues.apache.org/jira/browse/SPARK-27423 Project: Spark Issue

[jira] [Resolved] (SPARK-27357) Cast timestamps to/from dates independently from time zones

2019-04-09 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-27357. Resolution: Not A Problem > Cast timestamps to/from dates independently from time zones >

[jira] [Created] (SPARK-27422) CurrentDate should return local date

2019-04-09 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27422: -- Summary: CurrentDate should return local date Key: SPARK-27422 URL: https://issues.apache.org/jira/browse/SPARK-27422 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27405) Restrict the range of generated random timestamps

2019-04-07 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27405: -- Summary: Restrict the range of generated random timestamps Key: SPARK-27405 URL: https://issues.apache.org/jira/browse/SPARK-27405 Project: Spark Issue Type:

[jira] [Created] (SPARK-27401) Refactoring conversion of Date/Timestamp to/from java.sql.Date/Timestamp

2019-04-06 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27401: -- Summary: Refactoring conversion of Date/Timestamp to/from java.sql.Date/Timestamp Key: SPARK-27401 URL: https://issues.apache.org/jira/browse/SPARK-27401 Project: Spark

[jira] [Created] (SPARK-27398) Get rid of sun.nio.cs.StreamDecoder in CreateJacksonParser

2019-04-06 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27398: -- Summary: Get rid of sun.nio.cs.StreamDecoder in CreateJacksonParser Key: SPARK-27398 URL: https://issues.apache.org/jira/browse/SPARK-27398 Project: Spark Issue

[jira] [Updated] (SPARK-27357) Cast timestamps to/from dates independently from time zones

2019-04-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27357: --- Summary: Cast timestamps to/from dates independently from time zones (was: Convert timestamps

[jira] [Created] (SPARK-27357) Convert timestamps to/from dates independently from time zones

2019-04-03 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27357: -- Summary: Convert timestamps to/from dates independently from time zones Key: SPARK-27357 URL: https://issues.apache.org/jira/browse/SPARK-27357 Project: Spark

[jira] [Created] (SPARK-27344) Support the LocalDate and Instant classes in Java Bean encoders

2019-04-02 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27344: -- Summary: Support the LocalDate and Instant classes in Java Bean encoders Key: SPARK-27344 URL: https://issues.apache.org/jira/browse/SPARK-27344 Project: Spark

[jira] [Created] (SPARK-27327) New JSON benchmarks: functions, dataset parsing

2019-03-30 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27327: -- Summary: New JSON benchmarks: functions, dataset parsing Key: SPARK-27327 URL: https://issues.apache.org/jira/browse/SPARK-27327 Project: Spark Issue Type:

[jira] [Created] (SPARK-27325) Support implicit encoders for LocalDate and Instant

2019-03-30 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27325: -- Summary: Support implicit encoders for LocalDate and Instant Key: SPARK-27325 URL: https://issues.apache.org/jira/browse/SPARK-27325 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26325) Interpret timestamp fields in Spark while reading json (timestampFormat)

2019-03-25 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801011#comment-16801011 ] Maxim Gekk commented on SPARK-26325: Can you try Z instead of 'Z'? > Interpret timestamp fields in

[jira] [Created] (SPARK-27252) Make current_date() independent from time zones

2019-03-22 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27252: -- Summary: Make current_date() independent from time zones Key: SPARK-27252 URL: https://issues.apache.org/jira/browse/SPARK-27252 Project: Spark Issue Type:

[jira] [Created] (SPARK-27242) Avoid using default time zone in formatting TIMESTAMP/DATE literals

2019-03-22 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27242: -- Summary: Avoid using default time zone in formatting TIMESTAMP/DATE literals Key: SPARK-27242 URL: https://issues.apache.org/jira/browse/SPARK-27242 Project: Spark

[jira] [Created] (SPARK-27222) Support Instant and LocalDate in Literal.apply

2019-03-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27222: -- Summary: Support Instant and LocalDate in Literal.apply Key: SPARK-27222 URL: https://issues.apache.org/jira/browse/SPARK-27222 Project: Spark Issue Type:

[jira] [Created] (SPARK-27212) Eliminate TimeZone to ZoneId conversion in stringToTimestamp

2019-03-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27212: -- Summary: Eliminate TimeZone to ZoneId conversion in stringToTimestamp Key: SPARK-27212 URL: https://issues.apache.org/jira/browse/SPARK-27212 Project: Spark

[jira] [Created] (SPARK-27199) Replace TimeZone by ZoneId in TimestampFormatter API

2019-03-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27199: -- Summary: Replace TimeZone by ZoneId in TimestampFormatter API Key: SPARK-27199 URL: https://issues.apache.org/jira/browse/SPARK-27199 Project: Spark Issue Type:

[jira] [Created] (SPARK-27109) Refactoring of TimestampFormatter and DateFormatter

2019-03-08 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27109: -- Summary: Refactoring of TimestampFormatter and DateFormatter Key: SPARK-27109 URL: https://issues.apache.org/jira/browse/SPARK-27109 Project: Spark Issue Type:

[jira] [Created] (SPARK-27057) Common trait for limit exec operators

2019-03-05 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27057: -- Summary: Common trait for limit exec operators Key: SPARK-27057 URL: https://issues.apache.org/jira/browse/SPARK-27057 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2019-03-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782778#comment-16782778 ] Maxim Gekk commented on SPARK-26016: > nothing reinterprets the bytes according to a different

[jira] [Resolved] (SPARK-24778) DateTimeUtils.getTimeZone method returns GMT time if timezone cannot be parsed

2019-03-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-24778. Resolution: Fixed Fix Version/s: 3.0.0 The issue has been fixed already by using ZoneId.of

[jira] [Updated] (SPARK-27035) Current time with microsecond resolution

2019-03-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27035: --- Description: Currently,  the CurrentTimestamp expression uses 

[jira] [Created] (SPARK-27035) Current time with microsecond resolution

2019-03-03 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27035: -- Summary: Current time with microsecond resolution Key: SPARK-27035 URL: https://issues.apache.org/jira/browse/SPARK-27035 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2019-03-02 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782542#comment-16782542 ] Maxim Gekk commented on SPARK-26016: [~srowen] JSON datasource uses Text datasource in schema

[jira] [Created] (SPARK-27031) Avoid double formatting in timestampToString

2019-03-02 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27031: -- Summary: Avoid double formatting in timestampToString Key: SPARK-27031 URL: https://issues.apache.org/jira/browse/SPARK-27031 Project: Spark Issue Type:

[jira] [Created] (SPARK-27008) Support java.time.LocalDate as an external type of DateType

2019-02-27 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27008: -- Summary: Support java.time.LocalDate as an external type of DateType Key: SPARK-27008 URL: https://issues.apache.org/jira/browse/SPARK-27008 Project: Spark

[jira] [Created] (SPARK-26978) Avoid magic time constants

2019-02-23 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26978: -- Summary: Avoid magic time constants Key: SPARK-26978 URL: https://issues.apache.org/jira/browse/SPARK-26978 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-26651) Use Proleptic Gregorian calendar

2019-02-22 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-26651. Resolution: Done > Use Proleptic Gregorian calendar > > >

[jira] [Created] (SPARK-26955) Align Spark's TimSort to JDK 11 TimSort

2019-02-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26955: -- Summary: Align Spark's TimSort to JDK 11 TimSort Key: SPARK-26955 URL: https://issues.apache.org/jira/browse/SPARK-26955 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-26953) Test TimSort for ArrayIndexOutOfBoundsException

2019-02-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26953: -- Summary: Test TimSort for ArrayIndexOutOfBoundsException Key: SPARK-26953 URL: https://issues.apache.org/jira/browse/SPARK-26953 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-26908) Fix toMillis

2019-02-17 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26908: -- Summary: Fix toMillis Key: SPARK-26908 URL: https://issues.apache.org/jira/browse/SPARK-26908 Project: Spark Issue Type: Improvement Components: SQL

[jira] [Updated] (SPARK-26903) Remove the TimeZone cache

2019-02-16 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26903: --- Summary: Remove the TimeZone cache (was: Remove the cache of TimeZones) > Remove the TimeZone

[jira] [Created] (SPARK-26903) Remove the cache of TimeZones

2019-02-16 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26903: -- Summary: Remove the cache of TimeZones Key: SPARK-26903 URL: https://issues.apache.org/jira/browse/SPARK-26903 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-26902) Support java.time.Instant as an external type of TimestampType

2019-02-16 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26902: -- Summary: Support java.time.Instant as an external type of TimestampType Key: SPARK-26902 URL: https://issues.apache.org/jira/browse/SPARK-26902 Project: Spark

[jira] [Created] (SPARK-26900) Simplify truncation to quarter of year

2019-02-16 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26900: -- Summary: Simplify truncation to quarter of year Key: SPARK-26900 URL: https://issues.apache.org/jira/browse/SPARK-26900 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26800) JDBC - MySQL nullable option is ignored

2019-02-10 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764665#comment-16764665 ] Maxim Gekk commented on SPARK-26800: Using NOT NULL for TIMESTAMP by default is non-standard

[jira] [Updated] (SPARK-26817) Use System.nanoTime to measure time intervals

2019-02-02 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26817: --- Description: Replace System.currentTimeMillis() by System.nanoTime in time intervals measurements.

[jira] [Created] (SPARK-26817) Use System.nanoTime to measure time intervals

2019-02-02 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26817: -- Summary: Use System.nanoTime to measure time intervals Key: SPARK-26817 URL: https://issues.apache.org/jira/browse/SPARK-26817 Project: Spark Issue Type:

[jira] [Created] (SPARK-26816) Benchmark for XORShiftRandom

2019-02-02 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26816: -- Summary: Benchmark for XORShiftRandom Key: SPARK-26816 URL: https://issues.apache.org/jira/browse/SPARK-26816 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-26651) Use Proleptic Gregorian calendar

2019-02-01 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26651: --- Labels: ReleaseNote (was: ) > Use Proleptic Gregorian calendar >

[jira] [Updated] (SPARK-26651) Use Proleptic Gregorian calendar

2019-02-01 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26651: --- Description: Spark 2.4 and previous versions use a hybrid calendar - Julian + Gregorian in

[jira] [Updated] (SPARK-26651) Use Proleptic Gregorian calendar

2019-02-01 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26651: --- Description: Spark 2.4 and previous versions use a hybrid calendar - Julian + Gregorian in

[jira] [Created] (SPARK-26805) Eliminate double checking of stringToDate and stringToTimestamp inputs

2019-01-31 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26805: -- Summary: Eliminate double checking of stringToDate and stringToTimestamp inputs Key: SPARK-26805 URL: https://issues.apache.org/jira/browse/SPARK-26805 Project: Spark

[jira] [Commented] (SPARK-26651) Use Proleptic Gregorian calendar

2019-01-28 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754437#comment-16754437 ] Maxim Gekk commented on SPARK-26651: [~cloud_fan] [~hvanhovell] [~srowen] [~hyukjin.kwon]

[jira] [Created] (SPARK-26740) Statistics for date and timestamp columns depend on system time zone

2019-01-26 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26740: -- Summary: Statistics for date and timestamp columns depend on system time zone Key: SPARK-26740 URL: https://issues.apache.org/jira/browse/SPARK-26740 Project: Spark

[jira] [Created] (SPARK-26720) Remove unused methods from DateTimeUtils

2019-01-24 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26720: -- Summary: Remove unused methods from DateTimeUtils Key: SPARK-26720 URL: https://issues.apache.org/jira/browse/SPARK-26720 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-26654) Use Timestamp/DateFormatter in CatalogColumnStat

2019-01-24 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751634#comment-16751634 ] Maxim Gekk commented on SPARK-26654: [~cloud_fan][~hvanhovell][~srowen] I do believe saving

[jira] [Created] (SPARK-26719) Get rid of java.util.Calendar in DateTimeUtils

2019-01-24 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26719: -- Summary: Get rid of java.util.Calendar in DateTimeUtils Key: SPARK-26719 URL: https://issues.apache.org/jira/browse/SPARK-26719 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26653) Use Proleptic Gregorian calendar in parsing JDBC lower/upper bounds

2019-01-20 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26653: --- Summary: Use Proleptic Gregorian calendar in parsing JDBC lower/upper bounds (was: Use

[jira] [Created] (SPARK-26657) Port DayWeek, DayOfWeek and WeekDay on Proleptic Gregorian calendar

2019-01-17 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26657: -- Summary: Port DayWeek, DayOfWeek and WeekDay on Proleptic Gregorian calendar Key: SPARK-26657 URL: https://issues.apache.org/jira/browse/SPARK-26657 Project: Spark

[jira] [Created] (SPARK-26656) Benchmark for date/time functions and expressions

2019-01-17 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26656: -- Summary: Benchmark for date/time functions and expressions Key: SPARK-26656 URL: https://issues.apache.org/jira/browse/SPARK-26656 Project: Spark Issue Type:

[jira] [Created] (SPARK-26654) Use Timestamp/DateFormatter in CatalogColumnStat

2019-01-17 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26654: -- Summary: Use Timestamp/DateFormatter in CatalogColumnStat Key: SPARK-26654 URL: https://issues.apache.org/jira/browse/SPARK-26654 Project: Spark Issue Type:

[jira] [Created] (SPARK-26653) Use Timestamp/DateFormatter in JDBCRelation

2019-01-17 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26653: -- Summary: Use Timestamp/DateFormatter in JDBCRelation Key: SPARK-26653 URL: https://issues.apache.org/jira/browse/SPARK-26653 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-26652) Use Proleptic Gregorian calendar in creation of Timestamp/Date literals from strings

2019-01-17 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26652: -- Summary: Use Proleptic Gregorian calendar in creation of Timestamp/Date literals from strings Key: SPARK-26652 URL: https://issues.apache.org/jira/browse/SPARK-26652

[jira] [Updated] (SPARK-26424) Use java.time API in timestamp/date functions

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26424: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-26651 > Use java.time API in

[jira] [Commented] (SPARK-26651) Use Proleptic Gregorian calendar

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745482#comment-16745482 ] Maxim Gekk commented on SPARK-26651: [~hvanhovell][~cloud_fan][~srowen] This is the umbrella ticket

[jira] [Updated] (SPARK-26178) Use java.time API for parsing timestamps and dates from CSV

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26178: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-26651 > Use java.time API for

[jira] [Updated] (SPARK-26243) Use java.time API for parsing timestamps and dates from JSON

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26243: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-26651 > Use java.time API for

[jira] [Updated] (SPARK-26374) Support new date/timestamp parser in HadoopFsRelationTest

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26374: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-26651 > Support new date/timestamp parser in

[jira] [Updated] (SPARK-26384) CSV schema inferring does not respect spark.sql.legacy.timeParser.enabled

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26384: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-26651 > CSV schema inferring does not

[jira] [Updated] (SPARK-26456) Cast date/timestamp by Date/TimestampFormatter

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26456: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-26651 > Cast date/timestamp by

[jira] [Updated] (SPARK-26503) Get rid of spark.sql.legacy.timeParser.enabled

2019-01-17 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26503: --- Issue Type: Sub-task (was: Task) Parent: SPARK-26651 > Get rid of

<    4   5   6   7   8   9   10   11   12   >