[jira] [Updated] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2023-05-24 Thread Robert Kruszewski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-21195: -- Labels: (was: bulk-closed) > MetricSystem should pick up dynamically registered

[jira] [Reopened] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2020-10-08 Thread Robert Kruszewski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski reopened SPARK-21195: --- This has been bulk closed due to inactivity but I am still interested in getting this

[jira] [Commented] (SPARK-27273) Compile break for UserDefinedType (DataType) subclasses with javac 8

2019-03-29 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805396#comment-16805396 ] Robert Kruszewski commented on SPARK-27273: --- This is when compiling a subclass of

[jira] [Created] (SPARK-27273) Compile break for UserDefinedType (DataType) subclasses with javac 8

2019-03-25 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-27273: - Summary: Compile break for UserDefinedType (DataType) subclasses with javac 8 Key: SPARK-27273 URL: https://issues.apache.org/jira/browse/SPARK-27273

[jira] [Created] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

2018-03-27 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-23802: - Summary: PropagateEmptyRelation can leave query plan in unresolved state Key: SPARK-23802 URL: https://issues.apache.org/jira/browse/SPARK-23802 Project:

[jira] [Reopened] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-11-06 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski reopened SPARK-21195: --- > MetricSystem should pick up dynamically registered metrics in sources >

[jira] [Commented] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-11-06 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240274#comment-16240274 ] Robert Kruszewski commented on SPARK-21195: --- Thanks, indeed I responded that I missed the last

[jira] [Commented] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-11-06 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240267#comment-16240267 ] Robert Kruszewski commented on SPARK-21195: --- What's the justification for `won't fix` here? >

[jira] [Commented] (SPARK-20952) ParquetFileFormat should forward TaskContext to its forkjoinpool

2017-08-16 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129327#comment-16129327 ] Robert Kruszewski commented on SPARK-20952: --- I don't care about what actual filesystem impls

[jira] [Closed] (SPARK-21400) Spark shouldn't ignore user defined output committer in append mode

2017-07-26 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski closed SPARK-21400. - Resolution: Fixed Fix Version/s: 2.3.0 > Spark shouldn't ignore user defined

[jira] [Commented] (SPARK-21400) Spark shouldn't ignore user defined output committer in append mode

2017-07-26 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102638#comment-16102638 ] Robert Kruszewski commented on SPARK-21400: --- Fixed in

[jira] [Commented] (SPARK-21334) Fix metrics for external shuffle service

2017-07-20 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095095#comment-16095095 ] Robert Kruszewski commented on SPARK-21334: --- I think this is a dupe of

[jira] [Updated] (SPARK-21400) Spark shouldn't ignore user defined output committer in append mode

2017-07-13 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-21400: -- Component/s: (was: Spark Core) SQL > Spark shouldn't ignore user

[jira] [Created] (SPARK-21400) Spark shouldn't ignore user defined output committer in append mode

2017-07-13 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-21400: - Summary: Spark shouldn't ignore user defined output committer in append mode Key: SPARK-21400 URL: https://issues.apache.org/jira/browse/SPARK-21400

[jira] [Updated] (SPARK-21319) UnsafeExternalRowSorter.RowComparator memory leak

2017-07-05 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-21319: -- Labels: (was: patch) > UnsafeExternalRowSorter.RowComparator memory leak >

[jira] [Updated] (SPARK-21319) UnsafeExternalRowSorter.RowComparator memory leak

2017-07-05 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-21319: -- Flags: (was: Patch) > UnsafeExternalRowSorter.RowComparator memory leak >

[jira] [Commented] (SPARK-20952) ParquetFileFormat should forward TaskContext to its forkjoinpool

2017-06-23 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060972#comment-16060972 ] Robert Kruszewski commented on SPARK-20952: --- [~zsxwing] updated the issue title and pr with

[jira] [Updated] (SPARK-20952) ParquetFileFormat should forward TaskContext to its forkjoinpool

2017-06-23 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-20952: -- Summary: ParquetFileFormat should forward TaskContext to its forkjoinpool (was:

[jira] [Created] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-06-23 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-21195: - Summary: MetricSystem should pick up dynamically registered metrics in sources Key: SPARK-21195 URL: https://issues.apache.org/jira/browse/SPARK-21195

[jira] [Comment Edited] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043714#comment-16043714 ] Robert Kruszewski edited comment on SPARK-20952 at 6/9/17 2:12 AM: ---

[jira] [Comment Edited] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043714#comment-16043714 ] Robert Kruszewski edited comment on SPARK-20952 at 6/9/17 2:13 AM: ---

[jira] [Comment Edited] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043714#comment-16043714 ] Robert Kruszewski edited comment on SPARK-20952 at 6/9/17 12:28 AM:

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043714#comment-16043714 ] Robert Kruszewski commented on SPARK-20952: --- Right, but how do I pass it downstream? >

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043712#comment-16043712 ] Robert Kruszewski commented on SPARK-20952: --- It doesn't but things underneath it do. It's weird

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043704#comment-16043704 ] Robert Kruszewski commented on SPARK-20952: --- No modifications, it's this code

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043669#comment-16043669 ] Robert Kruszewski commented on SPARK-20952: --- I am not really attached to the solution. Would be

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043623#comment-16043623 ] Robert Kruszewski commented on SPARK-20952: --- This is already an issue though on driver side

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-02 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035195#comment-16035195 ] Robert Kruszewski commented on SPARK-20952: --- 2 is already happening on executors where the Task

[jira] [Commented] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-02 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034427#comment-16034427 ] Robert Kruszewski commented on SPARK-20952: --- You're right that this needs a bit of

[jira] [Created] (SPARK-20952) TaskContext should be an InheritableThreadLocal

2017-06-01 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-20952: - Summary: TaskContext should be an InheritableThreadLocal Key: SPARK-20952 URL: https://issues.apache.org/jira/browse/SPARK-20952 Project: Spark

[jira] [Closed] (SPARK-19213) FileSourceScanExec uses SparkSession from HadoopFsRelation creation time instead of the active session at execution time

2017-05-10 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski closed SPARK-19213. - Resolution: Won't Fix Not really necessary and can lead to confusing results >

[jira] [Commented] (SPARK-20683) Make table uncache chaining optional

2017-05-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003464#comment-16003464 ] Robert Kruszewski commented on SPARK-20683: --- +1, have the similar use case where we do cache

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003339#comment-16003339 ] Robert Kruszewski commented on SPARK-20682: --- Awesome, didn't know that there are nohive

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003321#comment-16003321 ] Robert Kruszewski commented on SPARK-20682: --- Would it make sense to bring hive-storage-api to

[jira] [Commented] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972400#comment-15972400 ] Robert Kruszewski commented on SPARK-20364: --- Looks like parquet doesn't differentiate between

[jira] [Comment Edited] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972400#comment-15972400 ] Robert Kruszewski edited comment on SPARK-20364 at 4/18/17 9:36 AM:

[jira] [Closed] (SPARK-19632) Allow configuring non-hive and non-local SessionState and ExternalCatalog

2017-02-16 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski closed SPARK-19632. - Resolution: Won't Fix > Allow configuring non-hive and non-local SessionState and

[jira] [Commented] (SPARK-19632) Allow configuring non-hive and non-local SessionState and ExternalCatalog

2017-02-16 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870649#comment-15870649 ] Robert Kruszewski commented on SPARK-19632: --- Thanks, I must have missed that. > Allow

[jira] [Created] (SPARK-19632) Allow configuring non-hive and non-local SessionState and ExternalCatalog

2017-02-16 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-19632: - Summary: Allow configuring non-hive and non-local SessionState and ExternalCatalog Key: SPARK-19632 URL: https://issues.apache.org/jira/browse/SPARK-19632

[jira] [Updated] (SPARK-19213) FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution

2017-01-13 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-19213: -- Description: If you look at

[jira] [Updated] (SPARK-19213) FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution

2017-01-13 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-19213: -- Description: If you look at

[jira] [Updated] (SPARK-19213) FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution

2017-01-13 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-19213: -- Description: If you look at

[jira] [Updated] (SPARK-19213) FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution

2017-01-13 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-19213: -- Description: If you look at

[jira] [Updated] (SPARK-19213) FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution

2017-01-13 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-19213: -- Description: If you look at

[jira] [Created] (SPARK-19213) FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution

2017-01-13 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-19213: - Summary: FileSourceScanExec usese sparksession from hadoopfsrelation creation time instead of the one active at time of execution Key: SPARK-19213 URL:

[jira] [Updated] (SPARK-17213) Parquet String Pushdown for Non-Eq Comparisons Broken

2016-12-01 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17213: -- Affects Version/s: (was: 2.0.0) 2.1.0 > Parquet String

[jira] [Commented] (SPARK-17478) Create spark.eventLog.dir if it does not exist

2016-09-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478087#comment-15478087 ] Robert Kruszewski commented on SPARK-17478: --- Thanks for the pointers and apologies that I

[jira] [Created] (SPARK-17478) Create spark.eventLog.dir if it does not exist

2016-09-09 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17478: - Summary: Create spark.eventLog.dir if it does not exist Key: SPARK-17478 URL: https://issues.apache.org/jira/browse/SPARK-17478 Project: Spark

[jira] [Commented] (SPARK-17226) Allow defining multiple date formats per column in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435709#comment-15435709 ] Robert Kruszewski commented on SPARK-17226: --- Anything in particular you have in mind? I should

[jira] [Updated] (SPARK-17226) Allow defining multiple date formats per column in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17226: -- Description: Useful to have fallbacks in case of messy input and different columns can

[jira] [Updated] (SPARK-17225) Support multiple null values in csv files

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17225: -- Component/s: SQL > Support multiple null values in csv files >

[jira] [Updated] (SPARK-17224) Support skipping multiple header rows in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17224: -- Component/s: SQL > Support skipping multiple header rows in csv >

[jira] [Updated] (SPARK-17225) Support multiple null values in csv files

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17225: -- Affects Version/s: 2.0.0 > Support multiple null values in csv files >

[jira] [Updated] (SPARK-17226) Allow defining multiple date formats per column in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17226: -- Component/s: SQL > Allow defining multiple date formats per column in csv >

[jira] [Updated] (SPARK-17224) Support skipping multiple header rows in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17224: -- Affects Version/s: 2.0.0 > Support skipping multiple header rows in csv >

[jira] [Updated] (SPARK-17226) Allow defining multiple date formats per column in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17226: -- Affects Version/s: 2.0.0 > Allow defining multiple date formats per column in csv >

[jira] [Updated] (SPARK-17227) Allow configuring record delimiter in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17227: -- Affects Version/s: 2.0.0 > Allow configuring record delimiter in csv >

[jira] [Updated] (SPARK-17227) Allow configuring record delimiter in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17227: -- Component/s: SQL > Allow configuring record delimiter in csv >

[jira] [Updated] (SPARK-17222) Support multline csv records

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17222: -- Affects Version/s: 2.0.0 > Support multline csv records >

[jira] [Updated] (SPARK-17222) Support multline csv records

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17222: -- Component/s: SQL > Support multline csv records > > >

[jira] [Updated] (SPARK-17222) Support multline csv records

2016-08-24 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-17222: -- Description: Below should be read as one record and currently it won't be since files

[jira] [Created] (SPARK-17227) Allow configuring record delimiter in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17227: - Summary: Allow configuring record delimiter in csv Key: SPARK-17227 URL: https://issues.apache.org/jira/browse/SPARK-17227 Project: Spark Issue

[jira] [Created] (SPARK-17226) Allow defining multiple date formats per column in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17226: - Summary: Allow defining multiple date formats per column in csv Key: SPARK-17226 URL: https://issues.apache.org/jira/browse/SPARK-17226 Project: Spark

[jira] [Created] (SPARK-17225) Support multiple null values in csv files

2016-08-24 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17225: - Summary: Support multiple null values in csv files Key: SPARK-17225 URL: https://issues.apache.org/jira/browse/SPARK-17225 Project: Spark Issue

[jira] [Created] (SPARK-17224) Support skipping multiple header rows in csv

2016-08-24 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17224: - Summary: Support skipping multiple header rows in csv Key: SPARK-17224 URL: https://issues.apache.org/jira/browse/SPARK-17224 Project: Spark Issue

[jira] [Created] (SPARK-17222) Support multline csv records

2016-08-24 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17222: - Summary: Support multline csv records Key: SPARK-17222 URL: https://issues.apache.org/jira/browse/SPARK-17222 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-17216) Even timeline for a stage doesn't core 100% of the bar timeline bar in chrome

2016-08-24 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-17216: - Summary: Even timeline for a stage doesn't core 100% of the bar timeline bar in chrome Key: SPARK-17216 URL: https://issues.apache.org/jira/browse/SPARK-17216

[jira] [Created] (SPARK-16984) executeTake tries all partitions if first parition is empty

2016-08-09 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-16984: - Summary: executeTake tries all partitions if first parition is empty Key: SPARK-16984 URL: https://issues.apache.org/jira/browse/SPARK-16984 Project: Spark

[jira] [Created] (SPARK-16623) Datasources should expose ability to define schema conversions they're performing to save data

2016-07-19 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-16623: - Summary: Datasources should expose ability to define schema conversions they're performing to save data Key: SPARK-16623 URL:

[jira] [Commented] (SPARK-16569) Use Cython to speed up Pyspark internals

2016-07-15 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15379310#comment-15379310 ] Robert Kruszewski commented on SPARK-16569: --- Cython only improves performance if you have type

[jira] [Commented] (SPARK-16511) SparkLauncher should allow setting working directory for spark-submit process

2016-07-12 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374024#comment-15374024 ] Robert Kruszewski commented on SPARK-16511: --- >From the comments on original PR are we fine

[jira] [Updated] (SPARK-16511) SparkLauncher should allow setting working directory for spark-submit process

2016-07-12 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-16511: -- Description: Setting directory on java ProcessBuilder is only legitimate way of

[jira] [Created] (SPARK-16511) SparkLauncher should allow setting working directory for spark-submit process

2016-07-12 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-16511: - Summary: SparkLauncher should allow setting working directory for spark-submit process Key: SPARK-16511 URL: https://issues.apache.org/jira/browse/SPARK-16511

[jira] [Reopened] (SPARK-9843) Catalyst: Allow adding custom optimizers

2015-12-08 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski reopened SPARK-9843: -- Made a new PR. Let me know if anyone have feedback > Catalyst: Allow adding custom

[jira] [Created] (SPARK-9843) Catalyst: Allow adding custom optimizers

2015-08-11 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-9843: Summary: Catalyst: Allow adding custom optimizers Key: SPARK-9843 URL: https://issues.apache.org/jira/browse/SPARK-9843 Project: Spark Issue Type: