Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16021
@dongjoon-hyun Thanks for letting me know! I'm not familiar with a backport
policy in spark though, if we can, I think it'd be better to do because it
seems there are still some of users
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/16030
[SPARK-18108][SQL] Fix a bug to fail partition schema inference
## What changes were proposed in this pull request?
This pr is to fix a bug to fail partition schema inference;
```
scala
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
This query passed in the released spark-2.0.2, so it seems this regression
is involved with SPARK-18510.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@brkyvz @tdas Could you check this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@brkyvz Thanks for your comment! okay, I'll fix in a that way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
I'm looking into the failures.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@brkyvz How about this fix?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/15257
I think it is difficult to support Seq and JList because of type erasure.
If it is possible to add a new interface below, we can support these types via
`ScalaReflection.schemaFor[T
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
Many thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16605#discussion_r96547709
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3073,7 +3073,12 @@ object functions {
*/
def udf[RT: TypeTag, A1
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16605#discussion_r96549098
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1428,6 +1428,134 @@ class DataFrameSuite extends QueryTest with
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16605#discussion_r96551693
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1428,6 +1428,134 @@ class DataFrameSuite extends QueryTest with
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16213#discussion_r96556871
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -225,6 +225,74 @@ abstract class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16605
@dongjoon-hyun okay, I applied your comments into this pr. Could you check
again to satisfy your intentions?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16605#discussion_r96745940
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala
---
@@ -84,7 +86,9 @@ case class ScalaUDF(
case 1
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16605#discussion_r96789868
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala
---
@@ -84,7 +86,9 @@ case class ScalaUDF(
case 1
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16605
oh, yea. I didn't find that and I think it's a good point.
IMO `WrappedArray` is implicitly used inside for implicit conversions, so
users do not use `WrappedArray` directly for UD
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
@tdas ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16605
okay. But, if this issue finished, I'm planning to take SPARK-12823 in a
similar way.
Do u think also it's not also worth trying struct? cc: @cloud-fan
---
If your project is set
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16605
The workaround @cloud-fan said is good to me though, IMO the most critical
issue here is that this cast exception happens not in analysis phases but in
runtime. So, at least I think we should modify
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16605
okay, I'll update this pr in that way, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/15945#discussion_r97087686
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -79,34 +78,21 @@ class QueryExecution(val sparkSession
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/15945#discussion_r97096455
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/PhysicalOptimizer.scala
---
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
@tdas ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/14481
Any update?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
@brkyvz @tdas ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/15928
@hvanhovell ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/14038
@liancheng ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/15945
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/16698
[CORE][DOCS] Update a help message for --files in spark-submit
## What changes were proposed in this pull request?
This pr is to update a help message for `--files` in spark-submit because
it
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16698
Nitpicking though, I think it helps for some users. Could someone check
this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
okay, I'll update soon!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and w
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16213#discussion_r97912489
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisCheckpointer.scala
---
@@ -64,7 +64,18 @@ private[kinesis] class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
Many thanks! Also, congrats, commiter
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
great, haha! Many thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/16707
[SPARK-19338][SQL] Add UDF names in explain
## What changes were proposed in this pull request?
This pr added a variable for a UDF name in `ScalaUDF`.
Then, if the variable filled
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16707
Aha, SGTM. I'll fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wish
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16707#discussion_r97937869
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala ---
@@ -125,7 +125,7 @@ class UDFRegistration private[sql] (functionRegistry
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16707
It seems the latest test failure does not relate to this pr..., I'll test
again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/13300
This pr seems stale and inactive. I know this kind of API changes has lower
priorities now. So, how about closing this pr for now and setting `LATER` in
the corresponding JIRA? Thought? cc: @rxin
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/13300
Yea, I also think `json` and `csv` stuffs should be consistent and they'd
be better to have the same code structure and behaviour as @HyukjinKwon said.
Since we do not have `DataFrameReader
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16707
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/13300
Aha, I see. Anyway, we need to keep discussion not here but the JIRA!
(because this is the closed..)
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16698
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16707
@gatorsmile @rxin okay, it seems ready to merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/16733
[SPARK-19392][SQL] Fix the bug that throws an exception when using numeric
types in Oracle JDBC
## What changes were proposed in this pull request?
In OracleDialect, if you use Numeric types in
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16733
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16733
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16733
@gatorsmile okay, I'll do
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16733
oh, sorry. I'll add the test soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
en
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16733#discussion_r99255011
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala ---
@@ -29,7 +29,12 @@ private case object OracleDialect extends JdbcDialect
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16733#discussion_r99255047
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
---
@@ -726,11 +726,14 @@ class JDBCSuite extends SparkFunSuite
test
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16733#discussion_r99261637
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala ---
@@ -29,7 +29,12 @@ private case object OracleDialect extends JdbcDialect
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16733#discussion_r99273860
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala ---
@@ -29,7 +29,12 @@ private case object OracleDialect extends JdbcDialect
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/14038
@liancheng ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/15928
@hvanhovell @rxin Could you assign some committers to review this? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16610
@hvanhovell Could you give me some insights?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/15945
@cloud-fan ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16733
Yea, I sent the e-mail, but I didn't get a reply from him;
http://apache-spark-developers-list.1001551.n3.nabble.com/Re-Oracle-JDBC-Spark-SQL-Key-Not-Found-Scale-td20835.html
---
If your proje
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
This is not a bug in `VectorizedParquetReader ` as @liancheng said, and
the root cause is that wrongly inferred types are passed into the reader in
`VectorizedParquetReader#initBatch`
https
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@brkyvz Thanks! Does the latest fix apply your suggestion?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/16114
[SPARK-18620][Streaming][Kinesis] Flatten input rates in timeline for
streaming + kinesis
## What changes were proposed in this pull request?
This pr is to make input rates in timeline more
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
Thanks your review, @brkyvz ! I'm checking your comments now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project doe
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r90600966
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -56,6 +56,38 @@ private[kinesis] class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r90601340
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -221,6 +221,12 @@ private[kinesis] class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r90619967
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala
---
@@ -969,4 +969,15 @@ class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r90620033
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -174,22 +185,18 @@ case class DataSource
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r90620001
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -174,22 +185,18 @@ case class DataSource
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r90620083
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -132,13 +132,24 @@ case class DataSource
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@brkyvz How should we handle this query? I think we have some overheads to
check the constant value (`1480617712537`) in path matches all the data in
DataFrame and to throw an exception if illegal
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@brkyvz yea, I think so, but the issue of column ordering seems to be out
of scope in this original ticket. So, why don't we open a new jira and discuss
there?
---
If your project is set up f
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r90752283
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -189,7 +189,15 @@ case class DataSource
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r90754922
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -56,6 +56,31 @@ private[kinesis] class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16114
@srowen Do u know qualified maintainers on this component?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
The failure seems to be not related to this pr?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r90758182
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -56,6 +56,27 @@ private[kinesis] class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16114
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r90758322
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -56,6 +56,27 @@ private[kinesis] class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
okay, I'll do
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r91208525
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -68,9 +69,16 @@ private[kinesis] class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r91210953
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -68,9 +69,16 @@ private[kinesis] class
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16114#discussion_r91419246
--- Diff:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala
---
@@ -68,7 +68,16 @@ private[kinesis] class
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/16213
[SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END to finish reading
closed shards
## What changes were proposed in this pull request?
This pr is to fix an issue occurred when resharding
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
I'm looking into the failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16114
@brkyvz Could you also check this pr #16213? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16213#discussion_r91829740
--- Diff:
external/kinesis-asl/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/CheckpointerShim.java
---
@@ -0,0 +1,43
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16213#discussion_r91829798
--- Diff:
external/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisStreamSuite.scala
---
@@ -225,6 +225,74 @@ abstract class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16213
Thanks for these comments! ya, I do not like this approach, too. But, since
those who reshard streams always hit this issue and resharding is important for
load-balancing in Kinesis streams
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
What's the status of this pr? If I've something to do, plz let me know.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
okay, I'll try to fix in that way, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r92543342
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala
---
@@ -969,4 +969,17 @@ class
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@cloud-fan Does the latest fix satisfy what you suggested?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
@liancheng As for `DataFrameReader.dataSchema()` and
`DataFrameReader.partitoinSchema()`, did you mean we add new interfaces there
for users to set user-defined data and partition schema
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r92594503
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala
---
@@ -49,9 +49,12 @@ case class HadoopFsRelation
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/16030#discussion_r92600244
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala
---
@@ -49,10 +51,16 @@ case class HadoopFsRelation
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/16030
okay
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
301 - 400 of 3608 matches
Mail list logo
