Rui Wang created SPARK-47176:
Summary: Have a ResolveAllExpressionsUpWithPruning helper function
Key: SPARK-47176
URL: https://issues.apache.org/jira/browse/SPARK-47176
Project: Spark
Issue
Rui Wang created SPARK-46740:
Summary: Only convert to ParquetFileScan for normal Hive Parquet
table
Key: SPARK-46740
URL: https://issues.apache.org/jira/browse/SPARK-46740
Project: Spark
Issue
Rui Wang created SPARK-46610:
Summary: Create table should throw exception when no value for a
key in options
Key: SPARK-46610
URL: https://issues.apache.org/jira/browse/SPARK-46610
Project: Spark
Rui Wang created SPARK-46600:
Summary: Move shared code between SqlConf and SqlApiConf to
another object
Key: SPARK-46600
URL: https://issues.apache.org/jira/browse/SPARK-46600
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-46448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-46448:
-
Summary: InlineTable column should be nullable when converted to Decimal
might cause null (was:
Rui Wang created SPARK-46448:
Summary: InlineTable column should be nullable when converted to
Decimal and overflow
Key: SPARK-46448
URL: https://issues.apache.org/jira/browse/SPARK-46448
Project: Spark
Rui Wang created SPARK-46285:
Summary: Add foreachWithSubqueries
Key: SPARK-46285
URL: https://issues.apache.org/jira/browse/SPARK-46285
Project: Spark
Issue Type: Improvement
Rui Wang created SPARK-46145:
Summary: spark.catalog.listTables does not throw exception when
the table or view is not found
Key: SPARK-46145
URL: https://issues.apache.org/jira/browse/SPARK-46145
Rui Wang created SPARK-45752:
Summary: Unreferenced CTE should all be checked by CheckAnalysis0
Key: SPARK-45752
URL: https://issues.apache.org/jira/browse/SPARK-45752
Project: Spark
Issue Type:
Rui Wang created SPARK-45421:
Summary: Catch AnalysisException over InlineCTE
Key: SPARK-45421
URL: https://issues.apache.org/jira/browse/SPARK-45421
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-45242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang resolved SPARK-45242.
--
Fix Version/s: 4.0.0
Resolution: Fixed
https://github.com/apache/spark/pull/43010
> Use
Rui Wang created SPARK-45242:
Summary: Use DataFrame ID to semantically validate CollectMetrics
Key: SPARK-45242
URL: https://issues.apache.org/jira/browse/SPARK-45242
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-45012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-45012:
-
Affects Version/s: 4.0.0
(was: 3.5.0)
> CheckAnalysis should throw inlined
Rui Wang created SPARK-45012:
Summary: CheckAnalysis should throw inlined plan in
AnalysisException
Key: SPARK-45012
URL: https://issues.apache.org/jira/browse/SPARK-45012
Project: Spark
Issue
Rui Wang created SPARK-44921:
Summary: Remove SqlBaseLexer.tokens from codebase
Key: SPARK-44921
URL: https://issues.apache.org/jira/browse/SPARK-44921
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-44258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang resolved SPARK-44258.
--
Resolution: Fixed
> Move Metadata to sql/api
>
>
> Key:
[
https://issues.apache.org/jira/browse/SPARK-42887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang resolved SPARK-42887.
--
Resolution: Fixed
> Simple DataType interface
> -
>
> Key:
Rui Wang created SPARK-44574:
Summary: Errors that moved into sq/api should also use Analysis
Key: SPARK-44574
URL: https://issues.apache.org/jira/browse/SPARK-44574
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-44507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang resolved SPARK-44507.
--
Resolution: Won't Fix
> SCSC does not depend on AnalysisException
>
Rui Wang created SPARK-44507:
Summary: SCSC does not depend on AnalysisException
Key: SPARK-44507
URL: https://issues.apache.org/jira/browse/SPARK-44507
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44496:
Summary: Move Interfaces needed by SCSC to sql/api
Key: SPARK-44496
URL: https://issues.apache.org/jira/browse/SPARK-44496
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44475:
Summary: Relocate DataType and Parser to sql/api
Key: SPARK-44475
URL: https://issues.apache.org/jira/browse/SPARK-44475
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44326:
Summary: Move utils that are used from Scala client to the common
modules
Key: SPARK-44326
URL: https://issues.apache.org/jira/browse/SPARK-44326
Project: Spark
Rui Wang created SPARK-44324:
Summary: Move CaseInsensitiveMap to sql/api
Key: SPARK-44324
URL: https://issues.apache.org/jira/browse/SPARK-44324
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44315:
Summary: Move DefinedByConstructorParams to sql/api
Key: SPARK-44315
URL: https://issues.apache.org/jira/browse/SPARK-44315
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-44281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang resolved SPARK-44281.
--
Resolution: Fixed
> Move QueryCompilation error that used by DataType to sql/api as DataTypeErrors
>
[
https://issues.apache.org/jira/browse/SPARK-44281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-44281:
-
Summary: Move QueryCompilation error that used by DataType to sql/api as
DataTypeErrors (was: Move
Rui Wang created SPARK-44281:
Summary: Move QueryCompilcation error that used by DataType to
sql/api as DataTypeErrors
Key: SPARK-44281
URL: https://issues.apache.org/jira/browse/SPARK-44281
Project:
Rui Wang created SPARK-44274:
Summary: Move out util functions used by ArtifactManager to
common/utils
Key: SPARK-44274
URL: https://issues.apache.org/jira/browse/SPARK-44274
Project: Spark
Rui Wang created SPARK-44273:
Summary: Spark Connect Scala Client do not depend on Spark
Key: SPARK-44273
URL: https://issues.apache.org/jira/browse/SPARK-44273
Project: Spark
Issue Type: Task
Rui Wang created SPARK-44271:
Summary: Move util functions from DataType to ResolveDefaultColumns
Key: SPARK-44271
URL: https://issues.apache.org/jira/browse/SPARK-44271
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-44261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739248#comment-17739248
]
Rui Wang commented on SPARK-44261:
--
probably wait a bit for https://github.com/apache/spark/pull/41794
[
https://issues.apache.org/jira/browse/SPARK-44261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739247#comment-17739247
]
Rui Wang commented on SPARK-44261:
--
I probably need to work on this one. I got a chance to talk to
[
https://issues.apache.org/jira/browse/SPARK-44266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-44266:
-
Summary: Move Util.truncatedString to sql/api (was: Move
Util.truncatedString)
> Move
Rui Wang created SPARK-44266:
Summary: Move Util.truncatedString
Key: SPARK-44266
URL: https://issues.apache.org/jira/browse/SPARK-44266
Project: Spark
Issue Type: Sub-task
Components:
[
https://issues.apache.org/jira/browse/SPARK-44160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang resolved SPARK-44160.
--
Resolution: Won't Fix
> Extract shared code from StructType
> ---
>
>
[
https://issues.apache.org/jira/browse/SPARK-42887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739245#comment-17739245
]
Rui Wang commented on SPARK-42887:
--
[~panbingkun] yes please! any help is very welcome!
Basically I
Rui Wang created SPARK-44258:
Summary: Move Metadata to sql/api
Key: SPARK-44258
URL: https://issues.apache.org/jira/browse/SPARK-44258
Project: Spark
Issue Type: Sub-task
Components:
Rui Wang created SPARK-44255:
Summary: Relocate StorageLevel to common/utils
Key: SPARK-44255
URL: https://issues.apache.org/jira/browse/SPARK-44255
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-44255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-44255:
-
Issue Type: Task (was: Bug)
> Relocate StorageLevel to common/utils
>
Rui Wang created SPARK-44254:
Summary: Move QueryExecutionErrors to sql/api
Key: SPARK-44254
URL: https://issues.apache.org/jira/browse/SPARK-44254
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44227:
Summary: Extract SchemaUtils from StructField
Key: SPARK-44227
URL: https://issues.apache.org/jira/browse/SPARK-44227
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44220:
Summary: Move StringConcat to sql/api
Key: SPARK-44220
URL: https://issues.apache.org/jira/browse/SPARK-44220
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44205:
Summary: Extract catalyst code from DecimalType
Key: SPARK-44205
URL: https://issues.apache.org/jira/browse/SPARK-44205
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-44164:
Summary: Extract toAttribute method from StructField to Util class
Key: SPARK-44164
URL: https://issues.apache.org/jira/browse/SPARK-44164
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-44160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-44160:
-
Description: The StructType has some methods that require CatalystParser
and Catalyst expression. We
Rui Wang created SPARK-44160:
Summary: Extract shared code from StructType
Key: SPARK-44160
URL: https://issues.apache.org/jira/browse/SPARK-44160
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-43891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735348#comment-17735348
]
Rui Wang commented on SPARK-43891:
--
Sorry I am bit busy recently and will follow up on this ticket.
>
Rui Wang created SPARK-44082:
Summary: Generate operator does not update reference set properly
Key: SPARK-44082
URL: https://issues.apache.org/jira/browse/SPARK-44082
Project: Spark
Issue Type:
Rui Wang created SPARK-44030:
Summary: Move out json and expression private methods from DataType
Key: SPARK-44030
URL: https://issues.apache.org/jira/browse/SPARK-44030
Project: Spark
Issue
Rui Wang created SPARK-43979:
Summary: CollectedMetrics should be treated as the same one for
self-join
Key: SPARK-43979
URL: https://issues.apache.org/jira/browse/SPARK-43979
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-43920:
-
Description: We need a sql/api module to host public API like DataType,
Row, etc. This module can be
Rui Wang created SPARK-43920:
Summary: Create sql/api module
Key: SPARK-43920
URL: https://issues.apache.org/jira/browse/SPARK-43920
Project: Spark
Issue Type: Task
Components:
[
https://issues.apache.org/jira/browse/SPARK-43919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-43919:
-
Summary: Extract JSON functionality out of Row (was: Extract JSON
functionality to Row)
> Extract
Rui Wang created SPARK-43919:
Summary: Extract JSON functionality to Row
Key: SPARK-43919
URL: https://issues.apache.org/jira/browse/SPARK-43919
Project: Spark
Issue Type: Task
Rui Wang created SPARK-43891:
Summary: Support SHOW VIEWS IN . when not
is not the current selected catalog
Key: SPARK-43891
URL: https://issues.apache.org/jira/browse/SPARK-43891
Project: Spark
Rui Wang created SPARK-43888:
Summary: Relocate Logging to common/utils
Key: SPARK-43888
URL: https://issues.apache.org/jira/browse/SPARK-43888
Project: Spark
Issue Type: Task
Rui Wang created SPARK-43779:
Summary: To_date should load the EvalMode in main thread
Key: SPARK-43779
URL: https://issues.apache.org/jira/browse/SPARK-43779
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-43779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-43779:
-
Summary: ParseToDate should load the EvalMode in main thread (was: To_date
should load the EvalMode in
[
https://issues.apache.org/jira/browse/SPARK-43265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721850#comment-17721850
]
Rui Wang commented on SPARK-43265:
--
[~LuciferYang]
The eventual goal is that Spark Connect Scala
Rui Wang created SPARK-43431:
Summary: Spark Connect Scala Client should not expose
`collectResult`
Key: SPARK-43431
URL: https://issues.apache.org/jira/browse/SPARK-43431
Project: Spark
Issue
Rui Wang created SPARK-43428:
Summary: Move some class utils to common/utils
Key: SPARK-43428
URL: https://issues.apache.org/jira/browse/SPARK-43428
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-43265:
Summary: Move Error framework to a common utils module
Key: SPARK-43265
URL: https://issues.apache.org/jira/browse/SPARK-43265
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-43165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-43165:
-
Affects Version/s: 3.5.0
(was: 3.4.0)
> Move canWrite to DataTypeUtils
>
Rui Wang created SPARK-43168:
Summary: Remove get PhysicalDataType method from Datatype class
Key: SPARK-43168
URL: https://issues.apache.org/jira/browse/SPARK-43168
Project: Spark
Issue Type:
Rui Wang created SPARK-43165:
Summary: Move canWrite to DataTypeUtils
Key: SPARK-43165
URL: https://issues.apache.org/jira/browse/SPARK-43165
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-43130:
Summary: Move InternalType to PhysicalDataType
Key: SPARK-43130
URL: https://issues.apache.org/jira/browse/SPARK-43130
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-43110:
Summary: Move asIntegral to PhysicalDataType
Key: SPARK-43110
URL: https://issues.apache.org/jira/browse/SPARK-43110
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-43103:
Summary: Move Integral to PhysicalDataType
Key: SPARK-43103
URL: https://issues.apache.org/jira/browse/SPARK-43103
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-43058:
Summary: Move Numeric to PhysicalDataType
Key: SPARK-43058
URL: https://issues.apache.org/jira/browse/SPARK-43058
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-43019:
Summary: Move Ordering to PhysicalDataType
Key: SPARK-43019
URL: https://issues.apache.org/jira/browse/SPARK-43019
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-42895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703815#comment-17703815
]
Rui Wang commented on SPARK-42895:
--
Of course though it probably need a better error message locally.
[
https://issues.apache.org/jira/browse/SPARK-42895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703814#comment-17703814
]
Rui Wang commented on SPARK-42895:
--
In fact spark.stop() does not stop the remote spark session but
Rui Wang created SPARK-42892:
Summary: Move sameType and relevant methods out of DataType
Key: SPARK-42892
URL: https://issues.apache.org/jira/browse/SPARK-42892
Project: Spark
Issue Type:
Rui Wang created SPARK-42887:
Summary: Simple DataType interface
Key: SPARK-42887
URL: https://issues.apache.org/jira/browse/SPARK-42887
Project: Spark
Issue Type: Task
Components: SQL
Rui Wang created SPARK-42878:
Summary: Named Table should support options
Key: SPARK-42878
URL: https://issues.apache.org/jira/browse/SPARK-42878
Project: Spark
Issue Type: Task
Rui Wang created SPARK-42876:
Summary: DataType's physicalDataType should be private[sql]
Key: SPARK-42876
URL: https://issues.apache.org/jira/browse/SPARK-42876
Project: Spark
Issue Type: Task
Rui Wang created SPARK-42722:
Summary: Python Connect def schema() should not cache the schema
Key: SPARK-42722
URL: https://issues.apache.org/jira/browse/SPARK-42722
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang reassigned SPARK-42688:
Assignee: (was: Rui Wang)
> Rename Connect proto Request client_id to session_id
>
[
https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-42688:
-
Summary: Rename Connect proto Request client_id to session_id (was: Rename
client_id to session_id)
>
Rui Wang created SPARK-42688:
Summary: Rename client_id to session_id
Key: SPARK-42688
URL: https://issues.apache.org/jira/browse/SPARK-42688
Project: Spark
Issue Type: Task
Rui Wang created SPARK-42667:
Summary: Spark Connect: newSession API
Key: SPARK-42667
URL: https://issues.apache.org/jira/browse/SPARK-42667
Project: Spark
Issue Type: Task
Components:
Rui Wang created SPARK-42640:
Summary: Remove stale entries from the excluding rules for
CompabilitySuite
Key: SPARK-42640
URL: https://issues.apache.org/jira/browse/SPARK-42640
Project: Spark
Rui Wang created SPARK-42609:
Summary: Support Grouping/Grouping_Set
Key: SPARK-42609
URL: https://issues.apache.org/jira/browse/SPARK-42609
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-42542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang reassigned SPARK-42542:
Assignee: Rui Wang
> Support Pivot without providing pivot column values
>
[
https://issues.apache.org/jira/browse/SPARK-42576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang reassigned SPARK-42576:
Assignee: Rui Wang
> Add 2nd groupBy method to Dataset
> -
>
>
[
https://issues.apache.org/jira/browse/SPARK-42561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang reassigned SPARK-42561:
Assignee: Rui Wang
> Add TempView APIs to Dataset
>
>
>
Rui Wang created SPARK-42569:
Summary: Throw unsupported exceptions for non-supported API
Key: SPARK-42569
URL: https://issues.apache.org/jira/browse/SPARK-42569
Project: Spark
Issue Type: Task
Rui Wang created SPARK-42544:
Summary: Spark Connect Scala Client: support parameterized SQL
Key: SPARK-42544
URL: https://issues.apache.org/jira/browse/SPARK-42544
Project: Spark
Issue Type:
Rui Wang created SPARK-42542:
Summary: Support Pivot without providing pivot column values
Key: SPARK-42542
URL: https://issues.apache.org/jira/browse/SPARK-42542
Project: Spark
Issue Type:
Rui Wang created SPARK-42541:
Summary: Support Pivot with provided pivot column values
Key: SPARK-42541
URL: https://issues.apache.org/jira/browse/SPARK-42541
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-42529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated SPARK-42529:
-
Summary: Support Cube and Rollup (was: Support Cube,Rollup,Pivot)
> Support Cube and Rollup
>
Rui Wang created SPARK-42529:
Summary: Support Cube,Rollup,Pivot
Key: SPARK-42529
URL: https://issues.apache.org/jira/browse/SPARK-42529
Project: Spark
Issue Type: Sub-task
Components:
Rui Wang created SPARK-42520:
Summary: Spark Connect Scala Client: Window
Key: SPARK-42520
URL: https://issues.apache.org/jira/browse/SPARK-42520
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-42467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691264#comment-17691264
]
Rui Wang commented on SPARK-42467:
--
Yes we gonna need to support cube/rollup/groupingsets along with
Rui Wang created SPARK-42481:
Summary: Implement agg.{max,min,mean,count,avg,sum}
Key: SPARK-42481
URL: https://issues.apache.org/jira/browse/SPARK-42481
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-42468:
Summary: Implement agg by (String, String)*
Key: SPARK-42468
URL: https://issues.apache.org/jira/browse/SPARK-42468
Project: Spark
Issue Type: Sub-task
Rui Wang created SPARK-42467:
Summary: Spark Connect Scala Client: GroupBy and Aggregation
Key: SPARK-42467
URL: https://issues.apache.org/jira/browse/SPARK-42467
Project: Spark
Issue Type: Task
Rui Wang created SPARK-42453:
Summary: Implement function max in Scala client
Key: SPARK-42453
URL: https://issues.apache.org/jira/browse/SPARK-42453
Project: Spark
Issue Type: Task
Rui Wang created SPARK-42443:
Summary: Remove unused object in DataFrameAggregateSuite
Key: SPARK-42443
URL: https://issues.apache.org/jira/browse/SPARK-42443
Project: Spark
Issue Type: Task
1 - 100 of 300 matches
Mail list logo
