Allison Wang created SPARK-48205:
Summary: Remove the private[sql] modifier for Python data sources
Key: SPARK-48205
URL: https://issues.apache.org/jira/browse/SPARK-48205
Project: Spark
Allison Wang created SPARK-48064:
Summary: Improve error messages for routine related errors
Key: SPARK-48064
URL: https://issues.apache.org/jira/browse/SPARK-48064
Project: Spark
Issue
Allison Wang created SPARK-48014:
Summary: Change the makeFromJava error in EvaluatePython to a
user-facing error
Key: SPARK-48014
URL: https://issues.apache.org/jira/browse/SPARK-48014
Project:
Allison Wang created SPARK-47921:
Summary: Fix ExecuteJobTag creation in ExecuteHolder
Key: SPARK-47921
URL: https://issues.apache.org/jira/browse/SPARK-47921
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-47367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-47367:
-
Summary: Support Python data source API with Spark Connect (was: Support
Python data source
[
https://issues.apache.org/jira/browse/SPARK-47367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-47367:
-
Summary: Support Python data source API in Spark Connect (was: Support
Python data source API
Allison Wang created SPARK-47367:
Summary: Support Python data source API with Spark Connect
Key: SPARK-47367
URL: https://issues.apache.org/jira/browse/SPARK-47367
Project: Spark
Issue
Allison Wang created SPARK-47346:
Summary: Make daemon mode configurable when creating Python workers
Key: SPARK-47346
URL: https://issues.apache.org/jira/browse/SPARK-47346
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-46973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-46973:
-
Description: Improve v2 table lookup performance when a table is already in
the v1 table cache.
[
https://issues.apache.org/jira/browse/SPARK-46973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-46973:
-
Summary: Skip V2 table lookup when a table is in V1 table cache (was: Add
table cache for V2
Allison Wang created SPARK-46973:
Summary: Add table cache for V2 tables
Key: SPARK-46973
URL: https://issues.apache.org/jira/browse/SPARK-46973
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-46818:
Summary: Improve error messages for range with non-foldable input
Key: SPARK-46818
URL: https://issues.apache.org/jira/browse/SPARK-46818
Project: Spark
Allison Wang created SPARK-46618:
Summary: Improve error messages for DATA_SOURCE_NOT_FOUND error
Key: SPARK-46618
URL: https://issues.apache.org/jira/browse/SPARK-46618
Project: Spark
Issue
Allison Wang created SPARK-46616:
Summary: Disallow re-registration of statically registered data
sources
Key: SPARK-46616
URL: https://issues.apache.org/jira/browse/SPARK-46616
Project: Spark
Allison Wang created SPARK-46568:
Summary: Python data source options should be a case insensitive
dictionary
Key: SPARK-46568
URL: https://issues.apache.org/jira/browse/SPARK-46568
Project: Spark
Allison Wang created SPARK-46565:
Summary: Improve Python data source error classes and messages
Key: SPARK-46565
URL: https://issues.apache.org/jira/browse/SPARK-46565
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-46540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-46540:
-
Summary: Respect column names when Python data source read function outputs
named Row objects
Allison Wang created SPARK-46540:
Summary: Respects named arguments when Python data source read
function outputs Row objects
Key: SPARK-46540
URL: https://issues.apache.org/jira/browse/SPARK-46540
[
https://issues.apache.org/jira/browse/SPARK-46540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-46540:
-
Summary: Respect named arguments when Python data source read function
outputs Row objects
Allison Wang created SPARK-46522:
Summary: Block Python data source registration with name conflicts
Key: SPARK-46522
URL: https://issues.apache.org/jira/browse/SPARK-46522
Project: Spark
Allison Wang created SPARK-46520:
Summary: Support overwrite mode for Python data source write
Key: SPARK-46520
URL: https://issues.apache.org/jira/browse/SPARK-46520
Project: Spark
Issue
Allison Wang created SPARK-46452:
Summary: Add a new API in DSv2 DataWriter to write an iterator of
records
Key: SPARK-46452
URL: https://issues.apache.org/jira/browse/SPARK-46452
Project: Spark
Allison Wang created SPARK-46375:
Summary: Add documentation for Python data source API
Key: SPARK-46375
URL: https://issues.apache.org/jira/browse/SPARK-46375
Project: Spark
Issue Type:
Allison Wang created SPARK-46290:
Summary: Change saveMode to overwrite for DataSourceWriter
constructor
Key: SPARK-46290
URL: https://issues.apache.org/jira/browse/SPARK-46290
Project: Spark
Allison Wang created SPARK-46273:
Summary: Support INSERT INTO/OVERWRITE using DSv2 sources
Key: SPARK-46273
URL: https://issues.apache.org/jira/browse/SPARK-46273
Project: Spark
Issue Type:
Allison Wang created SPARK-46272:
Summary: Support CTAS using DSv2 sources
Key: SPARK-46272
URL: https://issues.apache.org/jira/browse/SPARK-46272
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-46253:
Summary: Plan Python data source read using mapInArrow
Key: SPARK-46253
URL: https://issues.apache.org/jira/browse/SPARK-46253
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-46057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-46057:
-
Description: This is an umbrella ticket to support SQL user-defined
functions. (was: This is
Allison Wang created SPARK-46057:
Summary: Support SQL user-defined functions
Key: SPARK-46057
URL: https://issues.apache.org/jira/browse/SPARK-46057
Project: Spark
Issue Type: Umbrella
Allison Wang created SPARK-46043:
Summary: Support create table using DSv2 sources
Key: SPARK-46043
URL: https://issues.apache.org/jira/browse/SPARK-46043
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-46013:
Summary: Improve basic datasource examples
Key: SPARK-46013
URL: https://issues.apache.org/jira/browse/SPARK-46013
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45940:
Summary: Add InputPartition to DataSourceReader interface
Key: SPARK-45940
URL: https://issues.apache.org/jira/browse/SPARK-45940
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-45861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786487#comment-17786487
]
Allison Wang commented on SPARK-45861:
--
[~panbingkun] again, thanks for working on this. Let me
Allison Wang created SPARK-45931:
Summary: Refine docstring of `mapInPandas`
Key: SPARK-45931
URL: https://issues.apache.org/jira/browse/SPARK-45931
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45930:
Summary: Allow non-deterministic Python UDFs in
MapInPandas/MapInArrow
Key: SPARK-45930
URL: https://issues.apache.org/jira/browse/SPARK-45930
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-45927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45927:
-
Summary: Update `path` handling in Python data source (was: Remove `path`
from data source
Allison Wang created SPARK-45927:
Summary: Remove `path` from data source constructor
Key: SPARK-45927
URL: https://issues.apache.org/jira/browse/SPARK-45927
Project: Spark
Issue Type:
Allison Wang created SPARK-45914:
Summary: Support `commit` and `abort` API for Python data source
write
Key: SPARK-45914
URL: https://issues.apache.org/jira/browse/SPARK-45914
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-45525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45525:
-
Description: Add a new command and logical rules (similar to V1Writes and
V2Writes) to support
[
https://issues.apache.org/jira/browse/SPARK-45600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45600:
-
Description: Currently, registered data sources are stored in `sharedState`
and can be accessed
[
https://issues.apache.org/jira/browse/SPARK-45600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45600:
-
Summary: Make data source registration session level (was: Separate the
Python data source
[
https://issues.apache.org/jira/browse/SPARK-45600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45600:
-
Summary: Make Python data source registration session level (was: Make
data source
Allison Wang created SPARK-45865:
Summary: Add user guide for window operations
Key: SPARK-45865
URL: https://issues.apache.org/jira/browse/SPARK-45865
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45864:
Summary: Add user guide for groupby and aggregate
Key: SPARK-45864
URL: https://issues.apache.org/jira/browse/SPARK-45864
Project: Spark
Issue Type:
Allison Wang created SPARK-45863:
Summary: Add user guide for column selections
Key: SPARK-45863
URL: https://issues.apache.org/jira/browse/SPARK-45863
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45862:
Summary: Add user guide for basic dataframe operations
Key: SPARK-45862
URL: https://issues.apache.org/jira/browse/SPARK-45862
Project: Spark
Issue Type:
Allison Wang created SPARK-45861:
Summary: Add user guide for dataframe creation
Key: SPARK-45861
URL: https://issues.apache.org/jira/browse/SPARK-45861
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45783:
Summary: Improve exception message when no remote url is set
Key: SPARK-45783
URL: https://issues.apache.org/jira/browse/SPARK-45783
Project: Spark
Issue
Allison Wang created SPARK-45773:
Summary: Refine docstring of `SparkSession.builder.config`
Key: SPARK-45773
URL: https://issues.apache.org/jira/browse/SPARK-45773
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-45765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45765:
-
Description:
Currently, the error message is super confusing when a user tries to load
[
https://issues.apache.org/jira/browse/SPARK-45765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang resolved SPARK-45765.
--
Resolution: Invalid
> Improve error messages when loading multiple paths in PySpark
>
Allison Wang created SPARK-45765:
Summary: Improve error messages when loading multiple paths in
PySpark
Key: SPARK-45765
URL: https://issues.apache.org/jira/browse/SPARK-45765
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-45764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45764:
-
Description:
We should consider adding a copy button next to the pyspark code blocks.
For
[
https://issues.apache.org/jira/browse/SPARK-45764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781887#comment-17781887
]
Allison Wang commented on SPARK-45764:
--
cc [~podongfeng] WDYT?
> Make code block copyable
>
Allison Wang created SPARK-45764:
Summary: Make code block copyable
Key: SPARK-45764
URL: https://issues.apache.org/jira/browse/SPARK-45764
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-45713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45713:
-
Description:
Support registering Python data sources.
Users can register a Python data source
[
https://issues.apache.org/jira/browse/SPARK-45713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45713:
-
Description:
Support registering Python data sources.
Users can register a Python data source
Allison Wang created SPARK-45713:
Summary: Support registering Python data sources
Key: SPARK-45713
URL: https://issues.apache.org/jira/browse/SPARK-45713
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45639:
-
Description:
Allow users to read from a Python data source using
[
https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45639:
-
Summary: Support loading Python data sources in DataFrameReader (was:
Support Python data
Allison Wang created SPARK-45654:
Summary: Add Python data source write API
Key: SPARK-45654
URL: https://issues.apache.org/jira/browse/SPARK-45654
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45639:
-
Description:
Allow users to read from a Python data source using
[
https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45639:
-
Description:
Allow users to read from a Python data source using
[
https://issues.apache.org/jira/browse/SPARK-45639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45639:
-
Description:
Allow users to read from a Python data source using
Allison Wang created SPARK-45639:
Summary: Support Python data source in DataFrameReader
Key: SPARK-45639
URL: https://issues.apache.org/jira/browse/SPARK-45639
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-45524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45524:
-
Description:
Add API for data source and data source reader and add Catalyst + execution
[
https://issues.apache.org/jira/browse/SPARK-45023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1954#comment-1954
]
Allison Wang commented on SPARK-45023:
--
[~abhinavofficial] this proposal is on hold, given the
[
https://issues.apache.org/jira/browse/SPARK-45023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang resolved SPARK-45023.
--
Resolution: Won't Do
> SPIP: Python Stored Procedures
> --
>
>
Allison Wang created SPARK-45600:
Summary: Separate the Python data source logic from DataFrameReader
Key: SPARK-45600
URL: https://issues.apache.org/jira/browse/SPARK-45600
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-45559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45559:
-
Description:
Support `spark.read.schema(...)` for Python data source read.
Add test cases
Allison Wang created SPARK-45597:
Summary: Support creating table using a Python data source in SQL
Key: SPARK-45597
URL: https://issues.apache.org/jira/browse/SPARK-45597
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-45584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45584:
-
Description:
When there are subqueries in TakeOrderedAndProjectExec, the query can throw
this
Allison Wang created SPARK-45584:
Summary: Execution fails when there are subqueries in
TakeOrderedAndProjectExec
Key: SPARK-45584
URL: https://issues.apache.org/jira/browse/SPARK-45584
Project:
[
https://issues.apache.org/jira/browse/SPARK-45560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45560:
-
Summary: Support spark.read.load() with non-empty path for Python data
source API (was:
[
https://issues.apache.org/jira/browse/SPARK-45559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45559:
-
Summary: Support spark.read.schema(...) for Python data source API (was:
Support
Allison Wang created SPARK-45560:
Summary: Support spark.read.load() with paths for Python data
source API
Key: SPARK-45560
URL: https://issues.apache.org/jira/browse/SPARK-45560
Project: Spark
Allison Wang created SPARK-45559:
Summary: Support df.read.schema(...) for Python data source API
Key: SPARK-45559
URL: https://issues.apache.org/jira/browse/SPARK-45559
Project: Spark
Issue
Allison Wang created SPARK-45526:
Summary: Refine docstring of `options` for dataframe reader and
writer
Key: SPARK-45526
URL: https://issues.apache.org/jira/browse/SPARK-45526
Project: Spark
Allison Wang created SPARK-45525:
Summary: Initial support for Python data source write API
Key: SPARK-45525
URL: https://issues.apache.org/jira/browse/SPARK-45525
Project: Spark
Issue Type:
Allison Wang created SPARK-45524:
Summary: Initial support for Python data source read API
Key: SPARK-45524
URL: https://issues.apache.org/jira/browse/SPARK-45524
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45509:
-
Description:
SPARK-45220 discovers a behavior difference for a self-join scenario between
[
https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45509:
-
Description:
SPARK-45220 discovers a behavior difference for a self-join scenario between
[
https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45509:
-
Description:
SPARK-45220 discovers a behavior difference for a self-join scenario between
Allison Wang created SPARK-45509:
Summary: Investigate the behavior difference in self-join
Key: SPARK-45509
URL: https://issues.apache.org/jira/browse/SPARK-45509
Project: Spark
Issue Type:
Allison Wang created SPARK-45505:
Summary: Refactor analyzeInPython function to make it reusable
Key: SPARK-45505
URL: https://issues.apache.org/jira/browse/SPARK-45505
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-44076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-44076:
-
Affects Version/s: 4.0.0
(was: 3.5.0)
> SPIP: Python Data Source API
Allison Wang created SPARK-45442:
Summary: Refine docstring of `DataFrame.show`
Key: SPARK-45442
URL: https://issues.apache.org/jira/browse/SPARK-45442
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45428:
Summary: Add Matomo analytics to all released docs pages
Key: SPARK-45428
URL: https://issues.apache.org/jira/browse/SPARK-45428
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772395#comment-17772395
]
Allison Wang commented on SPARK-45428:
--
cc [~podongfeng]
> Add Matomo analytics to all released
[
https://issues.apache.org/jira/browse/SPARK-44729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-44729:
-
Description:
We should add the canonical link to the PySpark docs page
[
https://issues.apache.org/jira/browse/SPARK-45264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767743#comment-17767743
]
Allison Wang commented on SPARK-45264:
--
[~podongfeng] do we have ways to bypass such pandas version
Allison Wang created SPARK-45264:
Summary: Configurable error when generating Python docs
Key: SPARK-45264
URL: https://issues.apache.org/jira/browse/SPARK-45264
Project: Spark
Issue Type:
Allison Wang created SPARK-45260:
Summary: Refine docstring of count_distinct
Key: SPARK-45260
URL: https://issues.apache.org/jira/browse/SPARK-45260
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45259:
Summary: Refine docstring of `count`
Key: SPARK-45259
URL: https://issues.apache.org/jira/browse/SPARK-45259
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45258:
Summary: Refine docstring of `sum`
Key: SPARK-45258
URL: https://issues.apache.org/jira/browse/SPARK-45258
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-45220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-45220:
-
Description:
Refine the docstring of `DataFrame.join`.
The examples should also include: left
Allison Wang created SPARK-45223:
Summary: Refine docstring of `Column.when`
Key: SPARK-45223
URL: https://issues.apache.org/jira/browse/SPARK-45223
Project: Spark
Issue Type: Sub-task
Allison Wang created SPARK-45222:
Summary: Refine docstring of `DataFrameReader.json`
Key: SPARK-45222
URL: https://issues.apache.org/jira/browse/SPARK-45222
Project: Spark
Issue Type:
Allison Wang created SPARK-45221:
Summary: Refine docstring of `DataFrameReader.parquet`
Key: SPARK-45221
URL: https://issues.apache.org/jira/browse/SPARK-45221
Project: Spark
Issue Type:
Allison Wang created SPARK-45220:
Summary: Refine docstring of `DataFrame.join`
Key: SPARK-45220
URL: https://issues.apache.org/jira/browse/SPARK-45220
Project: Spark
Issue Type: Sub-task
1 - 100 of 393 matches
Mail list logo
