Martin Grund created SPARK-47862:
Summary: Connect generated proots can't be pickled
Key: SPARK-47862
URL: https://issues.apache.org/jira/browse/SPARK-47862
Project: Spark
Issue Type:
Martin Grund created SPARK-47812:
Summary: Support Serializing Spark Sessions in ForEachBAtch
Key: SPARK-47812
URL: https://issues.apache.org/jira/browse/SPARK-47812
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-47336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827077#comment-17827077
]
Martin Grund commented on SPARK-47336:
--
I think the general idea is great! I would like to propose
Martin Grund created SPARK-47227:
Summary: Spark Connect Documentation
Key: SPARK-47227
URL: https://issues.apache.org/jira/browse/SPARK-47227
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-47081:
Summary: Support Query Execution Progress Messages
Key: SPARK-47081
URL: https://issues.apache.org/jira/browse/SPARK-47081
Project: Spark
Issue Type:
Martin Grund created SPARK-45852:
Summary: Gracefully deal with recursion exception during Spark
Connect logging
Key: SPARK-45852
URL: https://issues.apache.org/jira/browse/SPARK-45852
Project: Spark
Martin Grund created SPARK-45808:
Summary: Improve error details for Spark Connect Client in Python
Key: SPARK-45808
URL: https://issues.apache.org/jira/browse/SPARK-45808
Project: Spark
Martin Grund created SPARK-45798:
Summary: Assert server-side session ID in Spark Connect
Key: SPARK-45798
URL: https://issues.apache.org/jira/browse/SPARK-45798
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-45167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Grund updated SPARK-45167:
-
Issue Type: Bug (was: Improvement)
> Python Spark Connect client does not call `releaseAll`
>
Martin Grund created SPARK-45167:
Summary: Python Spark Connect client does not call `releaseAll`
Key: SPARK-45167
URL: https://issues.apache.org/jira/browse/SPARK-45167
Project: Spark
Issue
Martin Grund created SPARK-45048:
Summary: Add additional tests for Python client
Key: SPARK-45048
URL: https://issues.apache.org/jira/browse/SPARK-45048
Project: Spark
Issue Type:
Martin Grund created SPARK-44931:
Summary: Fix JSON Serailization for Spark Connect Event Listener
Key: SPARK-44931
URL: https://issues.apache.org/jira/browse/SPARK-44931
Project: Spark
Martin Grund created SPARK-44815:
Summary: Cache Schema of DF
Key: SPARK-44815
URL: https://issues.apache.org/jira/browse/SPARK-44815
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-44814:
Summary: Test to trigger protobuf 4.23.3 crash
Key: SPARK-44814
URL: https://issues.apache.org/jira/browse/SPARK-44814
Project: Spark
Issue Type:
Martin Grund created SPARK-44740:
Summary: Allow configuring the session ID for a spark connect
client in the remote string
Key: SPARK-44740
URL: https://issues.apache.org/jira/browse/SPARK-44740
Martin Grund created SPARK-44738:
Summary: Spark Connect Reattach misses metadata propagation
Key: SPARK-44738
URL: https://issues.apache.org/jira/browse/SPARK-44738
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-44528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Grund updated SPARK-44528:
-
Summary: Spark Connect DataFrame does not allow to add custom instance
attributes and check for
[
https://issues.apache.org/jira/browse/SPARK-44528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Grund updated SPARK-44528:
-
Description:
```
df = spark.range(10)
df._test = 10
assert(hasattr(df, "_test"))
Martin Grund created SPARK-44528:
Summary: Spark Connect DataFrame does not allow to add custom
instance attributes
Key: SPARK-44528
URL: https://issues.apache.org/jira/browse/SPARK-44528
Project:
[
https://issues.apache.org/jira/browse/SPARK-44528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Grund updated SPARK-44528:
-
Description:
```
df = spark.range(10)
df._test = 10
```
Treats `df._test` like a column
Martin Grund created SPARK-44505:
Summary: DataSource v2 Scans should not require planning the input
partitions on explain
Key: SPARK-44505
URL: https://issues.apache.org/jira/browse/SPARK-44505
Martin Grund created SPARK-43958:
Summary: Channel Builder for Golang client
Key: SPARK-43958
URL: https://issues.apache.org/jira/browse/SPARK-43958
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-43909:
Summary: Golang Repository Workflows
Key: SPARK-43909
URL: https://issues.apache.org/jira/browse/SPARK-43909
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-43895:
Summary: Skeleton Golang Repository
Key: SPARK-43895
URL: https://issues.apache.org/jira/browse/SPARK-43895
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-43894:
Summary: df.cache() not working
Key: SPARK-43894
URL: https://issues.apache.org/jira/browse/SPARK-43894
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-43509:
Summary: Support creating multiple sessions for Spark Connect in
PySpark
Key: SPARK-43509
URL: https://issues.apache.org/jira/browse/SPARK-43509
Project: Spark
Martin Grund created SPARK-43430:
Summary: ExecutePlanRequest should have the ability to set request
options.
Key: SPARK-43430
URL: https://issues.apache.org/jira/browse/SPARK-43430
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720864#comment-17720864
]
Martin Grund commented on SPARK-43351:
--
1.1 we had the same questions with Python and Scala and
Martin Grund created SPARK-43332:
Summary: Allow ChannelBuilder extensions
Key: SPARK-43332
URL: https://issues.apache.org/jira/browse/SPARK-43332
Project: Spark
Issue Type: Improvement
Martin Grund created SPARK-43249:
Summary: df.sql() should send metrics back()
Key: SPARK-43249
URL: https://issues.apache.org/jira/browse/SPARK-43249
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707580#comment-17707580
]
Martin Grund commented on SPARK-41628:
--
I think this needs a bit more discussion. Generally, I
Martin Grund created SPARK-42853:
Summary: Update the Spark Doc to match the new website style
Key: SPARK-42853
URL: https://issues.apache.org/jira/browse/SPARK-42853
Project: Spark
Issue
Martin Grund created SPARK-42816:
Summary: Increase max message size to 128MB
Key: SPARK-42816
URL: https://issues.apache.org/jira/browse/SPARK-42816
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-42733:
Summary: df.write.format().save() should support calling with no
path or table name
Key: SPARK-42733
URL: https://issues.apache.org/jira/browse/SPARK-42733
Project:
[
https://issues.apache.org/jira/browse/SPARK-42374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688394#comment-17688394
]
Martin Grund commented on SPARK-42374:
--
Yes, that is correct. There is not built-in authentication.
[
https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688393#comment-17688393
]
Martin Grund commented on SPARK-39375:
--
[~tgraves] Currently the Python UDFs are implemented
Martin Grund created SPARK-42156:
Summary: Support client-side retries in Spark Connect Python client
Key: SPARK-42156
URL: https://issues.apache.org/jira/browse/SPARK-42156
Project: Spark
Martin Grund created SPARK-42029:
Summary: Distribution build for Spark Connect does not work with
Spark Shell
Key: SPARK-42029
URL: https://issues.apache.org/jira/browse/SPARK-42029
Project: Spark
Martin Grund created SPARK-42028:
Summary: Support Pandas DF to Spark DF with Nanosecond Timestamps
Key: SPARK-42028
URL: https://issues.apache.org/jira/browse/SPARK-42028
Project: Spark
Martin Grund created SPARK-42027:
Summary: CreateDataframe from Pandas with Struct and Timestamp
Key: SPARK-42027
URL: https://issues.apache.org/jira/browse/SPARK-42027
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-41919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655346#comment-17655346
]
Martin Grund commented on SPARK-41919:
--
There is a compatible way of doing this by adding another
[
https://issues.apache.org/jira/browse/SPARK-41918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655345#comment-17655345
]
Martin Grund commented on SPARK-41918:
--
Renaming fields is WIRE compatible and most likely this is
[
https://issues.apache.org/jira/browse/SPARK-41911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655343#comment-17655343
]
Martin Grund commented on SPARK-41911:
--
I think the first part would be to identify which messages
[
https://issues.apache.org/jira/browse/SPARK-41910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655336#comment-17655336
]
Martin Grund commented on SPARK-41910:
--
Didn't we just have the discussion on why we wanted to use
[
https://issues.apache.org/jira/browse/SPARK-41755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655334#comment-17655334
]
Martin Grund commented on SPARK-41755:
--
While not obvious, this is not needed. We can just add a
[
https://issues.apache.org/jira/browse/SPARK-41812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653649#comment-17653649
]
Martin Grund commented on SPARK-41812:
--
On the Spark side we eagerly resolve the column and return
[
https://issues.apache.org/jira/browse/SPARK-41815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653643#comment-17653643
]
Martin Grund edited comment on SPARK-41815 at 1/2/23 3:56 PM:
--
The reason
[
https://issues.apache.org/jira/browse/SPARK-41815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653643#comment-17653643
]
Martin Grund commented on SPARK-41815:
--
The reason seems to be that when Spark Connect serializes
Martin Grund created SPARK-41803:
Summary: log() function variations are missing
Key: SPARK-41803
URL: https://issues.apache.org/jira/browse/SPARK-41803
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17652936#comment-17652936
]
Martin Grund commented on SPARK-41743:
--
Running the doc tests actually passes for me.
>
[
https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17652934#comment-17652934
]
Martin Grund edited comment on SPARK-41743 at 12/29/22 8:24 PM:
In the
[
https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17652934#comment-17652934
]
Martin Grund commented on SPARK-41743:
--
In the following example, I cannot reproduce this:
```
df
Martin Grund created SPARK-41738:
Summary: Client ID should be mixed into SparkSession cache
Key: SPARK-41738
URL: https://issues.apache.org/jira/browse/SPARK-41738
Project: Spark
Issue
Martin Grund created SPARK-41664:
Summary: Support streaming client data to create large DataFrames
Key: SPARK-41664
URL: https://issues.apache.org/jira/browse/SPARK-41664
Project: Spark
Martin Grund created SPARK-41662:
Summary: Minimal support for pickled Python UDFs
Key: SPARK-41662
URL: https://issues.apache.org/jira/browse/SPARK-41662
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41661:
Summary: Support for Python UDFs
Key: SPARK-41661
URL: https://issues.apache.org/jira/browse/SPARK-41661
Project: Spark
Issue Type: Umbrella
Martin Grund created SPARK-41629:
Summary: Support for protocol extensions
Key: SPARK-41629
URL: https://issues.apache.org/jira/browse/SPARK-41629
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-41625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Grund updated SPARK-41625:
-
Description: We need to design how support for structured streaming will
look like in Spark
Martin Grund created SPARK-41628:
Summary: Support async query execution
Key: SPARK-41628
URL: https://issues.apache.org/jira/browse/SPARK-41628
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41627:
Summary: Spark Connect Server Development
Key: SPARK-41627
URL: https://issues.apache.org/jira/browse/SPARK-41627
Project: Spark
Issue Type: Umbrella
Martin Grund created SPARK-41626:
Summary: Document validation choices for local and remote input
validation
Key: SPARK-41626
URL: https://issues.apache.org/jira/browse/SPARK-41626
Project: Spark
Martin Grund created SPARK-41625:
Summary: Feature parity: Streaming support
Key: SPARK-41625
URL: https://issues.apache.org/jira/browse/SPARK-41625
Project: Spark
Issue Type: Umbrella
Martin Grund created SPARK-41624:
Summary: Support Python logging
Key: SPARK-41624
URL: https://issues.apache.org/jira/browse/SPARK-41624
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41623:
Summary: Support Catalog.uncacheTable
Key: SPARK-41623
URL: https://issues.apache.org/jira/browse/SPARK-41623
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41620:
Summary: Support Catalog.registerFunction
Key: SPARK-41620
URL: https://issues.apache.org/jira/browse/SPARK-41620
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41619:
Summary: Support Catalog.refreshTable
Key: SPARK-41619
URL: https://issues.apache.org/jira/browse/SPARK-41619
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41622:
Summary: Support Catalog.setCurrentDatabase
Key: SPARK-41622
URL: https://issues.apache.org/jira/browse/SPARK-41622
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41621:
Summary: Support Catalog.setCurrentCatalog
Key: SPARK-41621
URL: https://issues.apache.org/jira/browse/SPARK-41621
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41618:
Summary: Support Catalog.recoverPartitions
Key: SPARK-41618
URL: https://issues.apache.org/jira/browse/SPARK-41618
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41617:
Summary: Support Catalog.listTables
Key: SPARK-41617
URL: https://issues.apache.org/jira/browse/SPARK-41617
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41616:
Summary: Support Catalog.listFunctions
Key: SPARK-41616
URL: https://issues.apache.org/jira/browse/SPARK-41616
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41614:
Summary: Support Catalog.listColumns
Key: SPARK-41614
URL: https://issues.apache.org/jira/browse/SPARK-41614
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41615:
Summary: Support Catalog.listDatabases
Key: SPARK-41615
URL: https://issues.apache.org/jira/browse/SPARK-41615
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41610:
Summary: Support Catalog.getFunction
Key: SPARK-41610
URL: https://issues.apache.org/jira/browse/SPARK-41610
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41612:
Summary: Support Catalog.isCached
Key: SPARK-41612
URL: https://issues.apache.org/jira/browse/SPARK-41612
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41613:
Summary: Support Catalog.listCatalogs
Key: SPARK-41613
URL: https://issues.apache.org/jira/browse/SPARK-41613
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41611:
Summary: Support Catalog.getTable
Key: SPARK-41611
URL: https://issues.apache.org/jira/browse/SPARK-41611
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41608:
Summary: Support Catalog.functionExists
Key: SPARK-41608
URL: https://issues.apache.org/jira/browse/SPARK-41608
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41609:
Summary: Support Catalog.getDatabase
Key: SPARK-41609
URL: https://issues.apache.org/jira/browse/SPARK-41609
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41607:
Summary: Support Catalog.dropTempView
Key: SPARK-41607
URL: https://issues.apache.org/jira/browse/SPARK-41607
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41604:
Summary: Support Catalog.currentCatalog
Key: SPARK-41604
URL: https://issues.apache.org/jira/browse/SPARK-41604
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41606:
Summary: Support Catalog dropGlobalTempView
Key: SPARK-41606
URL: https://issues.apache.org/jira/browse/SPARK-41606
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41605:
Summary: Support Catalog.currentDatabase
Key: SPARK-41605
URL: https://issues.apache.org/jira/browse/SPARK-41605
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41602:
Summary: Support Catalog.createExternalTable
Key: SPARK-41602
URL: https://issues.apache.org/jira/browse/SPARK-41602
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41603:
Summary: Support Catalog.createTable
Key: SPARK-41603
URL: https://issues.apache.org/jira/browse/SPARK-41603
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41601:
Summary: Support Catalog.clearCache
Key: SPARK-41601
URL: https://issues.apache.org/jira/browse/SPARK-41601
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41600:
Summary: Support Catalog.cacheTable
Key: SPARK-41600
URL: https://issues.apache.org/jira/browse/SPARK-41600
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41560:
Summary: Document how to add new functions
Key: SPARK-41560
URL: https://issues.apache.org/jira/browse/SPARK-41560
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41537:
Summary: Protobuf backwards compatibility testing
Key: SPARK-41537
URL: https://issues.apache.org/jira/browse/SPARK-41537
Project: Spark
Issue Type:
Martin Grund created SPARK-41533:
Summary: GRPC Errors on the client should be cleaned up
Key: SPARK-41533
URL: https://issues.apache.org/jira/browse/SPARK-41533
Project: Spark
Issue Type:
Martin Grund created SPARK-41532:
Summary: DF operations that involve multiple data frames should
fail if sessions don't match
Key: SPARK-41532
URL: https://issues.apache.org/jira/browse/SPARK-41532
Martin Grund created SPARK-41531:
Summary: Debugging and Stability
Key: SPARK-41531
URL: https://issues.apache.org/jira/browse/SPARK-41531
Project: Spark
Issue Type: Umbrella
Martin Grund created SPARK-41366:
Summary: DF.groupby.agg() API should be compatible
Key: SPARK-41366
URL: https://issues.apache.org/jira/browse/SPARK-41366
Project: Spark
Issue Type:
Martin Grund created SPARK-41362:
Summary: Better type errors when passing wrong parameters
Key: SPARK-41362
URL: https://issues.apache.org/jira/browse/SPARK-41362
Project: Spark
Issue Type:
Martin Grund created SPARK-41351:
Summary: Column does not support !=
Key: SPARK-41351
URL: https://issues.apache.org/jira/browse/SPARK-41351
Project: Spark
Issue Type: Sub-task
Martin Grund created SPARK-41326:
Summary: Bug in Deduplicate Python transformation
Key: SPARK-41326
URL: https://issues.apache.org/jira/browse/SPARK-41326
Project: Spark
Issue Type:
Martin Grund created SPARK-41325:
Summary: Add missing avg() to DF group
Key: SPARK-41325
URL: https://issues.apache.org/jira/browse/SPARK-41325
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-41297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Grund reassigned SPARK-41297:
Assignee: Martin Grund
> Support string sql expressions in DF.where()
>
Martin Grund created SPARK-41301:
Summary: SparkSession.range should treat end as optional
Key: SPARK-41301
URL: https://issues.apache.org/jira/browse/SPARK-41301
Project: Spark
Issue Type:
Martin Grund created SPARK-41300:
Summary: Unset Read.schema is incorrectly read when unset
Key: SPARK-41300
URL: https://issues.apache.org/jira/browse/SPARK-41300
Project: Spark
Issue Type:
1 - 100 of 149 matches
Mail list logo