[ 
https://issues.apache.org/jira/browse/SPARK-61404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-61404:
-----------------------------------
    Labels: pull-request-available  (was: )

> Add ai_query for remote model inference in PySpark
> --------------------------------------------------
>
>                 Key: SPARK-61404
>                 URL: https://issues.apache.org/jira/browse/SPARK-61404
>             Project: Spark
>          Issue Type: New Feature
>          Components: PySpark
>    Affects Versions: 5.0.0
>            Reporter: Alejandro Morgante
>            Priority: Major
>              Labels: pull-request-available
>
> Add a PySpark column function that sends a prompt to a registered model and 
> returns the response as text. This lets DataFrame transformations summarize 
> or classify text using the same expression with Databricks Model Serving or 
> Amazon Bedrock.
> Register a Databricks endpoint and use its alias to summarize each document:
> {code:python}
> from pyspark.sql.ai import ai_query, register_model
> from pyspark.sql.functions import col, concat, lit
> register_model(
>     "summarizer",
>     "databricks-gpt-oss-20b",
>     provider="databricks",
>     profile="my-workspace",
>     auth_type="databricks-cli",
> )
> df.withColumn(
>     "summary",
>     ai_query(
>         endpoint="summarizer",
>         request=concat(lit("Summarize in one sentence: "), 
> col("description")),
>     ),
> )
> {code}
> The draft uses an iterator Pandas UDF and resolves credentials on the 
> workers. A null prompt yields null. Spark task retries and repeated actions 
> can make additional billable model calls.
> The prototype passed end-to-end tests on local Spark 4.3 with Arrow against 
> Databricks Model Serving (GPT OSS 20B) and Amazon Bedrock Converse (Nova 
> Micro).
> [Draft PR #60311|https://github.com/apache/spark/pull/60311]. Feedback is 
> welcome on the API and the scope of the first version.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to