yuxiqian commented on code in PR #4520:
URL: https://github.com/apache/flink-cdc/pull/4520#discussion_r3910317397
##########
docs/content.zh/docs/core-concept/ai-model.md:
##########
@@ -43,13 +43,24 @@ AI 模型可用于 transform 表达式中的文本生成、文本分析和 embed
| `AI_MASK(model, input, entities)` | 对指定实体类型进行脱敏。 |
`masked_text`、`detected_entities` |
| `AI_EMBED(model, input)` | 生成 embedding 向量。 | 不返回 JSON，而是返回 `ARRAY<FLOAT>`。 |
+以下多模态函数从 `BYTES` 字段读取图片数据：
+
+| 函数 | 说明 | 返回类型 |
+|------|------|----------|
+| `AI_IMAGE_COMPLETE(model, image, prompt)` | 根据图片和自然语言 prompt 生成文本。 |
`STRING` |
+| `AI_IMAGE_EMBED(model, image)` | 将图片转换为 embedding 向量。 | `ARRAY<FLOAT>` |
+
+OpenAI-compatible 模型客户端通过标准 vision chat 支持 `AI_IMAGE_COMPLETE`。客户端会识别
PNG、JPEG、GIF 和 WebP 图片，并将图片编码为 Base64 data URL。图片为 `NULL` 时直接返回
`NULL`，且不会调用模型；图片为空或格式无法识别时，会在发送请求前报错。
+
+`AI_IMAGE_EMBED` 当前只提供框架函数和 provider capability。OpenAI-compatible 模型客户端不实现图片
embedding，社区发行包目前也没有可用于生产的图片 embedding provider。需要图片向量化的用户需要等待后续 provider 实现。
Review Comment:
Shall we mention that OpenAI API does not provide standard image embedding
protocols here?
##########
flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/parser/TransformParser.java:
##########
Review Comment:
Seems this method requires model name string to be a compile-time literal
string. Besides introducing more complexity in `JaninoCompiler`, It forbids the
following use case:
```sql
AI_COMPLETE(
IF(important_job, 'ultimate_model', 'cheap_model'), ...
)
```
Do you think it's a good idea to postpone the capability check from SQL
compile time (the first we met `CreateTableEvent`) to evaluation time (first
`DataChangeEvent`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
