This is an automated email from the ASF dual-hosted git repository.
bhavanisudha pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c8dfba84 [HUDI-2830] Docs for Commit Notifications (#4095)
c8dfba84 is described below
commit c8dfba84c651f078032f0ee28123cad20df5d8ea
Author: Kyle Weller <kywe...@gmail.com>
AuthorDate: Wed Nov 24 20:56:42 2021 -0700
[HUDI-2830] Docs for Commit Notifications (#4095)
* setup initial page
* outline for doc complete
* placed the commit_notifications as a sub-head in the writing data page
---
website/docs/writing_data.md | 29 ++++++++++++++++++++++++++++-
1 file changed, 28 insertions(+), 1 deletion(-)
diff --git a/website/docs/writing_data.md b/website/docs/writing_data.md
index 575bafc..2a103bf 100644
--- a/website/docs/writing_data.md
+++ b/website/docs/writing_data.md
@@ -338,7 +338,34 @@ inputDF.write.format("hudi")
.save(basePath)
```
-
+### Commit Notifications
+Apache Hudi provides the ability to post a callback notification about a write
commit. This may be valuable if you need
+an event notification stream to take actions with other services after a Hudi
write commit.
+You can push a write commit callback notification into HTTP endpoints or to a
Kafka server.
+
+#### HTTP Endpoints
+You can push a commit notification to an HTTP URL and can specify custom
values by extending a callback class defined below.
+
+| Config | Description | Required | Default |
+| ----------- | ------- | ------- | ------ |
+| TURN_CALLBACK_ON | Turn commit callback on/off | optional | false
(*callbacks off*) |
+| CALLBACK_HTTP_URL | Callback host to be sent along with callback messages |
required | N/A |
+| CALLBACK_HTTP_TIMEOUT_IN_SECONDS | Callback timeout in seconds | optional |
3 |
+| CALLBACK_CLASS_NAME | Full path of callback class and must be a subclass of
HoodieWriteCommitCallback class,
org.apache.hudi.callback.impl.HoodieWriteCommitHttpCallback by default |
optional | org.apache.hudi.callback.impl.HoodieWriteCommitHttpCallback |
+| CALLBACK_HTTP_API_KEY_VALUE | Http callback API key | optional |
hudi_write_commit_http_callback |
+| | | | |
+
+#### Kafka Endpoints
+You can push a commit notification to a Kafka topic so it can be used by other
real time systems.
+
+| Config | Description | Required | Default |
+| ----------- | ------- | ------- | ------ |
+| TOPIC | Kafka topic name to publish timeline activity into. | required | N/A
|
+| PARTITION | It may be desirable to serialize all changes into a single Kafka
partition for providing strict ordering. By default, Kafka messages are keyed
by table name, which guarantees ordering at the table level, but not globally
(or when new partitions are added) | required | N/A |
+| RETRIES | Times to retry the produce | optional | 3 |
+| ACKS | kafka acks level, all by default to ensure strong durability |
optional | all |
+| BOOTSTRAP_SERVERS | Bootstrap servers of kafka cluster, to be used for
publishing commit metadata | required | N/A |
+| | | | |
## Flink SQL Writer
The hudi-flink module defines the Flink SQL connector for both hudi source and
sink.
