shuaijinchao opened a new issue #5636: URL: https://github.com/apache/apisix/issues/5636
### Issue description Splunk is a machine data search engine. According to the latest database search engine ranking [1], Splunk has been ranked second, and Splunk can be used to collect, index and retrieve various application data. Splunk, like ElasticSearch, is quasi-real-time and can provide an uninterrupted data stream of search results. I think Apache APISIX can provide a plugin that supports pushing request logs to Splunk, expands the data observability of APISIX, and reduces the cost of Splunk users using APISIX as a gateway. ### Name - splunk-hec-logging ### Configuration ```json { "endpoint":{ "uri":"https://hec-splunk.company.com/services/collector", "token":"BD274822-96AA-4DA6-90EC-18940FB2414C", "channel":"FE0ECFAD-13D5-401B-847D-77833BD77131", "ssl":true }, "inactive_timeout":10, "max_retry_count":0, "buffer_duration":60, "retry_delay":1, "batch_max_size":1 } ``` - `endpoint` Splunk HTTP Event Collector(HEC) endpoint - `endpoint.uri` Splunk HTTP Event Collector(HEC) endpoint request uri - `endpoint.token` Splunk HTTP Event Collector(HEC) endpoint request access token [2] - `endpoint.channel` Splunk HTTP Event Collector(HEC) endpoint channel identifier (GUID) [3] - `endpoint.ssl` Enable SSL Verify - `max_retry_count` max number of retries before removing from the processing pipe line - `retry_delay` number of seconds the process execution should be delayed if the execution fails - `buffer_duration` max age in seconds of the oldest entry in a batch before the batch must be processed - `inactive_timeout` max age in seconds when the buffer will be flushed if inactive - `batch_max_size` max size of each batch ### Details #### Configuration process 1. Add and set up the Http Event Controller (HEC) through the Splunk console and get the access token. 2. If HEC enable the indexer to confirm, you must specify a channel and obtain the channel ID. 3. Set the request URI, access token, and channel ID of HEC to the plugin configuration #### HTTP Request process 1. Obtain and assemble request information data in the APISIX Log stage, data format refer to [4] 2. Add the assembled request data to the batch queue 3. When the threshold of the batch queue is triggered, the requested data is submitted in batches to Splunk's HEC [1] https://db-engines.com/en/ranking/search+engine [2] https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector#Create_an_Event_Collector_token_on_Splunk_Enterprise [3] https://docs.splunk.com/Documentation/Splunk/latest/Data/AboutHECIDXAck#About_channels_and_sending_data [4] https://docs.splunk.com/Documentation/Splunk/latest/Data/FormateventsforHTTPEventCollector#Event_metadata ### Environment - apisix version (cmd: `apisix version`): - OS (cmd: `uname -a`): - OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`): - etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API): - apisix-dashboard version, if have: - the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner): - luarocks version, if the issue is about installation (cmd: `luarocks --version`): -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@apisix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org