[
https://issues.apache.org/jira/browse/SPARK-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ke Li updated SPARK-28864:
--
Description:
Alibaba Log Service is a big data service which has been widely used in Alibaba
Group and thousands of customers of Alibaba Cloud. The core storage engine of
Log Service is named Loghub which is a large scale distributed storage system
which provides producer and consumer to push and pull data like Kafka, AWS
Kinesis and Azure Eventhub does.
Log Service provides a complete solution to help user collect data from both on
premise and cloud data sources. More than 10 PB data is sent to and consumed
from Loghub every day. And hundreds of thousands of users of Alibaba are
working with Log Service for building their DevOPS and big data system. A large
part of these users of Log Service are using Spark Streaming, Spark SQL and
Spark Structured Streaming as well.
Happy to hear any comments.
was:
Alibaba Log Service is a big data service which has been widely used in Alibaba
Group and thousands of customers of Alibaba Cloud. The core storage engine of
Log Service is named Loghub which is a large scale distributed storage system
which provides producer and consumer to push and pull data like Kafka, AWS
Kinesis and Azure Eventhub does.
There are a lot of users of Log Service are using Spark Streaming, Spark SQL
and Spark Structured Streaming to analysis data collected from both on premise
and cloud data sources.
Happy to hear any comments.
> Add spark connector for Alibaba Log Service
> ---
>
> Key: SPARK-28864
> URL: https://issues.apache.org/jira/browse/SPARK-28864
> Project: Spark
> Issue Type: New Feature
> Components: Input/Output
>Affects Versions: 3.0.0
>Reporter: Ke Li
>Priority: Major
>
> Alibaba Log Service is a big data service which has been widely used in
> Alibaba Group and thousands of customers of Alibaba Cloud. The core storage
> engine of Log Service is named Loghub which is a large scale distributed
> storage system which provides producer and consumer to push and pull data
> like Kafka, AWS Kinesis and Azure Eventhub does.
> Log Service provides a complete solution to help user collect data from both
> on premise and cloud data sources. More than 10 PB data is sent to and
> consumed from Loghub every day. And hundreds of thousands of users of Alibaba
> are working with Log Service for building their DevOPS and big data system. A
> large part of these users of Log Service are using Spark Streaming, Spark SQL
> and Spark Structured Streaming as well.
> Happy to hear any comments.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
