Re: GitHub is out of order

Yea.. please ignore my duplicated comments if they exist. I didn't know it's globally happening but I thought a problem specific to me so I left duplicated comments multiple times. 2018년 10월 22일 (월) 오후 12:40, Dongjoon Hyun 님이 작성: > Hi, All. > > Currently, GitHub is out of order. Apache Spark

GitHub is out of order

Hi, All. Currently, GitHub is out of order. Apache Spark repo is also affected. Newly filed pull requests to Apache Spark repository seem to disappear repeatedly, too. https://status.github.com/messages Bests, Dongjoon.

Re: queryable state & streaming

It doesn't seem Spark has workarounds other than storing output into external storages, so +1 on having this. My major concern on implementing queryable state in structured streaming is "Are all states available on executors at any time while query is running?" Querying state shouldn't affect the

Re: Plan on Structured Streaming in next major/minor release?

Yeah, the main intention of this thread is to collect interest on possible feature list for structured streaming. From what I can see in Spark community, most of the discussions as well as contributions are for SQL, and I'd wish to see similar activeness / efforts on structured streaming.

Re: Support SqlStreaming in spark

The code of SQLStreaming has been pushed: https://github.com/apache/spark/pull/22575 -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: data source api v2 refactoring

I have pushed a patch for SQLStreaming, which just resolved the problem just discussed. the Jira: https://issues.apache.org/jira/browse/SPARK-24630 the Patch: https://github.com/apache/spark/pull/22575 SQLStreaming just defined the table API for StructStreaming, and the Table APIs for

Re: Plan on Structured Streaming in next major/minor release?

Hi Jungtaek, I just tried to start the discussion in the dev list along time ago. I enumerated some uses cases as Michael proposed here . The discussion didn't

Re: Plan on Structured Streaming in next major/minor release?

Stavros, if my memory is right, you were trying to drive queryable state, right? Could you summary the progress and the reason why the progress got stopped? 2018년 10월 21일 (일) 오후 10:27, Stavros Kontopoulos < stavros.kontopou...@lightbend.com>님이 작성: > That is a very interesting list thanks. I

Re: Plan on Structured Streaming in next major/minor release?

That is a very interesting list thanks. I could create a design doc as a starting pointing for discussion if this is a feature we would like to have. Regards, Stavros On Sun, Oct 21, 2018 at 3:04 PM, JackyLee wrote: > Thanks for raising them. > > FYI, I believe this open issues could also be

Re: Plan on Structured Streaming in next major/minor release?

Thanks for raising them. FYI, I believe this open issues could also be considered: https://issues.apache.org/jira/browse/SPARK-24630 An new ability to express Struct Streaming on pure SQL. -- Sent from: