Yes I understand what Reynold stated (as Michael Armbrust stated earlier),
and I agree it's major great thing that improvements on CORE/SQL also
benefit to SS as well.
I just concerned that both of SQL / SS are being impacted with DSv2, but
things are going differently between SQL and SS. SQL is
Agree with rxin. Maybe we should consider about these PRs, especially those
large PRs, after DataSource V2 API is ready.
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
-
To unsubscribe e-mail:
OK, good to know, and that all makes sense. Thanks for clearing up my
concern.
One of great things about Spark is, as you pointed out, that improvements
to core components benefit multiple features at once.
On Mon, Jan 14, 2019 at 8:36 PM Reynold Xin wrote:
> BTW the largest change to SS right
BTW the largest change to SS right now is probably the entire data source API
v2 effort, which aims to unify streaming and batch from data source
perspective, and provide a reliable, expressive source/sink API.
On Mon, Jan 14, 2019 at 5:34 PM, Reynold Xin < r...@databricks.com > wrote:
>
>
There are a few things to keep in mind:
1. Structured Streaming isn't an independent project. It actually (by design)
depends on all the rest of Spark SQL, and virtually all improvements to Spark
SQL benefit Structured Streaming.
2. The project as far as I can tell is relatively mature for
As an observer, this thread is interesting and concerning. Is there an
emerging consensus that Structured Streaming is somehow not relevant
anymore? Or is it just that folks consider it "complete enough"?
Structured Streaming was billed as the replacement to DStreams. If
committers, generally
Cody, I guess I already addressed your comments in the PR (#22138). The
approach was changed to address your concern, and after that Gabor helped
to review the PR. Please take a look again when you have time to get into.
2019년 1월 15일 (화) 오전 1:01, Cody Koeninger 님이 작성:
> I feel like I've already
Sad to hear that. While I understand such thing can be happened for any
project, it feels me to a kind of bad sign that non-experimental major
feature which has no alternative is getting lost on interest.
I also fully agree that there isn't a way to make people work on it (I also
had encountered
I feel like I've already said my piece on
https://github.com/apache/spark/pull/22138 let me know if you have
more questions.
As for SS in general, I don't have a production SS deployment, so I'm
less comfortable with reviewing large changes to it. But if no other
committers are working on it...
Yes you're preaching to the choir here. SS does seem somewhat
abandoned by those that have worked on it. I have also been at times
frustrated that some areas fall into this pattern.
There isn't a way to make people work on it, and I personally am not
interested in it nor have a background in SS.
Sean, this is actually a fail-back on pinging committers. I know who can
review and merge in SS area, and pinged to them, didn't work. Even there's
a PR which approach was encouraged by committer and reviewed the first
phase, and no review.
That's not the first time I have faced the situation,
But it's true that imho there's less activity in SS in general. Should be
noted. Maybe it's also because committers are busy for other stuffs.
Yea, I agree that one actionable strategy for now might be to make the PR
description as clear as possible to make the review easier, and then ping
them
Jungtaek, the best strategy is to find who wrote the code you are
modifying (use Github history or git blame) and ping them directly on
the PR. I don't know this code well myself.
It also helps if you can address why the functionality is important,
and describe compatibility implications.
Most
I'm sorry but let me remind this, as non-SS PRs are being reviewed
accordingly, whereas many of SS PRs (regardless of who create) are still
not reviewed and merged in time.
2019년 1월 3일 (목) 오전 7:57, Jungtaek Lim 님이 작성:
> Spark devs, happy new year!
>
> I would like to remind this kindly, since
Spark devs, happy new year!
I would like to remind this kindly, since there was actually no review
after initiating the thread.
Thanks,
Jungtaek Lim (HeartSaVioR)
2018년 12월 12일 (수) 오후 11:12, Vaclav Kosar 님이 작성:
> I am also waiting for any finalization of my PR [3]. I seems that SS PRs
> are
I am also waiting for any finalization of my PR [3]. I seems that SS PRs
are not being reviewed much these days.
[3] https://github.com/apache/spark/pull/21919
On 12. 12. 18 14:37, Dongjin Lee wrote:
If it is possible, could you review my PR on Kafka's header
functionality[^1] also? It was
If it is possible, could you review my PR on Kafka's header
functionality[^1] also? It was added in Kafka 0.11.0.0 but still not
supported in Spark.
Thanks,
Dongjin
[^1]: https://github.com/apache/spark/pull/22282
[^2]: https://issues.apache.org/jira/browse/KAFKA-4208
On Wed, Dec 12, 2018 at
Hi devs,
Would I kindly ask for reviewing on PRs for Structured Streaming? I have 5
open pull requests on SS side [1] (earliest PR was opened around 4 months
so far), and there looks like couple of PR for others [2] which looks good
to be reviewed, too.
Thanks in advance,
Jungtaek Lim
18 matches
Mail list logo
