Hi, I would like to propose two changes:
1. Renaming “Runtime / Operators” to “Runtime / Task” or something like “Runtime / Processing”. “Runtime / Operators” was confusing me, since it sounded like it covers concrete implementations of the operators, like “WindowOperator” or various join implementations. 2. I think we should add additional component for benchmarks and benchmarking infrastructure. While this is more complicated topic (because of the setup and how is it running), it should be on the same level as correctness tests. Piotrek > On 20 Feb 2019, at 10:53, Robert Metzger <rmetz...@apache.org> wrote: > > Thanks a lot Timo! > > I will start a vote Chesnay! > > On Wed, Feb 20, 2019 at 10:11 AM Timo Walther <twal...@apache.org> wrote: > >> +1 for the vote. Btw I can help cleaning up the "Table API & SQL" >> component. It seems to be the biggest with 1229 Issues. >> >> Thanks, >> Timo >> >> Am 20.02.19 um 10:09 schrieb Chesnay Schepler: >>> I would prefer if you'd start a vote with a new cleaned up proposal. >>> >>> On 18.02.2019 15:23, Robert Metzger wrote: >>>> I added "Runtime / Configuration" to the proposal: >>>> >> https://cwiki.apache.org/confluence/display/FLINK/Proposal+for+new+JIRA+Components >>>> >>>> >>>> Since this discussion has been open for 10 days, I assume we have >>>> reached >>>> consensus here. I will soon start renaming components. >>>> >>>> On Wed, Feb 13, 2019 at 10:51 AM Chesnay Schepler <ches...@apache.org> >>>> wrote: >>>> >>>>> The only parent I can think of is "Infrastructure", but I don't quite >>>>> like it :/ >>>>> >>>>> +1 for "Runtime / Configuration"; this is too general to be placed in >>>>> coordination imo. >>>>> >>>>> On 12.02.2019 18:25, Robert Metzger wrote: >>>>>> Thanks a lot for your feedback Chesnay! >>>>>> >>>>>> re build/travis/release: Do you have a good idea for a common >>>>>> parent for >>>>>> "Build System", "Travis" and "Release System"? >>>>>> >>>>>> re legacy: Okay, I see your point. I will keep the Legacy Components >>>>> prefix. >>>>>> re library: I think I don't have a argument here. My proposal is >>>>>> based on >>>>>> what I felt as being right :) I added the "Library / " prefix to the >>>>>> proposal. >>>>>> >>>>>> re core/config: From the proposed components, I see the best match >>>>>> with >>>>>> "Runtime / Coordination", but I agree that this example is >>>>>> difficult to >>>>>> place into my proposed scheme. Do you think we should introduce >>>>>> "Runtime >>>>> / >>>>>> Configuration" as a component? >>>>>> >>>>>> >>>>>> I updated the proposal accordingly! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Feb 12, 2019 at 12:19 PM Chesnay Schepler <ches...@apache.org >>> >>>>>> wrote: >>>>>> >>>>>>> re build/travis/release: No, I'm against merging build system, travis >>>>>>> and release system. >>>>>>> >>>>>>> re legacy: So going forward you're proposing to move dropped features >>>>>>> into the legacy bucket and make it impossible to search for specific >>>>>>> issues for that component? There's 0 overhead to having these >>>>>>> components, so I really don't get the benefit here, but see the >>>>> overhead. >>>>>>> I don't buy the argument of "people will not open issues if the >>>>>>> component doesn't exist", they will just leave the component field >>>>>>> blank >>>>>>> or add a random one (that would be wrong). In fact, if you had a >>>>>>> storm/tez component (that users would adhere to) then it would be >>>>>>> _easier_ to figure out whether an issue can be rejected right away. >>>>>>> >>>>>>> re library: If you are against a library category, what's your >>>>>>> argument >>>>>>> for a connector category? >>>>>>> >>>>>>> re tests: I don't mind "tests" being removed from tickets about test >>>>>>> instabilities, but you specified the migration as "rename E2E tests" >>>>>>> which is not equivalent. >>>>>>> Under what category would you file modifications to >>>>> flink-test-utils-junit? >>>>>>> I would propose to not differentiate between e2e and other tests; I >>>>>>> would go along with "Test infrastructure", and remove the major >>>>>>> "Tests" >>>>>>> category. >>>>>>> >>>>>>> re core/config: As an example, where (under Runtime) would you >>>>>>> place the >>>>>>> introduction of the ConfigOption class? >>>>>>> >>>>>>> On 11.02.2019 11:31, Robert Metzger wrote: >>>>>>>> Thanks a lot for your feedback! >>>>>>>> >>>>>>>> @Timo: >>>>>>>> I've followed your suggestions and updated the proposed names in the >>>>>>> wiki. >>>>>>>> Regarding a new "SQL/Connectors" component: I (with admittedly >>>>>>>> not much >>>>>>>> knowledge) would not add this component at the moment, and put >>>>>>>> the SQL >>>>>>>> stuff into the respective connector component. >>>>>>>> It is probably pretty difficult for a user to decide whether a but >>>>>>> belongs >>>>>>>> to "SQL/Connector" to "Connectors/Kafka" when Kafka in SQL does not >>>>> work. >>>>>>>> @Chesnay: >>>>>>>> - You are suggesting to rename "Build System" to "Maven" and still >>>>> merge >>>>>>> it >>>>>>>> with "Travis", "Release System" etc. as in the proposal? >>>>>>>> >>>>>>>> - "Runtime / Control Plan" vs "Runtime / Coordination" -- I >>>>>>>> changed the >>>>>>>> proposal >>>>>>>> >>>>>>>> - Re. "Documentation": Yes, I think that would be better in the long >>>>> run. >>>>>>>> We are already in a situation where there are groups within the >>>>> community >>>>>>>> focusing on certain areas of the code (such as SQL, the runtime, >>>>>>>> connectors). Those groups will monitor their components, but it will >>>>> be a >>>>>>>> lot of overhead for them to monitor the "Documentation" component. >>>>>>>> We can also try to assign documentation components to both >>>>>>> "Documentation" >>>>>>>> and the affected component, such as "Runtime / Metrics". >>>>>>>> >>>>>>>> - Removed "Misc / " prefix. >>>>>>>> >>>>>>>> - "Legacy Components": Usually legacy components usually have >>>>>>>> very few >>>>>>>> tickets. "Flink on Tez" has 13, "Storm Compat" ~30, and JIRA has >>>>>>>> a bulk >>>>>>>> edit feature :) >>>>>>>> The benefit of having it generalized is that people will probably >>>>>>>> not >>>>> add >>>>>>>> tickets to it. >>>>>>>> >>>>>>>> - "Libraries /" prefix: I don't think that it is necessary. Some >>>>>>> libraries >>>>>>>> might grow in the future (like the Table API), then we need to >>>>>>>> rename. >>>>>>>> the "flink-libraries" module does contain stuff like the sql >>>>>>>> client or >>>>>>> the >>>>>>>> python api, which are already covered by other components in my >>>>> proposal >>>>>>> -- >>>>>>>> so going with the maven module structure is not an argument here. >>>>>>>> >>>>>>>> - "End to end infrastructure" and "Tests: The same argument as >>>>>>>> with the >>>>>>>> "Documentation" applies here. The maintainers of Kafka, Metrics, .. >>>>>>> should >>>>>>>> get visibility into "their" test instabilities through "their" >>>>>>> components. >>>>>>>> Not many people will feel responsible for the "Tests" component. >>>>>>>> >>>>>>>> For "Core" and "Configuration", I will move the tickets to the >>>>>>> appropriate >>>>>>>> components in "Runtime /". >>>>>>>> >>>>>>>> For "API / Scala": Good point. I will add that component. >>>>>>>> >>>>>>>> How to do it? I will just go through the pain and do it. >>>>>>>> >>>>>>>> >>>>>>>> Best, >>>>>>>> Robert >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Feb 8, 2019 at 2:40 PM Chesnay Schepler <ches...@apache.org >>> >>>>>>> wrote: >>>>>>>>> Some concerns: >>>>>>>>> >>>>>>>>> Travis and build system / release system are entirely different. I >>>>> would >>>>>>>>> even keep the release system away from the build-system, as it >>>>>>>>> is more >>>>>>>>> about the release scripts and documentation, while the latter is >>>>>>>>> about >>>>>>>>> maven. Actually I'd just rename build-system to maven. >>>>>>>>> >>>>>>>>> Control Plane is a term I've never heard before in this context; >>>>>>>>> I'd >>>>>>>>> replace it with Coordination. >>>>>>>>> >>>>>>>>> The "Documentation" descriptions refers to it as a "Fallback >>>>> component". >>>>>>>>> In other words, if I make a change to the metrics documentation I >>>>>>>>> shouldn't use this component any more? >>>>>>>>> >>>>>>>>> I don't see the benefit of a `Misc` major category. I'd attribute >>>>>>>>> everything that doesn't have a major category implicitly to "Misc". >>>>>>>>> >>>>>>>>> Not a fan of a generalized "Legacy components" category; this seems >>>>>>>>> unnecessary. It's also a bit weird going forward as we'd have to >>>>>>>>> touch >>>>>>>>> every JIRA for a component if we drop it. >>>>>>>>> >>>>>>>>> How come gelly/CEP don't have a Major category (libraries?) >>>>>>>>> >>>>>>>>> "End to end infrastructure" is not equivalent to "E2E tests". >>>>>>>>> Infrastructure is not about fixing failing tests, which is what we >>>>>>>>> partially used this component for so far. >>>>>>>>> >>>>>>>>> I don't believe you can get rid of the generic "Tests" component; >>>>>>>>> consider any changes to the `flink-test-utils-junit` module. >>>>>>>>> >>>>>>>>> You propose deleting "Core" and "Configuration" but haven't >>>>>>>>> listed any >>>>>>>>> migration paths. >>>>>>>>> >>>>>>>>> If there's a API / Python category there should also be a API / >>>>>>>>> Scala >>>>>>>>> category. This could also include the shala-shell. Note that the >>>>>>>>> existing Scala API category is not mentioned anywhere in the >>>>>>>>> document. >>>>>>>>> >>>>>>>>> How do you actually want to do the migration? >>>>>>>>> >>>>>>>>> On 08.02.2019 13:13, Timo Walther wrote: >>>>>>>>>> Hi Robert, >>>>>>>>>> >>>>>>>>>> thanks for starting this discussion. I was also about to suggest >>>>>>>>>> splitting the `Table API & SQL` component because it contains >>>>>>>>>> already >>>>>>>>>> more than 1000 issues. >>>>>>>>>> >>>>>>>>>> My comments: >>>>>>>>>> >>>>>>>>>> - Rename "SQL/Shell" to "SQL/Client" because the long-term goal >>>>>>>>>> might >>>>>>>>>> not only be a CLI interface. I would keep the generic name "SQL >>>>>>>>>> Client" for now. This is also what is written in FLIPs, >>>>> presentations, >>>>>>>>>> and documentation. >>>>>>>>>> - Rename "SQL/Query Planner" to "SQL/Planner" a query is read-only >>>>>>>>>> operation but we support things like INSERT INTO etc.. Planner is >>>>> more >>>>>>>>>> generic. >>>>>>>>>> - Rename "Gelly" to "Graph Processing". New users don't know what >>>>>>>>>> Gelly means. This is the only component that has a "feature >>>>>>>>>> name". I >>>>>>>>>> don't know if we want to stick with that in the future. >>>>>>>>>> - Not sure about this: Introduce a "SQL/Connectors"? Because SQL >>>>>>>>>> connectors are tightly bound to SQL internals but also to the >>>>>>>>>> connector itself. >>>>>>>>>> - Rename "Connectors/HCatalog" to "Connectors/Hive". This name is >>>>> more >>>>>>>>>> generic and reflects the efforts about Hive Metastore and catalog >>>>>>>>>> integration that is currenlty taking place. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Timo >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Am 08.02.19 um 12:39 schrieb Robert Metzger: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I am currently trying to improve how the Flink community is >>>>>>>>>>> handling >>>>>>>>>>> incoming pull requests and JIRA tickets. >>>>>>>>>>> >>>>>>>>>>> I've looked at how other big communities are handling such a high >>>>>>>>>>> number of >>>>>>>>>>> contributions, and I found that many are using GitHub labels >>>>>>>>>>> extensively. >>>>>>>>>>> An integral part of the label use is to tag PRs with the >>>>>>>>>>> component / >>>>>>>>>>> area >>>>>>>>>>> they belong to. I think the most obvious and logical way of >>>>>>>>>>> tagging >>>>>>>>>>> the PRs >>>>>>>>>>> is by using the JIRA components. This will force us to keep >>>>>>>>>>> the JIRA >>>>>>>>>>> tickets well-organized, if we want the PRs to be organized :) >>>>>>>>>>> I will soon start a separate discussion for the GitHub labels. >>>>>>>>>>> >>>>>>>>>>> Let's first discuss the JIRA components. >>>>>>>>>>> >>>>>>>>>>> I've created the following Wiki page with my proposal of the new >>>>>>>>>>> component, >>>>>>>>>>> and how to migrate from the existing components: >>>>>>>>>>> >>>>> >> https://cwiki.apache.org/confluence/display/FLINK/Proposal+for+new+JIRA+Components >>>>> >>>>>>>>>>> Please comment here or directly in the Wiki to let me know >>>>>>>>>>> what you >>>>>>>>>>> think. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Robert >>>>>>>>>>> >>>>> >> >>