+1 to everything Sean said.
> On May 28, 2026, at 11:58 AM, Sean Owen <[email protected]> wrote:
>
> Asif, I'm not familiar with your specific changes, but, I think you are
> misreading how a big OSS project works, and not helping anyone including
> yourself.
>
> Thousands of people have proposed changes to Spark in its 12 years; there are
> tens of thousands of JIRAs and pull requests. From experience, most are noise.
> Yours may or may not be. But, you will not be nearly the first person that
> said "hey my change is important and made sense to me! why wasn't it
> accepted?"
> The explanation is simply not going to be because a 'cartel' got together and
> decided it would be, what, easier to go steal, modify, duplicate your work,
> in public, and alienate you, instead of just pressing 'merge' on your work?
> The far more likely explanation is that nobody has even looked at it, not in
> any detail.
>
> There are no paid staff here. There are no rewards for contributing, or
> merging a PR. There is nobody that is obligated to review your proposals,
> either.
> I'm sure some great PRs have been overlooked as a result, but that's just
> life. The game here is to play nice and make clear succinct changes that
> stand a chance of getting attention. People do it all the time.
>
> Committers, certainly, have far more influence in the project, by design.
> Committers get that status by contributing a lot. Often people that
> contribute a lot are people for whom this is part of a day job. Those people
> have a set of things they want to work on and not others. They have
> credibility to get attention for reviews and power to merge. None of that is
> nefarious. It's how OSS projects just work.
> People get committer bits regularly. We just added like 3 of them?
>
> I bet you aren't the first to feel this way at some point, but I dont' recall
> any message like yours claiming a conspiracy to steal credit, no, because it
> just does not map to how an OSS project like this works.
> You want your code reviewed, you want appreciation here, but 99% of what
> people know about you so far is this angry message alleging a conspiracy.
>
> I'd drop this line and focus on working constructively with people whose
> interests clearly overlap with yours, like Peter, instead of writing to the
> whole list about cartels.
>
>
>
> On Thu, May 28, 2026 at 10:33 AM Asif Shahid <[email protected]
> <mailto:[email protected]>> wrote:
>> Hi Peter,
>> Pls see inline for comments/ replies
>>
>> On Thu, May 28, 2026 at 6:11 AM Peter Toth <[email protected]
>> <mailto:[email protected]>> wrote:
>>> Hey Asif,
>>>
>>> Are you referring to https://github.com/apache/spark/pull/49154/changes vs.
>>> https://github.com/apache/spark/pull/55644/changes? Those are definitely
>>> solving the same issue but I can assure you I wouldn't take any code from
>>> your PR without consulting with you first.
>> Yes Indeed Peter, I am referring to those.
>> As for the fix, itself, is not indicative of any thing as its a one liner,
>> test has uncanny resemblance.
>>
>>> As far as I remember, I opened SPARK-56694 /
>>> https://github.com/apache/spark/pull/55644 because I ran into that minor
>>> bug during the implementation of https://github.com/apache/spark/pull/55298.
>>
>>> Sorry, I didn't check whether a ticket or PR already existed.
>>
>> The below I am addressing to the whole cartel.:
>> I have experienced this before, as recent as couple of months back (
>> https://issues.apache.org/jira/browse/SPARK-54386)
>> I have experienced, my personal effort ( going into weeks) to debug,
>> reproduce issue reliably , being hijacked by members, without even
>> discussing the fix proposed, ( by opening new PRs). ( If interested, I can
>> provide details of the PRs / issues I am talking about)
>> I have seen a perfectly valid PR being nixed , by following comment which
>> essentially said
>> " my code of making the cache lookup more effective , would result in
>> greater chances of stale cache being picked, which already spark suffers
>> from."
>> Now the PR was related to collapsing the projects in analysis phase, and
>> side effect was cache pick up being more sensitive.
>> So this is such a frivolous reason to nix the PR , because "staleness" is an
>> underlying existing issue which had nothing to do with my PR. And its more
>> amusing , that if a DB is giving even one wrong result in millions, that
>> makes all the results a suspect in any case. It does not matter at what
>> frequency this occurs. To me the real reason was code complexity ( & more
>> likely the loss of control of the code to the outsider).
>>
>> The reason I call this open source community as cartel, is because, I have
>> seen the way it works pretty closely and have experienced it in the email
>> exchanges which happen on this group.
>> For the same PR , same issue, if advertently or inadvertently , other
>> person ( especially a member) gets his changes pushed, by the virtue of his
>> standing/position and the "for profit" company the person works, how would
>> you give the credit to the original person who discovered the issue first /
>> provided the fix?
>> Why are issues filed by some immediately worked upon by members ( some of
>> whom claim to be working full time on spark) ? Is it because certain
>> companies / groups ( for profit companies, mind you ) exert undue control,
>> or the petty newbee has to be in the good books of members ( with the hope
>> that at some point they will also reach that position of power ?)
>>
>> Given the AI advent and such occurrences, how will you give due credit to
>> the original creators and how do you plan to prevent some member for taking
>> up idea of any old open PR ( which for reasons of complexity and non
>> technical reasons) , polishing it up and pushing it as their own?
>>
>> I am also curious , am I the only one who is troubled by all this, or there
>> are others who have experienced it?
>>
>> Regards
>> Asif
>>
>>> If you have further improvements please feel free to open a PR.
>>>
>>> Best,
>>> Peter
>>>
>>> On Thu, May 28, 2026 at 8:20 AM Asif Shahid <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>> Hi,
>>>> I had filed a bug
>>>> https://issues.apache.org/jira/browse/SPARK-45866
>>>>
>>>> I had also opened a PR for the same.
>>>>
>>>> Now I see that the ticket I filed is still open, but the issue has been
>>>> fixed using a new ticket
>>>> https://issues.apache.org/jira/browse/SPARK-56694
>>>>
>>>> and on top of that the bug test and ofcourse the fix ( which in any case
>>>> would be same) has been taken from my PR for
>>>> https://github.com/apache/spark/pull/49154/changes#diff-137d880ff73623bf7a452bb84f9c3dbbb27ba929e7f5e070c6bff68cfc8ec71f
>>>>
>>>> To me this is clear unethical conduct of cartel member, unless I am
>>>> missing some valid reason.
>>>>
>>>> And the irony is that the fix is still incomplete, as I just found and
>>>> filed a new ticket
>>>> https://issues.apache.org/jira/browse/SPARK-57126
>>>>
>>>> I know that atleast some cartel members are insecure and think of OSS as
>>>> their fiefdom, but this sort of behaviour , I never expected.
>>>> Regards
>>>> Asif
