Re: [DISCUSS] SPIP: FunctionCatalog

If people have such a big concern about reflection, we can follow the current Spark Java UDF and Transport

Re: [DISCUSS] assignee practice on committers+ (possible issue on preemption)

I agree, Assignee has been used primarily to give recognition to the contributor who ended up submitting the patch which got merged. Typically jira's remain unassigned - even if it were to be assigned, it conveys no meaning or ownership or ongoing work : IMO it is equivalent to an unassigned

Re: [DISCUSS] SPIP: FunctionCatalog

I agree with Ryan on the questions around the expressivity of the Invoke method. It is not clear to me how the Invoke method can be used to declare UDFs with type-parameterized parameters. For example: a UDF to get the Nth element of an array (regardless of the Array element type) or a UDF to

Re: [DISCUSS] assignee practice on committers+ (possible issue on preemption)

I don't believe Assignee has ever been used for anything except to give a bit of informal credit to the person who drove most of the work on the issue, when it's resolved. If that's the question - does Assignee mean only that person can work on the issue? then no, it has never meant that. You say

Re: Please use Jekyll via "bundle exec" from now on

Nice fix. Thanks! On Thu, Feb 18, 2021 at 7:13 PM Hyukjin Kwon wrote: > Thanks Attlila for fixing and sharing this. > > 2021년 2월 18일 (목) 오후 6:17, Attila Zsolt Piros 님이 > 작성: > >> Hello everybody, >> >> To pin the exact same version of Jekyll across all the contributors, Ruby >> Bundler is

Re: [DISCUSS] assignee practice on committers+ (possible issue on preemption)

(Actually the real world case was fixed somehow and I wouldn't like to point out a fixed one. I just would like to make sure what I think is correct and is considered as "consensus".) Just consider the case as simple - someone files two different JIRA issues for new features and assigns to

(send this email to unsubscribe)

Re: [DISCUSS] SPIP: FunctionCatalog

I agree with you that it is better in many cases to directly call a method. But it it not better in all cases, which is why I don’t think it is the right general-purpose choice. First, if codegen isn’t used for some reason, the reflection overhead is really significant. That gets much better when

Re: Bug?

I am not sure if the problem persists in 3.x. On Thu, Feb 18, 2021 at 12:14 PM Dongjoon Hyun wrote: > Thank you for sharing, Tyson. > > Spark 2.4.4 looks too old to me. Do you think it will occur at 3.x? > > Bests, > Dongjoon. > > > On Thu, Feb 18, 2021 at 11:07 AM Tyson wrote: > >> We

Re: Bug?

Thank you for sharing, Tyson. Spark 2.4.4 looks too old to me. Do you think it will occur at 3.x? Bests, Dongjoon. On Thu, Feb 18, 2021 at 11:07 AM Tyson wrote: > We observed an interesting stack trace that I'd like to share with you. > The logging level is WARN, but it appears to be causing

Bug?

We observed an interesting stack trace that I'd like to share with you. The logging level is WARN, but it appears to be causing task failures. Please let me know if anyone has any insights. It appears to be a integer overflow issue from looking at the code in Spark 2.4.4 WARN TaskSetManager

Re: Auto-closing PRs or How to get reviewers' attention

Enrico - do feel free to reopen the PRs or email people directly, unless you are told otherwise. On Thu, Feb 18, 2021 at 9:09 AM, Nicholas Chammas < nicholas.cham...@gmail.com > wrote: > > On Thu, Feb 18, 2021 at 10:34 AM Sean Owen < srowen@ gmail. com ( > sro...@gmail.com ) > wrote: > > >>

Re: Auto-closing PRs or How to get reviewers' attention

On Thu, Feb 18, 2021 at 10:34 AM Sean Owen wrote: > There is no way to force people to review or commit something of course. > And keep in mind we get a lot of, shall we say, unuseful pull requests. > There is occasionally some blowback to closing someone's PR, so the path of > least resistance

Re: Auto-closing PRs or How to get reviewers' attention

On Thu, Feb 18, 2021 at 9:58 AM Enrico Minack wrote: > *What is the approved way to ...* > > *... prevent it from being auto-closed?* Committing and commenting to the > PR does not prevent it from being closed the next day. > Committing and commenting should prevent the PR from being closed. It

Re: Auto-closing PRs or How to get reviewers' attention

Holden is absolutely correct - pinging relevant individuals is probably your best bet. I skim the 40-50 PRs that have activity each day and look into a few that look like I would know something about by the title, but, easy to miss something I could weigh in on. There is no way to force people to

Re: [DISCUSS] assignee practice on committers+ (possible issue on preemption)

I think it's OK to raise particular instances. It's hard for me to evaluate further in the abstract. I don't think we use Assignee much at all, except to kinda give credit when something is done. No piece of code or work can be solely owned by one person; this is just ASF policy. I think we've

Re: Auto-closing PRs or How to get reviewers' attention

Git blame is a good way to figure out likely potential reviewers (eg who’s been working in the area). Another is who filed the JIRA if it’s not you. On Thu, Feb 18, 2021 at 6:58 AM Enrico Minack wrote: > Hi Spark Developers, > > I have a fundamental question on the process of contributing to

Auto-closing PRs or How to get reviewers' attention

Hi Spark Developers, I have a fundamental question on the process of contributing to Apache Spark from outside the circle of committers. I have gone through a number of pull requests and I always found it hard to get feedback, especially from committers. I understand there is a very high

Re: Please use Jekyll via "bundle exec" from now on

Thanks Attlila for fixing and sharing this. 2021년 2월 18일 (목) 오후 6:17, Attila Zsolt Piros 님이 작성: > Hello everybody, > > To pin the exact same version of Jekyll across all the contributors, Ruby > Bundler is introduced. > This way the differences in the generated documentation, which were caused >

Please use Jekyll via "bundle exec" from now on

Hello everybody, To pin the exact same version of Jekyll across all the contributors, Ruby Bundler is introduced. This way the differences in the generated documentation, which were caused by using different Jekyll versions, are avoided. Regarding its usage this simply means an extra prefix