Live Code Reviews, Coding, and Dev Tools

Tomorrow afternoon @ 3pm pacific I'll be doing some dev tools poking for Beam and Spark - https://www.youtube.com/watch?v=6cTmC_fP9B0 for mention-bot. On Friday I'll be doing my normal code reviews - https://www.youtube.com/watch?v=O4rRx-3PTiM On Monday July 30th @ 9:30am I'll be doing some more

Re: Build timeout -- continuous-integration/appveyor/pr — AppVeyor build failed

Eh, I believe we have been using this since we decided to use it in SPARK-17200. (probably you mean Travis CI(?)) It's still two clicks of the mouse to close and reopen :-) but yea I get It could be bothering and it's a workaround for clarification. As far as I know, we should open an INFRA JIRA t

Re: Build timeout -- continuous-integration/appveyor/pr — AppVeyor build failed

out of curiosity: why are we using appveyor again? closing and reopening PRs solely to retrigger builds seems... cumbersome. shane On Tue, Jul 24, 2018 at 6:09 PM, Hyukjin Kwon wrote: > looks we are getting close indeed ... > Fortunately(?), looks still a bit unusual yet though given the his

Re: Build timeout -- continuous-integration/appveyor/pr — AppVeyor build failed

looks we are getting close indeed ... Fortunately(?), looks still a bit unusual yet though given the history ( https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/history) As far as I know, simple workaround is just to close and reopen the PR and it retriggers the build. I believe this

Re: Build timeout -- continuous-integration/appveyor/pr — AppVeyor build failed

revisiting this thread... i pushed a small change to some R test code ( https://github.com/apache/spark/pull/21864), and the appveyor build timed out after 90 minutes: https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/build/2440-master to be honest, i don't have a lot of time to deb

[build system] upped build retention for GHPRB builds

the PRB was set to rotate build logs out after two weeks, but due to the sheer number of builds (yay! a great thing!), i just went in and upped it to 30 days. we've got plenty of disk space on our jenkins master, so this shouldn't be a problem. i'll keep an eye on things and adjust if needed. s

[DISCUSS] SPIP: APIs for Table Metadata Operations

The recently adopted SPIP to standardize logical plans requires a way for to plug in providers for table metadata operations, so that the new plans can create and drop tables. I proposed an API to do this in a follow-up SPIP on APIs for Table Metadata Operations

[SPARK-24581] Design: BarrierTaskContext.barrier()

Hi All, This is a follow up work of [SPARK-24374 ] SPIP: Support Barrier Execution Mode in Apache Spark. https://docs.google.com/document/d/1r07-vU5JTH6s1jJ6azkmK0K5it6jwpfO6b_K3mJmxR4/edit?usp=sharing We need to provide a communication barrier f