I'm not sure what you mean. The dependencies are downloaded by SBT and
Maven like in any other project, and nothing about it is specific to Spark.
The worker machines cache artifacts that are downloaded from these, but
this is a function of Maven and SBT, not Spark. You may find that the
initial do
Hi Sean,
Thanks very much for pointing out the roadmap. ;-). Then I think we will
continue to focus on our test environment.
For the networking problems, I mean that we can access Maven Central, and
jobs cloud download the required jar package with a high network speed.
What we want to know is th
@Sean Owen , thanks for your reply.
I agree with you basically, two points I have to say :)
First, maybe I didn't express clear enough, now we download from Maven
Central in our test system, seems the community jenkins ci tests never
download the jar packages from maven centry repo, our question i
a couple of workers needed a bit more time to finish booting up, so no need
for my excursion tomorrow. :)
builds be building, things look happy.
On Thu, Aug 15, 2019 at 6:46 PM Shane Knapp wrote:
> it's back up! some of the workers didn't come back cleanly, so i'll have
> to hit up the colo t
it's back up! some of the workers didn't come back cleanly, so i'll have
to hit up the colo tomorrow and persuade them in person.
On Thu, Aug 15, 2019 at 6:45 PM Wenchen Fan wrote:
> Thanks for tracking it Shane!
>
> On Fri, Aug 16, 2019 at 7:41 AM Shane Knapp wrote:
>
>> just got an update:
>
Thanks for tracking it Shane!
On Fri, Aug 16, 2019 at 7:41 AM Shane Knapp wrote:
> just got an update:
>
> there was a problem w/the replacement part, and they're trying to fix it.
> if that's successful, the expect to have power restored within the hour.
>
> if that doesn't work, a new (new) re
just got an update:
there was a problem w/the replacement part, and they're trying to fix it.
if that's successful, the expect to have power restored within the hour.
if that doesn't work, a new (new) replacement part is scheduled to arrive
at 8am tomorrow.
shane
On Thu, Aug 15, 2019 at 2:07 PM
quick update:
it's been 4 hours, the colo is still down, and i haven't gotten any news
yet as to when they're planning on getting power restored.
once i hear something i will let everyone know what's up.
On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp wrote:
> the berkeley colo had a major power
+1 for that.
Kazuaki volunteered for 2.3.4 release last month. AFAIK, he has been
preparing that.
-
https://lists.apache.org/thread.html/6fafeefb7715e8764ccfe5d30c90d7444378b5f4f383ec95e2f1d7de@%3Cdev.spark.apache.org%3E
I believe we can handle them after 2.4.4 RC1 (or concurrently.)
Hi, Ka
While we're on the topic:
In theory, branch 2.3 is meant to be unsupported as of right about now.
There are 69 fixes in branch 2.3 since 2.3.3 was released in Februrary:
https://issues.apache.org/jira/projects/SPARK/versions/12344844
Some look moderately important.
Should we also, or first, cut
I think the right goal is to fix the remaining issues first. If we set up
CI/CD it will only tell us there are still some test failures. If it's
stable, and not hard to add to the existing CI/CD, yes it could be done
automatically later. You can continue to test on ARM independently for now.
It so
Hi all,
I want to discuss spark ARM CI again, we took some tests on arm instance
based on master and the job includes
https://github.com/theopenlab/spark/pull/13 and k8s integration
https://github.com/theopenlab/spark/pull/17/ , there are several things I
want to talk about:
First, about the fai
Yeah, we will probably drop Python 2 entirely after 3.0.0. Python 2 is
already deprecated.
On Thu, 15 Aug 2019, 18:25 Driesprong, Fokko, wrote:
> Sorry for the late reply, was a bit busy lately, but I still would like to
> share my thoughts on this.
>
> For Apache Airflow we're dropping support
I mean python 2 _will be_ deprecated in Spark 3.
On Thu, 15 Aug 2019, 18:37 Hyukjin Kwon, wrote:
> Yeah, we will probably drop Python 2 entirely after 3.0.0. Python 2 is
> already deprecated.
>
> On Thu, 15 Aug 2019, 18:25 Driesprong, Fokko,
> wrote:
>
>> Sorry for the late reply, was a bit bus
Sorry for the late reply, was a bit busy lately, but I still would like to
share my thoughts on this.
For Apache Airflow we're dropping support for Python 2 in the next major
release. We're now supporting Python 3.5+. Mostly because:
- Easier to maintain and test, and less if/else construction
15 matches
Mail list logo
