Hi everyone,
We have fixed the legal issues, as well as a bug found by @Nadav please
review and vote on the release candidate #3 for the version
0.2.0-incubating, as follows
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging are
+1 approve. Tested multiple times and after a long round of fixing and
testing over and over.
Cheers,
Nadav
On 29 May 2018 at 07:38, Yaniv Rodenski wrote:
> Hi everyone,
>
> We have fixed the legal issues, as well as a bug found by @Nadav please
> review and vote on the release candidate #3 fo
+1 looks good to me
On Tue, May 29, 2018, 14:39 Nadav Har Tzvi wrote:
> +1 approve. Tested multiple times and after a long round of fixing and
> testing over and over.
>
> Cheers,
> Nadav
>
>
> On 29 May 2018 at 07:38, Yaniv Rodenski wrote:
>
> > Hi everyone,
> >
> > We have fixed the legal iss
Unit tests passing and build was successful on the branch
"version-0.2.0-incubating-rc3"
+1 from me
Cheers,
Kirupa
On Tue, May 29, 2018 at 3:06 PM, guy peleg wrote:
> +1 looks good to me
>
> On Tue, May 29, 2018, 14:39 Nadav Har Tzvi wrote:
>
> > +1 approve. Tested multiple times and after a
+1 from me
Unit Tests and Build ran fine.
Tested on HDP (VM) but had trouble allocating containers (didn't have that
before). Apparently Centos VMs are known to have this problem. Disabled
physical memory check (yarn.nodemanager.pmem-check-enabled) and ran jobs
successfully.
On Tue, May 29
Yaniv, Eyal, this might be related to the same issue you faced with HDP.
Can you confirm?
On Tue, May 29, 2018, 17:58 Arun Manivannan wrote:
> +1 from me
>
> Unit Tests and Build ran fine.
>
> Tested on HDP (VM) but had trouble allocating containers (didn't have that
> before). Apparently Cento
The pmem disabling is just temporary. I'll do a detailed analysis and get
back with a proper solution.
Any hints on this front is highly appreciated.
Cheers
Arun
On Wed, May 30, 2018, 01:10 Nadav Har Tzvi wrote:
> Yaniv, Eyal, this might be related to the same issue you faced with HDP.
> Can y
Yaniv and I just tested it. It worked flawlessly on my end (HDP docker on
AWS). Both Spark-Scala and PySpark.
It worked on Yaniv's HDP cluster as well.
Worth noting:
1. HDP 2.6.4
2. Cluster has total of 32GB memory available
3. Each container is allocated 1G memory.
4. Amaterasu.properties:
zk=san
Thanks a lot, Nadav. Will get home and spend some more time on this. I was
in a rush and did this poor workaround. My VM is just 8 GB.
Cheers
Arun
On Wed, May 30, 2018, 12:27 Nadav Har Tzvi wrote:
> Yaniv and I just tested it. It worked flawlessly on my end (HDP docker on
> AWS). Both Spark-Sca
Hi All,
Just a kind reminder for our mentors, we need your vote on this one :)
Cheers,
Yaniv
On Wed, May 30, 2018 at 3:25 PM, Arun Manivannan wrote:
> Thanks a lot, Nadav. Will get home and spend some more time on this. I was
> in a rush and did this poor workaround. My VM is just 8 GB.
>
> Ch
Great improvement!
license/notice/disclaimer: ok
headers: ~fine (missing in some non-code files, it would be great to add
those -- non-blocking)
binaries: there's a gradle binary, but that may be ruled as acceptable
(non-blocking from me)
Not sure why we need both .tar and .tar.gz. I'd get rid of
Thanks Davor,
We will fix those for the next release.
Cheers,
Yaniv
On Fri, Jun 1, 2018 at 7:23 AM, Davor Bonaci wrote:
> Great improvement!
>
> license/notice/disclaimer: ok
> headers: ~fine (missing in some non-code files, it would be great to add
> those -- non-blocking)
> binaries: there's
One thing to clarify, the tar.gz is the source code as I've seen this is
released in other projects. if that is not needed I'm happy to drop it from
future releases.
On Fri, Jun 1, 2018 at 5:01 PM, Yaniv Rodenski wrote:
> Thanks Davor,
>
> We will fix those for the next release.
>
> Cheers,
> Ya
No problem with .tar.gz. But, you don't need .tar then.
On Fri, Jun 1, 2018 at 12:04 AM, Yaniv Rodenski wrote:
> One thing to clarify, the tar.gz is the source code as I've seen this is
> released in other projects. if that is not needed I'm happy to drop it from
> future releases.
>
> On Fri, J
Hi Davor,
Sorry for keeping my emails short and undescriptive, I have only one hand
for the next month due to a skateboarding accident.
Based on what I've seen in other projects, it seems common to release bot
the binaries (in Amaterasu those are released in the .tar file) as well as
a source (whi
Gentlemen,
Apologies for coming back late. The issue was just the minimum container
size that was configured in my cluster
(yarn.scheduler.maximum-allocation-mb). It was set at 1 GB.
I didn't specify any spark specific memory parameters during my run (the
memory defaults that the SparkSetupProv
Hi Arun,
Fair point, but I think it can be configured via env/[env_name]/spark.yml
am I wrong?
Anyway, ideally I think we should try and have per job configurations in
the environment rather than in the amaterasu.properties
Cheers,
Yaniv
On Sun, 3 Jun 2018 at 1:32 am, Arun Manivannan wrote:
>
Ah. Missed the part where we could configure parameters at a job level.
Thanks a lot for clarifying, Yaniv.
Cheers,
Arun
On Sun, Jun 3, 2018 at 6:37 PM Yaniv Rodenski wrote:
> Hi Arun,
>
> Fair point, but I think it can be configured via env/[env_name]/spark.yml
> am I wrong?
> Anyway, ideally
Hi All,
Just following up with @JB and @Olivier, any chance you guys can have a
look and vote?
Cheers,
Yaniv
On Mon, Jun 4, 2018 at 12:08 AM, Arun Manivannan wrote:
> Ah. Missed the part where we could configure parameters at a job level.
> Thanks a lot for clarifying, Yaniv.
>
> Cheers,
> Aru
>
> Based on what I've seen in other projects, it seems common to release bot
> the binaries (in Amaterasu those are released in the .tar file) as well as
> a source (which are in the -source.tar.gz file)
> The main question is if we do or do not need to release both.
>
Sorry, my mistake. It is pe
+1
On Mon, 4 Jun 2018 at 17:02, Yaniv Rodenski wrote:
> Hi All,
>
> Just following up with @JB and @Olivier, any chance you guys can have a
> look and vote?
>
> Cheers,
> Yaniv
>
> On Mon, Jun 4, 2018 at 12:08 AM, Arun Manivannan wrote:
>
> > Ah. Missed the part where we could configure parame
+1
I checked the signature, the headers: it looks good.
Thanks !
Regards
JB
On 02/06/2018 17:32, Arun Manivannan wrote:
> Gentlemen,
>
> Apologies for coming back late. The issue was just the minimum container
> size that was configured in my cluster
> (yarn.scheduler.maximum-allocation-mb).
I'm happy to announce that we have unanimously approved this release.
There are 7 approving votes, 3 of which are binding:
* Nadav Har Tzvi
* Guy Peleg
* Kirupa Devarajan
* Arun Manivannan
IPMC votes
* Davor Bonaci
* Olivier Lamy
* Jean-Baptiste Onofré
There are no disapproving votes.
23 matches
Mail list logo
