Re: [VOTE] Apache Giraph 1.1.0 RC1
Hi! as per prior suggestion I've recut RC2 with the only extra fix included: https://issues.apache.org/jira/browse/GIRAPH-961 Given the scope of the fix, I firmly believe that the testing we've conducted so far should be 100% applicable to the new RC. Still, I started a new vote thread to keep things clean. Claudio, can you please recast your vote in that other new thread? Maja, Eugene, we really need you guys to vote. Please do so. We're sooo close to finally push 1.1.0 out. Thanks, Roman. On Mon, Nov 10, 2014 at 4:01 AM, Claudio Martella claudio.marte...@gmail.com wrote: Yes, I did re-run the build this weekend, and it built succesfully for the default profile and the hadoop_2 one. I ran a couple of examples on the cluster, and it ran succesfully. I'm +1. On Tue, Nov 4, 2014 at 8:10 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella claudio.marte...@gmail.com wrote: I am indeed having some problems. mvn install will fail because the test is opening too many files: [snip] I have to investigate why this happens. I'm not using a different ulimit than what I have on my Mac OS X by default. Where are you building yours? This is really weird. I have not issues whatsoever on Mac OS X with the following setup: $ uname -a Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version 12.4.1: Tue May 21 17:04:50 PDT 2013; root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64 $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 2560 pipe size(512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 709 virtual memory (kbytes, -v) unlimited $ mvn --version Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T13:58:10-07:00) Maven home: /Users/shapor/dist/apache-maven-3.2.3 Java version: 1.7.0_51, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: mac os x, version: 10.8.4, arch: x86_64, family: mac Thanks, Roman. -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
Yes, I did re-run the build this weekend, and it built succesfully for the default profile and the hadoop_2 one. I ran a couple of examples on the cluster, and it ran succesfully. I'm +1. On Tue, Nov 4, 2014 at 8:10 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella claudio.marte...@gmail.com wrote: I am indeed having some problems. mvn install will fail because the test is opening too many files: [snip] I have to investigate why this happens. I'm not using a different ulimit than what I have on my Mac OS X by default. Where are you building yours? This is really weird. I have not issues whatsoever on Mac OS X with the following setup: $ uname -a Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version 12.4.1: Tue May 21 17:04:50 PDT 2013; root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64 $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 2560 pipe size(512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 709 virtual memory (kbytes, -v) unlimited $ mvn --version Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T13:58:10-07:00) Maven home: /Users/shapor/dist/apache-maven-3.2.3 Java version: 1.7.0_51, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: mac os x, version: 10.8.4, arch: x86_64, family: mac Thanks, Roman. -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
I am indeed having some problems. mvn install will fail because the test is opening too many files: Caused by: java.io.FileNotFoundException: /private/var/folders/5b/8yx5dbyn40nbt_70syjs86chgp/T/giraph-hive-1415098102276/metastore_db/seg0/c90.dat (Too many open files in system) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:241) at org.apache.derby.impl.io.DirRandomAccessFile.init(Unknown Source) at org.apache.derby.impl.io.DirRandomAccessFile4.init(Unknown Source) at org.apache.derby.impl.io.DirFile4.getRandomAccessFile(Unknown Source) at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at org.apache.derby.impl.store.raw.data.RAFContainer.createContainer(Unknown Source) at org.apache.derby.impl.store.raw.data.RAFContainer4.createContainer(Unknown Source) at org.apache.derby.impl.store.raw.data.FileContainer.createIdent(Unknown Source) at org.apache.derby.impl.store.raw.data.RAFContainer.createIdentity(Unknown Source) at org.apache.derby.impl.services.cache.ConcurrentCache.create(Unknown Source) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(Unknown Source) at org.apache.derby.impl.store.raw.xact.Xact.addContainer(Unknown Source) at org.apache.derby.impl.store.access.heap.Heap.create(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.createConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.createConglomerate(Unknown Source) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.createConglomerate(Unknown Source) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.createDictionaryTables(Unknown Source) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.boot(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source) at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown Source) at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source) ... 96 more I have to investigate why this happens. I'm not using a different ulimit than what I have on my Mac OS X by default. Where are you building yours? On Sat, Nov 1, 2014 at 11:49 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: Ping! Any progress on testing the current RC? Thanks, Roman. On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella claudio.marte...@gmail.com wrote: Oh, thanks for the info! On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella claudio.marte...@gmail.com wrote: Hi Roman, thanks again for this. I have had a look at the staging site so far (our cluster has been down whole week... universities...), and I was wondering if you have an insight why some of the docs are missing, e.g. gora and rexster documentation. None of them are missing. The links moved to a User Docs - Modules though: http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/gora.html http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/rexster.html and so forth. Thanks, Roman. -- Claudio Martella -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
On Mon, Nov 3, 2014 at 4:51 PM, Maja Kabiljo majakabi...@fb.com wrote: We¹ve been running code which is the same as release candidate plus fix on GIRAPH-961 in production for 5 days now, no problems. This is hadoop_facebook profile, using only hive-io from all io modules. Great! This tells me that once I cut RC2 with GIRAPH-961 you guys will be ready to vote! Thanks, Roman.
Re: [VOTE] Apache Giraph 1.1.0 RC1
On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella claudio.marte...@gmail.com wrote: I am indeed having some problems. mvn install will fail because the test is opening too many files: [snip] I have to investigate why this happens. I'm not using a different ulimit than what I have on my Mac OS X by default. Where are you building yours? This is really weird. I have not issues whatsoever on Mac OS X with the following setup: $ uname -a Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version 12.4.1: Tue May 21 17:04:50 PDT 2013; root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64 $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 2560 pipe size(512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 709 virtual memory (kbytes, -v) unlimited $ mvn --version Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T13:58:10-07:00) Maven home: /Users/shapor/dist/apache-maven-3.2.3 Java version: 1.7.0_51, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: mac os x, version: 10.8.4, arch: x86_64, family: mac Thanks, Roman.
Re: [VOTE] Apache Giraph 1.1.0 RC1
We¹ve been running code which is the same as release candidate plus fix on GIRAPH-961 in production for 5 days now, no problems. This is hadoop_facebook profile, using only hive-io from all io modules. On 11/1/14, 3:49 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: Ping! Any progress on testing the current RC? Thanks, Roman. On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella claudio.marte...@gmail.com wrote: Oh, thanks for the info! On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella claudio.marte...@gmail.com wrote: Hi Roman, thanks again for this. I have had a look at the staging site so far (our cluster has been down whole week... universities...), and I was wondering if you have an insight why some of the docs are missing, e.g. gora and rexster documentation. None of them are missing. The links moved to a User Docs - Modules though: https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs /giraph-1.1.0-RC1/site/gora.htmlk=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=RGg 8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0Am=8PzjCy0QzsbRm9lbAnj 1Sreanb81jw%2FnRRX1Zju8ZvM%3D%0As=aabb0575b0830bb2c1b05645279b426e8789e fca3a6049073b214e2fbf832ec7 https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs /giraph-1.1.0-RC1/site/rexster.htmlk=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar= RGg8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0Am=8PzjCy0QzsbRm9lb Anj1Sreanb81jw%2FnRRX1Zju8ZvM%3D%0As=08f4a813900872e6085eea6d3569bf7db0 078c050d4aac784c0d61ba8f70504d and so forth. Thanks, Roman. -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
Ping! Any progress on testing the current RC? Thanks, Roman. On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella claudio.marte...@gmail.com wrote: Oh, thanks for the info! On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella claudio.marte...@gmail.com wrote: Hi Roman, thanks again for this. I have had a look at the staging site so far (our cluster has been down whole week... universities...), and I was wondering if you have an insight why some of the docs are missing, e.g. gora and rexster documentation. None of them are missing. The links moved to a User Docs - Modules though: http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/gora.html http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/rexster.html and so forth. Thanks, Roman. -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
Hi Roman, thanks again for this. I have had a look at the staging site so far (our cluster has been down whole week... universities...), and I was wondering if you have an insight why some of the docs are missing, e.g. gora and rexster documentation. Thanks, Claudio On Fri, Oct 31, 2014 at 6:38 AM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Wed, Oct 29, 2014 at 6:51 PM, Maja Kabiljo majakabi...@fb.com wrote: Roman, again thanks for taking care of the release. We found one issue https://issues.apache.org/jira/browse/GIRAPH-961 - any application using MasterLoggingAggregator fails without this fix. Can we backport it to the release? This looks like a really good idea to me. I will be re-cutting the RC over the weekend. For now I'd really, really, really ask everybody to once again consider issue like GIRAPH-961 so that we don't have to re-cut multiple times. Thanks, Roman. -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella claudio.marte...@gmail.com wrote: Hi Roman, thanks again for this. I have had a look at the staging site so far (our cluster has been down whole week... universities...), and I was wondering if you have an insight why some of the docs are missing, e.g. gora and rexster documentation. None of them are missing. The links moved to a User Docs - Modules though: http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/gora.html http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/rexster.html and so forth. Thanks, Roman.
Re: [VOTE] Apache Giraph 1.1.0 RC1
Oh, thanks for the info! On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella claudio.marte...@gmail.com wrote: Hi Roman, thanks again for this. I have had a look at the staging site so far (our cluster has been down whole week... universities...), and I was wondering if you have an insight why some of the docs are missing, e.g. gora and rexster documentation. None of them are missing. The links moved to a User Docs - Modules though: http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/gora.html http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/rexster.html and so forth. Thanks, Roman. -- Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
Roman, again thanks for taking care of the release. We found one issue https://issues.apache.org/jira/browse/GIRAPH-961 - any application using MasterLoggingAggregator fails without this fix. Can we backport it to the release? Thanks, Maja On 10/26/14, 12:25 AM, Roman Shaposhnik ro...@shaposhnik.org wrote: This vote is for Apache Giraph, version 1.1.0 release It fixes the following issues: https://urldefense.proofpoint.com/v1/url?u=http://s.apache.org/a8Xk=ZVNjl DMF0FElm4dQtryO4A%3D%3D%0Ar=RGg8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJod KY%3D%0Am=vuRjLsufH81dyuj4l%2BAg5A3PGGSsvGxYyv1pMk0nLgA%3D%0As=cbb0287e4 058ca62c8f79b319121949555bdfba84aaf17a71e3191bde97e8110 *** Please download, test and vote by Mon 11/3 noon PST Note that we are voting upon the source (tag): release-1.1.0-RC1 Source and binary files are available at: https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs/g iraph-1.1.0-RC1/k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=RGg8bUFUf%2FM2K95hnYD 1RGWK1CQ%2BbcclArMcjzJodKY%3D%0Am=vuRjLsufH81dyuj4l%2BAg5A3PGGSsvGxYyv1pM k0nLgA%3D%0As=d120562d308957a39a2f31942102a17fc1c243c3450f92793536568d601 7230c Staged website is available at: https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs/g iraph-1.1.0-RC1/site/k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=RGg8bUFUf%2FM2K9 5hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0Am=vuRjLsufH81dyuj4l%2BAg5A3PGGSsvGxY yv1pMk0nLgA%3D%0As=3dfee9d283999801f5c4c52b6ab99ef60bfa517884f693b8b31124 2bb1371871 Maven staging repo is available at: https://urldefense.proofpoint.com/v1/url?u=https://repository.apache.org/c ontent/repositories/orgapachegiraph-1002k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A r=RGg8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0Am=vuRjLsufH81dyuj 4l%2BAg5A3PGGSsvGxYyv1pMk0nLgA%3D%0As=fb75b2940fd8ea0b956b85303801d08020b be748d0d93ec82cc2cf18729f9b74 Please notice, that as per earlier agreement two sets of artifacts are published differentiated by the version ID: * version ID 1.1.0 corresponds to the artifacts built for the hadoop_1 profile * version ID 1.1.0-hadoop2 corresponds to the artifacts built for hadoop_2 profile. The tag to be voted upon (release-1.1.0-RC1): https://urldefense.proofpoint.com/v1/url?u=https://git-wip-us.apache.org/r epos/asf?p%3Dgiraph.git%3Ba%3Dcommit%3Bh%3D1f0fc23c26ce3addb746e3e57cc155f 82afbab87k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=RGg8bUFUf%2FM2K95hnYD1RGWK1C Q%2BbcclArMcjzJodKY%3D%0Am=vuRjLsufH81dyuj4l%2BAg5A3PGGSsvGxYyv1pMk0nLgA% 3D%0As=1b7468aca53943fc0794e58a1b884342dae353c5da294d32e9b698cf0abfe254 The KEYS file containing PGP keys we use to sign the release: https://urldefense.proofpoint.com/v1/url?u=http://svn.apache.org/repos/asf /bigtop/dist/KEYSk=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=RGg8bUFUf%2FM2K95hnY D1RGWK1CQ%2BbcclArMcjzJodKY%3D%0Am=vuRjLsufH81dyuj4l%2BAg5A3PGGSsvGxYyv1p Mk0nLgA%3D%0As=5f9023144fb07f177de59c5843259cbd8dfb9ba0a2d25444140781a36f b45010 Thanks, Roman.
[VOTE] Apache Giraph 1.1.0 RC1
This vote is for Apache Giraph, version 1.1.0 release It fixes the following issues: http://s.apache.org/a8X *** Please download, test and vote by Mon 11/3 noon PST Note that we are voting upon the source (tag): release-1.1.0-RC1 Source and binary files are available at: http://people.apache.org/~rvs/giraph-1.1.0-RC1/ Staged website is available at: http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/ Maven staging repo is available at: https://repository.apache.org/content/repositories/orgapachegiraph-1002 Please notice, that as per earlier agreement two sets of artifacts are published differentiated by the version ID: * version ID 1.1.0 corresponds to the artifacts built for the hadoop_1 profile * version ID 1.1.0-hadoop2 corresponds to the artifacts built for hadoop_2 profile. The tag to be voted upon (release-1.1.0-RC1): https://git-wip-us.apache.org/repos/asf?p=giraph.git;a=commit;h=1f0fc23c26ce3addb746e3e57cc155f82afbab87 The KEYS file containing PGP keys we use to sign the release: http://svn.apache.org/repos/asf/bigtop/dist/KEYS Thanks, Roman.
