[jira] [Commented] (GIRAPH-346) Top Level POM

[Brian Femiano (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463564#comment-13463564
 ] 

Brian Femiano commented on GIRAPH-346:
--

For faster development turnaround this would be crucial. Users must still 
remember to 'mvn clean install' from the parent pom, otherwise formats-contrib 
will not utilize the latest changes. Running mvn clean package won't cut it, at 
least until we push artifacts to a known repo. I agree for everyday operations 
it would be smoother. 

Often I wonder if format-contrib is advertised well enough to newer users. Have 
a base build with a parent would help this. I'm all for it. 

> Top Level POM
> -
>
> Key: GIRAPH-346
> URL: https://issues.apache.org/jira/browse/GIRAPH-346
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
>
> Most Maven projects that have multiple modules (as we do with main / formats) 
> have a top level pom.xml. This allows you avoid redundancy and also to build 
> everything from top level all at once. I'm using a lot of the formats stuff, 
> especially playing with hive / hcatalog and it's a pain having to constantly 
> compile everything twice.
> So I propose we move all the main code under a subdir, say giraph/ or main/. 
> Then add a pom.xml that both the giraph/ and the formats/ subdirs can point 
> to.
> Note we will still build completely separate jars. If people want to just 
> build giraph-main without formats they can do so directly from the 
> subdirectory.
> Let me know if you guys +1 the idea and I'll whip it up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-346) Top Level POM

[Eugene Koontz (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463562#comment-13463562
 ] 

Eugene Koontz commented on GIRAPH-346:
--

I think it's worth exploring. We might refer to HBase's experiences with the 
same journey that they took, starting with Gary Helmling's work on security, to 
a more modular pom.xml:

https://issues.apache.org/jira/browse/HBASE-4336



> Top Level POM
> -
>
> Key: GIRAPH-346
> URL: https://issues.apache.org/jira/browse/GIRAPH-346
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
>
> Most Maven projects that have multiple modules (as we do with main / formats) 
> have a top level pom.xml. This allows you avoid redundancy and also to build 
> everything from top level all at once. I'm using a lot of the formats stuff, 
> especially playing with hive / hcatalog and it's a pain having to constantly 
> compile everything twice.
> So I propose we move all the main code under a subdir, say giraph/ or main/. 
> Then add a pom.xml that both the giraph/ and the formats/ subdirs can point 
> to.
> Note we will still build completely separate jars. If people want to just 
> build giraph-main without formats they can do so directly from the 
> subdirectory.
> Let me know if you guys +1 the idea and I'll whip it up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-346) Top Level POM

[Nitay Joffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463527#comment-13463527
 ] 

Nitay Joffe commented on GIRAPH-346:


This is output of 'time mvn clean verify' in giraph-formats-contrib:

real2m38.763s
user1m41.244s
sys 0m13.127s

> Top Level POM
> -
>
> Key: GIRAPH-346
> URL: https://issues.apache.org/jira/browse/GIRAPH-346
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
>
> Most Maven projects that have multiple modules (as we do with main / formats) 
> have a top level pom.xml. This allows you avoid redundancy and also to build 
> everything from top level all at once. I'm using a lot of the formats stuff, 
> especially playing with hive / hcatalog and it's a pain having to constantly 
> compile everything twice.
> So I propose we move all the main code under a subdir, say giraph/ or main/. 
> Then add a pom.xml that both the giraph/ and the formats/ subdirs can point 
> to.
> Note we will still build completely separate jars. If people want to just 
> build giraph-main without formats they can do so directly from the 
> subdirectory.
> Let me know if you guys +1 the idea and I'll whip it up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-346) Top Level POM

[Nitay Joffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463526#comment-13463526
 ] 

Nitay Joffe commented on GIRAPH-346:


Yeah we can do that, but I think if you want to just build giraph-main you can 
just go into that subdirectory and do all your work there, no? Anyways I can 
add a profile for that part if that is what you guys prefer, but I actually 
think it's better to make it by default build everything so that when anyone 
checks in code they run mvn clean verify at top level and it verifies 
everything.

> Top Level POM
> -
>
> Key: GIRAPH-346
> URL: https://issues.apache.org/jira/browse/GIRAPH-346
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
>
> Most Maven projects that have multiple modules (as we do with main / formats) 
> have a top level pom.xml. This allows you avoid redundancy and also to build 
> everything from top level all at once. I'm using a lot of the formats stuff, 
> especially playing with hive / hcatalog and it's a pain having to constantly 
> compile everything twice.
> So I propose we move all the main code under a subdir, say giraph/ or main/. 
> Then add a pom.xml that both the giraph/ and the formats/ subdirs can point 
> to.
> Note we will still build completely separate jars. If people want to just 
> build giraph-main without formats they can do so directly from the 
> subdirectory.
> Let me know if you guys +1 the idea and I'll whip it up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-346) Top Level POM

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463519#comment-13463519
 ] 

Avery Ching commented on GIRAPH-346:


I think this is good, how long does it take to build the formats?  Would be 
great to have the default build only giraph and then add an option to build the 
formats, but for commits we could try to build everything.

Would like to hear what others think.

> Top Level POM
> -
>
> Key: GIRAPH-346
> URL: https://issues.apache.org/jira/browse/GIRAPH-346
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
>
> Most Maven projects that have multiple modules (as we do with main / formats) 
> have a top level pom.xml. This allows you avoid redundancy and also to build 
> everything from top level all at once. I'm using a lot of the formats stuff, 
> especially playing with hive / hcatalog and it's a pain having to constantly 
> compile everything twice.
> So I propose we move all the main code under a subdir, say giraph/ or main/. 
> Then add a pom.xml that both the giraph/ and the formats/ subdirs can point 
> to.
> Note we will still build completely separate jars. If people want to just 
> build giraph-main without formats they can do so directly from the 
> subdirectory.
> Let me know if you guys +1 the idea and I'll whip it up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-345) Publishing snapshots to maven repo

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463517#comment-13463517
 ] 

Avery Ching commented on GIRAPH-345:


This would be a nice thing.

> Publishing snapshots to maven repo
> --
>
> Key: GIRAPH-345
> URL: https://issues.apache.org/jira/browse/GIRAPH-345
> Project: Giraph
>  Issue Type: Bug
>Reporter: Nitay Joffe
>Priority: Minor
>
> Should we publish every successful build from builds.apache.org to apache's 
> snapshots repo (https://repository.apache.org/content/repositories/snapshots) 
> ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (GIRAPH-346) Top Level POM

[Nitay Joffe (JIRA)]

Nitay Joffe created GIRAPH-346:
--

 Summary: Top Level POM
 Key: GIRAPH-346
 URL: https://issues.apache.org/jira/browse/GIRAPH-346
 Project: Giraph
  Issue Type: Improvement
Reporter: Nitay Joffe
Priority: Minor


Most Maven projects that have multiple modules (as we do with main / formats) 
have a top level pom.xml. This allows you avoid redundancy and also to build 
everything from top level all at once. I'm using a lot of the formats stuff, 
especially playing with hive / hcatalog and it's a pain having to constantly 
compile everything twice.

So I propose we move all the main code under a subdir, say giraph/ or main/. 
Then add a pom.xml that both the giraph/ and the formats/ subdirs can point to.

Note we will still build completely separate jars. If people want to just build 
giraph-main without formats they can do so directly from the subdirectory.

Let me know if you guys +1 the idea and I'll whip it up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (GIRAPH-345) Publishing snapshots to maven repo

[Nitay Joffe (JIRA)]

Nitay Joffe created GIRAPH-345:
--

 Summary: Publishing snapshots to maven repo
 Key: GIRAPH-345
 URL: https://issues.apache.org/jira/browse/GIRAPH-345
 Project: Giraph
  Issue Type: Bug
Reporter: Nitay Joffe
Priority: Minor


Should we publish every successful build from builds.apache.org to apache's 
snapshots repo (https://repository.apache.org/content/repositories/snapshots) ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-328) Outgoing messages from current superstep should be grouped at the sender by owning worker, not by partition

[Eli Reisman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eli Reisman updated GIRAPH-328:
---

Attachment: GIRAPH-328-5.patch

Rebased, fixed errors, passes 'mvn verify'
 etc.

Ready for review.



> Outgoing messages from current superstep should be grouped at the sender by 
> owning worker, not by partition
> ---
>
> Key: GIRAPH-328
> URL: https://issues.apache.org/jira/browse/GIRAPH-328
> Project: Giraph
>  Issue Type: Improvement
>  Components: bsp, graph
>Affects Versions: 0.2.0
>Reporter: Eli Reisman
>Assignee: Eli Reisman
>Priority: Minor
> Fix For: 0.2.0
>
> Attachments: GIRAPH-328-1.patch, GIRAPH-328-2.patch, 
> GIRAPH-328-3.patch, GIRAPH-328-4.patch, GIRAPH-328-5.patch
>
>
> Currently, outgoing messages created by the Vertex#compute() cycle on each 
> worker are stored and grouped by the partitionId on the destination worker to 
> which the messages belong. This results in messages being duplicated on the 
> wire per partition on a given receiving worker that has delivery vertices for 
> those messages.
> By partitioning the outgoing, current-superstep messages by destination 
> worker, we can split them into partitions at insertion into a MessageStore on 
> the destination worker. What we trade in come compute time while inserting at 
> the receiver side, we gain in fine grained control over the real number of 
> messages each worker caches outbound for any given worker before flushing, 
> and how those flush messages are aggregated for delivery as well. 
> Potentially, it allows for a great reduction in duplicate messages sent in 
> situations like Vertex#sendMessageToAllEdges() -- see GIRAPH-322, GIRAPH-314. 
> You get the idea.
> This might be a poor idea, and it can certainly use some additional 
> refinement, but it passes mvn verify and may even run ;) It interoperates 
> with the disk spill code, but not as well as it could. Consider this a 
> request for comment on the idea (and the approach) rather than a finished 
> product.
> Comments/ideas/help welcome! Thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-344) mvn test fails with java 1.7

[Eugene Koontz (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463195#comment-13463195
 ] 

Eugene Koontz commented on GIRAPH-344:
--

Hi Nitay, I wonder if it's a zookeeper-specific issue. I'd suggest building and 
running Zookeeper with JDK 1.7 and see if that works with its own "ant clean 
test".

-Eugene


> mvn test fails with java 1.7
> 
>
> Key: GIRAPH-344
> URL: https://issues.apache.org/jira/browse/GIRAPH-344
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Nitay Joffe
>Priority: Minor
>
> On my Mac:
> JAVA_HOME=/Library/Java/Home mvn clean test
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> JAVA_HOME=/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean 
> test
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> Full output: https://gist.github.com/3783472
> Surefire reports doesn't have anything useful: https://gist.github.com/3783480

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-211) Add secure authentication to Netty IPC

[Eugene Koontz (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463193#comment-13463193
 ] 

Eugene Koontz commented on GIRAPH-211:
--

Thanks Avery. I appreciate your help. I am hoping to 1) address your comments 
and 2) get mvn verify to work with all profiles in a new patch in the next 2 
days.
-Eugene

> Add secure authentication to Netty IPC
> --
>
> Key: GIRAPH-211
> URL: https://issues.apache.org/jira/browse/GIRAPH-211
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Fix For: 0.2.0
>
> Attachments: GIRAPH-211.patch, GIRAPH-211.patch, GIRAPH-211.patch, 
> GIRAPH-211.patch, GIRAPH-211.patch, GIRAPH-211.patch, GIRAPH-211.patch, 
> GIRAPH-211-proposal.txt
>
>
> Gianmarco De Francisci Morales asked on the user list:
> bq. I am getting the exception in the subject when running my giraph program
> bq. on a cluster with Kerberos authentication.
> This leads to the idea of having Kerberos authentication supported within 
> GIRAPH. Hopefully it would use our fast GIRAPH-37 IPC, but could also 
> interoperate with Hadoop security.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-211) Add secure authentication to Netty IPC

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463180#comment-13463180
 ] 

Avery Ching commented on GIRAPH-211:


Let me know if you need any help on this one.  I'm looking forward into putting 
this in!

> Add secure authentication to Netty IPC
> --
>
> Key: GIRAPH-211
> URL: https://issues.apache.org/jira/browse/GIRAPH-211
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Eugene Koontz
>Assignee: Eugene Koontz
> Fix For: 0.2.0
>
> Attachments: GIRAPH-211.patch, GIRAPH-211.patch, GIRAPH-211.patch, 
> GIRAPH-211.patch, GIRAPH-211.patch, GIRAPH-211.patch, GIRAPH-211.patch, 
> GIRAPH-211-proposal.txt
>
>
> Gianmarco De Francisci Morales asked on the user list:
> bq. I am getting the exception in the subject when running my giraph program
> bq. on a cluster with Kerberos authentication.
> This leads to the idea of having Kerberos authentication supported within 
> GIRAPH. Hopefully it would use our fast GIRAPH-37 IPC, but could also 
> interoperate with Hadoop security.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-274) Jobs still failing due to tasks timeout during INPUT_SUPERSTEP

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463176#comment-13463176
 ] 

Hudson commented on GIRAPH-274:
---

Integrated in Giraph-trunk-Commit #207 (See 
[https://builds.apache.org/job/Giraph-trunk-Commit/207/])
GIRAPH-274: Jobs still failing due to tasks timeout during
INPUT_SUPERSTEP. (nitayj via aching) (Revision 1390102)

 Result = SUCCESS
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390102
Files : 
* /giraph/trunk/CHANGELOG
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/partition/Partition.java


> Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
> --
>
> Key: GIRAPH-274
> URL: https://issues.apache.org/jira/browse/GIRAPH-274
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Jaeho Shin
>Assignee: Nitay Joffe
> Fix For: 0.2.0
>
> Attachments: GIRAPH-274-2.patch, GIRAPH-274-2.patch, 
> GIRAPH-274.3.patch, GIRAPH-274-alt-1.patch, GIRAPH-274.patch, GIRAPH-274.txt
>
>
> Even after GIRAPH-267, jobs were failing during INPUT_SUPERSTEP when some 
> workers don't get to reserve an input split, while others were loading 
> vertices for a long time.  (related to GIRAPH-246 and GIRAPH-267)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-312) Giraph needs an admin script

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463153#comment-13463153
 ] 

Avery Ching commented on GIRAPH-312:


Hi Eli, sorry for the delay on this.  It looks good to me, can you please 
rebase it?

> Giraph needs an admin script
> 
>
> Key: GIRAPH-312
> URL: https://issues.apache.org/jira/browse/GIRAPH-312
> Project: Giraph
>  Issue Type: New Feature
>  Components: conf and scripts, zookeeper
>Affects Versions: 0.2.0
>Reporter: Eli Reisman
>Assignee: Eli Reisman
>Priority: Minor
> Fix For: 0.2.0
>
> Attachments: GIRAPH-312-1.patch, GIRAPH-312-2.patch, 
> GIRAPH-312-3.patch, GIRAPH-312-3.patch, GIRAPH-312-4.patch, GIRAPH-312-5.patch
>
>
> Our zookeeper instances have very long uptimes on our cluster, and failed job 
> trees are never cleaned from memory. There is a separate shell script to do 
> this, but its not picky about which node trees it erases, and on some systems 
> some Giraph users may not have access to it.
> This patch will add a shell script to activate a new class which will use 
> Giraph conf file options or our normal -Dgiraph.XYZ command-line opts to get 
> the ZK quorum info, and clean out the remnants from its memory of old failed 
> and killed jobs. They do pile up over time.
> This led to the larger idea that Giraph needs a general giraph-admin shell 
> script as a home for stuff like this. Jakob suggested it would be a good idea 
> to put this into such a script since then admin groups can be created so that 
> not every Giraph client can run it. This script currently only has code to 
> start up the zk cleaner, but can have more options added to it as JIRA's to 
> add features crop up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-274) Jobs still failing due to tasks timeout during INPUT_SUPERSTEP

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463149#comment-13463149
 ] 

Avery Ching commented on GIRAPH-274:


Great, committed.

> Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
> --
>
> Key: GIRAPH-274
> URL: https://issues.apache.org/jira/browse/GIRAPH-274
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Jaeho Shin
>Assignee: Nitay Joffe
> Fix For: 0.2.0
>
> Attachments: GIRAPH-274-2.patch, GIRAPH-274-2.patch, 
> GIRAPH-274.3.patch, GIRAPH-274-alt-1.patch, GIRAPH-274.patch, GIRAPH-274.txt
>
>
> Even after GIRAPH-267, jobs were failing during INPUT_SUPERSTEP when some 
> workers don't get to reserve an input split, while others were loading 
> vertices for a long time.  (related to GIRAPH-246 and GIRAPH-267)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-274) Jobs still failing due to tasks timeout during INPUT_SUPERSTEP

[Nitay Joffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463121#comment-13463121
 ] 

Nitay Joffe commented on GIRAPH-274:


Oh interesting the GraphState wasn't set and was triggering some error? Weird I 
didn't see it. Anyways +1 looks fine to me.

> Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
> --
>
> Key: GIRAPH-274
> URL: https://issues.apache.org/jira/browse/GIRAPH-274
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Jaeho Shin
>Assignee: Nitay Joffe
> Fix For: 0.2.0
>
> Attachments: GIRAPH-274-2.patch, GIRAPH-274-2.patch, 
> GIRAPH-274.3.patch, GIRAPH-274-alt-1.patch, GIRAPH-274.patch, GIRAPH-274.txt
>
>
> Even after GIRAPH-267, jobs were failing during INPUT_SUPERSTEP when some 
> workers don't get to reserve an input split, while others were loading 
> vertices for a long time.  (related to GIRAPH-246 and GIRAPH-267)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-274) Jobs still failing due to tasks timeout during INPUT_SUPERSTEP

[Avery Ching (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avery Ching updated GIRAPH-274:
---

Attachment: GIRAPH-274.3.patch

+1, with minor changes.

Had to make a few small changes to get this to pass unittests and be refreshed 
with trunk.  If you're fine with this [~nitay], then I'll commit on your behalf.

> Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
> --
>
> Key: GIRAPH-274
> URL: https://issues.apache.org/jira/browse/GIRAPH-274
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Jaeho Shin
>Assignee: Nitay Joffe
> Fix For: 0.2.0
>
> Attachments: GIRAPH-274-2.patch, GIRAPH-274-2.patch, 
> GIRAPH-274.3.patch, GIRAPH-274-alt-1.patch, GIRAPH-274.patch, GIRAPH-274.txt
>
>
> Even after GIRAPH-267, jobs were failing during INPUT_SUPERSTEP when some 
> workers don't get to reserve an input split, while others were loading 
> vertices for a long time.  (related to GIRAPH-246 and GIRAPH-267)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-326) Writing input splits to ZooKeeper in parallel

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463082#comment-13463082
 ] 

Avery Ching commented on GIRAPH-326:


+1.  Really minor thing.

In 

public Void call() {

There are a bunch of log messages that start with "createInputSplits: " and 
should be "call: ".  Can you fix that and then commit? 

> Writing input splits to ZooKeeper in parallel
> -
>
> Key: GIRAPH-326
> URL: https://issues.apache.org/jira/browse/GIRAPH-326
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Maja Kabiljo
> Attachments: GIRAPH-326.patch, GIRAPH-326.patch
>
>
> (Posting issue and the patch from a colleague)
> Writing input splits to zookeeper can take a lot of time. From his 
> experiments: serial 2m45s, with 16 cores 15s.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-344) mvn test fails with java 1.7

[Nitay Joffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nitay Joffe updated GIRAPH-344:
---

Description: 
On my Mac:

JAVA_HOME=/Library/Java/Home mvn clean test
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 


JAVA_HOME=/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean 
test
[INFO] 
[INFO] BUILD FAILURE
[INFO] 

Full output: https://gist.github.com/3783472

Surefire reports doesn't have anything useful: https://gist.github.com/3783480

  was:
On my Mac:

JAVA_HOME=/Library/Java/Home mvn clean test
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 


JAVA_HOME=/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean 
test
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
https://gist.github.com/3783472

Surefire reports doesn't have anything useful: https://gist.github.com/3783480


> mvn test fails with java 1.7
> 
>
> Key: GIRAPH-344
> URL: https://issues.apache.org/jira/browse/GIRAPH-344
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Nitay Joffe
>Priority: Minor
>
> On my Mac:
> JAVA_HOME=/Library/Java/Home mvn clean test
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> JAVA_HOME=/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean 
> test
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> Full output: https://gist.github.com/3783472
> Surefire reports doesn't have anything useful: https://gist.github.com/3783480

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-344) mvn test fails with java 1.7

[Nitay Joffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nitay Joffe updated GIRAPH-344:
---

Description: 
On my Mac:

JAVA_HOME=/Library/Java/Home mvn clean test
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 


JAVA_HOME=/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean 
test
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
https://gist.github.com/3783472

Surefire reports doesn't have anything useful: https://gist.github.com/3783480

  was:
On my Mac:

JAVA_HOME=/Library/Java/Home mvn clean test
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 


/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean test
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
https://gist.github.com/3783472

Surefire reports doesn't have anything useful: https://gist.github.com/3783480


> mvn test fails with java 1.7
> 
>
> Key: GIRAPH-344
> URL: https://issues.apache.org/jira/browse/GIRAPH-344
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Nitay Joffe
>Priority: Minor
>
> On my Mac:
> JAVA_HOME=/Library/Java/Home mvn clean test
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> JAVA_HOME=/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean 
> test
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> https://gist.github.com/3783472
> Surefire reports doesn't have anything useful: https://gist.github.com/3783480

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (GIRAPH-344) mvn test fails with java 1.7

[Nitay Joffe (JIRA)]

Nitay Joffe created GIRAPH-344:
--

 Summary: mvn test fails with java 1.7
 Key: GIRAPH-344
 URL: https://issues.apache.org/jira/browse/GIRAPH-344
 Project: Giraph
  Issue Type: Bug
Affects Versions: 0.2.0
Reporter: Nitay Joffe
Priority: Minor


On my Mac:

JAVA_HOME=/Library/Java/Home mvn clean test
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 


/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents/Home mvn clean test
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
https://gist.github.com/3783472

Surefire reports doesn't have anything useful: https://gist.github.com/3783480

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-340) Added client/server ExecutionHandlers to Netty to avoid and added WrappedAdaptiveReceiveBufferSizePredictorFactory to debug/predict the size of the incoming messages

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463055#comment-13463055
 ] 

Avery Ching commented on GIRAPH-340:


Can someone take a quick look at this?  It's not very long, but is an important 
Netty improvement.

> Added client/server ExecutionHandlers to Netty to avoid and added 
> WrappedAdaptiveReceiveBufferSizePredictorFactory to debug/predict the size of 
> the incoming messages 
> --
>
> Key: GIRAPH-340
> URL: https://issues.apache.org/jira/browse/GIRAPH-340
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-340.2.patch, GIRAPH-340.patch
>
>
> - Added client and server ExecutionHandlers to Netty to avoid doing the 
> business logic with I/O threads (This used to clog up the pipe)
> - Added WrappedAdaptiveReceiveBufferSizePredictorFactory to debug/predict the 
> size of the incoming messages and provide better performance tuning on the 
> server instead of receiving a bunch of small messages.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-339) Optimization to SimpleMessageStore#addPartitionMessages to avoid getting the partition map for every vertex

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463049#comment-13463049
 ] 

Hudson commented on GIRAPH-339:
---

Integrated in Giraph-trunk-Commit #206 (See 
[https://builds.apache.org/job/Giraph-trunk-Commit/206/])
GIRAPH-339: Optimization to SimpleMessageStore#addPartitionMessages to
avoid getting the partition map for every vertex. (Revision 1390018)

 Result = SUCCESS
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390018
Files : 
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java


> Optimization to SimpleMessageStore#addPartitionMessages to avoid getting the 
> partition map for every vertex
> ---
>
> Key: GIRAPH-339
> URL: https://issues.apache.org/jira/browse/GIRAPH-339
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-339.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-329) Improve network performance

[Avery Ching (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463040#comment-13463040
 ] 

Avery Ching commented on GIRAPH-329:


Thanks for your comments Eli, I'm glad you liked it.

> Improve network performance
> ---
>
> Key: GIRAPH-329
> URL: https://issues.apache.org/jira/browse/GIRAPH-329
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-329.patch, nettyImprovement.png, 
> NettyImprovements.xlsx
>
>
> I have been working on some network improvements that appear to give 1.5x to 
> 2x.  Unfortunately, this is a very large change and very hard to split out.  
> Will give an update soon.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-337) Make a specific Giraph configuration for Class caching and specific Giraph configuration

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463027#comment-13463027
 ] 

Hudson commented on GIRAPH-337:
---

Integrated in Giraph-trunk-Commit #205 (See 
[https://builds.apache.org/job/Giraph-trunk-Commit/205/])
GIRAPH-337: Make a specific Giraph configuration for Class caching and
specific Giraph configuration (Revision 1390014)

 Result = SUCCESS
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390014
Files : 
* /giraph/trunk/CHANGELOG
* /giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java
* /giraph/trunk/src/main/java/org/apache/giraph/GiraphRunner.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfigurable.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfiguration.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
* /giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java
* /giraph/trunk/src/main/java/org/apache/giraph/bsp/BspInputFormat.java
* /giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
* /giraph/trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java
* /giraph/trunk/src/main/java/org/apache/giraph/comm/SendMessageCache.java
* /giraph/trunk/src/main/java/org/apache/giraph/comm/ServerData.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStore.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
* /giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClientServer.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java
* /giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyServer.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientServer.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/ResponseClientHandler.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/WorkerRequestReservedMap.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/SendPartitionMessagesRequest.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/SendPartitionMutationsRequest.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/SendVertexRequest.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/WritableRequest.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/examples/GeneratedVertexReader.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleVertexWithWorkerContext.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspService.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceMaster.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspUtils.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/EdgeListVertex.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/GiraphJob.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/HashMapVertex.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/MasterCompute.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/MasterThread.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/MutableVertex.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/SimpleMutableVertex.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/SimpleVertex.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/Vertex.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/VertexMutations.java
* /giraph/trunk/src/main/java/org/apache/giraph/graph/VertexResolver.java
* 
/giraph/trunk/src/main/java/org/apache/giraph/graph/partition/BasicPartitionOwn

[jira] [Commented] (GIRAPH-336) style violations in giraph-formats-contrib

[Brian Femiano (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462964#comment-13462964
 ] 

Brian Femiano commented on GIRAPH-336:
--

K. 

Once the GIRAPH-343 is approved and committed I'll continue the style check 
updates in the new package. 

> style violations in giraph-formats-contrib
> --
>
> Key: GIRAPH-336
> URL: https://issues.apache.org/jira/browse/GIRAPH-336
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Greg Malewicz
>Assignee: Brian Femiano
> Attachments: style_violations.txt
>
>
> A list of 590 style violations in giraph-formats-contrib as reported by "mvn 
> install" is attached.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-336) style violations in giraph-formats-contrib

[Nitay Joffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462957#comment-13462957
 ] 

Nitay Joffe commented on GIRAPH-336:


Brian see GIRAPH-343

> style violations in giraph-formats-contrib
> --
>
> Key: GIRAPH-336
> URL: https://issues.apache.org/jira/browse/GIRAPH-336
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Greg Malewicz
>Assignee: Brian Femiano
> Attachments: style_violations.txt
>
>
> A list of 590 style violations in giraph-formats-contrib as reported by "mvn 
> install" is attached.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-343) Use published hcatalog jars.

[Nitay Joffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nitay Joffe updated GIRAPH-343:
---

Attachment: GIRAPH-343-2.patch

Forgot to move code from src/hcatalog to src/main. Here's full patch.

> Use published hcatalog jars.
> 
>
> Key: GIRAPH-343
> URL: https://issues.apache.org/jira/browse/GIRAPH-343
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
> Attachments: GIRAPH-343-2.patch, GIRAPH-343.patch
>
>
> Shouldn't need special profile now that hcatalog has started pushing their 
> jars to maven.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-343) Use published hcatalog jars.

[Nitay Joffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nitay Joffe updated GIRAPH-343:
---

Attachment: GIRAPH-343.patch

> Use published hcatalog jars.
> 
>
> Key: GIRAPH-343
> URL: https://issues.apache.org/jira/browse/GIRAPH-343
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Nitay Joffe
>Priority: Minor
> Attachments: GIRAPH-343.patch
>
>
> Shouldn't need special profile now that hcatalog has started pushing their 
> jars to maven.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-329) Improve network performance

[Eli Reisman (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462932#comment-13462932
 ] 

Eli Reisman commented on GIRAPH-329:


Thanks again for doing this, after struggling with 214 I know how much work it 
is! Knowing this problem well, I have learned so much from reading your 
solution. I think in 214 I had an idea of the way I wanted it to be and just 
wouldn't let go. The 329 solution is just right.

Anyway in conclusion...shame on me! And thanks again Avery!


> Improve network performance
> ---
>
> Key: GIRAPH-329
> URL: https://issues.apache.org/jira/browse/GIRAPH-329
> Project: Giraph
>  Issue Type: Improvement
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-329.patch, nettyImprovement.png, 
> NettyImprovements.xlsx
>
>
> I have been working on some network improvements that appear to give 1.5x to 
> 2x.  Unfortunately, this is a very large change and very hard to split out.  
> Will give an update soon.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (GIRAPH-343) Use published hcatalog jars.

[Nitay Joffe (JIRA)]

Nitay Joffe created GIRAPH-343:
--

 Summary: Use published hcatalog jars.
 Key: GIRAPH-343
 URL: https://issues.apache.org/jira/browse/GIRAPH-343
 Project: Giraph
  Issue Type: Improvement
Reporter: Nitay Joffe
Priority: Minor


Shouldn't need special profile now that hcatalog has started pushing their jars 
to maven.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-274) Jobs still failing due to tasks timeout during INPUT_SUPERSTEP

[Nitay Joffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462729#comment-13462729
 ] 

Nitay Joffe commented on GIRAPH-274:


Posted cleaned up GIRAPH-274-2.patch.

> Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
> --
>
> Key: GIRAPH-274
> URL: https://issues.apache.org/jira/browse/GIRAPH-274
> Project: Giraph
>  Issue Type: Bug
>Affects Versions: 0.2.0
>Reporter: Jaeho Shin
>Assignee: Nitay Joffe
> Fix For: 0.2.0
>
> Attachments: GIRAPH-274-2.patch, GIRAPH-274-2.patch, 
> GIRAPH-274-alt-1.patch, GIRAPH-274.patch, GIRAPH-274.txt
>
>
> Even after GIRAPH-267, jobs were failing during INPUT_SUPERSTEP when some 
> workers don't get to reserve an input split, while others were loading 
> vertices for a long time.  (related to GIRAPH-246 and GIRAPH-267)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-339) Optimization to SimpleMessageStore#addPartitionMessages to avoid getting the partition map for every vertex

[Alessandro Presta (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462649#comment-13462649
 ] 

Alessandro Presta commented on GIRAPH-339:
--

This is good, +1.

> Optimization to SimpleMessageStore#addPartitionMessages to avoid getting the 
> partition map for every vertex
> ---
>
> Key: GIRAPH-339
> URL: https://issues.apache.org/jira/browse/GIRAPH-339
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-339.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-340) Added client/server ExecutionHandlers to Netty to avoid and added WrappedAdaptiveReceiveBufferSizePredictorFactory to debug/predict the size of the incoming messages

[Avery Ching (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avery Ching updated GIRAPH-340:
---

Attachment: GIRAPH-340.2.patch

Updated per GIRAPH-337 changes.

> Added client/server ExecutionHandlers to Netty to avoid and added 
> WrappedAdaptiveReceiveBufferSizePredictorFactory to debug/predict the size of 
> the incoming messages 
> --
>
> Key: GIRAPH-340
> URL: https://issues.apache.org/jira/browse/GIRAPH-340
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-340.2.patch, GIRAPH-340.patch
>
>
> - Added client and server ExecutionHandlers to Netty to avoid doing the 
> business logic with I/O threads (This used to clog up the pipe)
> - Added WrappedAdaptiveReceiveBufferSizePredictorFactory to debug/predict the 
> size of the incoming messages and provide better performance tuning on the 
> server instead of receiving a bunch of small messages.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (GIRAPH-337) Make a specific Giraph configuration for Class caching and specific Giraph configuration

[Avery Ching (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avery Ching updated GIRAPH-337:
---

Attachment: GIRAPH-337.3.patch

Updated per Eugene's comment about moving 
WrappedAdaptiveReceiveBufferSizePredictorFactory.java to GIRAPH-340.

> Make a specific Giraph configuration for Class caching and specific Giraph 
> configuration
> 
>
> Key: GIRAPH-337
> URL: https://issues.apache.org/jira/browse/GIRAPH-337
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-337.2.patch, GIRAPH-337.3.patch, GIRAPH-337.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: GIRAPH-337

[Avery Ching]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7247/
---

(Updated Sept. 25, 2012, 8:33 a.m.)


Review request for giraph.


Changes
---

Updated per Eugene's comment about moving 
WrappedAdaptiveReceiveBufferSizePredictorFactory.java to GIRAPH-340.


Description
---

As per Eugene's request.


This addresses bug GIRAPH-337.
https://issues.apache.org/jira/browse/GIRAPH-337


Diffs (updated)
-

  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphRunner.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfigurable.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfiguration.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/bsp/BspInputFormat.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/SendMessageCache.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/ServerData.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStore.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClientServer.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyServer.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientServer.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/ResponseClientHandler.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/handler/WorkerRequestReservedMap.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/SendPartitionMessagesRequest.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/SendPartitionMutationsRequest.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/SendVertexRequest.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/requests/WritableRequest.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/examples/GeneratedVertexReader.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java
 1389754 
  
http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java

[jira] [Commented] (GIRAPH-337) Make a specific Giraph configuration for Class caching and specific Giraph configuration

[Eugene Koontz (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462552#comment-13462552
 ] 

Eugene Koontz commented on GIRAPH-337:
--

+1 provided that a couple of unused parts in the patch that are for other 
sub-tasks of GIRAPH-329 are moved to their proper sub-tasks (as mentioned on 
https://reviews.apache.org/r/7247/)

> Make a specific Giraph configuration for Class caching and specific Giraph 
> configuration
> 
>
> Key: GIRAPH-337
> URL: https://issues.apache.org/jira/browse/GIRAPH-337
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-337.2.patch, GIRAPH-337.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-337) Make a specific Giraph configuration for Class caching and specific Giraph configuration

[Eugene Koontz (JIRA)]

[ 
https://issues.apache.org/jira/browse/GIRAPH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462545#comment-13462545
 ] 

Eugene Koontz commented on GIRAPH-337:
--

Looks good overall. The patch looks big, but most of the diff's text is due to 
moving methods and member variables from GiraphJob to GiraphConfiguration.

> Make a specific Giraph configuration for Class caching and specific Giraph 
> configuration
> 
>
> Key: GIRAPH-337
> URL: https://issues.apache.org/jira/browse/GIRAPH-337
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-337.2.patch, GIRAPH-337.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: GIRAPH-337

[Avery Ching]

> On Sept. 25, 2012, 8:19 a.m., Eugene Koontz wrote:
> > http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java,
> >  line 49
> > 
> >
> > We have to pass an additional second (configuration) param here because 
> > context's .getConfiguration() method returns an (overly general) 
> > Configuration object, but we need a ImmutableClassesGiraphConfiguration 
> > object.
> 
> Eugene Koontz wrote:
> Sorry, did not mean to open an issue here.

yes.


> On Sept. 25, 2012, 8:19 a.m., Eugene Koontz wrote:
> > http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java,
> >  line 783
> > 
> >
> > Not used in this patch, but I see it in GIRAPH-339. I understand why 
> > it's here in this patch since this patch was split out from GIRAPH-339.

=)


> On Sept. 25, 2012, 8:19 a.m., Eugene Koontz wrote:
> > http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/WrappedAdaptiveReceiveBufferSizePredictorFactory.java,
> >  line 1
> > 
> >
> > I don't see this class used anywhere.

I will move this to GIRAPH-340.  Sorry about mixing it in this patch.


> On Sept. 25, 2012, 8:19 a.m., Eugene Koontz wrote:
> > http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java,
> >  line 132
> > 
> >
> > This -N option is nice to have; the default 
> > "org.apache.giraph.benchmark.PageRankBenchmark" is a bit verbose.
> 
> Eugene Koontz wrote:
> Sorry, did not mean to open an issue here.

=)


- Avery


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7247/#review11878
---


On Sept. 25, 2012, 6:34 a.m., Avery Ching wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7247/
> ---
> 
> (Updated Sept. 25, 2012, 6:34 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> ---
> 
> As per Eugene's request.
> 
> 
> This addresses bug GIRAPH-337.
> https://issues.apache.org/jira/browse/GIRAPH-337
> 
> 
> Diffs
> -
> 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphRunner.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfigurable.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfiguration.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/bsp/BspInputFormat.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/SendMessageCache.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/ServerData.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java
>  1389724

Re: Review Request: GIRAPH-337

[Eugene Koontz]

> On Sept. 25, 2012, 8:19 a.m., Eugene Koontz wrote:
> > http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java,
> >  line 132
> > 
> >
> > This -N option is nice to have; the default 
> > "org.apache.giraph.benchmark.PageRankBenchmark" is a bit verbose.

Sorry, did not mean to open an issue here.


> On Sept. 25, 2012, 8:19 a.m., Eugene Koontz wrote:
> > http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java,
> >  line 49
> > 
> >
> > We have to pass an additional second (configuration) param here because 
> > context's .getConfiguration() method returns an (overly general) 
> > Configuration object, but we need a ImmutableClassesGiraphConfiguration 
> > object.

Sorry, did not mean to open an issue here.


- Eugene


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7247/#review11878
---


On Sept. 25, 2012, 6:34 a.m., Avery Ching wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7247/
> ---
> 
> (Updated Sept. 25, 2012, 6:34 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> ---
> 
> As per Eugene's request.
> 
> 
> This addresses bug GIRAPH-337.
> https://issues.apache.org/jira/browse/GIRAPH-337
> 
> 
> Diffs
> -
> 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphRunner.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfigurable.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfiguration.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/bsp/BspInputFormat.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/SendMessageCache.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/ServerData.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClientServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/WrappedAdaptive

Re: Review Request: GIRAPH-337

[Eugene Koontz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7247/#review11878
---



http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java


Not used in this patch, but I see it in GIRAPH-339. I understand why it's 
here in this patch since this patch was split out from GIRAPH-339.



http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java


This -N option is nice to have; the default 
"org.apache.giraph.benchmark.PageRankBenchmark" is a bit verbose.



http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java


We have to pass an additional second (configuration) param here because 
context's .getConfiguration() method returns an (overly general) Configuration 
object, but we need a ImmutableClassesGiraphConfiguration object.



http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/WrappedAdaptiveReceiveBufferSizePredictorFactory.java


I don't see this class used anywhere.


- Eugene Koontz


On Sept. 25, 2012, 6:34 a.m., Avery Ching wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7247/
> ---
> 
> (Updated Sept. 25, 2012, 6:34 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> ---
> 
> As per Eugene's request.
> 
> 
> This addresses bug GIRAPH-337.
> https://issues.apache.org/jira/browse/GIRAPH-337
> 
> 
> Diffs
> -
> 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/GiraphRunner.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfigurable.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/ImmutableClassesGiraphConfiguration.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/bsp/BspInputFormat.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/SendMessageCache.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/ServerData.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterClientServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyServer.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java
>  1389724 
>   
> http://svn.apache.org/repos/asf/giraph/trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientServer.java
>  1389724 
>   
> ht

[jira] [Updated] (GIRAPH-337) Make a specific Giraph configuration for Class caching and specific Giraph configuration

[Avery Ching (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GIRAPH-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avery Ching updated GIRAPH-337:
---

Attachment: GIRAPH-337.2.patch

One small change to RPCCommunciations.java to move 

import org.apache.giraph.ImmutableClassesGiraphConfiguration;

out of the munged imports.

> Make a specific Giraph configuration for Class caching and specific Giraph 
> configuration
> 
>
> Key: GIRAPH-337
> URL: https://issues.apache.org/jira/browse/GIRAPH-337
> Project: Giraph
>  Issue Type: Sub-task
>Reporter: Avery Ching
>Assignee: Avery Ching
> Attachments: GIRAPH-337.2.patch, GIRAPH-337.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira