Hi,
I'm not sure aggregators require necessarily high traffic. Aggregators are
aggregated locally on the worker before they are aggregated on the
(corresponding) master worker.
Anyway, assuming you want to proceed, my understanding is that you want
vertices on the same worker to share
Do you have any suggestions? Any would be much appreciated :)
Kind regards,
Alessio
--
Claudio Martella
with IllegalArgumentException upon
deserialization
Best,
Maja
--
Claudio Martella
Hello,
is anybody in the list able to get the standard job counters printed at the
end of the jobs when using pure YARN?
I can get the logs, but I cannot find the usual stats printed to the
command line on mapreduce.
Thanks,
Claudio
--
Claudio Martella
, disseminate, or otherwise use the
information contained in this communication. Internet communications
cannot be guaranteed to be timely, secure, error or virus-free. The
sender does not accept liability for any errors or omissions
--
Claudio Martella
,
Roman.
--
Claudio Martella
AM, Claudio Martella
claudio.marte...@gmail.com wrote:
I am indeed having some problems. mvn install will fail because the test
is
opening too many files:
[snip]
I have to investigate why this happens. I'm not using a different ulimit
than what I have on my Mac OS X by default. Where
/java/org/apache/giraph/comm/netty/SaslNettyServer.java:[108,11]
cannot find symbol
[ERROR] symbol: variable SaslPropertiesResolver
[ERROR] location: class org.apache.giraph.comm.netty.SaslNettyServer
--
Claudio Martella
, Oct 31, 2014 at 9:00 AM, Claudio Martella
claudio.marte...@gmail.com wrote:
Oh, thanks for the info!
On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik ro...@shaposhnik.org
wrote:
On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella
claudio.marte...@gmail.com wrote:
Hi Roman
of hashing or range).
In principle you can. You would need to organize splits so that they
contain all the data for each particular worker, and then assign relevant
splits to the corresponding worker.
Thanks for your time and help!
Cheers,
Martin
--
Claudio Martella
.
Thanks,
Roman.
--
Claudio Martella
Oh, thanks for the info!
On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik ro...@shaposhnik.org
wrote:
On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella
claudio.marte...@gmail.com wrote:
Hi Roman,
thanks again for this. I have had a look at the staging site so far (our
cluster has been
from workers. A
worker can handle zero or more partitions. You can make each worker handle
one partition, but we typically like multiple partitions since we can use
multi-threading per machine.
On 10/23/14, 9:04 AM, Claudio Martella wrote:
the way mappers (or containers) and hence
to be timely, secure, error or virus-free. The
sender does not accept liability for any errors or omissions
--
Claudio Martella
Dickinson
Street, Amherst MA 01002 | matthewcornell.org
--
Claudio Martella
@gmail.com wrote:
Thanks for help, let me look into sub projects.
Cheers !
On 22 May 2014 20:20, Claudio Martella claudio.marte...@gmail.comwrote:
You can have a look at the giraph-rexster subproject within the giraph
codebase. there is also some documentation on our site.
On Thu, May
Subject: n-ary relationship on Giraph
To: user@giraph.apache.org
Hi,
Does Giraph supports n-ary relationships? I need to store some properties
of triplet vertex - edge - vertex and be able to query with those
properties.
Sujan Perera
--
Claudio Martella
.
I'm more interested on *input/output formats* to be used between these
three?
BR
Arun
--
Claudio Martella
superstep each node writes a nodeID - label pair to an
HBase table.
Do you have any general hints where I can look at?
I absolutely have no clue where to start
Thanks for your help!
Regards
Pascal
--
Claudio Martella
message values.
Please help.
Nishant Gandhi
M.Tech. CSE
IIT Patna
--
Sent from Android Mobile
--
Claudio Martella
--
Claudio Martella
be able to increase that parameter, depending on the
message size.
Hope you can help me.
Regards Pascal
Hope this helps.
--
Claudio Martella
();
}
}
}
--
Claudio Martella
. This way the mapping of number to URL
would be stored in the Node.
Problem is that I have to assign the numbers in a sequential way to ensure
there are no holes and numbers are unique. No Idea if this is even possible
in Giraph.
Any input is welcome
cheers Martin
--
Claudio Martella
:
- Facebook
Anybody else?
Thanks!
Claudio
--
Claudio Martella
or in a *.xml file? I tried -Dgiraph.zkSessionMsecTimeout=90( googled)
in command line but failed. Thanks!
Best Regards,
Suijian
--
Claudio Martella
: Session 0x14543222b640009 for
server null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
Best Regards,
Suijian
--
Claudio Martella
is that it is slow.by using giraph will I be able in
improving its performances on very large graph?
I know these can seem very basic questions, but I'm pretty new to giraph
and I'm trying to understand it
Thank you
Angelo
--
Claudio Martella
:48 PM, Angelo Immediata wrote:
hi Claudio
so, if I understood correctly, it has no sense to use Giraph for shortest
path calculation in my scenario
Am I right?
2014-03-26 13:27 GMT+01:00 Claudio Martella claudio.marte...@gmail.com
:
It looks like you're expecting to use Giraph
is, essentially, an
implementation detail of Giraph.
What's the reason for you to need this info?
Thanks,
Roman.
--
Claudio Martella
/06 16:17:47 INFO job.JobProgressTracker: Data from 39 workers -
Compute superstep 0: 5854829 out of 4920 vertices computed; 181 out of
1521 partitions computed
Best Regards,
Suijian
--
Claudio Martella
stucture of the graph should be kept)? Thanks!
Best Regards,
Suijian
--
Claudio Martella
Summit.
Thanks,
Roman.
--
Claudio Martella
(july 2013)
entries on the mailing list and none existed by then. Have things changed
since then, or do I have to write my own?
When I write my own what's a good base class to start from?
cheers Martin
--
Claudio Martella
Yes, Giraph hijacks mapper tasks, and then does everything else on its
own.
On Fri, Feb 7, 2014 at 12:39 PM, Alexander Frolov
alexndr.fro...@gmail.comwrote:
On Fri, Feb 7, 2014 at 2:30 PM, Claudio Martella
claudio.marte...@gmail.com wrote:
On Fri, Feb 7, 2014 at 9:44 AM, Alexander
-g585511f
Hadoop: hadoop-0.20.203.0, hadoop-rdma-0.9.8
Best,
Alex
--
Claudio Martella
that executes the mapper function on the
records.
On Thu, Feb 6, 2014 at 2:28 PM, Claudio Martella
claudio.marte...@gmail.com wrote:
Hi Alex,
answers are inline.
On Thu, Feb 6, 2014 at 11:22 AM, Alexander Frolov
alexndr.fro...@gmail.com wrote:
Hi, folks!
I have started small research
On Thu, Feb 6, 2014 at 12:15 PM, Alexander Frolov
alexndr.fro...@gmail.comwrote:
On Thu, Feb 6, 2014 at 3:00 PM, Claudio Martella
claudio.marte...@gmail.com wrote:
On Thu, Feb 6, 2014 at 11:56 AM, Alexander Frolov
alexndr.fro...@gmail.com wrote:
Hi Claudio,
thank you.
If I
On Thu, Feb 6, 2014 at 3:04 PM, Alexander Frolov
alexndr.fro...@gmail.comwrote:
Claudio,
thank you very much for your help.
On Thu, Feb 6, 2014 at 4:06 PM, Claudio Martella
claudio.marte...@gmail.com wrote:
On Thu, Feb 6, 2014 at 12:15 PM, Alexander Frolov
alexndr.fro...@gmail.com
.
Best,
Alex
--
Claudio Martella
...@gmail.comwrote:
Yes but you can disable the counters per superstep, if you don't need the
data, and than I had around 2000 after which my algorithm stopped.
Cheers
Peter
On Jan 29, 2014 4:22 PM, Claudio Martella claudio.marte...@gmail.com
wrote:
the limit is currently defined
configurations wrong?
best regards,
Sebastian
--
Claudio Martella
claudio.marte...@gmail.com
algorithm??
Thanks
Jyoti
--
Claudio Martella
claudio.marte...@gmail.com
results in memory until the algorithm finishes.
--sebastian
--
Claudio Martella
claudio.marte...@gmail.com
try to pick it up back this month. Still very much
would
like to help make it happen.
Thanks,
Roman.
--
Claudio Martella
claudio.marte...@gmail.com
which
goes into Giraph in a later release.
Will there be a voting for such issues?
Mirko
--
Claudio Martella
claudio.marte...@gmail.com
the graph
Thanks
Pushparaj
--
Claudio Martella
claudio.marte...@gmail.com
.impl.MetricsSystemImpl: MapTask metrics system
stopped.
2013-11-20 15:22:55,122 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
shutdown complete.
Cheres,
--
Gianmarco
--
Claudio Martella
claudio.marte...@gmail.com
*
Can I use the RandomVertex and the RandomEdgeInputFormat to build random
graphs on the fly?
Thanks a lot in advance.
Best wishes
Mirko
--
Claudio Martella
claudio.marte...@gmail.com
about the *config parameters*, I have to provide for the
PseudoRandom ... InputFormat and where is the code for the *watts
strogatz model* you mentioned in a previous post?
Best wishes
Mirko
--
Claudio Martella
claudio.marte...@gmail.com
prediction could be accomplished
with Giraph?
Any input is highly appreciated :)
Thanks
Pascal
--
Claudio Martella
claudio.marte...@gmail.com
--
Claudio Martella
claudio.marte...@gmail.com
are really appreciated..
Regards
Jyoti
--
Claudio Martella
claudio.marte...@gmail.com
(NioClientSocketPipelineSink.java:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
--
Claudio Martella
how many mapper tasks do you have set for each node? how many workers are
you using for giraph?
On Fri, Oct 18, 2013 at 7:12 PM, YAN Da ya...@ust.hk wrote:
Dear Claudio Martella,
I don't quite get what you mean. Our cluster has 15 servers each with 24
cores, so ideally there can be 15*24
,
Jyotirmoy Sundi
Data Engineer,
Admobius
San Francisco, CA 94158
--
Best Regards,
Jyotirmoy Sundi
Data Engineer,
Admobius
San Francisco, CA 94158
--
Best Regards,
Jyotirmoy Sundi
Data Engineer,
Admobius
San Francisco, CA 94158
--
Claudio Martella
claudio.marte
.
Thanks for your help.
Best,
-Yi
--
Claudio Martella
claudio.marte...@gmail.com
of the message it is currenty processing.?
Thanks
Jyoti
--
Claudio Martella
claudio.marte...@gmail.com
: Failed map tasks=1
I appreciate any help. Maybe I did it wrong.
Andro.
--
Claudio Martella
claudio.marte...@gmail.com
ha ve build w/ hadoop_cdh4.1.2 parameter.
Something is changed, monday i report the result. Now the farm is closed.
Il giorno 05/ott/2013 14:06, Claudio Martella
claudio.marte...@gmail.com ha scritto:
Oh, right, -vof is in trunk. Anyway it looks like you built giraph for
the wrong profile
-achive first I would have avoided
the last post.
Now zk issue are resolved.
2013/10/7 Claudio Martella claudio.marte...@gmail.com
Can you try applying this one first?
http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff
On Mon, Oct 7, 2013 at 8:40 AM, Silvio Di gregorio
...@gmail.com wrote:
org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option:
-vof
in 1.0 version is
-of,--outputFormat arg Vertex output format
-op,--outputPath arg Vertex output path
2013/10/4 Claudio Martella claudio.marte...@gmail.com
did you
org.apache.hadoop.mapreduce.JobContext, but class was expected
at
org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
..
Thanks in advance
--
Claudio Martella
claudio.marte...@gmail.com
workload to work on ?
Thanks!
Wei
--
Claudio Martella
claudio.marte...@gmail.com
try to pass a different directory for writing, e.g.
/tmp/foobar?
On Mon, Sep 23, 2013 at 1:28 PM, Dionysis Logothetis
dlogothe...@gmail.comwrote:
offloadPartition: Failed to create directory
--
Claudio Martella
claudio.marte...@gmail.com
org.apache.giraph.graph.GraphTaskManager:
execute: 6 partitions to process with 1 compute thread(s), originally 1
thread(s) on superstep 7
Does this really mean that it uses only one thread?
Cheers,
Christian
--
Claudio Martella
claudio.marte...@gmail.com
is no more
heap space, and it crashes with OOM.
Has anybody had this problem with giraph failing to make HDFS directories?
--
Claudio Martella
claudio.marte...@gmail.com
? Is
there some agreement on the minimum amount of heap necessary for OOC
to succeed, to guide the choice of Mapper heap amount?
Either way, I will try increasing mapper heap again as much as
possible, which hopefully will run.
On 9/9/13, Claudio Martella claudio.marte...@gmail.com wrote:
did you
? (boolean) */
BooleanConfOption USE_SUPERSTEP_COUNTERS =
new BooleanConfOption(giraph.useSuperstepCounters, true,
Use superstep counters? (boolean));
On 9/9/13 6:43 AM, Claudio Martella wrote:
No, I used a different counters limit on that hadoop version. Setting
certain rules as input and can find
these missing links and create them?
Thanks,
Sushant
--
Claudio Martella
claudio.marte...@gmail.com
MasterCompute class?
Cheers,
Christian
2013/9/10 Claudio Martella claudio.marte...@gmail.com
one the command line, you can use the -D option after the GiraphRunner
class before the GiraphRunner specific parameters, e.g. -D giraph.
useSuperstepCounters=false
On Tue, Sep 10, 2013 at 1:15 PM
using the wrong binary. Currently I am including the giraph
dependencies with the jar, and running using hadoop jar.
On 9/7/13, Claudio Martella claudio.marte...@gmail.com wrote:
OOC is used also at input superstep. try to decrease the number of
partitions kept in memory.
On Sat, Sep 7
to
ensure it functions?
Regards,
Alexander Asplund
--
Claudio Martella
claudio.marte...@gmail.com
SQL table since Giraph is built on top of
Hadoop. But I do not seem to figure this part out. Do you have a class
similar to the DBInputFormat in Hadoop? Thank you very much for your help.
--
Claudio Martella
claudio.marte...@gmail.com
in the Giraph world?
Have a nice Sunday.
Best wishes
Mirko
--
--
Mirko Kämpf
*Trainer* @ Cloudera
tel: +49 *176 20 63 51 99*
skype: *kamir1604*
mi...@cloudera.com
On Wed, Aug 21, 2013 at 3:30 PM, Claudio Martella
claudio.marte...@gmail.com wrote:
As I said, the injection of the new
the snapshot is stored.
dataDir=/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
server.1=localhost:2888:3888
Thanks for any help,
Ken
--
Claudio Martella
claudio.marte...@gmail.com
--
Claudio Martella
claudio.marte...@gmail.com
Thanks for any help,
Ken
--
Claudio Martella
claudio.marte...@gmail.com
--
Claudio Martella
claudio.marte...@gmail.com
--
Claudio Martella
claudio.marte...@gmail.com
--
Claudio Martella
claudio.marte...@gmail.com
appreciated.
Thank you,
Ken
--
Pradeep Kumar
--
Claudio Martella
claudio.marte...@gmail.com
and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
server.1=localhost:2888:3888
Thanks for any help,
Ken
--
Claudio Martella
claudio.marte...@gmail.com
:port,zk5:port
Please suggest what is wrong with this invocation.
Thanks
Arun Ramani
--
Claudio Martella
claudio.marte...@gmail.com
zookeeper hostnames. These 5
hosts make a zookeeper quorum. Please let me know how to pass this.
Thanks
Arun Ramani
From: Claudio Martella claudio.marte...@gmail.com
Reply-To: user@giraph.apache.org user@giraph.apache.org
Date: Thursday, August 29, 2013 2:18 PM
To: user@giraph.apache.org
OK, then I'm going to open an issue for that.
On Mon, Aug 26, 2013 at 11:23 AM, Vivek Sembium vivek.semb...@gmail.comwrote:
Yes for the zookeeper problem I passed a seperate jar through -libjars
command. If I use additional jars zookeeper fails.
On Aug 26, 2013 2:51 PM, Claudio Martella
, but this only appears to work when we list one node
as a zookeeper.
On Mon, Aug 26, 2013 at 8:55 AM, Claudio Martella
claudio.marte...@gmail.com wrote:
OK, then I'm going to open an issue for that.
On Mon, Aug 26, 2013 at 11:23 AM, Vivek Sembium
vivek.semb...@gmail.comwrote:
Yes
, exactly. We couldn't get it to work otherwise.
On Mon, Aug 26, 2013 at 11:00 AM, Claudio Martella
claudio.marte...@gmail.com wrote:
you mean by running zookeeper independently?
On Mon, Aug 26, 2013 at 3:16 PM, Kyle Orlando
kyle.r.orla...@gmail.comwrote:
We were also experiencing
--
Claudio Martella
claudio.marte...@gmail.com
new splits and distribute to the workers from given a path in DFS?
Best Regards,
Marco Lotz
--
*From:* Claudio Martella claudio.marte...@gmail.com
*Sent:* 14 August 2013 15:25
*To:* user@giraph.apache.org
*Subject:* Re: Dynamic Graphs
Hi Marco,
Giraph
of tasks, and maybe abstraction of the
user-code-specific part into the API so that users are not concerned with
the actual message passing (message passing is done by the framework).
Thanks
Yang
--
Claudio Martella
claudio.marte...@gmail.com
$GIRAPH_HOME/giraph-core/target, $GIRAPH_HOME/giraph-hbase/target,
etc. Are these scripts out of date, or have I built my project
incorrectly?
Thanks
--
Kyle Orlando
Computer Engineering Major
University of Maryland
--
Claudio Martella
claudio.marte...@gmail.com
jar command above), and still, I couldn't find
aggregatorValues on my HDFS.
Could someone help me out, or at the very least rectify any
misconceptions and uncertainties that I have?
On Wed, Jul 24, 2013 at 12:25 PM, Claudio Martella
claudio.marte...@gmail.com wrote:
Hi Kyle,
you can
Am I the only one that recently is experiencing problems with zookeeper? I
get the workers failing to connect to zookeeper. I presume it is not
starting at all. I'm using trunk and hadoop 1.0.3. Used to work smoothly.
--
Claudio Martella
claudio.marte...@gmail.com
are detected by
Giraph, so why not allow also a global vertex ID factory...
Cheers,
Christian
--
Claudio Martella
claudio.marte...@gmail.com
, but I did not find any documentation
on these details. If this is documented somewhere, it would be helpful to
get a link.
Cheers,
Christian
--
Claudio Martella
claudio.marte...@gmail.com
? (a tab separated file with a .tsv
extension)
Is the input class correct?
Is the output class correct?
Thank you,
Peter
On 3 June 2013 01:05, Claudio Martella claudio.marte...@gmail.com wrote:
Hi Peter,
shortly, those are abstract classes, that's why you cannot instantiate
them. You'll
)
*
* at java.lang.reflect.Method.invoke(Method.java:597)*
* at org.apache.hadoop.util.RunJar.main(RunJar.java:156)*
On 3 June 2013 21:00, Claudio Martella claudio.marte...@gmail.com wrote:
with the only problem that you picked an abstract class again...
I advised you to use an inputformat that has
Master in Distributed Computing
marsty5.wordpress.com
--
Claudio Martella
claudio.marte...@gmail.com
reading! (hopefully ;p)
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing
--
Claudio Martella
claudio.marte...@gmail.com
JsonLongDoubleFloatDoubleVertexInputFormat.addInputPath(job, new
Path(/some/path));
That's not available though. Could someone point me in the right
direction with this?
Am I going about this all wrong?
Thanks for any help,
Matt
--
Claudio Martella
claudio.marte...@gmail.com
Giraph directly from the repo, but from the http mirror directly?
Thanks in advance,
Alexandros
--
Claudio Martella
claudio.marte...@gmail.com
)) {
urls.add(targetUrl);
edges.add(edge);
}
}
setEdges(edges);
}
}
Thanks,
Emre.
--
Claudio Martella
claudio.marte...@gmail.com
advertising this issue to potential students or people
interested in this project.
The related issue can be found here:
https://issues.apache.org/jira/browse/GIRAPH-549
An entry in the Giraph Wiki will be added soon.
Best,
Claudio
--
Claudio Martella
claudio.marte...@gmail.com
1 - 100 of 110 matches
Mail list logo
