nt M-CS) ;
user@spark.apache.org
Subject: Re: [spark-graphframes]: Generating incorrect edges
Hi Steve,
Thanks for your statement. I tend to use uuid myself to avoid collisions. This
built-in function generates random IDs that are highly likely to be unique
across systems. My concerns are
Hi Steve,
Thanks for your statement. I tend to use uuid myself to avoid
collisions. This built-in function generates random IDs that are highly
likely to be unique across systems. My concerns are on edge so to speak. If
the Spark application runs for a very long time or encounters restarts, the
Hi Mich,
I was just reading random questions on the user list when I noticed that you
said:
On 25 Apr 2024, at 2:12 AM, Mich Talebzadeh wrote:
1) You are using monotonically_increasing_id(), which is not
collision-resistant in distributed environments like Spark. Multiple hosts
can
"128G"
).set("spark.executor.memoryOverhead", "32G"
).set("spark.driver.cores", "16"
).set("spark.driver.memory", "64G"
)
I dont think b) applies as its a single machine.
Kind regards,
Jelle
Fr
o 100K records.
> Once I go past that amount of records the results become inconsistent and
> incorrect.
>
> Kind regards,
> Jelle Nijland
>
>
> --
> *From:* Mich Talebzadeh
> *Sent:* Wednesday, April 24, 2024 4:40 PM
> *To:* Nijland, J.G.W. (Jelle, Student M-CS) <
> j.g.
___
From: Mich Talebzadeh
Sent: Wednesday, April 24, 2024 4:40 PM
To: Nijland, J.G.W. (Jelle, Student M-CS)
Cc: user@spark.apache.org
Subject: Re: [spark-graphframes]: Generating incorrect edges
OK few observations
1) ID Generation Method: How are you generating unique IDs (UUIDs, seque
jl...@student.utwente.nl> wrote:
> tags: pyspark,spark-graphframes
>
> Hello,
>
> I am running pyspark in a podman container and I have issues with
> incorrect edges when I build my graph.
> I start with loading a source dataframe from a parquet directory on my
&
tags: pyspark,spark-graphframes
Hello,
I am running pyspark in a podman container and I have issues with incorrect
edges when I build my graph.
I start with loading a source dataframe from a parquet directory on my server.
The source dataframe has the following columns
Hello GraphFrame-community,
our company is very interested in using GraphFrames for large enterprise
tools. Therefore I would like to ask a few questions regarding the
architecture of GraphFrames:
1. In GraphFrames Quick-Start Guide it says that it's built on top of
SparkSQL. Is the
il.com>
> wrote:
>
>> Interesting!
>>
>> --
>> *From:* Robert Yokota <rayok...@gmail.com>
>> *Sent:* Sunday, April 2, 2017 9:40:07 AM
>> *To:* user@spark.apache.org
>> *Subject:* Graph Analytics on HBase with HGraphDB and Spark GraphFrames
>
t; *To:* user@spark.apache.org
> *Subject:* Graph Analytics on HBase with HGraphDB and Spark GraphFrames
>
> Hi,
>
> In case anyone is interested in analyzing graphs in HBase with Apache
> Spark GraphFrames, this might be helpful:
>
> https://yokota.blog/2017/04/02/graph-analytics-on-hbase-with
> -hgraphdb-and-spark-graphframes/
>
Interesting!
From: Robert Yokota <rayok...@gmail.com>
Sent: Sunday, April 2, 2017 9:40:07 AM
To: user@spark.apache.org
Subject: Graph Analytics on HBase with HGraphDB and Spark GraphFrames
Hi,
In case anyone is interested in analyzing graphs in HBase with
Hi,
In case anyone is interested in analyzing graphs in HBase with Apache Spark
GraphFrames, this might be helpful:
https://yokota.blog/2017/04/02/graph-analytics-on-hbase-
with-hgraphdb-and-spark-graphframes/
Hi
I am working on two different use cases where the basic problem is same but
scale is very different.
In case 1 we have two entities that can have many to many relation and we would
want to identify all subgraphs in the full graph and then further prune the
graph to find the best relation.
To:Divya Gehlot <divya.htco...@gmail.com>
> Cc:"user @spark" <user@spark.apache.org>
> Date:2016/08/02 17:06
> Subject:Re: Spark GraphFrames
> --
>
>
>
> Hi,
> Kay wrot
rk.apache.org>
Date: 2016/08/02 17:06
Subject: Re: Spark GraphFrames
Hi,
Kay wrote a procedure to use GraphFrames with Spark.
https://gist.github.com/kayousterhout/7008a8ebf2babeedc7ce6f8723fd1bf4
Kazuaki Ishizaki
From:Divya Gehlot <divya.htco...@gmail.com>
To
Hi,
Kay wrote a procedure to use GraphFrames with Spark.
https://gist.github.com/kayousterhout/7008a8ebf2babeedc7ce6f8723fd1bf4
Kazuaki Ishizaki
From: Divya Gehlot <divya.htco...@gmail.com>
To: "user @spark" <user@spark.apache.org>
Date: 2016/08/02 14:52
Hi,
Has anybody has worked with GraphFrames.
Pls let me know as I need to know the real case scenarios where It can used
.
Thanks,
Divya
18 matches
Mail list logo
