Ignite can also cache rdd > On 12 Jan 2016, at 13:06, Dmitry Goldenberg <dgoldenberg...@gmail.com> wrote: > > Jorn, you said Ignite or ... ? What was the second choice you were thinking > of? It seems that got omitted. > >> On Jan 12, 2016, at 2:44 AM, Jörn Franke <jornfra...@gmail.com> wrote: >> >> You can look at ignite as a HDFS cache or for storing rdds. >> >>> On 11 Jan 2016, at 21:14, Dmitry Goldenberg <dgoldenberg...@gmail.com> >>> wrote: >>> >>> We have a bunch of Spark jobs deployed and a few large resource files such >>> as e.g. a dictionary for lookups or a statistical model. >>> >>> Right now, these are deployed as part of the Spark jobs which will >>> eventually make the mongo-jars too bloated for deployments. >>> >>> What are some of the best practices to consider for maintaining and sharing >>> large resource files like these? >>> >>> Thanks.
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
