Re: how to use custom properties in spark app

2016-04-05 Thread Haroon Rasheed
Hi, You can have a custom properties file with Map like entries Key, Value pairs "URL"-> "IPaddress:port/user/" etc and put this file on HDFS or any location where Spark can access. Read the file as RDD as Map and read the values in program. You can also broadcast this in program if you need

how to use custom properties in spark app

2016-04-05 Thread yaoxiaohua
Hi bro, I am new in spark application develop, I need develop two app running on spark cluster. Now I have some arguments for the application. I can pass them as the program arguments when spark-submit, I want to find a new way. I