I think Stefano was asking for a different way to pass a generic
Configuration object, not just a subclass of it.
For example, in our use case, it would be helpful to broadcast around
generic objects/pojos.
Is that possible?
On Nov 10, 2014 6:46 PM, "Stephan Ewen" <[email protected]> wrote:
> Hi!
>
> I hope I understand correctly what you are trying to do: To have a config
> file available in the functions, you can simply do wither of the following:
>
> -----------------------
> Closure
> -----------------------
>
> Configuration conf = ...
>
> data.map(new RichMapFunction<String, Integer>() {
>
> public void open (Conficuration c) {
> // access the conf object here
> conf.getString(...);
> }
>
> public Integer map(String value) {
> // whatever
> }
> });
> -----------------------
>
>
> -----------------------
> Config Parameters
> -----------------------
>
> Configuration conf = ...
>
> data.map(new RichMapFunction<String, Integer>() {
>
> public void open (Conficuration c) {
> // access the c - it will will have all elements of the conf - see
> withParameters() below
> c.getString(...);
> }
>
> public Integer map(String value) {
> // whatever
> }
> })
> .withParameters(conf);
> -----------------------
>
>
> Stephan
>
>
> On Mon, Nov 10, 2014 at 5:36 PM, Stefano Bortoli <[email protected]>
> wrote:
>
>> Hi,
>>
>> trying to run some legacy code as part of Flink Job, I had to replicate
>> configurations files across my cluster. Not a big deal with a small
>> cluster, but it would be nice to have these configuration objects
>> broadcast-able. Namely, it would be nice to reuse the old "read from conf
>> file" logic to build objects that then could be serialized and used along
>> the processing through the broadcast mechanism.
>>
>> Do you think it will be possible? with the new Kryo serialization it
>> should not be extremely complicated.
>>
>> saluti,
>> Stefano
>>
>>
>>
>
