Re: Using shell environment variables

2019-08-24 Thread Zhu Zhu
Hi Abhishek, You need to export the environment variables on all the worker machines(not the machine to submit the job). Alternatively, if you are submitting the job to a yarn cluster, you can use flink conf prefix "containerized.taskmanager.env." to add environment variables to Flink's task mana

Re: type error with generics ..

2019-08-24 Thread Debasish Ghosh
Thanks .. I tried this .. DataStream ins = readStream(in, Data.class, serdeData).map((Data d) -> d).returns(new TypeHint(){}.getTypeInfo()); But still get the same error on this line .. (BTW I am not sure how to invoke returns on a DataStream and hence had to do a fake map - any suggestions here

Re: Using shell environment variables

2019-08-24 Thread Abhishek Jain
Hi Miki, Thanks for your reply. ParameterTool will only help in making the value accessible through ParameterTool.get(). However, I need a way of accessing the value using "System.getenv" since the underlying library uses it so. On Sat, 24 Aug 2019 at 23:04, miki haiat wrote: > Did you register

Re: Using shell environment variables

2019-08-24 Thread miki haiat
Did you register your system environment parameter ? You can find here several ways to use configuration data [1] 1. https://ci.apache.org/projects/flink/flink-docs-stable/dev/best_practices.html On Sat, Aug 24, 2019, 20:26 Abhishek Jain wrote: > Hi! > > I am using a library that depends on a

Using shell environment variables

2019-08-24 Thread Abhishek Jain
Hi! I am using a library that depends on a certain environment variable set (mandatorily). Now, I've exported this variable in my environment but somehow it's not being read by the task manager. Following is the exception I get when I try to run the job: Caused by: com.example.MyCustomException:

Re: type error with generics ..

2019-08-24 Thread Rong Rong
Hi Debasish, I think the error refers to the output of your source instead of your result of the map function. E.g. DataStream ins = readStream(in, Data.class, serdeData)*.returns(new TypeInformation);* DataStream simples = ins.map((Data d) -> new Simple(d.getName())) .returns(new TypeHint(){}.get

Re:[ANNOUNCE] Apache Flink 1.9.0 released

2019-08-24 Thread chaojianok
Congratulations and thanks! At 2019-08-22 20:03:26, "Tzu-Li (Gordon) Tai" wrote: >The Apache Flink community is very happy to announce the release of Apache >Flink 1.9.0, which is the latest major release. > >Apache FlinkĀ® is an open-source stream processing framework for >distributed, high-perfor

Re:[ANNOUNCE] Apache Flink 1.9.0 released

2019-08-24 Thread chaojianok
Congratulations and thanks! At 2019-08-22 20:03:26, "Tzu-Li (Gordon) Tai" wrote: The Apache Flink community is very happy to announce the release of Apache Flink 1.9.0, which is the latest major release. Apache FlinkĀ® is an open-source stream processing framework for distributed, high-perform