Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Ok gents thanks for clarification.

Regards,

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Tue, 7 Aug 2018 at 02:21, Hequn Cheng  wrote:

> Hi Mich,
>
> I think this is the behavior of the compiler. When run your job in local,
> you have to remove the provided or add jar to the lib path. But if run on
> cluster, you have to add the provided to ignore flink classes, since these
> classes are already exist in your installation version.
>
> Best, Hequn
>
> On Tue, Aug 7, 2018 at 1:34 AM, Mich Talebzadeh  > wrote:
>
>> Thanks Fabian,
>>
>> I looked at the maven and this is what it says *provided*
>>
>>
>> [image: image.png]
>> However, this jar file is not shipped with Flink? Is this deliberate?
>>
>> Thanks
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 6 Aug 2018 at 18:14, Fabian Hueske  wrote:
>>
>>> The problem is that you declared it as provided.
>>> This means the build tool assumes it will be there and therefore does
>>> not include it in the Jar file.
>>> By adding it to the lib folder you are providing the dependency.
>>>
>>> Best, Fabian
>>>
>>> 2018-08-06 18:58 GMT+02:00 Mich Talebzadeh :
>>>
 Hi,

 I resolved this issue of

 java.lang.NoClassDefFoundError: org/apache/flink/table/api/
 TableEnvironment

 By adding the jar file

 flink-table_2.11-1.5.0.jar

 To $FLINK_HOME/lib

 It compiles and run OK now.

 Rather strange as I had this dependency in my SBT

 libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
 "provided"


 HTH

 Dr Mich Talebzadeh



 LinkedIn * 
 https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
 *



 http://talebzadehmich.wordpress.com


 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Thu, 2 Aug 2018 at 08:26, Mich Talebzadeh 
 wrote:

> Thanks everyone for the advice
>
> This worked and passed the compilation error
>
> import org.apache.flink.table.api.TableEnvironment
> import org.apache.flink.table.api.scala._
> import org.apache.flink.api.scala._
>
> ….
>
> val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   tableEnv.registerDataStream("priceTable", dataStream, "key, ticker,
> timeissued, price")
>   sqltext  = "SELECT key from priceTable";
>   val result = tableEnv.sql(sqltext);
>
> Now I get this runtime error
>
> ….
> [success] Total time: 16 s, completed Aug 2, 2018 8:23:12 AM
> Completed compiling
> Thu Aug 2 08:23:12 BST 2018 , Running in **Standalone mode**
> Starting execution of program
> java.lang.NoClassDefFoundError:
> org/apache/flink/table/api/TableEnvironment$
> at md_streaming$.main(md_streaming.scala:140)
> at md_streaming.main(md_streaming.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.

Re: Converting a DataStream into a Table throws error

[Hequn Cheng]

Hi Mich,

I think this is the behavior of the compiler. When run your job in local,
you have to remove the provided or add jar to the lib path. But if run on
cluster, you have to add the provided to ignore flink classes, since these
classes are already exist in your installation version.

Best, Hequn

On Tue, Aug 7, 2018 at 1:34 AM, Mich Talebzadeh 
wrote:

> Thanks Fabian,
>
> I looked at the maven and this is what it says *provided*
>
>
> [image: image.png]
> However, this jar file is not shipped with Flink? Is this deliberate?
>
> Thanks
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 6 Aug 2018 at 18:14, Fabian Hueske  wrote:
>
>> The problem is that you declared it as provided.
>> This means the build tool assumes it will be there and therefore does not
>> include it in the Jar file.
>> By adding it to the lib folder you are providing the dependency.
>>
>> Best, Fabian
>>
>> 2018-08-06 18:58 GMT+02:00 Mich Talebzadeh :
>>
>>> Hi,
>>>
>>> I resolved this issue of
>>>
>>> java.lang.NoClassDefFoundError: org/apache/flink/table/api/Tab
>>> leEnvironment
>>>
>>> By adding the jar file
>>>
>>> flink-table_2.11-1.5.0.jar
>>>
>>> To $FLINK_HOME/lib
>>>
>>> It compiles and run OK now.
>>>
>>> Rather strange as I had this dependency in my SBT
>>>
>>> libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
>>> "provided"
>>>
>>>
>>> HTH
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * 
>>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> *
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 2 Aug 2018 at 08:26, Mich Talebzadeh 
>>> wrote:
>>>
 Thanks everyone for the advice

 This worked and passed the compilation error

 import org.apache.flink.table.api.TableEnvironment
 import org.apache.flink.table.api.scala._
 import org.apache.flink.api.scala._

 ….

 val dataStream =  streamExecEnv
.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
 SimpleStringSchema(), properties))
val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
   tableEnv.registerDataStream("priceTable", dataStream, "key, ticker,
 timeissued, price")
   sqltext  = "SELECT key from priceTable";
   val result = tableEnv.sql(sqltext);

 Now I get this runtime error

 ….
 [success] Total time: 16 s, completed Aug 2, 2018 8:23:12 AM
 Completed compiling
 Thu Aug 2 08:23:12 BST 2018 , Running in **Standalone mode**
 Starting execution of program
 java.lang.NoClassDefFoundError: org/apache/flink/table/api/
 TableEnvironment$
 at md_streaming$.main(md_streaming.scala:140)
 at md_streaming.main(md_streaming.scala)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(
 NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(
 DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.flink.client.program.PackagedProgram.
 callMainMethod(PackagedProgram.java:528)
 at org.apache.flink.client.program.PackagedProgram.
 invokeInteractiveModeForExecution(PackagedProgram.java:420)
 at org.apache.flink.client.program.ClusterClient.run(
 ClusterClient.java:404)
 at org.apache.flink.client.cli.CliFrontend.executeProgram(
 CliFrontend.java:781)
 at org.apache.flink.client.cli.CliFrontend.runProgram(
 CliFrontend.java:275)
 at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.
 java:210)
 at org.apache.flink.client.cli.CliFrontend.parseParameters(
 CliFrontend.java:1020)
 at org.apache.flink.client.cli.CliFrontend.lambda$main$9(
 CliFrontend.java:1096)
 at java.security.AccessController.doP

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Thanks Fabian,

I looked at the maven and this is what it says *provided*


[image: image.png]
However, this jar file is not shipped with Flink? Is this deliberate?

Thanks

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 6 Aug 2018 at 18:14, Fabian Hueske  wrote:

> The problem is that you declared it as provided.
> This means the build tool assumes it will be there and therefore does not
> include it in the Jar file.
> By adding it to the lib folder you are providing the dependency.
>
> Best, Fabian
>
> 2018-08-06 18:58 GMT+02:00 Mich Talebzadeh :
>
>> Hi,
>>
>> I resolved this issue of
>>
>> java.lang.NoClassDefFoundError: org/apache/flink/table/api/
>> TableEnvironment
>>
>> By adding the jar file
>>
>> flink-table_2.11-1.5.0.jar
>>
>> To $FLINK_HOME/lib
>>
>> It compiles and run OK now.
>>
>> Rather strange as I had this dependency in my SBT
>>
>> libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
>> "provided"
>>
>>
>> HTH
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 2 Aug 2018 at 08:26, Mich Talebzadeh 
>> wrote:
>>
>>> Thanks everyone for the advice
>>>
>>> This worked and passed the compilation error
>>>
>>> import org.apache.flink.table.api.TableEnvironment
>>> import org.apache.flink.table.api.scala._
>>> import org.apache.flink.api.scala._
>>>
>>> ….
>>>
>>> val dataStream =  streamExecEnv
>>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>>> SimpleStringSchema(), properties))
>>>val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>>   tableEnv.registerDataStream("priceTable", dataStream, "key, ticker,
>>> timeissued, price")
>>>   sqltext  = "SELECT key from priceTable";
>>>   val result = tableEnv.sql(sqltext);
>>>
>>> Now I get this runtime error
>>>
>>> ….
>>> [success] Total time: 16 s, completed Aug 2, 2018 8:23:12 AM
>>> Completed compiling
>>> Thu Aug 2 08:23:12 BST 2018 , Running in **Standalone mode**
>>> Starting execution of program
>>> java.lang.NoClassDefFoundError:
>>> org/apache/flink/table/api/TableEnvironment$
>>> at md_streaming$.main(md_streaming.scala:140)
>>> at md_streaming.main(md_streaming.scala)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
>>> at
>>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
>>> at
>>> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
>>> at
>>> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:781)
>>> at
>>> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:275)
>>> at
>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>>> at
>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1020)
>>> at
>>> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1096)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at javax.security.auth.Subject.doAs(Subject.java:422)
>>> at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
>>> at
>>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>>> at
>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1096)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.flink.table.api.TableEnvironment$
>>> at java.net.URLClassLo

Re: Converting a DataStream into a Table throws error

[Fabian Hueske]

The problem is that you declared it as provided.
This means the build tool assumes it will be there and therefore does not
include it in the Jar file.
By adding it to the lib folder you are providing the dependency.

Best, Fabian

2018-08-06 18:58 GMT+02:00 Mich Talebzadeh :

> Hi,
>
> I resolved this issue of
>
> java.lang.NoClassDefFoundError: org/apache/flink/table/api/Tab
> leEnvironment
>
> By adding the jar file
>
> flink-table_2.11-1.5.0.jar
>
> To $FLINK_HOME/lib
>
> It compiles and run OK now.
>
> Rather strange as I had this dependency in my SBT
>
> libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
> "provided"
>
>
> HTH
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 2 Aug 2018 at 08:26, Mich Talebzadeh 
> wrote:
>
>> Thanks everyone for the advice
>>
>> This worked and passed the compilation error
>>
>> import org.apache.flink.table.api.TableEnvironment
>> import org.apache.flink.table.api.scala._
>> import org.apache.flink.api.scala._
>>
>> ….
>>
>> val dataStream =  streamExecEnv
>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>> SimpleStringSchema(), properties))
>>val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>   tableEnv.registerDataStream("priceTable", dataStream, "key, ticker,
>> timeissued, price")
>>   sqltext  = "SELECT key from priceTable";
>>   val result = tableEnv.sql(sqltext);
>>
>> Now I get this runtime error
>>
>> ….
>> [success] Total time: 16 s, completed Aug 2, 2018 8:23:12 AM
>> Completed compiling
>> Thu Aug 2 08:23:12 BST 2018 , Running in **Standalone mode**
>> Starting execution of program
>> java.lang.NoClassDefFoundError: org/apache/flink/table/api/
>> TableEnvironment$
>> at md_streaming$.main(md_streaming.scala:140)
>> at md_streaming.main(md_streaming.scala)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.apache.flink.client.program.PackagedProgram.
>> callMainMethod(PackagedProgram.java:528)
>> at org.apache.flink.client.program.PackagedProgram.
>> invokeInteractiveModeForExecution(PackagedProgram.java:420)
>> at org.apache.flink.client.program.ClusterClient.run(
>> ClusterClient.java:404)
>> at org.apache.flink.client.cli.CliFrontend.executeProgram(
>> CliFrontend.java:781)
>> at org.apache.flink.client.cli.CliFrontend.runProgram(
>> CliFrontend.java:275)
>> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.
>> java:210)
>> at org.apache.flink.client.cli.CliFrontend.parseParameters(
>> CliFrontend.java:1020)
>> at org.apache.flink.client.cli.CliFrontend.lambda$main$9(
>> CliFrontend.java:1096)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(
>> UserGroupInformation.java:1836)
>> at org.apache.flink.runtime.security.HadoopSecurityContext.
>> runSecured(HadoopSecurityContext.java:41)
>> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.
>> java:1096)
>> Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.api.
>> TableEnvironment$
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 2 Aug 2018 at 03:07, vino yang  wrote:
>>
>>> Hi Mich,
>>>
>>> It seems that the type of y

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Hi,

I resolved this issue of

java.lang.NoClassDefFoundError: org/apache/flink/table/api/TableEnvironment

By adding the jar file

flink-table_2.11-1.5.0.jar

To $FLINK_HOME/lib

It compiles and run OK now.

Rather strange as I had this dependency in my SBT

libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
"provided"


HTH

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 2 Aug 2018 at 08:26, Mich Talebzadeh 
wrote:

> Thanks everyone for the advice
>
> This worked and passed the compilation error
>
> import org.apache.flink.table.api.TableEnvironment
> import org.apache.flink.table.api.scala._
> import org.apache.flink.api.scala._
>
> ….
>
> val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   tableEnv.registerDataStream("priceTable", dataStream, "key, ticker,
> timeissued, price")
>   sqltext  = "SELECT key from priceTable";
>   val result = tableEnv.sql(sqltext);
>
> Now I get this runtime error
>
> ….
> [success] Total time: 16 s, completed Aug 2, 2018 8:23:12 AM
> Completed compiling
> Thu Aug 2 08:23:12 BST 2018 , Running in **Standalone mode**
> Starting execution of program
> java.lang.NoClassDefFoundError:
> org/apache/flink/table/api/TableEnvironment$
> at md_streaming$.main(md_streaming.scala:140)
> at md_streaming.main(md_streaming.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:781)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:275)
> at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1020)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1096)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1096)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.table.api.TableEnvironment$
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 2 Aug 2018 at 03:07, vino yang  wrote:
>
>> Hi Mich,
>>
>> It seems that the type of your DataStream stream is always wrong.
>> If you want to specify four fields, usually the DataStream type should be
>> similar: DataStream[(Type1, Type2, Type3, Type4)], not DataStream[String],
>> you can try it.
>>
>> Thanks, vino
>>
>> 2018-08-02 6:44 GMT+08:00 Mich Talebzadeh :
>>
>>> Changed as suggested
>>>
>>>val  streamExecEnv =
>>> StreamExecutionEnvironment.getExecutionEnvironment
>>>  val dataStream =  streamExecEnv
>>>.addSource

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Apologies should read Vino and Timo

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Fri, 3 Aug 2018 at 09:14, Mich Talebzadeh 
wrote:

> Thanks a lot Timo.
>
> I will try the changes suggested.
>
> Appreciated
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Fri, 3 Aug 2018 at 03:56, vino yang  wrote:
>
>> Hi Mich,
>>
>> I have reviewed your code in the github you provided.
>>
>> I copied your code to org.apache.flink.table.examples.scala under
>> flink-examples-table. It passed the compilation and didn't report the
>> exception you provided, although there are other exceptions (it's about
>> hdfs, this is because of my environment).
>> First of all, the necessary dependencies are just a few:
>>
>> **
>> *org.apache.flink*
>> *flink-table_${scala.binary.version}*
>> *${project.version}*
>> **
>>
>> **
>> *org.apache.flink*
>> *flink-streaming-scala_${scala.binary.version}*
>> *${project.version}*
>> **
>>
>> **
>> *org.apache.flink*
>>
>> *flink-connector-kafka-0.11_${scala.binary.version}*
>> *${project.version}*
>> **
>>
>> Please remove irrelevant dependencies.
>>
>> In addition, your StreamExecutionEnvironment object has a problem with
>> its reference. The correct path should be:
>> Import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
>>
>> In addition, you specify four fields, but your stream object is
>> DataStream[String]. This is also a mistake. You are advised to parse it
>> first with a MapFunction. The example is as follows:
>>
>> *Val dataStream = streamExecEnv*
>> *  .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>> SimpleStringSchema(), properties))*
>>
>> *Val newDateStream = dataStream.map(new MapFunction[String,
>> Tuple4[String, String, String, String]] {*
>> *  Override def map(value: String): Tuple4[String, String, String,
>> String] = {*
>> *Return null*
>> *  }*
>> *})*
>>
>> *Val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)*
>> *//tableEnv.registerDataStream("table1", streamExecEnv, 'key,
>> 'ticker, 'timeissued, 'price)*
>> *tableEnv.registerDataStream("priceTable", newDateStream, 'key,
>> 'ticker, 'timeissued, 'price)*
>>
>> Thanks, vino.
>>
>> 2018-08-02 20:36 GMT+08:00 Mich Talebzadeh :
>>
>>> Appreciate if anyone had a chance to look at the Scala code in GitHub
>>> and advise
>>>
>>> https://github.com/michTalebzadeh/Flink
>>>
>>> Regards,
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * 
>>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> *
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 2 Aug 2018 at 09:06, Mich Talebzadeh 
>>> wrote:
>>>
 Thanks Timo,

 Did as suggested getting this compilation error

 [info] Compiling 1 Scala source to
 /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
 [error]
 /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:136:
 could not find implicit value for evidence parameter of type
 org.apache.flink.api.common.typeinfo.TypeInformation[String]
 [error].addSource(new
 FlinkKafkaConsumer011[String](topicsValue, new SimpleStringSchema(),
 properties))
 [error]  ^
 [error] one error found
 [error] (compile:compileIncremental) Compilation failed

 Dr Mich Talebzadeh



 LinkedIn * 
 https://www.linkedin.com/profile/

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Thanks a lot Timo.

I will try the changes suggested.

Appreciated

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Fri, 3 Aug 2018 at 03:56, vino yang  wrote:

> Hi Mich,
>
> I have reviewed your code in the github you provided.
>
> I copied your code to org.apache.flink.table.examples.scala under
> flink-examples-table. It passed the compilation and didn't report the
> exception you provided, although there are other exceptions (it's about
> hdfs, this is because of my environment).
> First of all, the necessary dependencies are just a few:
>
> **
> *org.apache.flink*
> *flink-table_${scala.binary.version}*
> *${project.version}*
> **
>
> **
> *org.apache.flink*
> *flink-streaming-scala_${scala.binary.version}*
> *${project.version}*
> **
>
> **
> *org.apache.flink*
>
> *flink-connector-kafka-0.11_${scala.binary.version}*
> *${project.version}*
> **
>
> Please remove irrelevant dependencies.
>
> In addition, your StreamExecutionEnvironment object has a problem with its
> reference. The correct path should be:
> Import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
>
> In addition, you specify four fields, but your stream object is
> DataStream[String]. This is also a mistake. You are advised to parse it
> first with a MapFunction. The example is as follows:
>
> *Val dataStream = streamExecEnv*
> *  .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))*
>
> *Val newDateStream = dataStream.map(new MapFunction[String,
> Tuple4[String, String, String, String]] {*
> *  Override def map(value: String): Tuple4[String, String, String,
> String] = {*
> *Return null*
> *  }*
> *})*
>
> *Val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)*
> *//tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
> 'timeissued, 'price)*
> *tableEnv.registerDataStream("priceTable", newDateStream, 'key,
> 'ticker, 'timeissued, 'price)*
>
> Thanks, vino.
>
> 2018-08-02 20:36 GMT+08:00 Mich Talebzadeh :
>
>> Appreciate if anyone had a chance to look at the Scala code in GitHub and
>> advise
>>
>> https://github.com/michTalebzadeh/Flink
>>
>> Regards,
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 2 Aug 2018 at 09:06, Mich Talebzadeh 
>> wrote:
>>
>>> Thanks Timo,
>>>
>>> Did as suggested getting this compilation error
>>>
>>> [info] Compiling 1 Scala source to
>>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>>> [error]
>>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:136:
>>> could not find implicit value for evidence parameter of type
>>> org.apache.flink.api.common.typeinfo.TypeInformation[String]
>>> [error].addSource(new FlinkKafkaConsumer011[String](topicsValue,
>>> new SimpleStringSchema(), properties))
>>> [error]  ^
>>> [error] one error found
>>> [error] (compile:compileIncremental) Compilation failed
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * 
>>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> *
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 2 Aug 2018 at 09:01, Timo Walther  wrote:
>>>
 Whenever you use Scala and there is a Scala specific class use it.

 remove: import
 org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
 add: import org.apache.flink.str

Re: Converting a DataStream into a Table throws error

[vino yang]

Hi Mich,

I have reviewed your code in the github you provided.

I copied your code to org.apache.flink.table.examples.scala under
flink-examples-table. It passed the compilation and didn't report the
exception you provided, although there are other exceptions (it's about
hdfs, this is because of my environment).
First of all, the necessary dependencies are just a few:

**
*org.apache.flink*
*flink-table_${scala.binary.version}*
*${project.version}*
**

**
*org.apache.flink*
*flink-streaming-scala_${scala.binary.version}*
*${project.version}*
**

**
*org.apache.flink*
*flink-connector-kafka-0.11_${scala.binary.version}*
*${project.version}*
**

Please remove irrelevant dependencies.

In addition, your StreamExecutionEnvironment object has a problem with its
reference. The correct path should be:
Import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment

In addition, you specify four fields, but your stream object is
DataStream[String]. This is also a mistake. You are advised to parse it
first with a MapFunction. The example is as follows:

*Val dataStream = streamExecEnv*
*  .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))*

*Val newDateStream = dataStream.map(new MapFunction[String,
Tuple4[String, String, String, String]] {*
*  Override def map(value: String): Tuple4[String, String, String,
String] = {*
*Return null*
*  }*
*})*

*Val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)*
*//tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
'timeissued, 'price)*
*tableEnv.registerDataStream("priceTable", newDateStream, 'key,
'ticker, 'timeissued, 'price)*

Thanks, vino.

2018-08-02 20:36 GMT+08:00 Mich Talebzadeh :

> Appreciate if anyone had a chance to look at the Scala code in GitHub and
> advise
>
> https://github.com/michTalebzadeh/Flink
>
> Regards,
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 2 Aug 2018 at 09:06, Mich Talebzadeh 
> wrote:
>
>> Thanks Timo,
>>
>> Did as suggested getting this compilation error
>>
>> [info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
>> streaming/target/scala-2.11/classes...
>> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
>> myPackage/md_streaming.scala:136: could not find implicit value for
>> evidence parameter of type org.apache.flink.api.common.
>> typeinfo.TypeInformation[String]
>> [error].addSource(new FlinkKafkaConsumer011[String](topicsValue,
>> new SimpleStringSchema(), properties))
>> [error]  ^
>> [error] one error found
>> [error] (compile:compileIncremental) Compilation failed
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 2 Aug 2018 at 09:01, Timo Walther  wrote:
>>
>>> Whenever you use Scala and there is a Scala specific class use it.
>>>
>>> remove: import org.apache.flink.streaming.api.environment.
>>> StreamExecutionEnvironment
>>> add: import org.apache.flink.streaming.api.scala._
>>>
>>> This will use org.apache.flink.streaming.api.scala.
>>> StreamExecutionEnvironment.
>>>
>>> Timo
>>>
>>> Am 02.08.18 um 09:47 schrieb Mich Talebzadeh:
>>>
>>> Tremendous. Many thanks.
>>>
>>> Put the sbt build file and the Scala code here
>>>
>>> https://github.com/michTalebzadeh/Flink
>>>
>>> Regards,
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * 
>>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> *
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Appreciate if anyone had a chance to look at the Scala code in GitHub and
advise

https://github.com/michTalebzadeh/Flink

Regards,

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 2 Aug 2018 at 09:06, Mich Talebzadeh 
wrote:

> Thanks Timo,
>
> Did as suggested getting this compilation error
>
> [info] Compiling 1 Scala source to
> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
> [error]
> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:136:
> could not find implicit value for evidence parameter of type
> org.apache.flink.api.common.typeinfo.TypeInformation[String]
> [error].addSource(new FlinkKafkaConsumer011[String](topicsValue,
> new SimpleStringSchema(), properties))
> [error]  ^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 2 Aug 2018 at 09:01, Timo Walther  wrote:
>
>> Whenever you use Scala and there is a Scala specific class use it.
>>
>> remove: import
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
>> add: import org.apache.flink.streaming.api.scala._
>>
>> This will use
>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.
>>
>> Timo
>>
>> Am 02.08.18 um 09:47 schrieb Mich Talebzadeh:
>>
>> Tremendous. Many thanks.
>>
>> Put the sbt build file and the Scala code here
>>
>> https://github.com/michTalebzadeh/Flink
>>
>> Regards,
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 2 Aug 2018 at 08:27, Timo Walther  wrote:
>>
>>> Hi Mich,
>>>
>>> could you share your project with us (maybe on github)? Then we can
>>> import it and debug what the problem is.
>>>
>>> Regards,
>>> Timo
>>>
>>> Am 02.08.18 um 07:37 schrieb Mich Talebzadeh:
>>>
>>> Hi Jorn,
>>>
>>> Here you go the dependencies
>>>
>>> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
>>> libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
>>> libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
>>> libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
>>> libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
>>> libraryDependencies += "org.apache.flink" %%
>>> "flink-connector-kafka-0.11" % "1.5.0"
>>> libraryDependencies += "org.apache.flink" %%
>>> "flink-connector-kafka-base" % "1.5.0"
>>> libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
>>> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
>>> libraryDependencies += "org.apache.flink" %% "flink-streaming-scala" %
>>> "1.5.0"
>>> libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
>>> "provided"
>>> libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"
>>>
>>> Thanks
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * 
>>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> *
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> dis

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Thanks Timo,

Did as suggested getting this compilation error

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:136:
could not find implicit value for evidence parameter of type
org.apache.flink.api.common.typeinfo.TypeInformation[String]
[error].addSource(new FlinkKafkaConsumer011[String](topicsValue,
new SimpleStringSchema(), properties))
[error]  ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 2 Aug 2018 at 09:01, Timo Walther  wrote:

> Whenever you use Scala and there is a Scala specific class use it.
>
> remove: import
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
> add: import org.apache.flink.streaming.api.scala._
>
> This will use
> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.
>
> Timo
>
> Am 02.08.18 um 09:47 schrieb Mich Talebzadeh:
>
> Tremendous. Many thanks.
>
> Put the sbt build file and the Scala code here
>
> https://github.com/michTalebzadeh/Flink
>
> Regards,
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 2 Aug 2018 at 08:27, Timo Walther  wrote:
>
>> Hi Mich,
>>
>> could you share your project with us (maybe on github)? Then we can
>> import it and debug what the problem is.
>>
>> Regards,
>> Timo
>>
>> Am 02.08.18 um 07:37 schrieb Mich Talebzadeh:
>>
>> Hi Jorn,
>>
>> Here you go the dependencies
>>
>> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
>> libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
>> libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
>> libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
>> libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
>> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-0.11"
>> % "1.5.0"
>> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-base"
>> % "1.5.0"
>> libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
>> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
>> libraryDependencies += "org.apache.flink" %% "flink-streaming-scala" %
>> "1.5.0"
>> libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
>> "provided"
>> libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"
>>
>> Thanks
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 2 Aug 2018 at 06:19, Jörn Franke  wrote:
>>
>>>
>>> How does your build.sbt looks especially dependencies?
>>> On 2. Aug 2018, at 00:44, Mich Talebzadeh 
>>> wrote:
>>>
>>> Changed as suggested
>>>
>>>val  streamExecEnv =
>>> StreamExecutionEnvironment.getExecutionEnvironment
>>>  val dataStream =  streamExecEnv
>>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>>> SimpleStringSchema(), properties))
>>>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>>   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
>>> 'timeissued, 'price)
>>>
>>> Still the same error
>>>
>>> [info] Compiling 1 Scala source to
>>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>>> [error]
>>> /home/hdu

Re: Converting a DataStream into a Table throws error

[Timo Walther]

Whenever you use Scala and there is a Scala specific class use it.

remove: import 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

add: import org.apache.flink.streaming.api.scala._

This will use 
org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.


Timo

Am 02.08.18 um 09:47 schrieb Mich Talebzadeh:

Tremendous. Many thanks.

Put the sbt build file and the Scala code here

https://github.com/michTalebzadeh/Flink

Regards,

Dr Mich Talebzadeh

LinkedIn 
/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/


http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk.Any and all responsibility for 
any loss, damage or destruction of data or any other property which 
may arise from relying on this email's technical content is explicitly 
disclaimed. The author will in no case be liable for any monetary 
damages arising from such loss, damage or destruction.




On Thu, 2 Aug 2018 at 08:27, Timo Walther > wrote:


Hi Mich,

could you share your project with us (maybe on github)? Then we
can import it and debug what the problem is.

Regards,
Timo

Am 02.08.18 um 07:37 schrieb Mich Talebzadeh:

Hi Jorn,

Here you go the dependencies

libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
libraryDependencies += "org.apache.flink" %%
"flink-connector-kafka-0.11" % "1.5.0"
libraryDependencies += "org.apache.flink" %%
"flink-connector-kafka-base" % "1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
libraryDependencies += "org.apache.kafka" % "kafka-clients" %
"0.11.0.0"
libraryDependencies += "org.apache.flink" %%
"flink-streaming-scala" % "1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-table" %
"1.5.0" % "provided"
libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"

Thanks

Dr Mich Talebzadeh

LinkedIn

/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/

http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk.Any and all responsibility
for any loss, damage or destruction of data or any other property
which may arise from relying on this email's technical content is
explicitly disclaimed. The author will in no case be liable for
any monetary damages arising from such loss, damage or destruction.



On Thu, 2 Aug 2018 at 06:19, Jörn Franke mailto:jornfra...@gmail.com>> wrote:


How does your build.sbt looks especially dependencies?
On 2. Aug 2018, at 00:44, Mich Talebzadeh
mailto:mich.talebza...@gmail.com>> wrote:


Changed as suggested

   val streamExecEnv =
StreamExecutionEnvironment.getExecutionEnvironment
     val dataStream =  streamExecEnv
   .addSource(new
FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))
  val tableEnv =
TableEnvironment.getTableEnvironment(streamExecEnv)
tableEnv.registerDataStream("table1", streamExecEnv, 'key,
'ticker, 'timeissued, 'price)

Still the same error

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]

/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:139:
overloaded method value registerDataStream with alternatives:
[error]   [T](name: String, dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T],
fields: String)Unit 
[error]   [T](name: String, dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T])Unit
[error]  cannot be applied to (String,
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,
Symbol, Symbol, Symbol, Symbol)
[error] tableEnv.registerDataStream("table1", streamExecEnv,
'key, 'ticker, 'timeissued, 'price)
[error]    ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Aug 1, 2018 11:40:47 PM

Thanks anyway.

Dr Mich Talebzadeh

LinkedIn

/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/

http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk.Any and all
responsibility for any loss, damage or destruction of data
or any other property which may arise from relying on this
email's technical content

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Tremendous. Many thanks.

Put the sbt build file and the Scala code here

https://github.com/michTalebzadeh/Flink

Regards,

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 2 Aug 2018 at 08:27, Timo Walther  wrote:

> Hi Mich,
>
> could you share your project with us (maybe on github)? Then we can import
> it and debug what the problem is.
>
> Regards,
> Timo
>
> Am 02.08.18 um 07:37 schrieb Mich Talebzadeh:
>
> Hi Jorn,
>
> Here you go the dependencies
>
> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
> libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
> libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
> libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
> libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-0.11"
> % "1.5.0"
> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-base"
> % "1.5.0"
> libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
> libraryDependencies += "org.apache.flink" %% "flink-streaming-scala" %
> "1.5.0"
> libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
> "provided"
> libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"
>
> Thanks
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 2 Aug 2018 at 06:19, Jörn Franke  wrote:
>
>>
>> How does your build.sbt looks especially dependencies?
>> On 2. Aug 2018, at 00:44, Mich Talebzadeh 
>> wrote:
>>
>> Changed as suggested
>>
>>val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
>>  val dataStream =  streamExecEnv
>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>> SimpleStringSchema(), properties))
>>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
>> 'timeissued, 'price)
>>
>> Still the same error
>>
>> [info] Compiling 1 Scala source to
>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>> [error]
>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:139:
>> overloaded method value registerDataStream with alternatives:
>> [error]   [T](name: String, dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
>> String)Unit 
>> [error]   [T](name: String, dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T])Unit
>> [error]  cannot be applied to (String,
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,
>> Symbol, Symbol, Symbol, Symbol)
>> [error]   tableEnv.registerDataStream("table1", streamExecEnv, 'key,
>> 'ticker, 'timeissued, 'price)
>> [error]^
>> [error] one error found
>> [error] (compile:compileIncremental) Compilation failed
>> [error] Total time: 3 s, completed Aug 1, 2018 11:40:47 PM
>>
>> Thanks anyway.
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Wed, 1 Aug 2018 at 23:34, Fabian Hueske  wrote:
>>
>>> Hi,
>>>
>>> You have to pass the StreamExecutionEnvironment to the
>>> getTableEnvironment() method, not the DataStream (or DataStreamSource).
>>> Change
>>>
>>> val tableEnv = TableEnvironment.getTa

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Thanks everyone for the advice

This worked and passed the compilation error

import org.apache.flink.table.api.TableEnvironment
import org.apache.flink.table.api.scala._
import org.apache.flink.api.scala._

….

val dataStream =  streamExecEnv
   .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))
   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
  tableEnv.registerDataStream("priceTable", dataStream, "key, ticker,
timeissued, price")
  sqltext  = "SELECT key from priceTable";
  val result = tableEnv.sql(sqltext);

Now I get this runtime error

….
[success] Total time: 16 s, completed Aug 2, 2018 8:23:12 AM
Completed compiling
Thu Aug 2 08:23:12 BST 2018 , Running in **Standalone mode**
Starting execution of program
java.lang.NoClassDefFoundError: org/apache/flink/table/api/TableEnvironment$
at md_streaming$.main(md_streaming.scala:140)
at md_streaming.main(md_streaming.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:781)
at
org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:275)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1020)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1096)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1096)
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.table.api.TableEnvironment$
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)


Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 2 Aug 2018 at 03:07, vino yang  wrote:

> Hi Mich,
>
> It seems that the type of your DataStream stream is always wrong.
> If you want to specify four fields, usually the DataStream type should be
> similar: DataStream[(Type1, Type2, Type3, Type4)], not DataStream[String],
> you can try it.
>
> Thanks, vino
>
> 2018-08-02 6:44 GMT+08:00 Mich Talebzadeh :
>
>> Changed as suggested
>>
>>val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
>>  val dataStream =  streamExecEnv
>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>> SimpleStringSchema(), properties))
>>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
>> 'timeissued, 'price)
>>
>> Still the same error
>>
>> [info] Compiling 1 Scala source to
>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>> [error]
>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:139:
>> overloaded method value registerDataStream with alternatives:
>> [error]   [T](name: String, dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
>> String)Unit 
>> [error]   [T](name: String, dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T])Unit
>> [error]  cannot be applied to (String,
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,
>> Symbol, Symbol, Symbol, Symbol)
>> [error]   tableEnv.registerDataStream("table1", streamExecEnv, 'key,
>> 'ticker, 'timeissued, 'price)
>> [error]^
>> [error] one error found
>> [error] (compile:compileIncremental) Compilatio

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Hi Jorn,

Here you go the dependencies

libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-0.11" %
"1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-base" %
"1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
libraryDependencies += "org.apache.flink" %% "flink-streaming-scala" %
"1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
"provided"
libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"

Thanks

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 2 Aug 2018 at 06:19, Jörn Franke  wrote:

>
> How does your build.sbt looks especially dependencies?
> On 2. Aug 2018, at 00:44, Mich Talebzadeh 
> wrote:
>
> Changed as suggested
>
>val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
>  val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
> 'timeissued, 'price)
>
> Still the same error
>
> [info] Compiling 1 Scala source to
> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
> [error]
> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:139:
> overloaded method value registerDataStream with alternatives:
> [error]   [T](name: String, dataStream:
> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
> String)Unit 
> [error]   [T](name: String, dataStream:
> org.apache.flink.streaming.api.datastream.DataStream[T])Unit
> [error]  cannot be applied to (String,
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,
> Symbol, Symbol, Symbol, Symbol)
> [error]   tableEnv.registerDataStream("table1", streamExecEnv, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
> [error] Total time: 3 s, completed Aug 1, 2018 11:40:47 PM
>
> Thanks anyway.
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 1 Aug 2018 at 23:34, Fabian Hueske  wrote:
>
>> Hi,
>>
>> You have to pass the StreamExecutionEnvironment to the
>> getTableEnvironment() method, not the DataStream (or DataStreamSource).
>> Change
>>
>> val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>>
>> to
>>
>> val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>
>> Best,
>> Fabian
>>
>> 2018-08-02 0:10 GMT+02:00 Mich Talebzadeh :
>>
>>> Hi,
>>>
>>> FYI, these are my imports
>>>
>>> import java.util.Properties
>>> import java.util.Arrays
>>> import org.apache.flink.api.common.functions.MapFunction
>>> import
>>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
>>> import org.apache.flink.streaming.api.scala
>>> import org.apache.flink.streaming.util.serialization.SimpleStringSchema
>>> import org.apache.flink.table.api.TableEnvironment
>>> import org.apache.flink.table.api.scala._
>>> import org.apache.flink.api.scala._
>>> import org.apache.kafka.clients.consumer.ConsumerConfig
>>> import org.apache.kafka.clients.consumer.ConsumerRecord
>>> import org.apache.kafka.clients.consumer.KafkaConsumer
>>> import org.apache.flink.core.fs.FileSystem
>>> import org.apache.flink.streaming.api.TimeCharacteristic
>>> import org.slf4j.LoggerFactory
>>> import
>>> org.apache.flink.streaming.connectors.kafka.{FlinkKafkaCo

Re: Converting a DataStream into a Table throws error

[Jörn Franke]

How does your build.sbt looks especially dependencies?
> On 2. Aug 2018, at 00:44, Mich Talebzadeh  wrote:
> 
> Changed as suggested
> 
>val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
>  val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new 
> SimpleStringSchema(), properties))
>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker, 
> 'timeissued, 'price)
> 
> Still the same error
> 
> [info] Compiling 1 Scala source to 
> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
> [error] 
> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:139:
>  overloaded method value registerDataStream with alternatives:
> [error]   [T](name: String, dataStream: 
> org.apache.flink.streaming.api.datastream.DataStream[T], fields: String)Unit 
> 
> [error]   [T](name: String, dataStream: 
> org.apache.flink.streaming.api.datastream.DataStream[T])Unit
> [error]  cannot be applied to (String, 
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment, 
> Symbol, Symbol, Symbol, Symbol)
> [error]   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker, 
> 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
> [error] Total time: 3 s, completed Aug 1, 2018 11:40:47 PM
> 
> Thanks anyway.
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
> 
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
> damage or destruction of data or any other property which may arise from 
> relying on this email's technical content is explicitly disclaimed. The 
> author will in no case be liable for any monetary damages arising from such 
> loss, damage or destruction.
>  
> 
> 
>> On Wed, 1 Aug 2018 at 23:34, Fabian Hueske  wrote:
>> Hi,
>> 
>> You have to pass the StreamExecutionEnvironment to the getTableEnvironment() 
>> method, not the DataStream (or DataStreamSource).
>> Change 
>> 
>> val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>> 
>> to
>> 
>> val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>> 
>> Best,
>> Fabian
>> 
>> 2018-08-02 0:10 GMT+02:00 Mich Talebzadeh :
>>> Hi,
>>> 
>>> FYI, these are my imports
>>> 
>>> import java.util.Properties
>>> import java.util.Arrays
>>> import org.apache.flink.api.common.functions.MapFunction
>>> import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
>>> import org.apache.flink.streaming.api.scala
>>> import org.apache.flink.streaming.util.serialization.SimpleStringSchema
>>> import org.apache.flink.table.api.TableEnvironment
>>> import org.apache.flink.table.api.scala._
>>> import org.apache.flink.api.scala._
>>> import org.apache.kafka.clients.consumer.ConsumerConfig
>>> import org.apache.kafka.clients.consumer.ConsumerRecord
>>> import org.apache.kafka.clients.consumer.KafkaConsumer
>>> import org.apache.flink.core.fs.FileSystem
>>> import org.apache.flink.streaming.api.TimeCharacteristic
>>> import org.slf4j.LoggerFactory
>>> import org.apache.flink.streaming.connectors.kafka.{FlinkKafkaConsumer011, 
>>> FlinkKafkaProducer011}
>>> import java.util.Calendar
>>> import java.util.Date
>>> import java.text.DateFormat
>>> import java.text.SimpleDateFormat
>>> import org.apache.log4j.Logger
>>> import org.apache.log4j.Level
>>> import sys.process.stringSeqToProcess
>>> import java.io.File
>>> 
>>> And this is the simple code
>>> 
>>> val properties = new Properties()
>>> properties.setProperty("bootstrap.servers", bootstrapServers)
>>> properties.setProperty("zookeeper.connect", zookeeperConnect)
>>> properties.setProperty("group.id", flinkAppName)
>>> properties.setProperty("auto.offset.reset", "latest")
>>> val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
>>> val dataStream =  streamExecEnv
>>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new 
>>> SimpleStringSchema(), properties))
>>>   val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>>>   tableEnv.registerDataStream("table1", dataStream, 'key, 'ticker, 
>>> 'timeissued, 'price)
>>> 
>>> And this is the compilation error
>>> 
>>> info] Compiling 1 Scala source to 
>>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>>> [error] 
>>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:138:
>>>  overloaded method value getTableEnvironment with alternatives:
>>> [error]   (executionEnvironment: 
>>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment)org.apache.flink.table.api.scala.StreamTableEnvironment
>>>  
>>> [error]   (executionEnvironment: 
>>> org.apache.flink.streaming.api.environm

Re: Converting a DataStream into a Table throws error

[vino yang]

Hi Mich,

It seems that the type of your DataStream stream is always wrong.
If you want to specify four fields, usually the DataStream type should be
similar: DataStream[(Type1, Type2, Type3, Type4)], not DataStream[String],
you can try it.

Thanks, vino

2018-08-02 6:44 GMT+08:00 Mich Talebzadeh :

> Changed as suggested
>
>val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
>  val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
> 'timeissued, 'price)
>
> Still the same error
>
> [info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
> streaming/target/scala-2.11/classes...
> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
> myPackage/md_streaming.scala:139: overloaded method value
> registerDataStream with alternatives:
> [error]   [T](name: String, dataStream: 
> org.apache.flink.streaming.api.datastream.DataStream[T],
> fields: String)Unit 
> [error]   [T](name: String, dataStream: org.apache.flink.streaming.
> api.datastream.DataStream[T])Unit
> [error]  cannot be applied to (String, org.apache.flink.streaming.
> api.environment.StreamExecutionEnvironment, Symbol, Symbol, Symbol,
> Symbol)
> [error]   tableEnv.registerDataStream("table1", streamExecEnv, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
> [error] Total time: 3 s, completed Aug 1, 2018 11:40:47 PM
>
> Thanks anyway.
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 1 Aug 2018 at 23:34, Fabian Hueske  wrote:
>
>> Hi,
>>
>> You have to pass the StreamExecutionEnvironment to the
>> getTableEnvironment() method, not the DataStream (or DataStreamSource).
>> Change
>>
>> val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>>
>> to
>>
>> val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>
>> Best,
>> Fabian
>>
>> 2018-08-02 0:10 GMT+02:00 Mich Talebzadeh :
>>
>>> Hi,
>>>
>>> FYI, these are my imports
>>>
>>> import java.util.Properties
>>> import java.util.Arrays
>>> import org.apache.flink.api.common.functions.MapFunction
>>> import org.apache.flink.streaming.api.environment.
>>> StreamExecutionEnvironment
>>> import org.apache.flink.streaming.api.scala
>>> import org.apache.flink.streaming.util.serialization.SimpleStringSchema
>>> import org.apache.flink.table.api.TableEnvironment
>>> import org.apache.flink.table.api.scala._
>>> import org.apache.flink.api.scala._
>>> import org.apache.kafka.clients.consumer.ConsumerConfig
>>> import org.apache.kafka.clients.consumer.ConsumerRecord
>>> import org.apache.kafka.clients.consumer.KafkaConsumer
>>> import org.apache.flink.core.fs.FileSystem
>>> import org.apache.flink.streaming.api.TimeCharacteristic
>>> import org.slf4j.LoggerFactory
>>> import org.apache.flink.streaming.connectors.kafka.{FlinkKafkaConsumer011,
>>> FlinkKafkaProducer011}
>>> import java.util.Calendar
>>> import java.util.Date
>>> import java.text.DateFormat
>>> import java.text.SimpleDateFormat
>>> import org.apache.log4j.Logger
>>> import org.apache.log4j.Level
>>> import sys.process.stringSeqToProcess
>>> import java.io.File
>>>
>>> And this is the simple code
>>>
>>> val properties = new Properties()
>>> properties.setProperty("bootstrap.servers", bootstrapServers)
>>> properties.setProperty("zookeeper.connect", zookeeperConnect)
>>> properties.setProperty("group.id", flinkAppName)
>>> properties.setProperty("auto.offset.reset", "latest")
>>> val  streamExecEnv = StreamExecutionEnvironment.
>>> getExecutionEnvironment
>>> val dataStream =  streamExecEnv
>>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>>> SimpleStringSchema(), properties))
>>>   val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>>>   tableEnv.registerDataStream("table1", dataStream, 'key, 'ticker,
>>> 'timeissued, 'price)
>>>
>>> And this is the compilation error
>>>
>>> info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
>>> streaming/target/scala-2.11/classes...
>>> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
>>> myPackage/md_streaming.scala:138: overloaded method value
>>> getTableEnvironment with 

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Changed as suggested

   val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
 val dataStream =  streamExecEnv
   .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))
  val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
  tableEnv.registerDataStream("table1", streamExecEnv, 'key, 'ticker,
'timeissued, 'price)

Still the same error

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:139:
overloaded method value registerDataStream with alternatives:
[error]   [T](name: String, dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T], fields:
String)Unit 
[error]   [T](name: String, dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T])Unit
[error]  cannot be applied to (String,
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,
Symbol, Symbol, Symbol, Symbol)
[error]   tableEnv.registerDataStream("table1", streamExecEnv, 'key,
'ticker, 'timeissued, 'price)
[error]^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Aug 1, 2018 11:40:47 PM

Thanks anyway.

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 1 Aug 2018 at 23:34, Fabian Hueske  wrote:

> Hi,
>
> You have to pass the StreamExecutionEnvironment to the
> getTableEnvironment() method, not the DataStream (or DataStreamSource).
> Change
>
> val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>
> to
>
> val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>
> Best,
> Fabian
>
> 2018-08-02 0:10 GMT+02:00 Mich Talebzadeh :
>
>> Hi,
>>
>> FYI, these are my imports
>>
>> import java.util.Properties
>> import java.util.Arrays
>> import org.apache.flink.api.common.functions.MapFunction
>> import
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
>> import org.apache.flink.streaming.api.scala
>> import org.apache.flink.streaming.util.serialization.SimpleStringSchema
>> import org.apache.flink.table.api.TableEnvironment
>> import org.apache.flink.table.api.scala._
>> import org.apache.flink.api.scala._
>> import org.apache.kafka.clients.consumer.ConsumerConfig
>> import org.apache.kafka.clients.consumer.ConsumerRecord
>> import org.apache.kafka.clients.consumer.KafkaConsumer
>> import org.apache.flink.core.fs.FileSystem
>> import org.apache.flink.streaming.api.TimeCharacteristic
>> import org.slf4j.LoggerFactory
>> import
>> org.apache.flink.streaming.connectors.kafka.{FlinkKafkaConsumer011,
>> FlinkKafkaProducer011}
>> import java.util.Calendar
>> import java.util.Date
>> import java.text.DateFormat
>> import java.text.SimpleDateFormat
>> import org.apache.log4j.Logger
>> import org.apache.log4j.Level
>> import sys.process.stringSeqToProcess
>> import java.io.File
>>
>> And this is the simple code
>>
>> val properties = new Properties()
>> properties.setProperty("bootstrap.servers", bootstrapServers)
>> properties.setProperty("zookeeper.connect", zookeeperConnect)
>> properties.setProperty("group.id", flinkAppName)
>> properties.setProperty("auto.offset.reset", "latest")
>> val  streamExecEnv =
>> StreamExecutionEnvironment.getExecutionEnvironment
>> val dataStream =  streamExecEnv
>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>> SimpleStringSchema(), properties))
>>   val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>>   tableEnv.registerDataStream("table1", dataStream, 'key, 'ticker,
>> 'timeissued, 'price)
>>
>> And this is the compilation error
>>
>> info] Compiling 1 Scala source to
>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>> [error]
>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:138:
>> overloaded method value getTableEnvironment with alternatives:
>> [error]   (executionEnvironment:
>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment)org.apache.flink.table.api.scala.StreamTableEnvironment
>> 
>> [error]   (executionEnvironment:
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)org.apache.flink.table.api.java.StreamTableEnvironment
>> 
>> [error]   (executionEnvironment:
>> org.apache.flink.api.scala.ExecutionEnvironment)org.apache.flink.table.api.scala.BatchTableE

Re: Converting a DataStream into a Table throws error

[Fabian Hueske]

Hi,

You have to pass the StreamExecutionEnvironment to the
getTableEnvironment() method, not the DataStream (or DataStreamSource).
Change

val tableEnv = TableEnvironment.getTableEnvironment(dataStream)

to

val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)

Best,
Fabian

2018-08-02 0:10 GMT+02:00 Mich Talebzadeh :

> Hi,
>
> FYI, these are my imports
>
> import java.util.Properties
> import java.util.Arrays
> import org.apache.flink.api.common.functions.MapFunction
> import org.apache.flink.streaming.api.environment.StreamExecutionEn
> vironment
> import org.apache.flink.streaming.api.scala
> import org.apache.flink.streaming.util.serialization.SimpleStringSchema
> import org.apache.flink.table.api.TableEnvironment
> import org.apache.flink.table.api.scala._
> import org.apache.flink.api.scala._
> import org.apache.kafka.clients.consumer.ConsumerConfig
> import org.apache.kafka.clients.consumer.ConsumerRecord
> import org.apache.kafka.clients.consumer.KafkaConsumer
> import org.apache.flink.core.fs.FileSystem
> import org.apache.flink.streaming.api.TimeCharacteristic
> import org.slf4j.LoggerFactory
> import org.apache.flink.streaming.connectors.kafka.{FlinkKafkaConsumer011,
> FlinkKafkaProducer011}
> import java.util.Calendar
> import java.util.Date
> import java.text.DateFormat
> import java.text.SimpleDateFormat
> import org.apache.log4j.Logger
> import org.apache.log4j.Level
> import sys.process.stringSeqToProcess
> import java.io.File
>
> And this is the simple code
>
> val properties = new Properties()
> properties.setProperty("bootstrap.servers", bootstrapServers)
> properties.setProperty("zookeeper.connect", zookeeperConnect)
> properties.setProperty("group.id", flinkAppName)
> properties.setProperty("auto.offset.reset", "latest")
> val  streamExecEnv = StreamExecutionEnvironment.get
> ExecutionEnvironment
> val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>   val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
>   tableEnv.registerDataStream("table1", dataStream, 'key, 'ticker,
> 'timeissued, 'price)
>
> And this is the compilation error
>
> info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
> streaming/target/scala-2.11/classes...
> [error] 
> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:138:
> overloaded method value getTableEnvironment with alternatives:
> [error]   (executionEnvironment: org.apache.flink.streaming.api
> .scala.StreamExecutionEnvironment)org.apache.flink.table.api.scala.StreamTableEnvironment
> 
> [error]   (executionEnvironment: org.apache.flink.streaming.api
> .environment.StreamExecutionEnvironment)org.apache.flink.table.api.java.StreamTableEnvironment
> 
> [error]   (executionEnvironment: org.apache.flink.api.scala.Exe
> cutionEnvironment)org.apache.flink.table.api.scala.BatchTableEnvironment
> 
> [error]   (executionEnvironment: org.apache.flink.api.java.Exec
> utionEnvironment)org.apache.flink.table.api.java.BatchTableEnvironment
> [error]  cannot be applied to (org.apache.flink.streaming.ap
> i.datastream.DataStreamSource[String])
> [error]   val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
> [error]   ^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
> [error] Total time: 3 s, completed Aug 1, 2018 11:02:33 PM
> Completed compiling
>
> which is really strange
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 1 Aug 2018 at 13:42, Fabian Hueske  wrote:
>
>> Hi I think you are mixing Java and Scala dependencies.
>>
>> org.apache.flink.streaming.api.datastream.DataStream is the DataStream
>> of the Java DataStream API.
>> You should use the DataStream of the Scala DataStream API.
>>
>> Best, Fabian
>>
>> 2018-08-01 14:01 GMT+02:00 Mich Talebzadeh :
>>
>>> Hi,
>>>
>>> I believed I tried Hequn's suggestion and tried again
>>>
>>> import org.apache.flink.table.api.Table
>>> import org.apache.flink.table.api.TableEnvironment
>>>
>>> *import org.apache.flink.table.api.scala._*
>>> Unfortunately I am still getting the same error!
>>>
>>> [info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
>>> streaming/target/scala-2.11/classes...
>>> [error] 
>>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scal

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Hi,

FYI, these are my imports

import java.util.Properties
import java.util.Arrays
import org.apache.flink.api.common.functions.MapFunction
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
import org.apache.flink.streaming.api.scala
import org.apache.flink.streaming.util.serialization.SimpleStringSchema
import org.apache.flink.table.api.TableEnvironment
import org.apache.flink.table.api.scala._
import org.apache.flink.api.scala._
import org.apache.kafka.clients.consumer.ConsumerConfig
import org.apache.kafka.clients.consumer.ConsumerRecord
import org.apache.kafka.clients.consumer.KafkaConsumer
import org.apache.flink.core.fs.FileSystem
import org.apache.flink.streaming.api.TimeCharacteristic
import org.slf4j.LoggerFactory
import org.apache.flink.streaming.connectors.kafka.{FlinkKafkaConsumer011,
FlinkKafkaProducer011}
import java.util.Calendar
import java.util.Date
import java.text.DateFormat
import java.text.SimpleDateFormat
import org.apache.log4j.Logger
import org.apache.log4j.Level
import sys.process.stringSeqToProcess
import java.io.File

And this is the simple code

val properties = new Properties()
properties.setProperty("bootstrap.servers", bootstrapServers)
properties.setProperty("zookeeper.connect", zookeeperConnect)
properties.setProperty("group.id", flinkAppName)
properties.setProperty("auto.offset.reset", "latest")
val  streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment
val dataStream =  streamExecEnv
   .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))
  val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
  tableEnv.registerDataStream("table1", dataStream, 'key, 'ticker,
'timeissued, 'price)

And this is the compilation error

info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:138:
overloaded method value getTableEnvironment with alternatives:
[error]   (executionEnvironment:
org.apache.flink.streaming.api.scala.StreamExecutionEnvironment)org.apache.flink.table.api.scala.StreamTableEnvironment

[error]   (executionEnvironment:
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)org.apache.flink.table.api.java.StreamTableEnvironment

[error]   (executionEnvironment:
org.apache.flink.api.scala.ExecutionEnvironment)org.apache.flink.table.api.scala.BatchTableEnvironment

[error]   (executionEnvironment:
org.apache.flink.api.java.ExecutionEnvironment)org.apache.flink.table.api.java.BatchTableEnvironment
[error]  cannot be applied to
(org.apache.flink.streaming.api.datastream.DataStreamSource[String])
[error]   val tableEnv = TableEnvironment.getTableEnvironment(dataStream)
[error]   ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Aug 1, 2018 11:02:33 PM
Completed compiling

which is really strange

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 1 Aug 2018 at 13:42, Fabian Hueske  wrote:

> Hi I think you are mixing Java and Scala dependencies.
>
> org.apache.flink.streaming.api.datastream.DataStream is the DataStream of
> the Java DataStream API.
> You should use the DataStream of the Scala DataStream API.
>
> Best, Fabian
>
> 2018-08-01 14:01 GMT+02:00 Mich Talebzadeh :
>
>> Hi,
>>
>> I believed I tried Hequn's suggestion and tried again
>>
>> import org.apache.flink.table.api.Table
>> import org.apache.flink.table.api.TableEnvironment
>>
>> *import org.apache.flink.table.api.scala._*
>> Unfortunately I am still getting the same error!
>>
>> [info] Compiling 1 Scala source to
>> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
>> [error]
>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:151:
>> overloaded method value fromDataStream with alternatives:
>> [error]   [T](dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
>> String)org.apache.flink.table.api.Table 
>> [error]   [T](dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
>> [error]  cannot be applied to
>> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
>> Symbol, Symbol, Symbol, Symbol)
>> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
>> 't

Re: Converting a DataStream into a Table throws error

[Fabian Hueske]

Hi I think you are mixing Java and Scala dependencies.

org.apache.flink.streaming.api.datastream.DataStream is the DataStream of
the Java DataStream API.
You should use the DataStream of the Scala DataStream API.

Best, Fabian

2018-08-01 14:01 GMT+02:00 Mich Talebzadeh :

> Hi,
>
> I believed I tried Hequn's suggestion and tried again
>
> import org.apache.flink.table.api.Table
> import org.apache.flink.table.api.TableEnvironment
>
> *import org.apache.flink.table.api.scala._*
> Unfortunately I am still getting the same error!
>
> [info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
> streaming/target/scala-2.11/classes...
> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
> myPackage/md_streaming.scala:151: overloaded method value fromDataStream
> with alternatives:
> [error]   [T](dataStream: 
> org.apache.flink.streaming.api.datastream.DataStream[T],
> fields: String)org.apache.flink.table.api.Table 
> [error]   [T](dataStream: org.apache.flink.streaming.
> api.datastream.DataStream[T])org.apache.flink.table.api.Table
> [error]  cannot be applied to 
> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
> Symbol, Symbol, Symbol, Symbol)
> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
> [error] Total time: 3 s, completed Aug 1, 2018 12:59:44 PM
> Completed compiling
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 1 Aug 2018 at 10:03, Timo Walther  wrote:
>
>> If these two imports are the only imports that you added, then you did
>> not follow Hequn's advice or the link that I sent you.
>>
>> You need to add the underscore imports to let Scala do its magic.
>>
>> Timo
>>
>>
>> Am 01.08.18 um 10:28 schrieb Mich Talebzadeh:
>>
>> Hi Timo,
>>
>> These are my two flink table related imports
>>
>> import org.apache.flink.table.api.Table
>> import org.apache.flink.table.api.TableEnvironment
>>
>> And these are my dependencies building with SBT
>>
>> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
>> libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
>> libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
>> libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
>> libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
>> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-0.11"
>> % "1.5.0"
>> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-base"
>> % "1.5.0"
>> libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
>> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
>> libraryDependencies += "org.apache.flink" %% "flink-streaming-java" %
>> "1.5.0" % "provided"
>>
>> *libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
>> "provided" *libraryDependencies += "org.apache.kafka" %% "kafka" %
>> "0.11.0.0"
>>
>> There appears to be conflict somewhere that cause this error
>>
>> [info] Compiling 1 Scala source to /home/hduser/dba/bin/flink/md_
>> streaming/target/scala-2.11/classes...
>> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
>> myPackage/md_streaming.scala:152: overloaded method value fromDataStream
>> with alternatives:
>> [error]   [T](dataStream: 
>> org.apache.flink.streaming.api.datastream.DataStream[T],
>> fields: String)org.apache.flink.table.api.Table 
>> [error]   [T](dataStream: org.apache.flink.streaming.
>> api.datastream.DataStream[T])org.apache.flink.table.api.Table
>> [error]  cannot be applied to 
>> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
>> Symbol, Symbol, Symbol, Symbol)
>> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
>> 'ticker, 'timeissued, 'price)
>> [error]^
>> [error] one error found
>> [error] (compile:compileIncremental) Compilation failed
>>
>> Thanks
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destru

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Hi,

I believed I tried Hequn's suggestion and tried again

import org.apache.flink.table.api.Table
import org.apache.flink.table.api.TableEnvironment

*import org.apache.flink.table.api.scala._*
Unfortunately I am still getting the same error!

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:151:
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T], fields:
String)org.apache.flink.table.api.Table 
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to
(org.apache.flink.streaming.api.datastream.DataStreamSource[String],
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
'ticker, 'timeissued, 'price)
[error]^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Aug 1, 2018 12:59:44 PM
Completed compiling


Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 1 Aug 2018 at 10:03, Timo Walther  wrote:

> If these two imports are the only imports that you added, then you did not
> follow Hequn's advice or the link that I sent you.
>
> You need to add the underscore imports to let Scala do its magic.
>
> Timo
>
>
> Am 01.08.18 um 10:28 schrieb Mich Talebzadeh:
>
> Hi Timo,
>
> These are my two flink table related imports
>
> import org.apache.flink.table.api.Table
> import org.apache.flink.table.api.TableEnvironment
>
> And these are my dependencies building with SBT
>
> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
> libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
> libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
> libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
> libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-0.11"
> % "1.5.0"
> libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-base"
> % "1.5.0"
> libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
> libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
> libraryDependencies += "org.apache.flink" %% "flink-streaming-java" %
> "1.5.0" % "provided"
>
> *libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
> "provided" *libraryDependencies += "org.apache.kafka" %% "kafka" %
> "0.11.0.0"
>
> There appears to be conflict somewhere that cause this error
>
> [info] Compiling 1 Scala source to
> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
> [error]
> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152:
> overloaded method value fromDataStream with alternatives:
> [error]   [T](dataStream:
> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
> String)org.apache.flink.table.api.Table 
> [error]   [T](dataStream:
> org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
> [error]  cannot be applied to
> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
> Symbol, Symbol, Symbol, Symbol)
> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> Thanks
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Wed, 1 Aug 2018 at 09:17, Timo Walther  wrote:
>
>> Hi Mich,
>>
>> I would check you imports again [1]. This is a pure compiler issue that
>> is unrelated to your actual data stream. Also check your pr

Re: Converting a DataStream into a Table throws error

[Timo Walther]

If these two imports are the only imports that you added, then you did 
not follow Hequn's advice or the link that I sent you.


You need to add the underscore imports to let Scala do its magic.

Timo


Am 01.08.18 um 10:28 schrieb Mich Talebzadeh:

Hi Timo,

These are my two flink table related imports

import org.apache.flink.table.api.Table
import org.apache.flink.table.api.TableEnvironment

And these are my dependencies building with SBT

libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
libraryDependencies += "org.apache.flink" %% 
"flink-connector-kafka-0.11" % "1.5.0"
libraryDependencies += "org.apache.flink" %% 
"flink-connector-kafka-base" % "1.5.0"

libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
libraryDependencies += "org.apache.flink" %% "flink-streaming-java" % 
"1.5.0" % "provided"
*libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" 
% "provided"

*libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"

There appears to be conflict somewhere that cause this error

[info] Compiling 1 Scala source to 
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error] 
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152: 
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream: 
org.apache.flink.streaming.api.datastream.DataStream[T], fields: 
String)org.apache.flink.table.api.Table 
[error]   [T](dataStream: 
org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to 
(org.apache.flink.streaming.api.datastream.DataStreamSource[String], 
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table = tableEnv.fromDataStream(dataStream, 
'key, 'ticker, 'timeissued, 'price)

[error]    ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

Thanks


Dr Mich Talebzadeh

LinkedIn 
/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/


http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk.Any and all responsibility for 
any loss, damage or destruction of data or any other property which 
may arise from relying on this email's technical content is explicitly 
disclaimed. The author will in no case be liable for any monetary 
damages arising from such loss, damage or destruction.




On Wed, 1 Aug 2018 at 09:17, Timo Walther > wrote:


Hi Mich,

I would check you imports again [1]. This is a pure compiler issue
that is unrelated to your actual data stream. Also check your
project dependencies.

Regards,
Timo

[1]

https://ci.apache.org/projects/flink/flink-docs-master/dev/table/common.html#implicit-conversion-for-scala

Am 01.08.18 um 09:30 schrieb Mich Talebzadeh:


Hi both,

I added the import as Hequn suggested.

My stream is very simple and consists of 4 values separated by
"," as below

05521df6-4ccf-4b2f-b874-eb27d461b305,IBM,2018-07-30T19:51:50,190.48

So this is what I have been trying to do

Code

val dataStream =  streamExecEnv
   .addSource(new FlinkKafkaConsumer011[String](topicsValue,
new SimpleStringSchema(), properties))
 //
 //
  val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
  val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
'ticker, 'timeissued, 'price)

note those four columns in Table1 definition

And this is the error being thrown

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]

/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152:
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T], fields:
String)org.apache.flink.table.api.Table 
[error]   [T](dataStream:

org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to
(org.apache.flink.streaming.api.datastream.DataStreamSource[String],
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table = tableEnv.fromDataStream(dataStream,
'key, 'ticker, 'timeissued, 'price)
[error]    ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

I suspect dataStream may not be compatible with this operation?

Regards,

Dr Mich Talebzadeh

LinkedIn

/https:/

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Hi Timo,

These are my two flink table related imports

import org.apache.flink.table.api.Table
import org.apache.flink.table.api.TableEnvironment

And these are my dependencies building with SBT

libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.2.1"
libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.6"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.6"
libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-0.11" %
"1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-connector-kafka-base" %
"1.5.0"
libraryDependencies += "org.apache.flink" %% "flink-scala" % "1.5.0"
libraryDependencies += "org.apache.kafka" % "kafka-clients" % "0.11.0.0"
libraryDependencies += "org.apache.flink" %% "flink-streaming-java" %
"1.5.0" % "provided"

*libraryDependencies += "org.apache.flink" %% "flink-table" % "1.5.0" %
"provided"*libraryDependencies += "org.apache.kafka" %% "kafka" % "0.11.0.0"

There appears to be conflict somewhere that cause this error

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152:
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T], fields:
String)org.apache.flink.table.api.Table 
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to
(org.apache.flink.streaming.api.datastream.DataStreamSource[String],
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
'ticker, 'timeissued, 'price)
[error]^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

Thanks


Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 1 Aug 2018 at 09:17, Timo Walther  wrote:

> Hi Mich,
>
> I would check you imports again [1]. This is a pure compiler issue that is
> unrelated to your actual data stream. Also check your project dependencies.
>
> Regards,
> Timo
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/common.html#implicit-conversion-for-scala
>
> Am 01.08.18 um 09:30 schrieb Mich Talebzadeh:
>
>
> Hi both,
>
> I added the import as Hequn suggested.
>
> My stream is very simple and consists of 4 values separated by "," as
> below
>
> 05521df6-4ccf-4b2f-b874-eb27d461b305,IBM,2018-07-30T19:51:50,190.48
>
> So this is what I have been trying to do
>
> Code
>
> val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>  //
>  //
>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   val table1: Table = tableEnv.fromDataStream(dataStream, 'key, 'ticker,
> 'timeissued, 'price)
>
> note those four columns in Table1 definition
>
> And this is the error being thrown
>
> [info] Compiling 1 Scala source to
> /home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
> [error]
> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152:
> overloaded method value fromDataStream with alternatives:
> [error]   [T](dataStream:
> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
> String)org.apache.flink.table.api.Table 
> [error]   [T](dataStream:
> org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
> [error]  cannot be applied to
> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
> Symbol, Symbol, Symbol, Symbol)
> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> I suspect dataStream may not be compatible with this operation?
>
> Regards,
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> 

Re: Converting a DataStream into a Table throws error

[Timo Walther]

Hi Mich,

I would check you imports again [1]. This is a pure compiler issue that 
is unrelated to your actual data stream. Also check your project 
dependencies.


Regards,
Timo

[1] 
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/common.html#implicit-conversion-for-scala


Am 01.08.18 um 09:30 schrieb Mich Talebzadeh:


Hi both,

I added the import as Hequn suggested.

My stream is very simple and consists of 4 values separated by "," as 
below


05521df6-4ccf-4b2f-b874-eb27d461b305,IBM,2018-07-30T19:51:50,190.48

So this is what I have been trying to do

Code

    val dataStream =  streamExecEnv
   .addSource(new FlinkKafkaConsumer011[String](topicsValue, new 
SimpleStringSchema(), properties))

 //
 //
  val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
  val table1: Table = tableEnv.fromDataStream(dataStream, 'key, 
'ticker, 'timeissued, 'price)


note those four columns in Table1 definition

And this is the error being thrown

[info] Compiling 1 Scala source to 
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error] 
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152: 
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream: 
org.apache.flink.streaming.api.datastream.DataStream[T], fields: 
String)org.apache.flink.table.api.Table 
[error]   [T](dataStream: 
org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to 
(org.apache.flink.streaming.api.datastream.DataStreamSource[String], 
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table = tableEnv.fromDataStream(dataStream, 
'key, 'ticker, 'timeissued, 'price)

[error]    ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

I suspect dataStream may not be compatible with this operation?

Regards,

Dr Mich Talebzadeh

LinkedIn 
/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/


http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk.Any and all responsibility for 
any loss, damage or destruction of data or any other property which 
may arise from relying on this email's technical content is explicitly 
disclaimed. The author will in no case be liable for any monetary 
damages arising from such loss, damage or destruction.




On Wed, 1 Aug 2018 at 04:51, Hequn Cheng > wrote:


Hi, Mich

You can try adding "import org.apache.flink.table.api.scala._", so
that the Symbol can be recognized as an Expression.

Best, Hequn

On Wed, Aug 1, 2018 at 6:16 AM, Mich Talebzadeh
mailto:mich.talebza...@gmail.com>> wrote:

Hi,

I am following this example


https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/table/common.html#integration-with-datastream-and-dataset-api

This is my dataStream which is built on a Kafka topic

//
    //Create a Kafka consumer
    //
    val dataStream =  streamExecEnv
   .addSource(new
FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))
 //
 //
  val tableEnv =
TableEnvironment.getTableEnvironment(streamExecEnv)
  val table1: Table = tableEnv.fromDataStream(dataStream,
'key, 'ticker, 'timeissued, 'price)

While compiling it throws this error

[error]

/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:169:
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T],
fields: String)org.apache.flink.table.api.Table 
[error]   [T](dataStream:

org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to
(org.apache.flink.streaming.api.datastream.DataStreamSource[String],
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table =
tableEnv.fromDataStream(dataStream, 'key, 'ticker,
'timeissued, 'price)
[error]    ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

The topic is very simple, it is comma separated prices. I
tried mapFunction and flatMap but neither worked!

Thanks,


Dr Mich Talebzadeh

LinkedIn

/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/

http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk.Any and all
responsibility for any loss, damage or destruction of data or
any other property which may arise from relying on this
email's technical content is explicitly disclaimed. The author
 

Re: Converting a DataStream into a Table throws error

[Mich Talebzadeh]

Hi both,

I added the import as Hequn suggested.

My stream is very simple and consists of 4 values separated by "," as below

05521df6-4ccf-4b2f-b874-eb27d461b305,IBM,2018-07-30T19:51:50,190.48

So this is what I have been trying to do

Code

val dataStream =  streamExecEnv
   .addSource(new FlinkKafkaConsumer011[String](topicsValue, new
SimpleStringSchema(), properties))
 //
 //
  val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
  val table1: Table = tableEnv.fromDataStream(dataStream, 'key, 'ticker,
'timeissued, 'price)

note those four columns in Table1 definition

And this is the error being thrown

[info] Compiling 1 Scala source to
/home/hduser/dba/bin/flink/md_streaming/target/scala-2.11/classes...
[error]
/home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:152:
overloaded method value fromDataStream with alternatives:
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T], fields:
String)org.apache.flink.table.api.Table 
[error]   [T](dataStream:
org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
[error]  cannot be applied to
(org.apache.flink.streaming.api.datastream.DataStreamSource[String],
Symbol, Symbol, Symbol, Symbol)
[error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
'ticker, 'timeissued, 'price)
[error]^
[error] one error found
[error] (compile:compileIncremental) Compilation failed

I suspect dataStream may not be compatible with this operation?

Regards,

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Wed, 1 Aug 2018 at 04:51, Hequn Cheng  wrote:

> Hi, Mich
>
> You can try adding "import org.apache.flink.table.api.scala._", so that
> the Symbol can be recognized as an Expression.
>
> Best, Hequn
>
> On Wed, Aug 1, 2018 at 6:16 AM, Mich Talebzadeh  > wrote:
>
>> Hi,
>>
>> I am following this example
>>
>>
>> https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/table/common.html#integration-with-datastream-and-dataset-api
>>
>> This is my dataStream which is built on a Kafka topic
>>
>>//
>> //Create a Kafka consumer
>> //
>> val dataStream =  streamExecEnv
>>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
>> SimpleStringSchema(), properties))
>>  //
>>  //
>>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>>   val table1: Table = tableEnv.fromDataStream(dataStream, 'key, 'ticker,
>> 'timeissued, 'price)
>>
>> While compiling it throws this error
>>
>> [error]
>> /home/hduser/dba/bin/flink/md_streaming/src/main/scala/myPackage/md_streaming.scala:169:
>> overloaded method value fromDataStream with alternatives:
>> [error]   [T](dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T], fields:
>> String)org.apache.flink.table.api.Table 
>> [error]   [T](dataStream:
>> org.apache.flink.streaming.api.datastream.DataStream[T])org.apache.flink.table.api.Table
>> [error]  cannot be applied to
>> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
>> Symbol, Symbol, Symbol, Symbol)
>> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
>> 'ticker, 'timeissued, 'price)
>> [error]^
>> [error] one error found
>> [error] (compile:compileIncremental) Compilation failed
>>
>> The topic is very simple, it is comma separated prices. I tried
>> mapFunction and flatMap but neither worked!
>>
>> Thanks,
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * 
>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> *
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>
>

Re: Converting a DataStream into a Table throws error

[vino yang]

Hi Mich,

The field specified by the fromDataStream API must match the number of
fields contained in the DataStream stream object, your DataStream's type is
just a string, example is here.[1]

[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/table/common.html#convert-a-datastream-or-dataset-into-a-table

Thanks, vino.

2018-08-01 6:16 GMT+08:00 Mich Talebzadeh :

> Hi,
>
> I am following this example
>
> https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/dev/table/common.html#integration-with-
> datastream-and-dataset-api
>
> This is my dataStream which is built on a Kafka topic
>
>//
> //Create a Kafka consumer
> //
> val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>  //
>  //
>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   val table1: Table = tableEnv.fromDataStream(dataStream, 'key, 'ticker,
> 'timeissued, 'price)
>
> While compiling it throws this error
>
> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
> myPackage/md_streaming.scala:169: overloaded method value fromDataStream
> with alternatives:
> [error]   [T](dataStream: 
> org.apache.flink.streaming.api.datastream.DataStream[T],
> fields: String)org.apache.flink.table.api.Table 
> [error]   [T](dataStream: org.apache.flink.streaming.
> api.datastream.DataStream[T])org.apache.flink.table.api.Table
> [error]  cannot be applied to 
> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
> Symbol, Symbol, Symbol, Symbol)
> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> The topic is very simple, it is comma separated prices. I tried
> mapFunction and flatMap but neither worked!
>
> Thanks,
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>

Re: Converting a DataStream into a Table throws error

[Hequn Cheng]

Hi, Mich

You can try adding "import org.apache.flink.table.api.scala._", so that the
Symbol can be recognized as an Expression.

Best, Hequn

On Wed, Aug 1, 2018 at 6:16 AM, Mich Talebzadeh 
wrote:

> Hi,
>
> I am following this example
>
> https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/dev/table/common.html#integration-with-
> datastream-and-dataset-api
>
> This is my dataStream which is built on a Kafka topic
>
>//
> //Create a Kafka consumer
> //
> val dataStream =  streamExecEnv
>.addSource(new FlinkKafkaConsumer011[String](topicsValue, new
> SimpleStringSchema(), properties))
>  //
>  //
>   val tableEnv = TableEnvironment.getTableEnvironment(streamExecEnv)
>   val table1: Table = tableEnv.fromDataStream(dataStream, 'key, 'ticker,
> 'timeissued, 'price)
>
> While compiling it throws this error
>
> [error] /home/hduser/dba/bin/flink/md_streaming/src/main/scala/
> myPackage/md_streaming.scala:169: overloaded method value fromDataStream
> with alternatives:
> [error]   [T](dataStream: 
> org.apache.flink.streaming.api.datastream.DataStream[T],
> fields: String)org.apache.flink.table.api.Table 
> [error]   [T](dataStream: org.apache.flink.streaming.
> api.datastream.DataStream[T])org.apache.flink.table.api.Table
> [error]  cannot be applied to 
> (org.apache.flink.streaming.api.datastream.DataStreamSource[String],
> Symbol, Symbol, Symbol, Symbol)
> [error]   val table1: Table = tableEnv.fromDataStream(dataStream, 'key,
> 'ticker, 'timeissued, 'price)
> [error]^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> The topic is very simple, it is comma separated prices. I tried
> mapFunction and flatMap but neither worked!
>
> Thanks,
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>