[jira] [Comment Edited] (FLINK-10674) DistinctAccumulator.remove lead to NPE
[
https://issues.apache.org/jira/browse/FLINK-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683156#comment-16683156
]
winifredtang edited comment on FLINK-10674 at 11/12/18 4:03 AM:
[~twalthr] [~ambition] Sorry, I haven't noticed before, so I created a pull
request too.
Our PR might be little different, I don't think when value = (currentCnt -
1L)<0, we should change it to 1. I think that when (currentCnt - 1L)<0, the
corresponding instance in distinctValueMap should be remove.
was (Author: winipanda):
[~twalthr] [~ambition] Sorry, I haven't noticed before, so I created a pull
request too.
> DistinctAccumulator.remove lead to NPE
> --
>
> Key: FLINK-10674
> URL: https://issues.apache.org/jira/browse/FLINK-10674
> Project: Flink
> Issue Type: Bug
> Components: Table API SQL
>Affects Versions: 1.6.1
> Environment: Flink 1.6.0
>Reporter: ambition
>Assignee: winifredtang
>Priority: Minor
> Labels: pull-request-available
> Attachments: image-2018-10-25-14-46-03-373.png
>
>
> Our online Flink Job run about a week，job contain sql ：
> {code:java}
> select `time`,
> lower(trim(os_type)) as os_type,
> count(distinct feed_id) as feed_total_view
> from my_table
> group by `time`, lower(trim(os_type)){code}
>
> then occur NPE:
>
> {code:java}
> java.lang.NullPointerException
> at scala.Predef$.Long2long(Predef.scala:363)
> at
> org.apache.flink.table.functions.aggfunctions.DistinctAccumulator.remove(DistinctAccumulator.scala:109)
> at NonWindowedAggregationHelper$894.retract(Unknown Source)
> at
> org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:124)
> at
> org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:39)
> at
> org.apache.flink.streaming.api.operators.LegacyKeyedProcessOperator.processElement(LegacyKeyedProcessOperator.java:88)
> at
> org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
> at
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
>
> View DistinctAccumulator.remove
> !image-2018-10-25-14-46-03-373.png!
>
> this NPE should currentCnt = null lead to, so we simple handle like :
> {code:java}
> def remove(params: Row): Boolean = {
> if(!distinctValueMap.contains(params)){
> true
> }else{
> val currentCnt = distinctValueMap.get(params)
> //
> if (currentCnt == null || currentCnt == 1) {
> distinctValueMap.remove(params)
> true
> } else {
> var value = currentCnt - 1L
> if(value < 0){
> value = 1
> }
> distinctValueMap.put(params, value)
> false
> }
> }
> }{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10674) DistinctAccumulator.remove lead to NPE
[
https://issues.apache.org/jira/browse/FLINK-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683156#comment-16683156
]
winifredtang commented on FLINK-10674:
--
[~twalthr] [~ambition] Sorry, I haven't noticed before, so I created a pull
request too.
> DistinctAccumulator.remove lead to NPE
> --
>
> Key: FLINK-10674
> URL: https://issues.apache.org/jira/browse/FLINK-10674
> Project: Flink
> Issue Type: Bug
> Components: Table API SQL
>Affects Versions: 1.6.1
> Environment: Flink 1.6.0
>Reporter: ambition
>Assignee: winifredtang
>Priority: Minor
> Labels: pull-request-available
> Attachments: image-2018-10-25-14-46-03-373.png
>
>
> Our online Flink Job run about a week，job contain sql ：
> {code:java}
> select `time`,
> lower(trim(os_type)) as os_type,
> count(distinct feed_id) as feed_total_view
> from my_table
> group by `time`, lower(trim(os_type)){code}
>
> then occur NPE:
>
> {code:java}
> java.lang.NullPointerException
> at scala.Predef$.Long2long(Predef.scala:363)
> at
> org.apache.flink.table.functions.aggfunctions.DistinctAccumulator.remove(DistinctAccumulator.scala:109)
> at NonWindowedAggregationHelper$894.retract(Unknown Source)
> at
> org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:124)
> at
> org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:39)
> at
> org.apache.flink.streaming.api.operators.LegacyKeyedProcessOperator.processElement(LegacyKeyedProcessOperator.java:88)
> at
> org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
> at
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
>
> View DistinctAccumulator.remove
> !image-2018-10-25-14-46-03-373.png!
>
> this NPE should currentCnt = null lead to, so we simple handle like :
> {code:java}
> def remove(params: Row): Boolean = {
> if(!distinctValueMap.contains(params)){
> true
> }else{
> val currentCnt = distinctValueMap.get(params)
> //
> if (currentCnt == null || currentCnt == 1) {
> distinctValueMap.remove(params)
> true
> } else {
> var value = currentCnt - 1L
> if(value < 0){
> value = 1
> }
> distinctValueMap.put(params, value)
> false
> }
> }
> }{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10674) DistinctAccumulator.remove lead to NPE
[
https://issues.apache.org/jira/browse/FLINK-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679335#comment-16679335
]
winifredtang commented on FLINK-10674:
--
[~twalthr] I have already modified the DistinctAccumulator.remove(params: Row),
but I'm not sure if it's the only cause, I am trying to reproduce the problem
to exam it.
> DistinctAccumulator.remove lead to NPE
> --
>
> Key: FLINK-10674
> URL: https://issues.apache.org/jira/browse/FLINK-10674
> Project: Flink
> Issue Type: Bug
> Components: flink-contrib
>Affects Versions: 1.6.1
> Environment: Flink 1.6.0
>Reporter: ambition
>Assignee: winifredtang
>Priority: Minor
> Attachments: image-2018-10-25-14-46-03-373.png
>
>
> Our online Flink Job run about a week，job contain sql ：
> {code:java}
> select `time`,
> lower(trim(os_type)) as os_type,
> count(distinct feed_id) as feed_total_view
> from my_table
> group by `time`, lower(trim(os_type)){code}
>
> then occur NPE:
>
> {code:java}
> java.lang.NullPointerException
> at scala.Predef$.Long2long(Predef.scala:363)
> at
> org.apache.flink.table.functions.aggfunctions.DistinctAccumulator.remove(DistinctAccumulator.scala:109)
> at NonWindowedAggregationHelper$894.retract(Unknown Source)
> at
> org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:124)
> at
> org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:39)
> at
> org.apache.flink.streaming.api.operators.LegacyKeyedProcessOperator.processElement(LegacyKeyedProcessOperator.java:88)
> at
> org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
> at
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
>
> View DistinctAccumulator.remove
> !image-2018-10-25-14-46-03-373.png!
>
> this NPE should currentCnt = null lead to, so we simple handle like :
> {code:java}
> def remove(params: Row): Boolean = {
> if(!distinctValueMap.contains(params)){
> true
> }else{
> val currentCnt = distinctValueMap.get(params)
> //
> if (currentCnt == null || currentCnt == 1) {
> distinctValueMap.remove(params)
> true
> } else {
> var value = currentCnt - 1L
> if(value < 0){
> value = 1
> }
> distinctValueMap.put(params, value)
> false
> }
> }
> }{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (FLINK-10795) STDDEV_POP error
[
https://issues.apache.org/jira/browse/FLINK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
winifredtang reassigned FLINK-10795:
Assignee: winifredtang (was: vinoyang)
> STDDEV_POP error
>
>
> Key: FLINK-10795
> URL: https://issues.apache.org/jira/browse/FLINK-10795
> Project: Flink
> Issue Type: Improvement
> Components: Table API SQL
>Affects Versions: 1.6.2
>Reporter: Flavio Pompermaier
>Assignee: winifredtang
>Priority: Major
> Attachments: FlinkTableApiError.java, test.tsv
>
>
> if using STDDEV_POP in the attached job the following error is thrown (with
> Flink 1.6.1):
>
> {code:java}
> Exception in thread "main"
> org.apache.flink.runtime.client.JobExecutionException:
> java.lang.NumberFormatException
> at
> org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:623)
> at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:235)
> at
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
> at
> org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:816)
> at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
> at org.apache.flink.api.java.DataSet.print(DataSet.java:1652)
> at
> it.okkam.datalinks.batch.flink.operations.FlinkTableApiError.main(FlinkTableApiError.java:466)
> Caused by: java.lang.NumberFormatException
> at java.math.BigDecimal.(BigDecimal.java:494)
> at java.math.BigDecimal.(BigDecimal.java:383)
> at java.math.BigDecimal.(BigDecimal.java:806)
> at java.math.BigDecimal.valueOf(BigDecimal.java:1274)
> at org.apache.calcite.runtime.SqlFunctions.sround(SqlFunctions.java:1242)
> at DataSetCalcRule$6909.flatMap(Unknown Source)
> at
> org.apache.flink.table.runtime.FlatMapRunner.flatMap(FlatMapRunner.scala:52)
> at
> org.apache.flink.table.runtime.FlatMapRunner.flatMap(FlatMapRunner.scala:31)
> at
> org.apache.flink.runtime.operators.chaining.ChainedFlatMapDriver.collect(ChainedFlatMapDriver.java:80)
> at
> org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
> at DataSetSingleRowJoinRule$6450.join(Unknown Source)
> at
> org.apache.flink.table.runtime.MapJoinLeftRunner.flatMap(MapJoinLeftRunner.scala:35)
> at
> org.apache.flink.runtime.operators.FlatMapDriver.run(FlatMapDriver.java:109)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
> at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (FLINK-10231) Add a view SQL DDL
[ https://issues.apache.org/jira/browse/FLINK-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668093#comment-16668093 ] winifredtang edited comment on FLINK-10231 at 10/31/18 2:50 AM: [~fhueske] Hello, could you give more details about it? I try to achieve a view SQL DDL. I wonder if the TableEnvironment.registerTableInternal(name: String, table: AbstractTable) might be deprecated too. Thanks a lot. was (Author: winipanda): Fabian Hueske Hello, could you give more details about it? I try to achieve a view SQL DDL. I wonder if the TableEnvironment.registerTableInternal(name: String, table: AbstractTable) might be deprecated too. Thanks a lot. > Add a view SQL DDL > -- > > Key: FLINK-10231 > URL: https://issues.apache.org/jira/browse/FLINK-10231 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Reporter: Timo Walther >Assignee: winifredtang >Priority: Major > > FLINK-10163 added initial view support for the SQL Client. However, for > supporting the [full definition of > views|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/AlterView] > (with schema, comments, etc.) we need to support native support for views in > the Table API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (FLINK-10231) Add a view SQL DDL
[ https://issues.apache.org/jira/browse/FLINK-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668093#comment-16668093 ] winifredtang edited comment on FLINK-10231 at 10/31/18 2:47 AM: Fabian Hueske Hello, could you give more details about it? I try to achieve a view SQL DDL. I wonder if the TableEnvironment.registerTableInternal(name: String, table: AbstractTable) might be deprecated too. Thanks a lot. was (Author: winipanda): [~fhueske] Hello, could you give more details about it? I try to achieve a view SQL DDL. I wonder if it would better for me to use the TableEnvironment.registerTableInternal(name: String, table: AbstractTable) instead of the TableEnvironment.registerTable(name: String, table: Table). Thanks a lot. > Add a view SQL DDL > -- > > Key: FLINK-10231 > URL: https://issues.apache.org/jira/browse/FLINK-10231 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Reporter: Timo Walther >Assignee: winifredtang >Priority: Major > > FLINK-10163 added initial view support for the SQL Client. However, for > supporting the [full definition of > views|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/AlterView] > (with schema, comments, etc.) we need to support native support for views in > the Table API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10231) Add a view SQL DDL
[ https://issues.apache.org/jira/browse/FLINK-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668093#comment-16668093 ] winifredtang commented on FLINK-10231: -- [~fhueske] Hello, could you give more details about it? I try to achieve a view SQL DDL. I wonder if it would better for me to use the TableEnvironment.registerTableInternal(name: String, table: AbstractTable) instead of the TableEnvironment.registerTable(name: String, table: Table). Thanks a lot. > Add a view SQL DDL > -- > > Key: FLINK-10231 > URL: https://issues.apache.org/jira/browse/FLINK-10231 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Reporter: Timo Walther >Priority: Major > > FLINK-10163 added initial view support for the SQL Client. However, for > supporting the [full definition of > views|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/AlterView] > (with schema, comments, etc.) we need to support native support for views in > the Table API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (FLINK-10231) Add a view SQL DDL
[ https://issues.apache.org/jira/browse/FLINK-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] winifredtang reassigned FLINK-10231: Assignee: winifredtang > Add a view SQL DDL > -- > > Key: FLINK-10231 > URL: https://issues.apache.org/jira/browse/FLINK-10231 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Reporter: Timo Walther >Assignee: winifredtang >Priority: Major > > FLINK-10163 added initial view support for the SQL Client. However, for > supporting the [full definition of > views|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/AlterView] > (with schema, comments, etc.) we need to support native support for views in > the Table API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (FLINK-10698) Create CatalogManager class manages all external catalogs and temporary meta objects
[
https://issues.apache.org/jira/browse/FLINK-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
winifredtang reassigned FLINK-10698:
Assignee: winifredtang
> Create CatalogManager class manages all external catalogs and temporary meta
> objects
>
>
> Key: FLINK-10698
> URL: https://issues.apache.org/jira/browse/FLINK-10698
> Project: Flink
> Issue Type: Sub-task
> Components: Table API SQL
>Affects Versions: 1.6.1
>Reporter: Xuefu Zhang
>Assignee: winifredtang
>Priority: Major
>
> Currently {{TableEnvironment}} manages a list of registered external catalogs
> as well as in-memory meta objects, and interacts with Calcite schema. It
> would be cleaner to delegate all those responsibilities to a dedicate class,
> especially when Flink's meta objects are also stored in a catalog.
> {{CatalogManager}} is responsible to manage all meta objects, including
> external catalogs, temporary meta objects, and Calcite schema.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (FLINK-10673) Table API / SQL UIDs not the only one
[
https://issues.apache.org/jira/browse/FLINK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
winifredtang reassigned FLINK-10673:
Assignee: winifredtang
> Table API / SQL UIDs not the only one
> -
>
> Key: FLINK-10673
> URL: https://issues.apache.org/jira/browse/FLINK-10673
> Project: Flink
> Issue Type: Bug
> Components: Table API SQL
>Affects Versions: 1.5.4, 1.6.1
> Environment: flink 1.5.0
>Reporter: Fan weiwen
>Assignee: winifredtang
>Priority: Major
>
> a job have two sql
> source is kafka
> sink is redis or other sink
> Asql
> {code:java}
> //代码占位符
> select
> reqIp as factorContenta,
> count(*) as eCount,
> 60 * 60 as expire
> from
> kafka_source
> where
> uri is not null
> group by
> hop(
> rowtime,
> interval '2' second,
> interval '60' minute
> ),
> reqIp
> {code}
> Bsql
> {code:java}
> //代码占位符
> select
> uid as factorContentb,
> count(*) as eCount,
> 60 * 60 as expire
> from
> kafka_source
> where
> uri is not null
> group by
> hop(
> rowtime,
> interval '2' second,
> interval '60' minute
> ),
> uid
> {code}
> now only start Asql stop Bsql sink have key 656.19.173.34
> then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
> kafka Don't delete)
> start Bsql from savepoint
> you will find sink have key 656.19.173.34 and 6630519 all exist
> Bsql fetch Asql savepoint result
> i think sql uids not the only one
> Who can help me see this problem?
> my test data is
> {code:java}
> //代码占位符
> {
> "reqIp" : "656.19.173.34",
> "rowtime" : 1537950912546,
> "uid" : 6630519,
> "uri" : "/web"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (FLINK-10265) Configure checkpointing behavior for SQL Client
[
https://issues.apache.org/jira/browse/FLINK-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
winifredtang reassigned FLINK-10265:
Assignee: winifredtang (was: vinoyang)
> Configure checkpointing behavior for SQL Client
> ---
>
> Key: FLINK-10265
> URL: https://issues.apache.org/jira/browse/FLINK-10265
> Project: Flink
> Issue Type: New Feature
> Components: SQL Client
>Reporter: Timo Walther
>Assignee: winifredtang
>Priority: Major
>
> The SQL Client environment file should expose checkpointing related
> properties:
> - enable checkpointing
> - checkpointing interval
> - mode
> - timeout
> - etc. see {{org.apache.flink.streaming.api.environment.CheckpointConfig}}
> Per-job selection of state backends and their configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (FLINK-10448) VALUES clause is translated into a separate operator per value
[
https://issues.apache.org/jira/browse/FLINK-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
winifredtang reassigned FLINK-10448:
Assignee: winifredtang (was: vinoyang)
> VALUES clause is translated into a separate operator per value
> --
>
> Key: FLINK-10448
> URL: https://issues.apache.org/jira/browse/FLINK-10448
> Project: Flink
> Issue Type: Bug
> Components: Table API SQL
>Affects Versions: 1.6.1
>Reporter: Timo Walther
>Assignee: winifredtang
>Priority: Major
>
> It seems that a SQL VALUES clause uses one operator per value under certain
> conditions which leads to a complicated job graph. Given that we need to
> compile code for every operator in the open method and have other overhead as
> well, this looks inefficient to me.
> For example, the following query creates and unions 6 operators together:
> {code}
> SELECT *
> FROM (
> VALUES
> (1, 'Bob', CAST(0 AS BIGINT)),
> (22, 'Alice', CAST(0 AS BIGINT)),
> (42, 'Greg', CAST(0 AS BIGINT)),
> (42, 'Greg', CAST(0 AS BIGINT)),
> (42, 'Greg', CAST(0 AS BIGINT)),
> (1, 'Bob', CAST(0 AS BIGINT)))
> AS UserCountTable(user_id, user_name, user_count)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (FLINK-5832) Support for simple hive UDF
[ https://issues.apache.org/jira/browse/FLINK-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] winifredtang reassigned FLINK-5832: --- Assignee: winifredtang (was: vinoyang) > Support for simple hive UDF > --- > > Key: FLINK-5832 > URL: https://issues.apache.org/jira/browse/FLINK-5832 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Reporter: Zhuoluo Yang >Assignee: winifredtang >Priority: Major > Labels: pull-request-available > > The first step of FLINK-5802 is to support simple Hive UDF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (FLINK-5802) Flink SQL calling Hive User-Defined Functions
[ https://issues.apache.org/jira/browse/FLINK-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] winifredtang reassigned FLINK-5802: --- Assignee: winifredtang > Flink SQL calling Hive User-Defined Functions > - > > Key: FLINK-5802 > URL: https://issues.apache.org/jira/browse/FLINK-5802 > Project: Flink > Issue Type: Sub-task > Components: Table API SQL >Reporter: Zhuoluo Yang >Assignee: winifredtang >Priority: Major > Labels: features > > It's important to call hive udf in Flink SQL. A great many udfs were written > in hive since last ten years. > It's really important to reuse the hive udfs. This feature will reduce the > cost of migration and bring more users to flink. > Spark SQL has already supported this function. > https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_spark-guide/content/calling-udfs.html > The Hive UDFs here include both built-in UDFs and customized UDFs. As many > business logic had been written in UDFs, the customized UDFs are more > important than the built-in UDFs. > Generally, there are three kinds of UDFs in Hive: UDF, UDTF and UDAF. > Here is the document of the Spark SQL: > http://spark.apache.org/docs/latest/sql-programming-guide.html#compatibility-with-apache-hive > > Spark code: > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala > https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (FLINK-10562) Relax (or document) table name constraints
[ https://issues.apache.org/jira/browse/FLINK-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] winifredtang reassigned FLINK-10562: Assignee: winifredtang (was: vinoyang) > Relax (or document) table name constraints > -- > > Key: FLINK-10562 > URL: https://issues.apache.org/jira/browse/FLINK-10562 > Project: Flink > Issue Type: Improvement > Components: Table API SQL >Affects Versions: 1.6.1 >Reporter: Flavio Pompermaier >Assignee: winifredtang >Priority: Minor > > At the moment it's not possible to register a table whose name starts with a > number (e.g. 1_test). Moreover this constraint is not reported in the > documentation. > I propose to enable table name escaping somehow in order to enable more > general scenarios like those having spaces in between (e.g. select * from 'my > table' ). > Best, > Flavio > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
