[jira] [Created] (FLINK-20768) Support routing field for Elasticsearch connector

wangsan created FLINK-20768: --- Summary: Support routing field for Elasticsearch connector Key: FLINK-20768 URL: https://issues.apache.org/jira/browse/FLINK-20768 Project: Flink Issue Type

[jira] [Created] (FLINK-20768) Support routing field for Elasticsearch connector

wangsan created FLINK-20768: --- Summary: Support routing field for Elasticsearch connector Key: FLINK-20768 URL: https://issues.apache.org/jira/browse/FLINK-20768 Project: Flink Issue Type

[jira] [Created] (FLINK-20624) Refactor StreamExecJoinRule、StreamExecIntervalJoinRule and StreamExecTemporalJoinRule

wangsan created FLINK-20624: --- Summary: Refactor StreamExecJoinRule、StreamExecIntervalJoinRule and StreamExecTemporalJoinRule Key: FLINK-20624 URL: https://issues.apache.org/jira/browse/FLINK-20624 Project

[jira] [Created] (FLINK-20624) Refactor StreamExecJoinRule、StreamExecIntervalJoinRule and StreamExecTemporalJoinRule

wangsan created FLINK-20624: --- Summary: Refactor StreamExecJoinRule、StreamExecIntervalJoinRule and StreamExecTemporalJoinRule Key: FLINK-20624 URL: https://issues.apache.org/jira/browse/FLINK-20624 Project

[jira] [Commented] (FLINK-18596) Derive format schema from table schema may get error result

[ https://issues.apache.org/jira/browse/FLINK-18596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157334#comment-17157334 ] wangsan commented on FLINK-18596: - I would like to fix this issue > Derive format schema from ta

[jira] [Updated] (FLINK-18596) Derive format schema from table schema may get error result

[ https://issues.apache.org/jira/browse/FLINK-18596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan updated FLINK-18596: Description: If rowtime attribute references a regular field, derive format schema from table schema may

[jira] [Created] (FLINK-18596) Derive format schema from table schema may get error result

wangsan created FLINK-18596: --- Summary: Derive format schema from table schema may get error result Key: FLINK-18596 URL: https://issues.apache.org/jira/browse/FLINK-18596 Project: Flink Issue

[jira] [Created] (FLINK-18596) Derive format schema from table schema may get error result

wangsan created FLINK-18596: --- Summary: Derive format schema from table schema may get error result Key: FLINK-18596 URL: https://issues.apache.org/jira/browse/FLINK-18596 Project: Flink Issue

[jira] [Commented] (FLINK-16710) Log Upload blocks Main Thread in TaskExecutor

[ https://issues.apache.org/jira/browse/FLINK-16710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070661#comment-17070661 ] wangsan commented on FLINK-16710: - Hi [~gjy], I can help with this issue if no one is working

[jira] [Created] (FLINK-15548) Make KeyedCoProcessOperatorWithWatermarkDelay extends KeyedCoProcessOperator instead of LegacyKeyedCoProcessOperator

wangsan created FLINK-15548: --- Summary: Make KeyedCoProcessOperatorWithWatermarkDelay extends KeyedCoProcessOperator instead of LegacyKeyedCoProcessOperator Key: FLINK-15548 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-15548) Make KeyedCoProcessOperatorWithWatermarkDelay extends KeyedCoProcessOperator instead of LegacyKeyedCoProcessOperator

wangsan created FLINK-15548: --- Summary: Make KeyedCoProcessOperatorWithWatermarkDelay extends KeyedCoProcessOperator instead of LegacyKeyedCoProcessOperator Key: FLINK-15548 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-13823) Incorrect debug log in CompileUtils

wangsan created FLINK-13823: --- Summary: Incorrect debug log in CompileUtils Key: FLINK-13823 URL: https://issues.apache.org/jira/browse/FLINK-13823 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-13823) Incorrect debug log in CompileUtils

wangsan created FLINK-13823: --- Summary: Incorrect debug log in CompileUtils Key: FLINK-13823 URL: https://issues.apache.org/jira/browse/FLINK-13823 Project: Flink Issue Type: Bug

zk connect loss

13:10:06.119 [main] ERROR - Failed to resolve default logging config file: config/java.util.logging.properties 13:10:37.097 [main] ERROR o.a.i.s.d.z.internal.ZookeeperClient - Operation failed with unexpected error, connection lost: org.apache.zookeeper.KeeperException$ConnectionLossException:

[jira] [Closed] (FLINK-11687) Remove useless code in StreamingJobGraphGenerator

[ https://issues.apache.org/jira/browse/FLINK-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan closed FLINK-11687. --- Resolution: Fixed > Remove useless code in StreamingJobGraphGenera

[jira] [Created] (FLINK-11687) Remove useless code in StreamingJobGraphGenerator

wangsan created FLINK-11687: --- Summary: Remove useless code in StreamingJobGraphGenerator Key: FLINK-11687 URL: https://issues.apache.org/jira/browse/FLINK-11687 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-11687) Remove useless code in StreamingJobGraphGenerator

wangsan created FLINK-11687: --- Summary: Remove useless code in StreamingJobGraphGenerator Key: FLINK-11687 URL: https://issues.apache.org/jira/browse/FLINK-11687 Project: Flink Issue Type: Task

Re: ZookeeperDiscovery block when communication error

Thank you! When I use zk discovery.I find many nodes in zookeeper path /jd/ . In my opinion.When new node join,Then a new /jd/ child node will be created,When the node join the cluster success,the /jd/ path will be removed.But in my cluster,That will be many remnant /jd/ nodes. -- Sent from:

Re: Why GridDiscoveryManager onSegmentation use StopNodeFailureHandler?

Thanks! The IgniteConfiguration.segmentationPolicy RESTART_JVM would be a little misleading. Exit java with some exit code ,The java application will ignore the exit code. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Failed to read data from remote connection

When check connections,Many nio socker will be create(one socker per node) ,Then direct memory will grow up with the node count? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: NPE when start

I will try to reproduce the excepiton. I am worried about it will happen as the cache remote listener rmtFilter @IgniteAsyncSupported public UUID remoteListen(@Nullable IgniteBiPredicate locLsnr, @Nullable IgnitePredicate rmtFilter, @Nullable int... types)

Re: ignite zk: Possible starvation in striped pool

Thank you！ I see that this is communication spi not discovery spi. But in other nodes there are many zk session timeout message or zk reconnect fail message. And the starvation message only print in the node which have the zk server(not cluster,just three zk node in one machine) in the same

ignite zk: Possible starvation in striped pool

10:38:31.577 [grid-timeout-worker-#55%DAEMON-NODE-10-153-106-16-8991%] WARN o.a.ignite.internal.util.typedef.G - >>> Possible starvation in striped pool. Thread name: sys-stripe-9-#10%DAEMON-NODE-10-153-106-16-8991% Queue: [] Deadlock: false Completed: 17156 Thread

NPE when start

When a client node start with zk discovery and persistence enabled,Some null point exceptions will be throw (when the node start on a new machine ) The exception traces as follows: 12:26:03.288 [zk-172_22_29_108_SEARCH_NODE_8100-EventThread] ERROR o.a.i.i.p.c.GridContinuousProcessor - Failed to

RE: Failed to read data from remote connection

Yeath, set a larger MaxDirectMemorySize . But, I am afraid of when nodes size be more larger.The directmemory will be larger with node sizes. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to read data from remote connection

Now the cluster have 100+ nodes, when 'Start check connection process' happens, Some node will throw oom with Direct buffer memory (java nio). When check connections,Many nio socker will be create ,Then oom happens? How to fix the oom except larger xmx? Thanks. -- Sent from:

Re: Failed to read data from remote connection

Do you shut down C++ node properly prior killing the process? Yeath, c++ node was killed by kill -9 .not sighup. It is a wrong ops,And I will use kill ops. Does this exceptions impacts cluster's functionality anyhow? I am not sure about the exceptions. My cluster will crash with oom

Re: Questions about UDTF in flink SQL

Hi Rong, Yes, what Jark described is exactly whet I need. Currently we have a work around for this problem, by using a UDF whose result type is a Map. I will took a look on your proposals and PR. Thanks for your help and suggestions. Best, Wangsan > On Dec 1, 2018, at 7:30 AM, Rong R

Questions about UDTF in flink SQL

, may be we should consider this feature in future? Best, wangsan

Questions about UDTF in flink SQL

, may be we should consider this feature in future? Best, wangsan

Re: Failed to read data from remote connection

As I restart cpp client many times concurrently ，may be zkcluster(ignite) has some node path has been closed. >From cpp client logs , I can see zkdiscovery watch 44 first,but the node has been closed

Re: Failed to read data from remote connection

I have a cluster with zookeeper discovery, Eg java server node s1,java client node jc1 and cpp client node cpp1 Sometimes when cpp1 restart ,s1 and jc1 will throw this exception many times Failed to process selector key [ses=GridSelectorNioSessionImpl And cpp1 with have many messages

Re: Why GridDiscoveryManager onSegmentation use StopNodeFailureHandler?

Can I use LifecycleEventType.AFTER_NODE_STOP to stop jvm? if segment event happens? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Why GridDiscoveryManager onSegmentation use StopNodeFailureHandler?

Can I config the FailureHandler when segment error happen? The IgniteConfiguration FailureHandler only used on other exception(error) happens? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Failed to read data from remote connection

When client （c++ node） restart mulit times, The server and other client will throw this excption ERROR o.a.i.s.c.tcp.TcpCommunicationSpi - Failed to read data from remote connection (will wait for 2000ms). org.apache.ignite.IgniteCheckedException: Failed to select events on selector.

Why GridDiscoveryManager onSegmentation use StopNodeFailureHandler?

Why GridDiscoveryManager onSegmentation use StopNodeFailureHandler? But not the StopNodeOrHaltFailureHandler ! In my case,when net problem happen, I wan't the jvm close? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ZookeeperDiscovery block when communication error

I have a server node in zone A ,then I start a client from zone B, Now access between A,B was controlled by firewall,The acl is B can access A,but A can not access B. So when client in zone B join the cluster,the communication will fail caused by firewall. But when client in zone B closed, The

ZookeeperDiscovery Waiting for local join event

I use ZookeeperDiscovery,but sometimes the cluster will be wried,client can not join the cluster,and the zookeeper node /jd will have many children nodes. When I restart a server node,the exception is: 19:14:06.675 [zk-192_0_0_1_DAEMON_NODE_8991-EventThread] INFO

persistence when disk full

I have two nodes (server and persistence true) in different machines.That node n1 in machine m1,node n2 in machine m2. When m1 disk full,Two node n1,n1 will be failed.Not only node n1? How can I only make node n1 failed? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[jira] [Assigned] (FLINK-10770) Some generated functions are not opened properly.

[ https://issues.apache.org/jira/browse/FLINK-10770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan reassigned FLINK-10770: --- Assignee: wangsan > Some generated functions are not opened prope

[jira] [Created] (FLINK-10770) Some generated functions are not opened properly.

wangsan created FLINK-10770: --- Summary: Some generated functions are not opened properly. Key: FLINK-10770 URL: https://issues.apache.org/jira/browse/FLINK-10770 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-10770) Some generated functions are not opened properly.

wangsan created FLINK-10770: --- Summary: Some generated functions are not opened properly. Key: FLINK-10770 URL: https://issues.apache.org/jira/browse/FLINK-10770 Project: Flink Issue Type: Bug

Re: npe: Failed to reinitialize local partition

Hi: I run the test in v2.5 (with npe patch) , The in_memory cache backups will be 0 when reconnect. When I use v2.7 ,It's ok! When v2.7 release will be released? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Reverse the order of fields in Flink SQL

merged. We have used that in our internal Flink version, and for now it works well. May be you can take a look at it. Best, wangsan > On Oct 24, 2018, at 9:31 AM, yinhua.dai wrote: > > Hi Timo, > > I write simple testing code for the issue, please checkout > https://gi

Re: npe: Failed to reinitialize local partition

IgniteTwoRegionsRebuildIndexTest.java The file is the test case. I use getOrCreateCache with name. But the cache template I have beed defined before.When reconnect ,It looks like the cache

Re: npe: Failed to reinitialize local partition

Tks! I applyed the patch from the issue.And the NPE was fixed. But another problems happen: As the issue says: Steps to reproduce: Ignite server region1 (with persistence) region2 (without persistence) client cache1a from region1 – with custom affinity cache2a fom region2 – with custom affinity

Re: npe: Failed to reinitialize local partition

Thanks! The ticket description is especially like my scene. I will apply the patch! I have a little confused about the persistence file. If I have a default region without persistence. and cache2a use default region. the db file(directory) won't be created. But if I have region1 (with

npe: Failed to reinitialize local partition

when a c++ node with cache(c++ defined) join the java server node(persistence true). it will throw this exception,When I clear server persistence home then restart.the exception won't throw Why? o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture - Failed to reinitialize local partitions

zookeeper jd(joindata) never delete

ignite version 2.5.0 discovery spi: ZookeeperDiscoverySpi when I start a client node c. c can join the cluster. but c just join the cluster never call any cache method. the jd path children willl never deleteed: -- Sent

[jira] [Updated] (FLINK-10553) Unified sink and source table name in SQL statement

[ https://issues.apache.org/jira/browse/FLINK-10553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan updated FLINK-10553: Summary: Unified sink and source table name in SQL statement (was: Unified table sink and source name

[jira] [Created] (FLINK-10553) Unified table sink and source name in SQL statement

wangsan created FLINK-10553: --- Summary: Unified table sink and source name in SQL statement Key: FLINK-10553 URL: https://issues.apache.org/jira/browse/FLINK-10553 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-10553) Unified table sink and source name in SQL statement

wangsan created FLINK-10553: --- Summary: Unified table sink and source name in SQL statement Key: FLINK-10553 URL: https://issues.apache.org/jira/browse/FLINK-10553 Project: Flink Issue Type: Bug

ignite.events().remoteListen unmarshal0 throw class not found

A server and a client, then client call: line_1: CacheNodeFilter rmtFilter = new CacheNodeFilter(serviceType, uuid); line_2: ignite.events().remoteListen(listenerDispatcher, rmtFilter, EventType.EVT_CACHE_OBJECT_EXPIRED); I am quite sure all the server and client have the class

Re: ignte cluster hang with GridCachePartitionExchangeManager

Yes , It was blocked when do cache operation in discovery event listeners when node left events arrival concurrently. I just do cache operation in another thread. Then the listener will not be blocked. The original cause may be that discovery event processor hold the server latch.when do cache

[jira] [Created] (FLINK-10290) Conversion error in StreamScan and BatchScan

wangsan created FLINK-10290: --- Summary: Conversion error in StreamScan and BatchScan Key: FLINK-10290 URL: https://issues.apache.org/jira/browse/FLINK-10290 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-10290) Conversion error in StreamScan and BatchScan

wangsan created FLINK-10290: --- Summary: Conversion error in StreamScan and BatchScan Key: FLINK-10290 URL: https://issues.apache.org/jira/browse/FLINK-10290 Project: Flink Issue Type: Bug

[jira] [Assigned] (FLINK-10290) Conversion error in StreamScan and BatchScan

[ https://issues.apache.org/jira/browse/FLINK-10290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan reassigned FLINK-10290: --- Assignee: wangsan > Conversion error in StreamScan and BatchS

Re: ignite cluster management

1. Note that you will have problem writing to caches when topology changes. Why not just query ignite.cluster(), expose node traits as node.attributes()? >> Yes, I use attributes for node join. But I want to know the stopped >> nodes(when node left) . Then I can do some operations such as

Re: ignte cluster hang with GridCachePartitionExchangeManager

Thanks! Can I do cache operations(update cache item) in another thread from discovery event listeners? And the operation(update cache item) will execute concurrently or execute before partition map exchange? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignte cluster hang with GridCachePartitionExchangeManager

; import org.apache.ignite.logger.slf4j.Slf4jLogger; import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi; import org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder; import org.junit.Test; import com.google.common.collect.Lists; /** * @author wangsan * @date

ignite multi thread stop bug

In my test case:see the attachment.I can reproduce the bug 1. testStartServer 2. testMultiClient 3. close testMultiClient 4. then testStartServer will Failed to wait for partition map exchang I need update cache in DiscoveryEvent. how to fix this?

ignite cluster management

I am using ignite to manage my cluster.when node join I save an item in my NodeCache, when node left I delete an item in my NodeCache with ignite event listener. Use custom NodeCache but not ignite cluster topology,The reason is I want to keep node status persistence when node left for some

Re: ignte cluster hang with GridCachePartitionExchangeManager

About question 2, debug level like this： I start a node，then b,c,d,e,f nodes in mulitithread. then close them all. in the debugs log, A server latch created with participantsSize=5 but only one countdown .then latch will be hang. simple logs is: >>> ++ > >>> Topology snapshot. >

ignte cluster hang with GridCachePartitionExchangeManager

Now my cluster topology is Node a,b,c,d all with persistence enable and peerclassloader false. b c d have different class(cache b) from a. 1.When any node crash with oom(memory or stack) .all nodes hang with " - Still waiting for initial partition map exchange " 2.When a start first, b,c,d start

Re: Side effect of DataStreamRel#translateToPlan

on same RelNode several times, the same operators are added in execution environment more than once, but actually we need that for only one time. Correct me if I misunderstood that. I will open an issue late this day, if this is indeed a problem. Best, wangsan > On Aug 21, 2018, at 10

Side effect of DataStreamRel#translateToPlan

environment repeatedly. Should we eliminate the side effect of DataStreamRel#translateToPlan ? Best, Wangsan appendix tenv.registerTableSource("test_source", sourceTable) val t = tenv.sqlQuery("SELECT * from test_source") println(tenv.explain(t)) pr

Side effect of DataStreamRel#translateToPlan

environment repeatedly. Should we eliminate the side effect of DataStreamRel#translateToPlan ? Best, Wangsan appendix tenv.registerTableSource("test_source", sourceTable) val t = tenv.sqlQuery("SELECT * from test_source") println(tenv.explain(t)) pr

how ignite c++ node set baselinetopology

I have a java node with persistence. Then when c++ node join.how to active and set blt with api? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: two data region with two nodes

I just want A module: node a1 a2 make cache cache_a be persistence and balance in A module only with persistence region p_region B module node b1,b2 with cache cache_b be non_persistence and distribute in B module.B will access cache_a and have same region p_region for persistence even cache_b

Re: two data region with two nodes

tks, I will try the oldest node select method to ensure only one node to process the event message. As mentioned earlier, in my project ,I have several modules。eg： module A , daemon node with cache nodecache,machinecache .all the cache persistent。 module B, search node with cache

Re: two data region with two nodes

Thank you for your answer, Maybe I use region config the wrong way. There are two apps(more apps with different roles) with different ignite configs, first App :set default region with persistence enable :set cache a,with nodefilter in first apps,default region second app :set

Re: Confusions About JDBCOutputFormat

implemented this at https://github.com/apache/flink/pull/6301 <https://github.com/apache/flink/pull/6301>, It would be kind of you to review this. Best, wangsan > On Jul 11, 2018, at 2:25 PM, Hequn Cheng wrote: > > Hi wangsan, > > What I mean is establishing a connection

Re: Confusions About JDBCOutputFormat

implemented this at https://github.com/apache/flink/pull/6301 <https://github.com/apache/flink/pull/6301>, It would be kind of you to review this. Best, wangsan > On Jul 11, 2018, at 2:25 PM, Hequn Cheng wrote: > > Hi wangsan, > > What I mean is establishing a connection

[jira] [Commented] (FLINK-9794) JDBCOutputFormat does not consider idle connection and multithreads synchronization

[ https://issues.apache.org/jira/browse/FLINK-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539468#comment-16539468 ] wangsan commented on FLINK-9794: Hi [~yanghua], I've opened a pull request for this issue, it would

[jira] [Created] (FLINK-9794) JDBCOutputFormat does not consider idle connection and multithreads synchronization

wangsan created FLINK-9794: -- Summary: JDBCOutputFormat does not consider idle connection and multithreads synchronization Key: FLINK-9794 URL: https://issues.apache.org/jira/browse/FLINK-9794 Project: Flink

[jira] [Created] (FLINK-9794) JDBCOutputFormat does not consider idle connection and multithreads synchronization

wangsan created FLINK-9794: -- Summary: JDBCOutputFormat does not consider idle connection and multithreads synchronization Key: FLINK-9794 URL: https://issues.apache.org/jira/browse/FLINK-9794 Project: Flink

Re: Confusions About JDBCOutputFormat

before the connection is closed? May be we could use a Timer to test the connection periodically and keep it alive. What do you think? I will open a jira and try to work on that issue. Best, wangsan > On Jul 10, 2018, at 8:38 PM, Hequn Cheng wrote: > > Hi wangsan, > > I

Re: Confusions About JDBCOutputFormat

before the connection is closed? May be we could use a Timer to test the connection periodically and keep it alive. What do you think? I will open a jira and try to work on that issue. Best, wangsan > On Jul 10, 2018, at 8:38 PM, Hequn Cheng wrote: > > Hi wangsan, > > I

[jira] [Resolved] (SPARK-22503) Using current processing time to generate windows in streaming processing

[ https://issues.apache.org/jira/browse/SPARK-22503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan resolved SPARK-22503. - Resolution: Not A Problem > Using current processing time to generate windows in streaming process

Confusions About JDBCOutputFormat

am wrong. —— wangsan

Confusions About JDBCOutputFormat

am wrong. —— wangsan

Re: ignite c++ support service grid?

can i use c++ disturbted closure on java ingite node -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite c++ support service grid?

>From the doc https://apacheignite-cpp.readme.io/docs , I did not find any service grid description. I just want to make sure ignite c++ don't support service grid(ignite service rpc) now, but maybe later. I have some c++ nodes do deeplearning which

client service fail when reconnect,all server nodes stopped then restart

i see some bugs such as https://issues.apache.org/jira/browse/IGNITE-2766 https://github.com/apache/ignite/pull/2627 when i apply the patch, cache can be reused when client reconnect. but

Unsubscribe

- To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Question about Timestamp in Flink SQL

value I should add the offset as opposite to what internalToTimestamp did. But the Processing time attribute can not keep consistent. Am I understanding that correctly? Best, wangsan > On 29 Nov 2017, at 4:43 PM, Timo Walther <twal...@apache.org> wrote: > > Hi Wangsan,

Re: Question about Timestamp in Flink SQL

event time with unix timestamp 0, then I got the Timestamp(-2880). I am confused why `internalToTimestamp` need to subtract the offset? Best, wangsan > On 28 Nov 2017, at 11:32 PM, Xingcan Cui <xingc...@gmail.com> wrote: > > Hi wangsan, > > in Flink, the ProcessingTime

Re: Hive integration in table API and SQL

tions, like join, on a streaming table and a batch table ? Best, wangsan > On 20 Nov 2017, at 9:15 PM, Timo Walther <twal...@apache.org> wrote: > > Timo

Hive integration in table API and SQL

attributes, so how can I implements this functionality. Do I need to implement my own dataset connectors to load data from external tables using JDBC and register the dataset as table, or should I provide an external catalog? Thanks, wangsan

[jira] [Updated] (SPARK-22503) Using current processing time to generate windows in streaming processing

[ https://issues.apache.org/jira/browse/SPARK-22503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangsan updated SPARK-22503: Affects Version/s: 2.2.0 > Using current processing time to generate windows in streaming process

[jira] [Created] (SPARK-22503) Using current processing time to generate windows in streaming processing

wangsan created SPARK-22503: --- Summary: Using current processing time to generate windows in streaming processing Key: SPARK-22503 URL: https://issues.apache.org/jira/browse/SPARK-22503 Project: Spark

Generate windows on processing time in Spark Structured Streaming

Hi all, How can I use current processing time to generate windows in streaming processing? window function's Scala doc says "For a streaming query, you may use the function current_timestamp to generate windows on processing time.” But when using current_timestamp as column in window

Re:Re: Exception in BucketingSink when cancelling Flink job

()' method is finished. Best, wangsan 在2017年09月29 01时52分, "Stephan Ewen"<step...@data-artisans.com>写道: Hi! Calling 'interrupt()' makes only sense before 'join()', because 'join()' blocks until the respective thread is finished. The 'interrupt()' call happens to cancel the task ou

Question about job canceling in Flink

Can anyone help? Best, wangsan

Exception in BucketingSink when cancelling Flink job

losed when cacelling a job? Best, wangsan