Re: Submitting an app via API

Ok thanks! That's superuseful On 7 Feb 2017 08:51, "Ufuk Celebi" wrote: Hey Flavio! Yes, the REST API call is the same and the environment for batch is RemoteEnvironment. – Ufuk On 6 February 2017 at 19:18:31, Flavio Pompermaier (pomperma...@okkam.it) wrote: > Hi Ufuk, >

Re: Submitting an app via API

Hey Flavio! Yes, the REST API call is the same and the environment for batch is  RemoteEnvironment. – Ufuk On 6 February 2017 at 19:18:31, Flavio Pompermaier (pomperma...@okkam.it) wrote: > Hi Ufuk, > does it work also for batch? > > On Mon, Feb 6, 2017 at 5:06 PM, Ufuk Celebi wrote: > > >

Re: Table API: java.sql.DateTime is not supported;

Hi, you can also use the CsvTableSource and read the DateTime fields as String. This will directly give you a table. You can implement a user-defined scalar function [1] to parse the String into a DateTime type. The benefit is that you stay in the Table API / SQL and don't have to deal with the

Re: Table API: java.sql.DateTime is not supported;

Hello Timo, Thanks for the clarification. This means that I *cannot use CsvTableSource*, as I have, in the example. Instead, I should: * Write custom Scalar function to convert STRINGs to other datatypes as required * Read the file as CsvInput, with all fields as STRINGs * Apply the

Re: Javadoc

See this thread: http://search-hadoop.com/m/Flink/VkLeQV4Igl2GAJjs?subj=JavaDoc+404 On Mon, Feb 6, 2017 at 2:08 PM, Colin Breame wrote: > Hello Ted, > > Thanks for the link. That looks like it but it renders rather badly > in my browser. > > There are two 404s on the

Re: Javadoc

Hello Ted, Thanks for the link. That looks like it but it renders rather badly in my browser. There are two 404s on the page which is likely causing the issue: https://ci.apache.org/projects/flink/flink-docs-master/api/java/stylesheet.css

Re: Javadoc

Have you looked under https://ci.apache.org/projects/flink/flink -docs-master/api/java ? e.g. suppose you look for API for ByteArrayOutputStreamWithPos:

Dealing with latency in Sink

What is the best way to dynamically adapt and tune down number of tasks created to write/read to a sink when sink slows down or the latency to sink increases? I am looking at the sink interface but don't see a way to influence flink to reduce the number of tasks or throttle the volume down to the

Javadoc

Hello, I'm looking for the javadoc for the APIs. Can someone tell me if this is hosted on the internet anywhere? Thanks Colin

Re: Submitting an app via API

Hi Ufuk, does it work also for batch? On Mon, Feb 6, 2017 at 5:06 PM, Ufuk Celebi wrote: > You can use RemoteStreamEnvironment or the REST APIs > (https://ci.apache.org/projects/flink/flink-docs- > release-1.3/monitoring/rest_api.html#submitting-programs). > > On Sun, Feb 5,

Re: Bug in Table api CsvTableSink

The error was just caused by a cut-and-paste on the shell instead of opening directly the file so bug solved :) On Mon, Feb 6, 2017 at 12:20 PM, Fabian Hueske wrote: > Hi Flavio, > > I checked the records on the current master and the CsvTableSink seemed to > work fine. >

Re: Submitting an app via API

Hi, Thanks a lot. On Mon, Feb 6, 2017 at 9:06 PM, Ufuk Celebi wrote: > You can use RemoteStreamEnvironment or the REST APIs > (https://ci.apache.org/projects/flink/flink-docs- > release-1.3/monitoring/rest_api.html#submitting-programs). > > On Sun, Feb 5, 2017 at 4:43 PM,

Re: Submitting an app via API

You can use RemoteStreamEnvironment or the REST APIs (https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html#submitting-programs). On Sun, Feb 5, 2017 at 4:43 PM, Luqman Ghani wrote: > Hi, > > On quickstart page of Flink docs, it suggests starting

Re: Many streaming jobs vs one

As Jonas said, for job upgrades having a single job that "multiplexes" multiple jobs means that all jobs will be offline at the same time. If all jobs share a single Flink cluster, it should be fine to use multiple jobs that share the resources. A down side of this will be that managing multiple

Re: To get Schema for jdbc database in Flink

I'm not sure how well this works for the streaming API. Looping in Chesnay, who worked on this. On Mon, Feb 6, 2017 at 11:09 AM, Punit Tandel wrote: > Hi , > > I was looking into flink streaming api and trying to implement the solution > for reading the data from jdbc

Re: Parallelism and max-parallelism

Thanks! Best regards, Dmitry On Mon, Feb 6, 2017 at 3:47 PM, Ufuk Celebi wrote: > Could you have a look at these PRs please: > > https://github.com/apache/flink/pull/3259 > > https://github.com/apache/flink/pull/3258 > > If you find that anything is missing, feel free to

Re: Parallelism and max-parallelism

Could you have a look at these PRs please: https://github.com/apache/flink/pull/3259 https://github.com/apache/flink/pull/3258 If you find that anything is missing, feel free to report it here. The PRs will be merged later today. On Mon, Feb 6, 2017 at 4:41 PM, Dmitry Golubets

Re: Chicago Hands on Apache Flink Workshop

Sounds very interesting. Thanks for sharing this and wish you all a great time. On Thu, Feb 2, 2017 at 2:56 AM, Trevor Grant wrote: > Any one who is going to be in or around Chicago 2/21: > > Joe Olson is putting on a workshop for our local Flink meeup- drop by if you >

Re: JavaDoc 404

Thanks for reporting this. I think Robert (cc'd) is working in fixing this, correct? On Sat, Feb 4, 2017 at 12:12 PM, Yassine MARZOUGUI wrote: > Hi, > > The JavaDoc link of BucketingSink in this page[1] yields to a 404 error. I > couldn't find the correct url. > The

Parallelism and max-parallelism

Hi guys, I would appreciate if someone could explain to me what's the difference between those two. The current description refers to "dynamic scaling", and yet I can't find anything about it in Flink's docs. Best regards, Dmitry

stream clustering in flink

Hi, we want to cluster a stream of Tweets using Flink. Every incoming tweet is compared to the last 100 tweets. After this comparison, a cluster ID is assigned to the tweet. We try to find out the best approach how to solve this: 1. Using a stream window of the last tweets seems to be

Re: Bug in Table api CsvTableSink

Hi Flavio, I checked the records on the current master and the CsvTableSink seemed to work fine. However, I had some issues when converting the DataSet[Row] into a table. You have to make sure that the TypeInformation for DataSet[Row] is RowTypeInfo and not GenericType[Row]. Can you check the

Re: Compiler error while using 'CsvTableSource'

Thanks, Timo. Do I need to add anything to the ticket? Please let me know. I will do the needful. -- N On Mon, Feb 6, 2017 at 2:25 PM, Timo Walther [via Apache Flink User Mailing List archive.] wrote: > I created an issue to make this a bit more

RE: 1.2 release date

Hi, Could you update List of contributors after that? ☺ Anton Solovev Software Engineer Office: +7 846 200 09 70 x 55621 Email: anton_solo...@epam.com Samara, Russia (GMT+4) epam.com

Fwd: To get Schema for jdbc database in Flink

Hi , I was looking into flink streaming api and trying to implement the solution for reading the data from jdbc database and writing them to jdbc databse again. At the moment i can see the datastream is returning Row from the database. dataStream.getType().getGenericParameters() retuning an

Re: How about Discourse (https://www.discourse.org/) for this mailing list

Instead of Nabble I will use PonyMail now :) Thanks. Didn't know it existed. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-about-Discourse-https-www-discourse-org-for-this-mailing-list-tp11448p11457.html Sent from the Apache Flink User

Re: How about Discourse (https://www.discourse.org/) for this mailing list

Hi Jonas, thanks for the suggestion. Critical infrastructure (repository, dev mailing list) of Apache projects must be hosted on Apache infrastructure. For example, Github is just mirroring the ASF git repositories. We integrated the mailing lists with Nabble (user [1], dev [2]) and there is

Re: Table API: java.sql.DateTime is not supported;

Hi, java.sql.Timestamps have to have a format like " -mm-dd hh:mm:ss.[fff...]". In your case you need to parse this as a String and write your own scalar function for parsing. Regards, Timo Am 04/02/17 um 17:46 schrieb nsengupta: "4/1/2014 0:11:00",40.769,-73.9549,"B02512"

Re: Improving Flink Performance

Hi Jonas, thanks for reporting back! Glad you solve the issue. Cheers, Fabian 2017-02-05 22:07 GMT+01:00 Jonas : > Using a profiler I found out that the main performance problem (80%) was > spent in a domain specific data structure. After implementing it with a > more >

Re: Compiler error while using 'CsvTableSource'

I created an issue to make this a bit more user-friendly in the future. https://issues.apache.org/jira/browse/FLINK-5714 Timo Am 05/02/17 um 06:08 schrieb nsengupta: Thanks, Till, for taking time to share your understanding. -- N On Sun, Feb 5, 2017 at 12:49 AM, Till Rohrmann [via Apache

Re: allowed lateness on windowed join?

Hi, Union is a super cheap operator in Flink. It does not scan the records, but just merges the streams. So the effort is very low. The built-in join operator works in the same way but does not expose allowed lateness. Cheers, Fabian

Re: 1.2 release date

Hi Tarandeep, afaik, Flink 1.2 will be released today. Cheers, Till On Sun, Feb 5, 2017 at 10:00 PM, Tarandeep Singh wrote: > Hi, > > Looking forward to 1.2 version of Flink (lots of exciting features have > been added). > Has the date finalized yet? > > Thanks, >