Re: Submit Flink Jobs to YARN running on AWS

If you use open vpn for accessing aws then you can use private IP of ec2 machine from your laptop. Thanks Ashutosh On Tue, Jun 7, 2016 at 11:00 PM, Shannon Carey wrote: > We're also starting to look at automating job deployment/start to Flink > running on EMR. There are a

Re: Facing error while running Hands-in training example “writing to Apache Kafka”

How are you packaging and deploying your jar ? I have tested with flink and kafka .9 . It works fine for me . Thanks Ashutosh On Wed, Jun 1, 2016 at 3:37 PM, ahmad Sa P wrote: > I did test it with Kafka 0.9.0.1, still the problem exists! > > On Wed, Jun 1, 2016 at 11:50 AM,

Re: Apache Beam and Flink

oduction to Apache Beam ( blog) >http://www.talend.com/blog/2016/05/02/introduction-to-apache-beam > > > I hope this helps. > > Thanks > > Slim Baltagi > > On May 26, 2016, at 2:20 AM, Ashutosh Kumar <kmr.ashutos...@gmail.com> > wrote: > > How does apache beam fits with flink ? Is it an alternative for flink or > complementary to it ? > > Thanks > Ashutosh > > >

Apache Beam and Flink

How does apache beam fits with flink ? Is it an alternative for flink or complementary to it ? Thanks Ashutosh

Re: join performance

Time unit can be in seconds as well. Is there specific need to get bursts hourly? On Fri, Apr 29, 2016 at 11:48 AM, Henry Cai wrote: > For the below standard stream/stream join, does flink store the results of > stream 1 and stream 2 into state store for the current hour and

Re: Kafka Test Error

I am using flink 1.0.0 with kafka 0.9 . I works fine for me. I use following dependency. org.apache.flink flink-connector-kafka-0.9_2.10 1.0.0 provided Thanks Ashutosh On Fri, Apr 1, 2016 at 10:46 AM, Zhun Shen

Re: Access to S3 from YARN on EC2

al problem rather than the combination > of the different libraries that are using there is a version compatibility > mismatch, so you will have to go back and check if there is any version > mismatch. Are you using scala or this is a java project ? > > On Mon, Mar 21, 2016 at 10:26 AM, Ashutosh

Re: Access to S3 from YARN on EC2

> Hi Ashutosh, > > I believe you need to add the hadoop-aws jar to your project. > > http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws/2.6.0 > > Thanks, > Tim > > On Sun, Mar 20, 2016 at 9:39 AM, Ashutosh Kumar < > ashutosh.disc...@gmail.com>

Re: Access to S3 from YARN on EC2

Do I need to add some jars in lib ? Thanks Ashutosh On Sun, Mar 20, 2016 at 4:30 PM, Ashutosh Kumar <ashutosh.disc...@gmail.com> wrote: > It is not there. > > Thanks > Ashutosh > > On Sun, Mar 20, 2016 at 2:58 PM, Robert Metzger <rmetz...@apache.org> > wr

Access to S3 from YARN on EC2

I have setup a 3 node YARN based cluster on EC2. I am running flink in cluster mode. I added these lines in core-site.xml fs.s3n.awsAccessKeyId accesskey fs.s3n.awsSecretAccessKey secret key fs.s3n.impl

Re: DataSet -> DataStream

As data is already collected, why do you want add one more layer of Kafka. Instead you can start processing your data. Thanks Ashutosh On Mar 11, 2016 4:19 AM, "Prez Cannady" wrote: > > I’d like to pour some data I’ve collected into a DataSet via JDBC into a > Kafka

Running Flink 1.0.0 on YARN

I have a yarn setup with 1 master and 2 slaves. When I run yarn session with bin/yarn-session.sh -n 2 -jm 1024 -tm 1024 and submit job with bin/flink run examples/batch/WordCount.jar , the job succeeds . It shows status on yarn UI http://x.x.x.x:8088/cluster . However it does not show

Looking for co founder /partner in Bangalore

I am planning to build a analytics platform based on Flink, Kafka , Camel, Zeppelin , Drill and Cassandra . I am looking for co founder/partner in Bangalore . I am sorry if this is not a right forum to express this. Thanks Ashutosh

Merging of streams

I found that merge method for datastream does not exist in latest version . What is the equivalent for it ? Shall I use union or join ? Thanks Ashutosh

How to prepare data for K means clustering

I saw example code for K means clustering . It takes input data points as pair of double values (1.2 2.3\n5.3 7.2\.). My question is how do I convert my business data to this format. I have customer data which has columns like house hold income , education and several others. I want to do

Re: Machine Learning on Apache Fink

. Thanks in advance. Ashutosh On Sat, Jan 9, 2016 at 3:32 PM, Ashutosh Kumar <ashutosh.disc...@gmail.com> wrote: > I am looking for some study material and examples on machine learning . > Are classification , recommendation and clustering libraries available ? > What is the tim