son
Sent: 07 January 2019 12:32
To: yeikel valdes ; Shashikant Bangera
Cc: user@spark.apache.org
Subject: RE: [EXTERNAL] RE: Re: Spark Kinesis Connector SSL issue
Hi,
The issue is that the KCL inside the Spark Streaming connector does not provide
a way to pass KCL configuration in, which means we
@spark.apache.org
Subject: [EXTERNAL] Re: Spark Kinesis Connector SSL issue
CAUTION EXTERNAL EMAIL
DO NOT open attachments or click on links from unknown senders or unexpected
emails.
Can you call this service with regular code(No Spark)?
On Mon, 07 Jan 2019 02:42:48 -0800
email: eppdev...@discover.com<mailto:eppdev...@discover.com>
Tel: +44 (0)
Mob: +44 (0) 7440783885
From: yeikel valdes [mailto:em...@yeikel.com]
Sent: 07 January 2019 12:15
To: Shashikant Bangera
Cc: user@spark.apache.org
Subject: [EXTERNAL] Re: Spark Kinesis Connector SSL issue
CAUTION EXTER
Can you call this service with regular code(No Spark)?
On Mon, 07 Jan 2019 02:42:48 -0800 shashikantbang...@discover.com wrote
Hi team,
please help , we are kind of blocked here.
Cheers,
Shashi
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
Hi team,
please help , we are kind of blocked here.
Cheers,
Shashi
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Hi Team,
we are trying access the endpoint thought library mentioned below and we get
the SSL error i think internally it use KCL library. so if I have to skip the
certificate is it possible through KCL utils call ? because I do not find any
provision to do that to set no-verify=false within
Hi! Sorry if this is a repost.
I'm using Spark + Kinesis ASL to process and persist stream data to
ElasticSearch. For the most part it works nicely.
There is a subtle issue I'm running into about how failures are handled.
For example's sake, let's say I am processing a Kinesis stream
(spark.streaming.rateLimit) to prevent spark from receiving data faster
that it can process.
On Mon, Aug 10, 2015 at 4:40 PM, Phil Kallos phil.kal...@gmail.com wrote:
Hi! Sorry if this is a repost.
I'm using Spark + Kinesis ASL to process and persist stream data to
ElasticSearch. For the most part it works
believe there is some work to fix this problem like Kafka direct API.
Not sure if this is it : https://issues.apache.org/jira/browse/SPARK-9215
Thanks,
Patanachai
On 08/06/2015 12:08 PM, phibit wrote:
Hi! I'm using Spark + Kinesis ASL to process and persist stream data to
ElasticSearch
Hi! I'm using Spark + Kinesis ASL to process and persist stream data to
ElasticSearch. For the most part it works nicely.
There is a subtle issue I'm running into about how failures are handled.
For example's sake, let's say I am processing a Kinesis stream that produces
400 records per second
hey mike-
as you pointed out here from my docs, changing the stream name is sometimes
problematic due to the way the Kinesis Client Library manages leases and
checkpoints, etc in DynamoDB.
I noticed this directly while developing the Kinesis connector which is why I
highlighted the issue
Hey Chris!
I was happy to see the documentation outlining that issue :-) However, I
must have got into a pretty terrible state because I had to delete and
recreate the kinesis streams as well as the DynamoDB tables.
Thanks for the reply, everything is sorted.
Mike
On Fri, May 8, 2015 at
- [Kinesis stream name]: The Kinesis stream that this streaming
application receives from
- The application name used in the streaming context becomes the
Kinesis application name
- The application name must be unique for a given account and region.
- The Kinesis
Hi All,
I am submitting the assembled fat jar file by the command:
bin/spark-submit --jars /spark-streaming-kinesis-asl_2.10-1.3.0.jar --class
com.xxx.Consumer -0.1-SNAPSHOT.jar
It reads the data file from kinesis using the stream name defined in a
configuration file. It turns out that it
Hey y'all,
While I haven't been able to get Spark + Kinesis integration working, I
pivoted to plan B: I now push data to S3 where I set up a DStream to
monitor an S3 bucket with textFileStream, and that works great.
I 3 Spark!
Best,
Vadim
ᐧ
On Mon, Apr 6, 2015 at 12:23 PM, Vadim Bichutskiy
be).
~ Jonathan Kelly
From: Vadim Bichutskiy vadim.bichuts...@gmail.com
Date: Friday, April 3, 2015 at 12:26 PM
To: Jonathan Kelly jonat...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis
@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis consumer and assemble it
into an uber jar following
http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
http://t.signauxtrois.com/e1t/c/5
To: Jonathan Kelly jonat...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis consumer and assemble it
into an uber jar following
http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
...@gmail.com
Date: Friday, April 3, 2015 at 12:26 PM
To: Jonathan Kelly jonat...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis consumer and assemble it
into an uber jar following
http
vadim.bichuts...@gmail.commailto:vadim.bichuts...@gmail.com
Date: Friday, April 3, 2015 at 12:26 PM
To: Jonathan Kelly jonat...@amazon.commailto:jonat...@amazon.com
Cc: user@spark.apache.orgmailto:user@spark.apache.org
user@spark.apache.orgmailto:user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all
...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis consumer and assemble it
into an uber jar following
http://spark.apache.org/docs/latest/streaming-kinesis-integration.html
and example
https
were not included in the assembly
(but yes, they should be).
~ Jonathan Kelly
From: Vadim Bichutskiy vadim.bichuts...@gmail.com
Date: Friday, April 3, 2015 at 12:26 PM
To: Jonathan Kelly jonat...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi
@spark.apache.orgmailto:user@spark.apache.org
user@spark.apache.orgmailto:user@spark.apache.org
Subject: Re: Spark + Kinesis
Thanks. So how do I fix it?
[https://mailfoogae.appspot.com/t?sender=admFkaW0uYmljaHV0c2tpeUBnbWFpbC5jb20%3Dtype=zerocontentguid=51a86f6a-7130-4760-aab3-f4368d8176b9]ᐧ
On Fri, Apr 3, 2015 at 3:43 PM
From: Vadim Bichutskiy vadim.bichuts...@gmail.com
Date: Friday, April 3, 2015 at 12:26 PM
To: Jonathan Kelly jonat...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis consumer and assemble
Kelly
From: Vadim Bichutskiy vadim.bichuts...@gmail.com
Date: Friday, April 3, 2015 at 12:26 PM
To: Jonathan Kelly jonat...@amazon.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Spark + Kinesis
Hi all,
Good news! I was able to create a Kinesis consumer and assemble
Hi all,
I am trying to write an Amazon Kinesis consumer Scala app that processes
data in the
Kinesis stream. Is this the correct way to specify *build.sbt*:
---
*import AssemblyKeys._*
*name := Kinesis Consumer*
*version := 1.0organization := com.myconsumerscalaVersion :=
...@gmail.commailto:vadim.bichuts...@gmail.com
Date: Thursday, April 2, 2015 at 9:53 AM
To: user@spark.apache.orgmailto:user@spark.apache.org
user@spark.apache.orgmailto:user@spark.apache.org
Subject: Spark + Kinesis
Hi all,
I am trying to write an Amazon Kinesis consumer Scala app that processes data
for that, and I temporarily moved on to other things for
now.
~ Jonathan Kelly
From: 'Vadim Bichutskiy' vadim.bichuts...@gmail.com
Date: Thursday, April 2, 2015 at 9:53 AM
To: user@spark.apache.org user@spark.apache.org
Subject: Spark + Kinesis
Hi all,
I am trying to write an Amazon
28 matches
Mail list logo
