I can't seem to get a listeners and advertised.listeners configuration for
server properties figured out so I can connect remotely with my producer
and consumers.
If I set it like this...
listeners=CLIENT://:9090,PLAINTEXT://:9092
advertised.listeners=CLIENT://:9090,PLAINTEXT://:9092
>From my ex
Hi All,
I am trying to understand different configurations to apply on my cluster.
I find the num.recovery.threads.per.data.dir=1,bit interesting.
In most of configuration example I find in the google, the value set as 1.
Everyone wants quick start and recovery, to achieve the same can I increase
t
Hi
Is there a python API for the source and sink connect implementation?
My source is - Filesystem
Sink - HDFS
Thanks
--
Swaapnika
Congratulations Onur!
On Tue, 7 Nov 2017 at 21:44 Jun Rao wrote:
> Affan,
>
> All known problems in the controller are described in the doc linked from
> https://issues.apache.org/jira/browse/KAFKA-5027.
>
> Thanks,
>
> Jun
>
> On Mon, Nov 6, 2017 at 11:00 PM, Affan Syed wrote:
>
> > Congrats O
https://issues.apache.org/jira/browse/KAFKA-6185
This was also reported by another person, Ben, attached to the end of this
ticket https://issues.apache.org/jira/browse/KAFKA-6042 which is a broker
lockup/FD issue, but a new ticket was requested. It was also suggested by
Ismael Juma there to discu
Previously deleting and recreating the topic has solved the problem.
Based on what you've said about the offset correction I did a quick test by
building kafka streams myself with the following code in
`GlobalStateManagerImpl#restoreState()`
while (offset < highWatermark) {
final Cons
I think how to use GDAX's API is orthogonal to using Kafka.
Kafka client has support for Java and Python.
On Tue, Nov 7, 2017 at 12:31 PM, Taha Arif wrote:
> Hello,
>
>
> I want to build a project that accesses the Gdax websocket in a real time
> stream, and push that data into Kafka to reforma
Hello,
I want to build a project that accesses the Gdax websocket in a real time
stream, and push that data into Kafka to reformat it and prep it for a ML
model.
*Can I use Kafka to access GDAX's API and pull data from it in realtime?*
or
*Should I pull data from their API to a text file in re
Did you delete and recreate the topic of the GlobalStore?
I did have a look into the code, too, and think there is a bug in
`GlobalStateManagerImpl#restoreState()`. I did some initial
investigation using an existing test, and the test passed without
transactional data but fails if the global store
Hello Tony,
You mentioned in 0.11.0.0 the
props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG,
1) while in 0.11.0.1 props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 2).
But from your logs it seems you set this config as 2 in both versions.
Right?
Anyways, I took a look into your logs and I think
Hello,
Got a bit further.
I registered a DNS using my router IP address and then I can connect to the
Kafka server from remote at my house through the ADSL.
So the "advertised.host.name" does not refer to an IP address but an actual
domain host name (am I correct in saying this).
Is there no
Disabling transactions doesn't seem to have changed anything. I've had a
read through the kafka streams source code, specifically the parts relating
to the restoration of the global stores and I can't see anything obvious I
should look at.
@Ted will do, here's a pastebin of the most recent run
htt
Henry,
I am not sure what do you mean by "waits for the leader of partition to
start up"? Leader election should not affect leader - follower starting up
process.
Guozhang
On Thu, Nov 2, 2017 at 4:56 PM, Henry Cai
wrote:
> We were on kafka 0.10.2.1. We tried to switch from unclean leader el
Alex:
In the future, please use pastebin if the log is not too large.
When people find this thread in mailing list archive, the attachment
wouldn't be there.
Thanks
On Tue, Nov 7, 2017 at 8:32 AM, Matthias J. Sax
wrote:
> Alex,
>
> I am not sure, but maybe it's a bug. I noticed that you read t
Can you reveal code snippet for BufferedConsumerClientAdapterImpl ?
I took a look at the logs. There was no log around 19:46:5x in either
server or controller log.
Thanks
On Tue, Nov 7, 2017 at 8:35 AM, Mana M wrote:
> Ted, did you get chance to look at the issue? I am also planning to update
It's almost certainly a typo in your command lines. Not sure how to help
without you posting them as requested.
Also, post the console output from when you created the topic.
Dale
-Original Message-
From: Donghun Kim [mailto:kimdho...@gmail.com]
Sent: Monday, November 6, 2017 7:03 PM
Ted, did you get chance to look at the issue? I am also planning to update
to latest version to find out if we still see the same issue.
On Fri, Nov 3, 2017 at 8:40 PM, Mana M wrote:
> Below are the logs:
>
> Consumer 1 logs, where issue can be seen: https://pastebin.com/PuQhud91
> Consumer 2 lo
Alex,
I am not sure, but maybe it's a bug. I noticed that you read transaction
data. Can you try to write to the topic without using transactions
and/or set the consumer into READ_UNCOMMITTED mode to verify? It only a
guess that it might be related to transactions and it would be great to
verify o
Affan,
All known problems in the controller are described in the doc linked from
https://issues.apache.org/jira/browse/KAFKA-5027.
Thanks,
Jun
On Mon, Nov 6, 2017 at 11:00 PM, Affan Syed wrote:
> Congrats Onur,
>
> Can you also share the document where all known problems are listed; I am
> as
Hi All,
I have a simple kafka streams app that seems to hang when restoring state
for a GlobalKTable. We're running in a test environment at the moment and
the topic it is loading from only has two messages in it, I don't know if
the very low volume of messages would affect the restore?
I've atta
Hi All,
I am trying to understand, what can max. no.of threads running on busy
kafka broker.
If I am missing any other configuration to consider at OS level please let
me know.
Thanks in advance.
bhanu
Hello, I'm working with an employer that is looking to hire a
permanent sysadmin to help maintain their Hadoop and Kafka clusters at
their Amsterdam, Netherlands office OR remote. Consequently I had
hoped that some members of this mailing list may like to discuss
further off-list using "JamesBTobi
Hi Guozang,
Thanks for looking into this. I was using 0.11.0.0 version of the library
earlier when I was getting the CommitFailed exception and the tasks were
terminating. The application config then was Replication Factor = 2, Num
Stream Threads = 1, Consumer Max Poll Records = 1000 & Consumer Ma
23 matches
Mail list logo