gt;
> Thanks,
>
> Ben
>
>
>
> On Fri, Jan 8, 2016 at 10:52 AM, David Montgomery <
> davidmontgom...@gmail.com
> > wrote:
>
> > Hi,
> >
> > I am on Digi Ocean FYI and not dealing with the issue of private and
> public
> > IP address.
Hi,
I am on Digi Ocean FYI and not dealing with the issue of private and public
IP address. I killed all kafka servers and rebuilt but have the same zk
cluster. One would think on kafka 0.8.2 that if I rebuild kafka, register
with ZK all would be right with the world but guess not. I have new b
Hi,
Why do I get this error in the logs for v8.x?
==> kafka-logs <==
tail: error reading ‘kafka-logs’: Is a directory
tail: kafka-logs: cannot follow end of this type of file; giving up on this
name
Here is what I have in the config file
log.dirs=/tmp/kafka-logs
Is this a major issue?
Thanks
H.
Why do I get the below when I use python kafka? Is this a buggy client?
LeaderNotAvailableError:
PartitionMetadata(topic='topic-impression-production', partition=0,
leader=-1, replicas=(), isr=(), error=5)
What is most disturbing I get the below when I run describe from the
command line.
n Davison <
> ben.davi...@7digital.com>
> > > wrote:
> > >
> > > > Hi David,
> > > >
> > > > Are you running in docker? Are you trying to connect from to a remote
> > > box?
> > > > We found we could connect locall
host.
>
> (I've just started using kafka also)
>
> We had the same issue and found out: host.name=<%=@ipaddress%> needed to
> be
> the FQDN of the box.
>
> Thanks,
>
> Ben
>
> On Thu, Dec 17, 2015 at 5:40 AM, David Montgomery <
> davidmontgom.
Hi,
I am very concerned about using kafka in production given the below
errors:
Now issues with myt zookeeper. Other services use ZK. Only kafka fails.
I have 2 kafka servers using 8.x. How do I resolve? I tried restarting
services for kafka. Below is my kafka server.properties file
'Traceb
ase check the size of that partition's
> directory on both the replicas. If the size differs by a substantial
> amount, it clears one thing- that your one-and-only replica fetcher thread
> on at least 1 broker is busy
>
> Regards,
> Prabhjot
> On Oct 10, 2015 12:56 AM, &q
replica fetcher thread (which would be 1 by default, as you've not set it
> explicitly) is busy.
> Try increasing the value of num.replica.fetchers to 4
>
> Also, please share your zoo.cfg
>
> Regards,
> Prabhjot
> On Oct 9, 2015 11:55 PM, "David Montgomery"
partition=0,
error=0, offset=2)]
success: [ProduceResponse(topic='topic-test-production', partition=0,
error=0, offset=3)]
fail: kafka.common.UnknownTopicOrPartitionError:
ProduceResponse(topic='topic-test-production', partition=1, error=3,
offset=-1)
Why would this happen?
Thanks
On Sa
socket error:
java.nio.channels.ClosedChannelException (kafka.consumer.SimpleConsumer)
On Sat, Oct 10, 2015 at 1:24 AM, David Montgomery wrote:
> Is this a total failure on kafka or python kakfa-python?
>
> I am using ubuntu 14.04 on digital ocean and zk 3.4.6
> Using UFW all zk and kafka servers hav
7;96.e.e.e', port=9092),
> BrokerMetadata(nodeId=4, host='96.c.c.c', port=9092)] *
>
> *2015-10-09
> 21:16:54,596.596.991062164:kafka.client:140735282348800:INFO:8705:Updating
> topic metadata: [TopicMetadata(topic='part_1_repl_3', error=0,
> partitions=[Par
, error=5,
partitions=[])]
On Fri, Oct 9, 2015 at 7:00 PM, David Montgomery
wrote:
> This is the python error
>
> kafka.common.LeaderNotAvailableError:
> TopicMetadata(topic='topic-test-production', error=5, partitions=[])
>
>
> On Fri, Oct 9, 2015 at 6:55 PM, Da
2015 at 9:03 PM, Prabhjot Bharaj
wrote:
> Hi,
>
> Please share the code snippet for Kafka python
>
> Regards,
> Prabhjot
> On Oct 9, 2015 4:30 PM, "David Montgomery"
> wrote:
>
> > This is the python error
> >
> > kafka.common.LeaderNotAva
This is the python error
kafka.common.LeaderNotAvailableError:
TopicMetadata(topic='topic-test-production', error=5, partitions=[])
On Fri, Oct 9, 2015 at 6:55 PM, David Montgomery
wrote:
> Well i masked my true domain but the hostnames are valid and do work. I
> do see the t
,2.zk.do.production.test:2181,3.zk.do.production.test:2181
>
> Is 2.zk.do.production.test or 3.zk.do.production.test a valid hostname??
>
> Regards,
> Prabhjot
> On Oct 9, 2015 1:37 PM, "David Montgomery"
> wrote:
>
> > PS I added one more server now 3
PS I added one more server now 3 kafka servers. still kafka is not
working. Is there a known bug?
On Fri, Oct 9, 2015 at 3:23 PM, David Montgomery
wrote:
> i,
>
> I have 3 ZK servers and 2 kafka servers with 2 partitions.
>
> I am using kafka_2.11-0.8.2.1
>
> Here is how
i,
I have 3 ZK servers and 2 kafka servers with 2 partitions.
I am using kafka_2.11-0.8.2.1
Here is how I am creating a topic:
bin/kafka-topics.sh --zookeeper
1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181
--create --topic topic-test-production --partitio
nload the wrapper ###
> > cd kafka_source_dir
> > gradle
> >
> > Now everything else will work
> >
> > ### Building a jar and running it ###
> > ./gradlew jar
> >
> >
> > Can you let us know where you saw the SBT instructions, so we ca
Just wondering I am getting this very disapointing error with kafka install.
git clone https://git-wip-us.apache.org/repos/asf/kafka.git
cd kafka
git checkout -b 0.8 remotes/origin/0.8
./sbt "++2.9.2 update"
Thanks
[warn]
http://repo1.maven.org/maven2/com/jsuereth/xsbt-gpg-plugin_2
is has to be larger than message.max.bytes, or
> a broker will accept messages and fail to replicate them. Leading to
> potential data loss.
>
> Gwen
>
> On Sat, Jul 11, 2015 at 9:08 PM, David Montgomery
> wrote:
> > I cant send this s simple payload using python.
I cant send this s simple payload using python.
topic: topic-test-development
payload: {"utcdt": "2015-07-12T03:59:36", "ghznezzhmx": "apple"}
No handlers could be found for logger "kafka.conn"
Traceback (most recent call last):
File "/home/ubuntu/workspace/feed-tests/tests/druid-adstar.py
Hi,
I am using druid that consumes from kafka 8. Using python-kafka I write to
kafka so 1 server and I have 2 partitions.
Given this setup,
1) How do I shard random? I assume a shard == partition...I hope.. and I
assune the client is responsible for sharding
For Example
rand = random.choice([0
> >
> > Otis
> > --
> > Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> > Solr & Elasticsearch Support * http://sematext.com/
> >
> >
> > On Tue, Dec 2, 2014 at 9:34 PM, YuanJia Li wrote:
> >
> > > Hi David，
ment.jmxremote=true
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false"' | tee -a
/var/kafka/bin/kafka-run-class.sh
echo 'export JMX_PORT=${JMX_PORT:-}' | tee -a
/var/kafka/bin/kafka-server-start.sh
Thanks
On Tue, Dec 2, 2014 at 9:58 PM, And
I am using kafka 8. I will try your suggestion. But if I run lsof -i
: should I not see a proccess running on that port? I am not seeing
anything.
On Tue, Dec 2, 2014 at 5:37 PM, yuanjia8947 wrote:
> hi David,
> which version do you use kafka?
> when I use kafka 0.8.0, I write jmxtrans "
Hi,
I am having a very difficult time trying to report kafka 8 metrics to
Graphite. Nothing is listening on and and no data in graphite. If
this method of graphite reporting is know to not work is there an
alternative to jmxtrans to get data to graphite?
I am using the deb file to install
Hi,
I need to solve the below problem I am using druid realtime as a consumer
for druid and they have zero issue on what is going on. Whyh would kafka
thow such an error? When I produce and consume from the python lib called
brod..I have no issues. Is the below an kafka issues or consumer issu
r.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 25 more
On Wed, Feb 26, 2014 at 12:40 PM, David Montgomery <
davidmontgom...@gmail.com> wrote:
> Hi,
>
> This
How do I fix?
In python
for offset, message in brod.Kafka.fetch(kafka,pixel_topic, offset=0):
print message
File
"/usr/local/lib/python2.7/dist-packages/brod-0.3.2-py2.7.egg/brod/blocking.py",
line 70, in _read
return callback(output)
File
"/usr/local/lib/python2.7/dist-packa
than the maximum size of 104857600 bytes.
(kafka.network.Processor)
On Wed, Feb 26, 2014 at 1:41 PM, David Montgomery wrote:
> I moved lib files..the consumer still gets the below error
> this is how I install kafka from the online docs.are the docs
> incorrect?
>
> tar -
opy scala-library.jar. The version depends on scala
> version that the kafka jar is built with.
>
> Thanks,
>
> Jun
>
>
> On Tue, Feb 25, 2014 at 9:21 PM, David Montgomery <
> davidmontgom...@gmail.com
> > wrote:
>
> > Below are the scala jar files I have on
the lib dir.
>
> Thanks,
>
> Jun
>
>
> On Tue, Feb 25, 2014 at 8:40 PM, David Montgomery <
> davidmontgom...@gmail.com
> > wrote:
>
> > Hi,
> >
> > This is how I start kafka.
> >
> > command = /var/lib/kafka-<%=@version%>
Hi,
This is how I start kafka.
command = /var/lib/kafka-<%=@version%>-src/bin/kafka-server-start.sh
/var/lib/kafka-<%=@version%>-src/config/server.properties
In another application I get teh below error. The suggestion is to add the
scalar libraries to the path. How do I do that?
thanks
ja
I am looking on how to get kafka to work with python and sending messages.
I am using python kafka to produce and consume messages.
https://github.com/mumrah/kafka-python
the messages that I am sending is a json string in python.
kafka = KafkaClient(kafka_domain, 9092)
producer = SimpleProdu
Hi,
Kafka 8 is giving issues.
I want to connect to kafka via python
Below is the error I get in kafka and in python. I just dont get why kafka
fails...how do I resolvehow do I just reset everything? Thanks
from kafka.client import KafkaClient
from kafka.producer import SimpleProducer,
itions this error
> is thrown.
>
> Have you created the topic manually before you try to produce/consume?
>
> Guozhang
>
>
> On Wed, Feb 12, 2014 at 4:05 PM, David Montgomery <
> davidmontgom...@gmail.com
> > wrote:
>
> > I just built a new kafka server
I just built a new kafka server 0.8.0. How I am getting the below in the
logs. What does it mean?
[2014-02-13 00:01:43,034] ERROR [KafkaApi-1392248055] Error while fetching
metadata for partition [topic-pixel,0] (kafka.server.KafkaApis)
kafka.common.LeaderNotAvailableException: Leader not avail
Hi,
I am using kafka 8.0.
when I set up kafkaand I look at the broker id in ZK...this is what
I get:
#my python script
print client.get_children('/brokers/ids/1391209402')
[ ]
should I not see the ip address?
Thanks
Hi,
I am using the python brod library to write to kafka 8.0.
I am on a 2 core server with 4 gigs of ram on ubuntu 12.04
I have zero clue what the below error means. How do I tweak kafka to get it
to work?
[2014-02-06 07:21:57,001] INFO Closing socket connection to
/222.127.xxx.xxx. (kafka.net
gt;
>
> On Wed, Feb 5, 2014 at 6:59 PM, David Montgomery
> wrote:
>
> > Hi,
> >
> > How does a kafka of one have issues becoming a leader? I am using kafka
> > 8.0. Should I just ignore this error? How would I resolve? I just have
> > one kafka server
Hi,
How does a kafka of one have issues becoming a leader? I am using kafka
8.0. Should I just ignore this error? How would I resolve? I just have
one kafka server in dev.
Below is how I start kafka:
/var/lib/kafka-0.8.0-src/bin/kafka-server-start.sh
/var/lib/kafka-0.8.0-src/config/server.pro
idea now why they are not
> being set as expected.
>
>
> On Fri, Jan 31, 2014 at 1:20 PM, Benjamin Black wrote:
>
> > kafka-run-class.sh in 0.8 does not define KAFKA_HEAP_OPTS. i think you
> > want KAFKA_OPTS.
> >
> >
> > On Fri, Jan 31, 2014 at 1:14 PM, Da
you sure the java opts are being set as you expect?
> On Jan 30, 2014 12:41 PM, "David Montgomery"
> wrote:
>
> > Hi,
> >
> > This is a dedicated machine on DO.But i can say I did not have a
> > problem with kafka 7.
> > I just upgraded the macine
Can you try running any java
> class with the same command line options you are using for kafka and
> confirm that that also doesn't work.
>
> -Jay
>
> On Thu, Jan 30, 2014 at 11:23 AM, David Montgomery <
> davidmontgom...@gmail.com> wrote:
>
> > Hi, <%20us
Hi, <%20us...@kafka.apache.org>
Why oh whhy can I nt start kafka 8? I am on a machine with 512 megs of ram
on digi ocean. What does one have to do to get kafka to work?
root@do-kafka-sf-development-20140130051956: export
KAFKA_HEAP_OPTS="-Xmx256M -Xms128M
root@do-kafka-sf-development-20140130
Hi,
On the Kafka cluster I would like like to get monitoring stats. I do not
know java per this page.
Monitoring
Our monitoring is done though a centralized monitoring system custom to
LinkedIn, but it keys off the JMX stats exposed from Kafka. To see what is
available the easiest thing is just
Hi,
My endpoint for kafka is in Europe where the a consumers gets the final
messages. But I want to write to kafka servers in SG, and the US east and
west and will then forward to kafka servers in the UK. How do I do that?
Is there a good tutorial? What are the keywords that describe what I wan
You may be able to do something if you are ready to use low level options.
>
>
> On Sun, Nov 10, 2013 at 2:53 PM, David Montgomery <
> davidmontgom...@gmail.com
> > wrote:
>
> > I am using kafka0. 7.x I should mention and is nit by choice because
> druid,
> >
n't support 0.8 so you have to use it with 0.7.x.
> On the other hand, kafka-python is the only python library I know of that
> supports 0.8.
>
> cheers,
> marc
>
>
> On Sat, Nov 9, 2013 at 6:34 PM, David Montgomery
> wrote:
>
> > Hi,
> >
> > Even
Exception("All servers failed to process request")
Exception: All servers failed to process request
On Sun, Nov 10, 2013 at 5:47 AM, David Montgomery wrote:
> Hi,
>
> I asked this question on StackOverflow. No response. No solution seems
> tp work...even for a python lib
Hi,
I asked this question on StackOverflow. No response. No solution seems tp
work...even for a python lib that was designed for gevent and kafka, Is
there any otehr libary that works with gevent wit python that works?
am trying to use gevent to write to kafka using brod on python 2.7.
Her
king. I do believe your client is pure python?
>>
>> Thanks
>>
>>
>> On Sun, Feb 17, 2013 at 1:15 PM, David Arthur wrote:
>>
>> Replying to both messages inline:
>>>
>>>
>>> On 2/16/13 9:07 PM, David Montgomery wrote:
>>
Sun, Feb 17, 2013 at 1:15 PM, David Arthur wrote:
> Replying to both messages inline:
>
>
> On 2/16/13 9:07 PM, David Montgomery wrote:
>
>> By the way..I assume that python-kafka is gevent safe?
>>
> No idea, I've never used greenlet/gevent before. If the qu
By the way..I assume that python-kafka is gevent safe?
Thanks
On Sun, Feb 17, 2013 at 10:04 AM, David Montgomery <
davidmontgom...@gmail.com> wrote:
> Ok...great...I see now about offsets. I see how I can manage on
> restarts. Thanks!
>
> So...considering I am in a dispos
You need to read the Kafka design docs. Kafka does not delete messages
>> just because a Consumer reads it. It does not track what messages have been
>> consumed by any Consumer.
>>
>> It is up to Consumers to start off where they left off, by always asking
>> for the righ
56 matches
Mail list logo
