You also wouldn't have any meta data about the file so I would avoid doing this.
> On Jun 15, 2014, at 20:51, Mark Roberts wrote:
>
> You would ship the contents of the file across as a message. In general this
> would mean that your maximum file size must be smaller than your maximum
> messag
You would ship the contents of the file across as a message. In general this
would mean that your maximum file size must be smaller than your maximum
message size. It would generally be a better choice to put a pointer to the
file in some shared location on the queue.
-Mark
> On Jun 15, 2014,
We have seen people sending files with tens of MB size like messages (i.e.
one file as a single message) using Kafka.
Guozhang
On Sun, Jun 15, 2014 at 7:41 PM, wrote:
> Hi Sometimes we want use kafka to send files(like textfile,xml...),
> but I didn't see this in documention.
> Can kaf
Hi Sometimes we want use kafka to send files(like textfile,xml...), but I
didn't see this in documention.
Can kafka use to tansfer files? If can ,how can I do
Thanks
yes, I gave it several minutes.
On Sat, Jun 14, 2014 at 2:18 PM, Michael G. Noll
wrote:
> Have you given Kafka some time to re-elect a new leader for the
> "missing" partition when you re-try steps 1-5?
>
> See here:
> > If you do, you should be able to go through steps
> > 1-8 without seeing L
None. What client(s) are you using? Maybe send more logs along for the
error and other info about the setup. If you revert back does everything
work?
On Jun 15, 2014 10:33 AM, "Max Kovgan" wrote:
> Thanks for the response, Joe.
>
> in the non-working setup, kafka user had all the permissions fo
+1 for detailed examination of metrics. You can see the main metrics here:
https://kafka.apache.org/documentation.html#monitoring
Jconsole is very helpful for looking quickly at what is going on.
Cheers, Robert
On Sun, Jun 15, 2014 at 7:49 AM, pushkar priyadarshi <
priyadarshi.push...@gmail.c
Hi Riyaz,
There are a number of reasons that you may be getting low performance.
Here are some questions to get started:
1. How big are your messages? To meet your throughput requirement you need
a minimum of 10K messages per second continuously. You specified a
replication factor of 3 so at a
and one more thing.using kafka metrices you can easily monitor at what rate
you are able to publish on to kafka and what speed your consumer(in this
case your spout) is able to drain messages out of kafka.it's possible that
due to slowly draining out even publishing rate in worst case might get
eff
what throughput are you getting from your kafka cluster alone?Storm
throughput can be dependent on what processing you are actually doing from
inside it.so must look at each component starting from kafka first.
Regards,
Pushkar
On Sat, Jun 14, 2014 at 8:44 PM, Shaikh Ahmed wrote:
> Hi,
>
> Dai
Thanks for the response, Joe.
in the non-working setup, kafka user had all the permissions for its own
directories and log files.
I ran normal tests via the command line, as user "kafka" and all worked.
I think my question is:
is there any non-AF_INET socket communication between kafka clients an
Off the top of my head sounds like directory permission/ownership issue
for data.
On Jun 15, 2014 5:05 AM, "Max Kovgan" wrote:
> hi.
> as subj. states, I'm trying to use kafka-0.8.1.1 with samza (0.7).
> Initially the work started as a dev. user, e.g. "devuser", and kafka
> would run as that user
hi.
as subj. states, I'm trying to use kafka-0.8.1.1 with samza (0.7).
Initially the work started as a dev. user, e.g. "devuser", and kafka
would run as that user, "and it's been good".
Then, during the deployment attempt, we're trying to run kafka under a
different user, "kafka".
After having th
13 matches
Mail list logo
