[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089096#comment-17089096
]
Yuexi Liu commented on KAFKA-9335:
--
[~vveeramani] I test it, the 2.4.1 fixed it
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Assignee: Boyang Chen
>Priority: Blocker
> Labels: bug
> Fix For: 2.4.1
>
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039049#comment-17039049
]
Vijay Veeramani commented on KAFKA-9335:
[~bchen225242] thanks for looking into this... Is there any work around for
this defect, till 2.4.1 release becomes available?
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Assignee: Boyang Chen
>Priority: Blocker
> Labels: bug
> Fix For: 2.4.1
>
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010149#comment-17010149
]
ASF GitHub Bot commented on KAFKA-9335:
---
guozhangwang commented on pull request #7904: KAFKA-9335: Fix
StreamPartitionAssignor regression in repartition topics counts
URL: https://github.com/apache/kafka/pull/7904
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Assignee: Boyang Chen
>Priority: Blocker
> Labels: bug
> Fix For: 2.4.1
>
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009915#comment-17009915
]
ASF GitHub Bot commented on KAFKA-9335:
---
abbccdda commented on pull request #7904: KAFKA-9335: Fix
StreamPartitionAssignor regression in repartition topics counts
URL: https://github.com/apache/kafka/pull/7904
This PR fixes the regression introduced in 2.4 from 2 refactoring PRs:
https://github.com/apache/kafka/pull/7249/
https://github.com/apache/kafka/pull/7419/
The bug was introduced by having a logical path leading
`numPartitionsCandidate` to be 0, which is assigned to `numPartitions` and
later being checked by `setNumPartitions`. In the subsequent check we will
throw illegal argument if the `numPartitions` is 0.
This bug is both impacting new 2.4 application and upgrades to 2.4 in
certain types of topology. The example in original JIRA was imported as a new
integration test to guard against such regression. We also verify that without
the bug fix application will still fail by running this integration test.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Assignee: Boyang Chen
>Priority: Blocker
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007853#comment-17007853
]
Boyang Chen commented on KAFKA-9335:
By applying your example, I could successfully reproduce the symptom. In fact,
this is not upgrade related but a 2.4 specific correctness bug.
The topology of the example used a chaining of 3 sub-topologies, which has a
dependency A -> B -> C all connected by repartition topics.
We gather the repartition topic metadata first, so that we created a mapping as
[A-output-rep, B-output-rep], and they all have no partitions defined yet.
Next, a lookup of C's input partition (= B-output-rep) will be informed that
the lookup fails, so it will try to look for the B's input partition
(=A-output-rep), here
as A-output is also a repartition topic which has no partition initialized
either, the passed-down assignment is invalid as we still use a numPartitions =
0 after the if block.
*To summarize, we need a chaining of 3 sub-topologies where both inputs for
non-source topologies are repartition topics, and the sink topology gets
initialized first.*
The explanation is bit tricky, but thanks so much for the reporting, and we
will propose a fix shortly.
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Priority: Major
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007641#comment-17007641
]
Boyang Chen commented on KAFKA-9335:
Sounds good, I will try to reproduce it today.
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Priority: Major
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007394#comment-17007394
]
Nitay Kufert commented on KAFKA-9335:
-
1. Yes
2. I tried with 12 but I believe it wouldn't matter
3. Pretty sure you don't needs any data
4. Tested it with 2.3.1 and it works, didn't try running it with 2.2 or 2.3
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Priority: Major
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007101#comment-17007101
]
Boyang Chen commented on KAFKA-9335:
Thanks for the report [~nitay.k]. Could you be more specific about the
reproducing process, for example:
# Does this reproduce on local env?
# How many topic partitions we need to configure for each input and output
topic?
# Do we need data flowing into the input topic when running this example?
# Does this only fail for 2.3.1 -> 2.4 upgrade? At least, does 2.2 and 2.3
could also reproduce issue with this example?
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Priority: Major
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006680#comment-17006680
]
Nitay Kufert commented on KAFKA-9335:
-
In the example I provided, the exception is thrown over the following key:
missing-partitions-example-odd_store_2-repartition
And regarding the number of partitions - the topic can't be found so I guess
that why it has 0 partitions
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Priority: Major
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9335) java.lang.IllegalArgumentException: Number of partitions must be at least 1.
[
https://issues.apache.org/jira/browse/KAFKA-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006659#comment-17006659
]
Dongjin Lee commented on KAFKA-9335:
{quote}java.lang.IllegalArgumentException: The number of partitions must be at
least 1.
{quote}
This Exception is thrown when an internal topic (i.e., 'xx-changelog' or
'xx-repartition') with 0 partition exists. Could you check the name of the
internal topics & their partition counts? It will be a great help for tracking
the reason.
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
>
>
> Key: KAFKA-9335
> URL: https://issues.apache.org/jira/browse/KAFKA-9335
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.4.0
>Reporter: Nitay Kufert
>Priority: Major
> Labels: bug
>
> Hey,
> When trying to upgrade our Kafka streams client to 2.4.0 (from 2.3.1) we
> encountered the following exception:
> {code:java}
> java.lang.IllegalArgumentException: Number of partitions must be at least 1.
> {code}
> It's important to notice that the exact same code works just fine at 2.3.1.
>
> I have created a "toy" example which reproduces this exception:
> [https://gist.github.com/nitayk/50da33b7bcce19ad0a7f8244d309cb8f]
> and I would love to get some insight regarding why its happening / ways to
> get around it
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
