2020-08-31 12:46:01 UTC - alex kurtser: Hello
Somebody has any guidance about partitioned topics. When should we use
partitioned topics instead of normal topics ?
What is the limit for a one regular topics? I mean for example how many
messages per sec one topic can produce/consume without needing to switch to
partitioned topic in order to prevent a bottleneck of single non partitioned
topic
----
2020-08-31 13:52:44 UTC - Vil: Personal understanding or thumb rule is, always
use a partitioned topic. If I need a unpartitioned. topic, I set partition
number to 1
+1 : alex kurtser
----
2020-08-31 13:53:23 UTC - Vil: i also remember reading in this slack that in a
near future ersion of pulsar the unpartitioned topics will be depracated
+1 : alex kurtser
----
2020-08-31 14:14:26 UTC - Nazia Firdous: Yes sir there is no issue in Broker
Through this Broker URL i can able to consume and produce the message but i am
not able to create clusters,tenants etc showing concurrent Timeout
exception,,Below are my code
```public class PulsarClientImpl {
private static final String SERVICE_URL = "<pulsar://67.160.195.238:6618>";
public PulsarClientImpl() {}
private static class PulsarClientObject {
private static PulsarClient pulsarClient = null;
static {
try {
pulsarClient = PulsarClient.builder()
.serviceUrl(SERVICE_URL)
.build();
} catch (PulsarClientException e) {
System.out.println(e.getMessage());
}
}
}
public static PulsarClient getInstance() {
return PulsarClientObject.pulsarClient;
}
}```
----
2020-08-31 16:18:20 UTC - Addison Higham: to add a bit more context @alex
kurtser
- as mentioned, you can have partitioned topics with a single partition and
later add more partitions, but you cannot convert an unpartitioned topic to
partitioned topic, hence suggesting all topics be partitioned topics even if
you only need a single partition
- partitioned topics are simply multiple topics "under the hood " that have a
distinct naming mechanism, so a partitioned topic 'my-topic' with 3 partitions
will have 3 "normal" topics called 'my-topic-partition-0',
'my-topic-partition-1', 'my-topic-partition-2', the `my-topic`. In the
implementation in both the broker and clients, some additional metadata is just
passed around to let clients know that topics are partitioned and it then
creates multiple producers/consumers for all the underlying topics.
- as far as performance, it is highly dependent on your pulsar cluster setup. A
really fast bookkeeper + large brokers can handle 100MB/s of writes a second on
a single topic, however, it doesn't let you to spread load across multiple
brokers as topics are tied to a single broker
----
2020-08-31 16:20:47 UTC - alex kurtser: Thanks for the explanations
----
2020-08-31 16:22:59 UTC - Evan Furman: reposting since it got lost:
<https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1598554550175300>
----
2020-08-31 16:28:08 UTC - Enrico Olivelli: As far as I know you should use
PulsarAdmin API, that accepts a http:// address
And not pulsar://
----
2020-08-31 16:32:39 UTC - Nazia Firdous: By mistake i posted Pulsar client
Program..
----
2020-08-31 16:32:50 UTC - Nazia Firdous: Below are my complete code
----
2020-08-31 16:33:48 UTC - Nazia Firdous: ```Pulsar Admin Program
public class PulsarAdminImpl {
private static final String URL = "<http://67.160.195.238:8080>";
public PulsarAdminImpl() {}
private static class PulsarAdminObject {
private static PulsarAdmin pulsarAdmin = null;
static {
try {
boolean useTls = false;
boolean tlsAllowInsecureConnection = false;
String tlsTrustCertsFilePath = null;
pulsarAdmin = PulsarAdmin
.builder()
.serviceHttpUrl(URL)
.tlsTrustCertsFilePath(tlsTrustCertsFilePath)
.allowTlsInsecureConnection(tlsAllowInsecureConnection)
//.connectionTimeout(60000, TimeUnit.SECONDS)
.build();
} catch (PulsarClientException e) {
System.out.println(e.getMessage());
}
}
}
public static PulsarAdmin getInstance() {
return PulsarAdminObject.pulsarAdmin;
}
}```
----
2020-08-31 16:34:06 UTC - Nazia Firdous: ```@Component
public class PulsarTenantsImpl {
public List<String> getTenantsList() {
PulsarAdmin pulsarAdmin = PulsarAdminImpl
.getInstance();
List<String> getTenantsList = null;
try {
getTenantsList = pulsarAdmin.tenants()
.getTenants();
if (getTenantsList.isEmpty()) {
return null;
}
System.out.println(getTenantsList);
} catch (PulsarAdminException e) {
System.out.println(e.getMessage());
}
return getTenantsList;
}```
----
2020-08-31 16:42:54 UTC - Nazia Firdous: I can able to instance the PulsarAdmin
----
2020-08-31 16:46:20 UTC - Nazia Firdous: But at the time getting the list of
tenants,creating tenants and clusters etc showing error message as
java.util.concurrent.TimeoutException
----
2020-08-31 20:06:18 UTC - Raman Gupta: Things to keep in mind from my own
investigations on this:
1. The `Reader` interface cannot currently be used on Partitioned topics (but
presumably could point to the specific underlying numbered topic representing a
partition directly, haven't tried this though).
2. There are some open questions about ordering when changing the number of
partitions on a partitioned topic. There was a thread about this on Thu.
----
2020-08-31 21:18:49 UTC - Nick Nezis: How do I recover a Bookkeeper node if the
journal has filled?
----
2020-08-31 22:12:32 UTC - Addison Higham: The reader interface can be used
against the underlying topics :slightly_smiling_face:
----
2020-09-01 01:59:45 UTC - ly_sffy: @ly_sffy has joined the channel
----
2020-09-01 07:38:49 UTC - Hrvoje Kovacevic: @Hrvoje Kovacevic has joined the
channel
----
