Awesome!
The auth params is a good option to allow the user custom authentication.
I will be taking a look!
Thanks for sharing!
Thanks,
Marco Robles (He/Him) | <https://www.wizeline.com/>
Software Engineer
marco.rob...@wizeline.com
Amado Nervo 2200, Esfera P6, Col. Ciudad del Sol,
Hi folks,
I have been working on the first version of PulsarIO Connector and I
recently raised the following PR-16634
<https://github.com/apache/beam/pull/16634> , I am sharing the PR here in
order to get feedback and possibly a review from you.
Thanks in advance.
--
*Marco
Hi folks,
Has anyone implemented a custom implementation of OffsetRange and
OffsetRangeTracker based on a Comparable Java class?
Thanks
--
*Marco Robles* *|* WIZELINE
Software Engineer
marco.rob...@wizeline.com
--
*This email and its contents (including any attachments) are being sent to
testing, but it seems the SDF
version for the testing is not being used (Pipeline tests
<https://github.com/apache/beam/blob/master/sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/KafkaIOIT.java>
).
Thanks in advance
--
*Marco Robles* *|* WIZELINE
Software Engineer
mar
<https://issues.apache.org/jira/browse/BEAM-12650> with the corresponding
changes.
Thanks
--
*Marco Robles* *|* WIZELINE
Software Engineer
marco.rob...@wizeline.com
Amado Nervo 2200, Esfera P6, Col. Ciudad del Sol, 45050 Zapopan, Jal.
--
*This email and its contents (including any attachments) are being
re any test related for testing the pipeline flow with the SDF
implementation in KafkaIOTest
<https://github.com/apache/beam/blob/master/sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/KafkaIOTest.java>
?
Thanks in advance
--
*Marco Robles* *|* WIZELINE
Software Engineer
marco.ro
ent/unzip/TalendP2UnzipOpenSourceRelease/org/apache/beam/beam-sdks-java-javadoc/0.6.0-tlnd/beam-sdks-java-javadoc-0.6.0-tlnd.jar-unzip/org/apache/beam/sdk/transforms/DoFn.html>
?
Thanks in advance
--
*Marco Robles* *|* WIZELINE
Software Engineer
marco.rob...@wizeline.com
--
*This email a
//github.com/apache/beam/blob/5e7e66f116da5b947cb271ccaeb9a6a15fb1d6f6/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java#L1359
> 2:
> https://pulsar.incubator.apache.org/api/client/2.8.0-SNAPSHOT/org/apache/pulsar/client/api/MessageId.html#latest
> 3:
> https://pulsar.incubator.a
imator, am I right?
--
*Marco Robles* *|* WIZELINE
Software Engineer
--
*This email and its contents (including any attachments) are being sent to
you on the condition of confidentiality and may be protected by legal
privilege. Access to this email by anyone other than the intended
.getPartitionsForTopic("my-topic").join();
> for (String p : partitions) {
> Reader reader = pulsarClient.newReader()
> .topic(p)
> .startMessageId(....)
> .create();
>
> ///...
> }
>
> Matteo
>
>
and can poll for
> the latest id so you can split the numerical range easily))
> 5) What value do you see it providing?
>
> 1:
> https://github.com/apache/beam/blob/03a1cca42ceeec2e963ec14c9bc344956a8683b3/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Watch.java#L885
&g
powerful since it can report the size when
> the restriction isn't being processed and also report work
> completed/remaining when the restriction is actively being processed.
>
> On Thu, Sep 2, 2021 at 12:40 PM Marco Robles
> wrote:
>
>> Hi folks,
>>
>>
of how much work would be processed)?
For the HasProgress, I supposed it should be implemented in the @NewTracker
method, right? Does anyone have experience using HasProgress?
Thanks in advance.
On Mon, Aug 16, 2021 at 11:49 AM Marco Robles
wrote:
> Does the client buffer/reserve any messa
is kind of confusing in that part when you are working
with SDF, Should it be treated as Unbounded for the source/reading part?
Thanks in advance
--
*Marco Robles* *|* WIZELINE
Software Engineer
marco.rob...@wizeline.com
Amado Nervo 2200, Esfera P6, Col. Ciudad del Sol, 45050 Zapopan, Jal.
:58 PM Luke Cwik wrote:
> Does the client buffer/reserve any messages?
> For example, you call consumer.receive() so it goes and fetches the next
> 100 messages assuming that you'll want them as well so the next 99
> consumer.receive() calls will be fast.
>
> On Fri, Au
wn in this client example
<https://pulsar.apache.org/docs/en/client-libraries-java/#consumer> (see
MessageListener example, above *Configure consumer* section)
On Fri, Aug 13, 2021 at 5:14 PM Luke Cwik wrote:
>
>
> On Thu, Aug 12, 2021 at 3:31 PM Marco Robles Pulido <
> m
any messages are still being received in
the queue with help of the client listener
<https://pulsar.apache.org/docs/en/concepts-messaging/#listeners>, so
every time the listener is invoked will do something like N + 1.
Thanks in advance
On Thu, Aug 12, 2021 at 5:18 PM Marco Rob
t the numerical range easily))
> 5) What value do you see it providing?
>
> 1:
> https://github.com/apache/beam/blob/03a1cca42ceeec2e963ec14c9bc344956a8683b3/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Watch.java#L885
>
> On Tue, Aug 3, 2021 at 1:17 PM Marco Robles Pulido <
> marco.rob...@wize
rs.
So, let me know your ideas/input about it. And maybe If i am wrong help to
clarify the SDF restrictions for KafkaIO.
Thanks,
--
*Marco Robles* *|* WIZELINE
Software Engineer
marco.rob...@wizeline.com
Amado Nervo 2200, Esfera P6, Col. Ciudad del Sol, 45050 Zapopan, Jal.
--
*This e
Hi everyone!
I am planning to start working on this ticket BEAM-8218
<https://issues.apache.org/jira/browse/BEAM-8218> - Implement Apache
PulsarIO
Is there any example about what is about? or any progress on this topic?
Thanks in advance,
--
*Marco Robles* *|* WIZELINE
Software En
Hi team,
This is Marco and I would like to be added to the Apache Beam contributor
list, my username is 'marroble'.
Thanks!
--
*This email and its contents (including any attachments) are being sent to
you on the condition of confidentiality and may be protected by legal
privilege. Access to t
21 matches
Mail list logo
