Hello Robert, Beam has documented only OffsetRangeTracker [1] for new SDF API. Since Beam is moving away from Source API, I thought it would be nice to develop IO connectors by using new SDFs. For this I need to create restriction tracker that follows new SDF API.
So I propose adding ByteKeyRange as new restriction class and ByteKeyRestrictionTracker as new restriction tracker class. In my implementation I’ve also used ByteKey class which are given to restriction. 1. https://github.com/apache/beam/blob/7eb7fd017a43353204eb8037603409dda7e0414a/sdks/python/apache_beam/io/restriction_trackers.py#L76 On 2022/02/11 18:27:23 Robert Bradshaw wrote: > Hi Sam! Glad to hear you're willing to contribute. > > Though the name is a bit different, I'm wondering if this is already > present as LexicographicKeyRangeTracker. > https://github.com/apache/beam/blob/release-2.35.0/sdks/python/apache_beam/io/range_trackers.py#L349 > > On Fri, Feb 11, 2022 at 9:54 AM Ahmet Altay > <al...@google.com<mailto:al...@google.com>> wrote: > > > > Hi Sami. Thank you for your interest. > > > > Adding people who might be able to comment: @Chamikara Jayalath @Lukasz Cwik > > > > On Thu, Feb 10, 2022 at 8:38 AM Sami Niemi > > <sa...@solita.fi<mailto:sa...@solita.fi>> wrote: > >> > >> Hello, > >> > >> > >> > >> I noticed that Python SDK only has implementation for OffsetRangeTracker > >> and OffsetRange while Java also has ByteKeyRange and -Tracker. > >> > >> > >> > >> I have currently created simple implementations of following Python > >> classes: > >> > >> ByteKey > >> ByteKeyRange > >> ByteKeyRestrictionTracker > >> > >> > >> > >> I would like to make contribution and make these available in Python SDK > >> in addition to OffsetRange and -Tracker. I would like to hear any thoughts > >> about this and should I make a contribution. > >> > >> > >> > >> Thank you, > >> > >> Sami Niemi > SAMI NIEMI Data Engineer +358 50 412 2115 sami.ni...@solita.fi<mailto:sami.ni...@solita.fi> SOLITA Eteläesplanadi 8 00130 Helsinki solita.fi<https://www.solita.fi>
