Also, will peer:// have an implementation of pinotFS ? On Thu, Jun 11, 2020 at 2:58 PM kishore g <g.kish...@gmail.com> wrote:
> +1 peer. > > unrelated to this - do we support multiple URI's? > > On Thu, Jun 11, 2020 at 2:51 PM Subbu Subramaniam <mcvsu...@apache.org> > wrote: > >> Hey Ting, >> >> I like the URI in metadata as "peer:///segmentName". This way, the URI >> remains parsable and we can use the scheme to check for a segment fetcher, >> >> thanks >> >> -Subbu >> >> On 2020/06/10 01:09:25, TING CHEN <tingc...@uber.com.INVALID> wrote: >> > As part of the deep store by-passing >> > < >> https://cwiki.apache.org/confluence/display/PINOT/By-passing+deep-store+requirement+for+Realtime+segment+completion >> > >> > project, a server is allowed to download segments from peer Pinot >> servers >> > during Low Level Consumer (LLC) instead of deep segment stores. To >> enable >> > this feature, we plan to add a new URI format for the field >> > *segment.realtime.download.url >> > *in LLCRealtimeSegmentZKMetadata. >> > >> > The new URI format serves the purpose of instructing a Pinot server to >> find >> > and download the segment from a peer server. Controller writes it to >> Helix >> > in case of segment upload failure or no deep store configured at all. We >> > proposed the following format options and want to hear your feedback: >> > >> > 1. peer:///segmentName; (my preference) >> > 2. simply an empty string *''* >> > >> > Both are in essence specially markers to indicate that the segment is >> not >> > found in deep store and servers have to download them from peer servers. >> > (1) has the benefit of better readability than (2) for debugging >> purposes. >> > >> > Please let me know what you think. >> > >> > Ting Chen >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org >> For additional commands, e-mail: dev-h...@pinot.apache.org >> >>
