Re: [druid-dev] Load rule doesn't honor intervals properly

[Gian Merlino]

Hi Pala,

That sounds like a bug to me - a patch would be welcome!

Btw, since we are trying to migrate the dev mailing list to Apache, please
cross post this sort of thing with d...@druid.incubator.apache.org, or even
only post to that list.

Gian

On Thu, Mar 29, 2018 at 5:43 PM, 'Pala Muthiah' via Druid Development <
druid-developm...@googlegroups.com> wrote:

> Hello folks,
>
> Anybody have insight on the below? Curious to know if there would be
> unforeseen side effects if we count even partial overlap as valid.
>
> On Mon, Mar 19, 2018 at 10:54 AM, pala.muthiah via Druid Development <
> druid-developm...@googlegroups.com> wrote:
>
>> Hi,
>>
>> In our deployment, we enabled background segment merging and found that
>> some of the data within the load period was actually getting dropped.
>>
>> My suspicion was that when a merged segment only partially overlaps with
>> a period (e.g: Rule says keep data from Jan 1st onwards, and i have a
>> segment that spans Dec 25th - Jan 2nd), for correctness that segment should
>> be kept but current implementation seems to drop it.
>>
>> I checked the code and found indeed Rules.eligibleForLoad() only keeps
>> segments that overlap fully.
>>
>> Is this a bug, or is there other reason behind this? In our case, we do
>> have data sources that are highly aggregated and therefore a single segment
>> could span a month for example.
>>
>> I can submit a patch but wanted to get proper context.
>>
>>
>> Thanks,
>> pala
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Druid Development" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/druid-development/QYMhjGup2RI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> druid-development+unsubscr...@googlegroups.com.
>> To post to this group, send email to druid-developm...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/druid-development/d8e3df1d-699e-4747-a681-ebba91327388%4
>> 0googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/druid-development/CALxr%3D2Wo9KKTYmFjbusc%2BsU0dt7K4k%3D
> P4JQX1T0sFe%3DLbBHXAA%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

Re: [druid-dev] Questions about Public API

[Gian Merlino]

Hi Jihoon,

The javadoc for @PublicApi explains the intent. They can change, but not in
a minor release. There should also
be some consensus around changing them. Once we are 1.0, then we should
only change PublicApis when going
to 2.0.

Btw, since we are trying to migrate the dev mailing list to Apache, please
cross post this sort of thing with
d...@druid.incubator.apache.org, or even only post to that list.

>From the javadoc:

> Signifies that the annotated entity is a public API for extension
authors. Public APIs may change in breaking ways
> only between major Druid release lines (e.g. 0.10.x -> 0.11.0), but
otherwise must remain stable. Public APIs may
> change at any time in non-breaking ways, however, such as by adding new
fields, methods, or constructors.
>
> Note that interfaces annotated with {@code PublicApi} but not with {@link
ExtensionPoint} are not meant to be
> subclassed in extensions. In this case, the annotation simply signifies
that the interface is stable for callers.
> In particular, since it is not meant to be subclassed, new non-default
methods may be added to an interface and
> new abstract methods may be added to a class.
>
> If a class or interface is annotated, then all public and protected
fields, methods, and constructors that class
> or interface are considered stable in this sense. If a class is not
annotated, but an individual field, method, or
> constructor is annotated, then only that particular field, method, or
constructor is considered a public API.
>
> Classes, fields, method, and constructors _not_ annotated with {@code
@PublicApi} may be modified or removed
> in any Druid release, unless they are annotated with {@link
ExtensionPoint} (which implies they are a public API
> as well).

Gian

On Thu, Mar 29, 2018 at 5:45 PM, Jihoon Son  wrote:

> Hi folks,
>
> I wonder what's the exact meaning of the 'PublicApi' annotation. From
> https://github.com/druid-io/druid/pull/4433,
>
> > @PublicApi which signifies something you're not meant to subclass, but
> that you can use for implementation.
>
> I can also see some methods can't be deleted because they are in some
> classes annotated with @PublicApi. Here is an example in TaskRunner.
>
> /**
>  * Start the state of the runner.
>  *
>  * This method is unused, but TaskRunner is {@link PublicApi}, so we
> cannot remove it.
>  */
> @SuppressWarnings("unused")
> void start();
>
> Does this mean @PublicApi classes must change in a backward-compatible
> way? Or can we change in a non-compatible way and call out when we release?
>
> If this is not defined yet, it would be good to start a discussion on this.
>
> Best,
> Jihoon
>
> --
> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to druid-development+unsubscr...@googlegroups.com.
> To post to this group, send email to druid-developm...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/druid-development/CACZfFK4_xEG-6JSdqARh3SRRVWCgoSVpK3Z48rqUhO
> TuqS-x3w%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>