> On Aug 13, 2015, at 23:07, abdullah alamoudi <[email protected]> wrote: > > I assume you meant “Dataset" record type definition and not "Datatype" and > you're right. they are the same and can be elevated one level higher :-).
OK, I will move it as a part of this change in this case. > > For everyone's information, here is a description of the problem: > In our regular secondary indexes, the structure of the tuple consists of > <SecondaryKeys,PrimaryKeys> and the Hyracks jobs get constructed based on > this assumption. However, our Metadata secondary indexes don't follow this > convention and so we end up with runtime errors whenever they are picked > for access. Correct, instead these indexes construct tuple in a form <SecondaryKey, StuffThatReallyNeeded>, which allows them to avoid probing the primary for the needed information, returning via index-only lookup. > > Cheers, > Abdullah. > > On Fri, Aug 14, 2015 at 8:59 AM, Ildar Absalyamov < > [email protected]> wrote: > >> @Abdullah, >> OK, now I see that change. IMHO this is a hack, but I will adopt that >> approach that as well to submit the code review. However this issue really >> needs to be addressed. >> >> Side question, when you introduced external indexes you have added a field >> “ExternalDetails” (which in turn is a nested record) into “Datatype" record >> type definition in addition to existing “InternalDetails” field (BTW I >> believe either of the fields exists at a particular point in time, maybe be >> can use UNION for them?). However both of them have nested field >> “GroupName”, which is used by one of the secondary metadata indexes. Is >> that the same GroupName in both cases? If yes, can we move this field 1 >> level up from {External\Internal}Details to avoid duplication? >> >>> On Aug 13, 2015, at 20:48, abdullah alamoudi <[email protected]> wrote: >>> >>> Ildar, >>> I remember that I had an issue with the secondary indexes of Metadata >> when >>> I enabled the prefix search for our indexes. I believe with the current >>> structure of the secondary indexes, they are unusable in a typical query >>> execution job and so I disabled secondary index access for Metadata >> indexes. >>> >>> I think we should stick to that at the moment at least. I also believe >> that >>> the point of the secondary indexes for Metadata is to validate ddl >>> operations and not to be used for dataset access within Hyracks. >>> >>> Cheers, >>> Abdullah. >>> >>> On Fri, Aug 14, 2015 at 12:10 AM, Ildar Absalyamov < >>> [email protected]> wrote: >>> >>>> End of this\early next week would be the estimate. >>>> >>>> 2015-08-13 13:53 GMT-07:00 Mike Carey <[email protected]>: >>>> >>>>> Okay - cool - future improvements needed, clearly. >>>>> Any ETA for the interim solution? >>>>> Thx! >>>>> Mike >>>>> >>>>> >>>>> On 8/13/15 1:42 PM, Ildar Absalyamov wrote: >>>>> >>>>>> With respect to metadata change I am fixing the issue with >> bootstrapped >>>>>> secondary metadata indexes. >>>>>> Strictly speaking they are not indexes per se, more similar to >>>>>> materialized >>>>>> views. >>>>>> Writing\reading to\from those indexes is done manually bypassing the >>>>>> regular index insert\delete codepath. This is really bad from >> dogfooding >>>>>> perspective, because it breaks the type validation logic when >> Select\INL >>>>>> Join rewrite rules are trying to fire. >>>>>> However I have an idea how to use them without breaking the >> validation >>>>>> logic. >>>>>> >>>>>> 2015-08-13 13:11 GMT-07:00 Jochen Wiedmann <[email protected] >>> : >>>>>> >>>>>> On Thu, Aug 13, 2015 at 10:08 PM, Mike Carey <[email protected]> >> wrote: >>>>>>> >>>>>>>> So where are we on feeds and metadata - the release blockers? >>>>>>>> (@Raman? Feed tutorial and API key? @Ildar? Your changes?) >>>>>>>> >>>>>>> I'd suggest to complete the "Verify distribution rights" section on >>>>>>> http://incubator.apache.org/projects/asterixdb.html, before cutting >> a >>>>>>> release. >>>>>>> >>>>>>> Jochen >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Any world that can produce the Taj Mahal, William Shakespeare, >>>>>>> and Stripe toothpaste can't be all bad. (C.R. MacNamara, One Two >> Three) >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Ildar >>>> >>> >>> >>> >>> -- >>> Amoudi, Abdullah. >> >> Best regards, >> Ildar >> >> > > > -- > Amoudi, Abdullah. Best regards, Ildar
