I assume you meant “Dataset" record type definition and not "Datatype" and you're right. they are the same and can be elevated one level higher :-).
For everyone's information, here is a description of the problem: In our regular secondary indexes, the structure of the tuple consists of <SecondaryKeys,PrimaryKeys> and the Hyracks jobs get constructed based on this assumption. However, our Metadata secondary indexes don't follow this convention and so we end up with runtime errors whenever they are picked for access. Cheers, Abdullah. On Fri, Aug 14, 2015 at 8:59 AM, Ildar Absalyamov < [email protected]> wrote: > @Abdullah, > OK, now I see that change. IMHO this is a hack, but I will adopt that > approach that as well to submit the code review. However this issue really > needs to be addressed. > > Side question, when you introduced external indexes you have added a field > “ExternalDetails” (which in turn is a nested record) into “Datatype" record > type definition in addition to existing “InternalDetails” field (BTW I > believe either of the fields exists at a particular point in time, maybe be > can use UNION for them?). However both of them have nested field > “GroupName”, which is used by one of the secondary metadata indexes. Is > that the same GroupName in both cases? If yes, can we move this field 1 > level up from {External\Internal}Details to avoid duplication? > > > On Aug 13, 2015, at 20:48, abdullah alamoudi <[email protected]> wrote: > > > > Ildar, > > I remember that I had an issue with the secondary indexes of Metadata > when > > I enabled the prefix search for our indexes. I believe with the current > > structure of the secondary indexes, they are unusable in a typical query > > execution job and so I disabled secondary index access for Metadata > indexes. > > > > I think we should stick to that at the moment at least. I also believe > that > > the point of the secondary indexes for Metadata is to validate ddl > > operations and not to be used for dataset access within Hyracks. > > > > Cheers, > > Abdullah. > > > > On Fri, Aug 14, 2015 at 12:10 AM, Ildar Absalyamov < > > [email protected]> wrote: > > > >> End of this\early next week would be the estimate. > >> > >> 2015-08-13 13:53 GMT-07:00 Mike Carey <[email protected]>: > >> > >>> Okay - cool - future improvements needed, clearly. > >>> Any ETA for the interim solution? > >>> Thx! > >>> Mike > >>> > >>> > >>> On 8/13/15 1:42 PM, Ildar Absalyamov wrote: > >>> > >>>> With respect to metadata change I am fixing the issue with > bootstrapped > >>>> secondary metadata indexes. > >>>> Strictly speaking they are not indexes per se, more similar to > >>>> materialized > >>>> views. > >>>> Writing\reading to\from those indexes is done manually bypassing the > >>>> regular index insert\delete codepath. This is really bad from > dogfooding > >>>> perspective, because it breaks the type validation logic when > Select\INL > >>>> Join rewrite rules are trying to fire. > >>>> However I have an idea how to use them without breaking the > validation > >>>> logic. > >>>> > >>>> 2015-08-13 13:11 GMT-07:00 Jochen Wiedmann <[email protected] > >: > >>>> > >>>> On Thu, Aug 13, 2015 at 10:08 PM, Mike Carey <[email protected]> > wrote: > >>>>> > >>>>>> So where are we on feeds and metadata - the release blockers? > >>>>>> (@Raman? Feed tutorial and API key? @Ildar? Your changes?) > >>>>>> > >>>>> I'd suggest to complete the "Verify distribution rights" section on > >>>>> http://incubator.apache.org/projects/asterixdb.html, before cutting > a > >>>>> release. > >>>>> > >>>>> Jochen > >>>>> > >>>>> > >>>>> -- > >>>>> Any world that can produce the Taj Mahal, William Shakespeare, > >>>>> and Stripe toothpaste can't be all bad. (C.R. MacNamara, One Two > Three) > >>>>> > >>>>> > >>>> > >>>> > >>> > >> > >> > >> -- > >> Best regards, > >> Ildar > >> > > > > > > > > -- > > Amoudi, Abdullah. > > Best regards, > Ildar > > -- Amoudi, Abdullah.
