So, we do the "project" in the RegionServer... Sounds like a winner. Go for it!
This brings me back to the point we discussed in the MDAM thread though. An Executor-in-RegionServer would be able to do this too. I'm wondering if we can build an Executor-in-RegionServer architecture on top of the various filters and coprocessors available in the RegionServer today. Well, really, the question I'm asking is: Is the current direction of filters and coprocessors a series of one-offs or do they fit together in a coherent way that we can make as powerful as the former Executor-in-DP2 architecture of SQL/MX? -----Original Message----- From: Eric Owhadi [mailto:[email protected]] Sent: Tuesday, March 29, 2016 8:57 AM To: [email protected] Subject: idea for performance improvement with alignFormat Good morning Trafodioneers, Align format is great from a storage, write and “sometime” read performance. The “sometime”, is what I think we could address. In alignFormat, if you do a select *, then yes, the performance will be better than regular format. But if you do a select a, on a wide row with many columns, alignFormat will bring back the full set of columns, like if it was a select *, when regular format would cherry pick. We could implement an HBase custom filter, using the transformCell() step to create a returned row in a new custom format (like aligned format but with only the requested columns), aware of alignFormat, that will pick inside an AlignFormat cell, only the column values requested. Here is the description of transformCell step of the Custom Filter framework. transformCell() : Once the cell has passed the check and is available, the transform call allows the filter to modify the cell, before it is added to the resulting row. Thought? Eric
