[
https://issues.apache.org/jira/browse/TAJO-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14121315#comment-14121315
]
ASF GitHub Bot commented on TAJO-907:
-------------------------------------
GitHub user hyunsik opened a pull request:
https://github.com/apache/tajo/pull/133
TAJO-907: Implement off-heap tuple block and zero-copy tuple.
Hi folks,
One week ago, I contributed run-time code generation for computation
efficiency and reduction of objection creations. This work has also the similar
purpose.
This patch includes off-heap memory and row block (a list of tuple) using
off-heap memory container. Also, it provides various utility classes for them.
In detail, this patch includes lots of changes.
* OffHeapMemory class
* OffHeapRowBlock class
* ZeroCopyTuple, which just points to actual row record stored in
OffHeapRowBlock.
* RowWriter interface
* RowBlockReader interface and OffHeapBlockReader
* TupleBuilder and BaseTupleBuilder
* HeapTuple, which keeps fields in byte array instead of Datum array.
* HeapTuple internally UnSafe to read and store field values in byte
array
* TupleComparatorCompiler, which is run-time code compiler for
TupleComparator
* It reduces branches even though tuple comparator actually should
consider complex logic with all null-first/last, ascending, and descending
orders.
* Others
Later, I'll replace current VTuple by HeapTuple or ZeroCopyTuple. Also, I'm
planning the improvement to replace current pull-iterator model of execution
engine by the push-based block iterator model. I'll describe it later in
another jira issue.
Thanks,
Hyunsik
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hyunsik/tajo TAJO-907
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/133.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #133
----
commit ded746bc691fa68b06019909c740a9fe945ab0a5
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-23T17:38:27Z
initial work for direct memory tuple.
commit c9414c012fdd388fb51e0a5bd81d2dca31930188
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-24T15:43:34Z
Implemented DirectRowBlock and its unit tests.
commit 943a29df75f9d583455002be4f65696052950b4d
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-24T16:11:52Z
Added benchmark test.
commit b6b3b4ffe39d4b2e5f75fb6749e8868751f51f17
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-24T16:31:22Z
Improved interfaces and refactored class hierarchy.
commit b424269b3c6e4728bea1c1e5248a30af611a6ead
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-24T23:29:58Z
added sort benchmark and add get() method to UnSafeTuple.
commit ef07dc9a93e66650f532e5fdc7753918f7bbf14f
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T04:23:59Z
Add timestamp data type support to RowOrientedRowBlock.
commit cb94b74a7f79cb074669f09252f8069fd8933db8
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T05:50:18Z
Cleanup unit tests and benchmarks.
commit 8d02ba7a1e2d0f4a8edb1cbc7910abc11e6e386d
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T06:29:35Z
Removed duplicated method cleanByteBuffer() from StorageUtil.
commit d783d65e9db97883f793ad24d7daabd36a514e3f
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T07:43:09Z
Refactored UnSafeTuple.
commit ab241625982296ed11813f0e9d95147fa4dddf21
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T11:25:33Z
Refactored SchemaUtil to tajo-catalog, improved DirectRawFileWriter.
commit 75949fa1455e75347b936bcce31c61a726581942
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T16:33:37Z
Implemented zero copy and zero (de)serialization file format.
commit 43d45be517e3a9b3c4d9c363a2e7e05e0def0acd
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-25T17:01:36Z
Implemented legacy addTuple method.
commit 6ed1df9818a79ac71f78bc9beb1083abbc50f02b
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-26T01:20:48Z
Implemented VTuple to UnSafeTuple encoder.
commit 72179ef4083c66ebe251482dd50913ef0ca19107
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-26T02:14:11Z
Improved RowOrientedRowBlock to take exiting Tuple.
commit d78d207455a697ca47c44ed6e4200c6b0b9ad07f
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-26T06:47:59Z
Fixed many bugs of DirectRawFileScanner.
commit 6b8f0d4484f3d546e7cff6dddc28dcc117171b59
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-26T07:43:07Z
External sort works well, but random access causes performance degradation.
commit afda01e3cae8ac88938d1a504c9faca0aca29845
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-27T14:49:08Z
Fixed many bugs and refactored tests.
commit a0859fd47b7e9d8c15e30a80c2d27e7d81eccfb3
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-28T04:07:33Z
Fixed reset bugs and resource leak in PairWiseMerger.
commit df91243afa6710de66a0386291969252bd0702f5
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-29T07:31:30Z
Implemented the basic of tuple comparator generation.
commit 31e418b2fbf2df2f1e192d34df22139f8c0a684f
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-29T14:46:09Z
Implemented TupleComparatorCompiler and UnSafeTupleTextComparator.
commit 738dd06421cd159c6c24390f29da94bb0a400602
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-29T15:07:16Z
Improving Null last and Null first of TupleComparatorCompiler.
commit 2930bd98d6011bcd45d3a2032c672e28594f3515
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-29T15:13:11Z
Renamed UnSafeComparator.
commit 06675b7cb1934563585127c62c5457459254a49b
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-29T15:19:10Z
Fixed a bug of UnSafeTupleTextComparator.
commit 1f0fa60929c2ae9858eb3f0bd116a9f2855f287e
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-29T16:33:46Z
Improved null first and last of comparator.
commit 8ab577460c2d6fa4983014963f574611014f9d51
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-31T06:50:07Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into
DirectMemTuple
commit 631dd21b54f9994736a7d7ca97f67717483567fe
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-31T12:53:51Z
Fixed offset mismatch bug.
commit e4e6fa3b7a8c7e818294e8fcaf20baba5f9cd9b5
Author: Hyunsik Choi <[email protected]>
Date: 2014-09-01T01:22:23Z
Passed all unit tests.
commit fcc59d38cb99afd97a2bec03f4c39344e416df00
Author: Hyunsik Choi <[email protected]>
Date: 2014-09-01T06:45:48Z
Improved TupleComparatorCompiler.
commit 6c9084a1f8e88fb22ed226bc81884bcf452b5439
Author: Hyunsik Choi <[email protected]>
Date: 2014-09-01T06:55:08Z
renamed UnSafeComparer.
commit 24200be8d2165eeb8c505d9fe3df305db3cce5b3
Author: Hyunsik Choi <[email protected]>
Date: 2014-09-01T12:01:14Z
Fixed isNull and many bugs related to ordering.
----
> Implement off-heap tuple block and zero-copy tuple
> --------------------------------------------------
>
> Key: TAJO-907
> URL: https://issues.apache.org/jira/browse/TAJO-907
> Project: Tajo
> Issue Type: Sub-task
> Components: storage
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.9.0
>
>
> The main purpose of this issue is to implement new Tuple structure using
> direct memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
