This is an automated email from the ASF dual-hosted git repository.
baunsgaard pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/systemds.git
from 1e3dff7600 [SYSTEMDS-3452] Fix warnings scalar reads, cleanup csv
default handling
new 4cebe8ed34 [SYSTEMDS-3444] Spark Write compressed format
new 07dd88aba3 [SYSTEMDS-3446] DDC Append
new 02a6a6c798 [SYSTEMDS-3450] DDCFOR Append
new 8e4f06a89a [SYSTEMDS-3448] Uncompressed Append
new 678b28a221 [MINOR] Disable Hadoop Write compression
new 657a1bacca [SYSTEMDS-3445] N Way combining Interface
new ead3c5e9fb [SYSTEMDS-3445] N Way combining Mapping
new 451e84c538 [MINOR] Fix rand instruction to return empty matrix if
sparsity == 0.0
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/sysds/parser/DMLTranslator.java | 6 +-
.../org/apache/sysds/parser/DataExpression.java | 2 +-
.../sysds/runtime/compress/colgroup/AColGroup.java | 34 ++-
.../compress/colgroup/AColGroupCompressed.java | 7 +
.../runtime/compress/colgroup/AMapToDataGroup.java | 13 +-
.../runtime/compress/colgroup/ColGroupConst.java | 9 +
.../runtime/compress/colgroup/ColGroupDDC.java | 51 ++++-
.../runtime/compress/colgroup/ColGroupDDCFOR.java | 14 +-
.../runtime/compress/colgroup/ColGroupEmpty.java | 8 +
.../colgroup/ColGroupLinearFunctional.java | 5 +
.../runtime/compress/colgroup/ColGroupOLE.java | 7 +-
.../runtime/compress/colgroup/ColGroupRLE.java | 7 +-
.../runtime/compress/colgroup/ColGroupSDC.java | 15 +-
.../runtime/compress/colgroup/ColGroupSDCFOR.java | 7 +-
.../compress/colgroup/ColGroupSDCSingle.java | 7 +-
.../compress/colgroup/ColGroupSDCSingleZeros.java | 7 +-
.../compress/colgroup/ColGroupSDCZeros.java | 7 +-
.../compress/colgroup/ColGroupUncompressed.java | 15 ++
.../compress/colgroup/dictionary/Dictionary.java | 1 -
.../compress/colgroup/mapping/AMapToData.java | 6 +
.../compress/colgroup/mapping/MapToBit.java | 63 +++++-
.../compress/colgroup/mapping/MapToByte.java | 48 ++++-
.../compress/colgroup/mapping/MapToChar.java | 54 ++++-
.../compress/colgroup/mapping/MapToCharPByte.java | 30 +++
.../compress/colgroup/mapping/MapToInt.java | 39 ++++
.../compress/colgroup/mapping/MapToZero.java | 18 ++
.../runtime/compress/colgroup/offset/AOffset.java | 32 +--
.../io/CompressWrap.java} | 18 +-
.../runtime/compress/io/CompressedWriteBlock.java | 26 ++-
.../runtime/compress/io/ReaderCompressed.java | 20 +-
.../runtime/compress/io/WriterCompressed.java | 75 +++++--
.../sysds/runtime/compress/lib/CLALibCombine.java | 85 ++++----
.../runtime/compress/lib/CLALibDecompress.java | 20 +-
.../sysds/runtime/compress/lib/CLALibSlice.java | 19 +-
.../runtime/compress/utils/DoubleCountHashMap.java | 32 +--
.../instructions/cp/DataGenCPInstruction.java | 7 +-
.../instructions/spark/WriteSPInstruction.java | 24 ++-
.../functions/ExtractBlockForBinaryReblock.java | 19 +-
.../spark/utils/RDDAggregateUtils.java | 23 +-
.../sysds/runtime/matrix/data/MatrixBlock.java | 46 ++++
.../compress/colgroup/ColGroupNegativeTests.java | 12 ++
.../component/compress/combine/CombineTest.java | 30 ++-
...OTestUtils.java => IOCompressionTestUtils.java} | 6 +-
.../sysds/test/component/compress/io/IOEmpty.java | 14 +-
.../sysds/test/component/compress/io/IOSpark.java | 240 ++++++++++++++++++---
.../sysds/test/component/compress/io/IOTest.java | 33 +--
.../component/compress/mapping/MappingTests.java | 96 ++++++++-
.../io/compressed/WriteCompressedTest.java | 95 ++++++++
.../compressed/WriteCompressedTest.dml} | 13 +-
49 files changed, 1205 insertions(+), 260 deletions(-)
copy
scripts/monitoring/src/app/modules/dashboard/dialog-dashboard/dialog-dashboard.component.scss
=>
src/main/java/org/apache/sysds/runtime/compress/colgroup/AMapToDataGroup.java
(80%)
copy
src/main/java/org/apache/sysds/runtime/{instructions/spark/functions/FilterNonEmptyBlocksFunction2.java
=> compress/io/CompressWrap.java} (73%)
rename
src/test/java/org/apache/sysds/test/component/compress/io/{IOTestUtils.java =>
IOCompressionTestUtils.java} (94%)
create mode 100644
src/test/java/org/apache/sysds/test/functions/io/compressed/WriteCompressedTest.java
copy src/test/scripts/functions/{builtin/lmDS.dml =>
io/compressed/WriteCompressedTest.dml} (90%)
