This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/setup-python-5
in repository https://gitbox.apache.org/repos/asf/systemds.git
omit ba68daf5e0 Bump actions/setup-python from 4 to 5
add c842072446 [MINOR] Bug fixes
add f1f37e724c [MINOR] Update syntax and deprecated in docker
add b41eccbdd1 [MINOR] C++ Build parallel
add 9781e1069b [MINOR] 100% test coverage of Dense-Sparse conversion of
Matrices
add 70fec49b27 [MINOR] LOG4j test ignore native support of HDFS
add 41db04537d [MINOR] Add Federated Timeouts
add a2aea092a8 [SYSTEMDS-3659] Federated GitHub Actions Fail
add d01c61d7d5 [MINOR] Performance improvement of dist
add 7f795aebd8 Bump actions/upload-artifact from 3 to 4 (#1962)
add 4b0133f4f3 Bump actions/download-artifact from 3 to 4 (#1963)
add 5898293b8d [SYSTEMDS-3660] GPU cache eviction operator and related
rewrite This patch introduces a new operator, _evict, to clean up the free
pointer cached in the lineage cache. A shift in the allocation pattern leads to
large eviction overhead and memory fragmentation. To address that, we
speculatively clear a fraction of the free pointers. Currently, we place a
_evict before every mini-batch processing.
add 96c620674f [MINOR] Write compressed test fix
add e6b54129f3 [MINOR] Reduce Epochs PararmservTest
add 1c26e2d299 [SYSTEMDS-3636] Improved ultra-sparse TSMM left w/ sparse
output
add 3b48c4ae5b [SYSTEMDS-3545] Linearized Img Sample Shear & Rotate
add 246eea9784 [MINOR] Uncompressed ColGroup Outer TSMM
add d9e6767adf Bump actions/setup-python from 4 to 5
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ba68daf5e0)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/setup-python-5 (d9e6767adf)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/javaTests.yml | 22 +-
.gitignore | 1 +
docker/entrypoint.sh | 2 +-
docker/testsysds.Dockerfile | 3 +-
scripts/builtin/dist.dml | 5 +-
scripts/builtin/img_rotate_linearized.dml | 62 ++
...{dist.dml => img_sample_pairing_linearized.dml} | 32 +-
.../builtin/{dist.dml => img_shear_linearized.dml} | 24 +-
scripts/builtin/img_transform_linearized.dml | 3 -
src/main/cpp/build.sh | 61 +-
docker/entrypoint.sh => src/main/cpp/build_BLAS.sh | 31 +-
docker/entrypoint.sh => src/main/cpp/build_HE.sh | 26 +-
docker/entrypoint.sh => src/main/cpp/build_mkl.sh | 32 +-
.../java/org/apache/sysds/common/Builtins.java | 3 +
src/main/java/org/apache/sysds/common/Types.java | 2 +-
.../apache/sysds/conf/ConfigurationManager.java | 4 +
.../java/org/apache/sysds/hops/OptimizerUtils.java | 6 +
src/main/java/org/apache/sysds/hops/UnaryOp.java | 3 +
.../hops/fedplanner/PrivacyConstraintLoader.java | 14 +-
.../org/apache/sysds/lops/rewrite/LopRewriter.java | 1 +
.../sysds/lops/rewrite/RewriteAddGPUEvictLop.java | 115 ++++
.../compress/colgroup/ColGroupUncompressed.java | 35 +-
.../context/SparkExecutionContext.java | 7 +-
.../federated/FederatedStatistics.java | 8 +-
.../controlprogram/federated/FederationUtils.java | 12 +-
.../monitoring/services/StatisticsService.java | 14 +-
.../runtime/instructions/CPInstructionParser.java | 5 +
.../runtime/instructions/cp/CPInstruction.java | 1 +
.../instructions/cp/EvictCPInstruction.java | 49 ++
.../apache/sysds/runtime/lineage/LineageCache.java | 70 +-
.../sysds/runtime/lineage/LineageCacheConfig.java | 5 +
.../runtime/lineage/LineageCacheEviction.java | 9 +
.../runtime/lineage/LineageCacheStatistics.java | 11 +-
.../runtime/lineage/LineageGPUCacheEviction.java | 8 +-
.../runtime/lineage/LineageSparkCacheEviction.java | 1 +
.../matrix/data/LibMatrixDenseToSparse.java | 20 +-
.../sysds/runtime/matrix/data/LibMatrixMult.java | 117 ++--
.../sysds/runtime/matrix/data/LibMatrixSketch.java | 62 +-
.../matrix/data/LibMatrixSparseToDense.java | 10 +-
.../sysds/runtime/matrix/data/MatrixBlock.java | 21 +-
.../java/org/apache/sysds/utils/Statistics.java | 2 +-
.../org/apache/sysds/test/AutomatedTestBase.java | 74 +-
src/test/java/org/apache/sysds/test/TestUtils.java | 6 +-
.../component/compress/colgroup/ColGroupTest.java | 21 +-
.../sysds/test/component/compress/io/IOTest.java | 28 +-
.../test/component/matrix/DenseAndSparseTest.java | 211 ++++++
.../test/component/matrix/MatrixMultiplyTest.java | 122 +++-
.../FullMatrixMultiplicationTransposeSelfTest.java | 27 +-
.../BuiltinImageSamplePairingLinearizedTest.java | 106 +++
.../primitives/FederatedCovarianceTest.java | 180 -----
.../primitives/FederatedQuantileTest.java | 215 ------
.../primitives/FederatedQuantileWeightsTest.java | 203 ------
.../{ => part1}/FederatedBinaryMatrixTest.java | 73 +-
.../{ => part1}/FederatedBinaryVectorTest.java | 71 +-
.../{ => part1}/FederatedBroadcastTest.java | 46 +-
.../{ => part1}/FederatedCastToFrameTest.java | 59 +-
.../{ => part1}/FederatedCastToMatrixTest.java | 81 +--
.../{ => part1}/FederatedCentralMomentTest.java | 109 +--
.../{ => part1}/FederatedColAggregateTest.java | 149 ++--
.../{ => part1}/FederatedConstructionTest.java | 72 +-
.../{ => part1}/FederatedLeftIndexTest.java | 130 ++--
.../{ => part1}/FederatedMisAlignedTest.java | 134 ++--
.../{ => part2}/FederatedMultiplyTest.java | 72 +-
.../{ => part2}/FederatedNegativeTest.java | 2 +-
.../primitives/{ => part2}/FederatedProdTest.java | 105 +--
.../primitives/part2/FederatedQuantileTest.java | 249 +++++++
.../part2/FederatedQuantileWeightsTest.java | 226 +++++++
.../{ => part2}/FederatedRCBindTest.java | 113 ++--
.../primitives/{ => part2}/FederatedRdiagTest.java | 117 ++--
.../{ => part2}/FederatedRemoveEmptyTest.java | 87 +--
.../{ => part2}/FederatedReplaceTest.java | 101 +--
.../{ => part2}/FederatedReshapeTest.java | 107 ++-
.../primitives/{ => part2}/FederatedRevTest.java | 105 +--
.../{ => part2}/FederatedRightIndexTest.java | 103 +--
.../{ => part2}/FederatedRowIndexTest.java | 101 +--
.../primitives/{ => part3}/FederatedSplitTest.java | 77 ++-
.../{ => part3}/FederatedStatisticsTest.java | 86 +--
.../primitives/{ => part3}/FederatedSumTest.java | 88 +--
.../{ => part3}/FederatedTokenizeTest.java | 101 +--
.../FederatedTransferLocalDataTest.java | 76 ++-
.../primitives/{ => part3}/FederatedTriTest.java | 98 +--
.../FederatedWeightedCrossEntropyTest.java | 104 +--
.../FederatedWeightedDivMatrixMultTest.java | 97 +--
.../{ => part3}/FederatedWeightedSigmoidTest.java | 84 +--
.../FederatedWeightedSquaredLossTest.java | 69 +-
.../FederatedWeightedUnaryMatrixMultTest.java | 69 +-
.../{ => part4}/FederatedLogicalTest.java | 254 ++++---
.../{ => part4}/FederatedRowAggregateTest.java | 135 ++--
.../primitives/part5/FederatedCovarianceTest.java | 193 ++++++
.../{ => part5}/FederatedCtableTest.java | 119 ++--
.../{ => part5}/FederatedFrameMapTest.java | 101 +--
.../{ => part5}/FederatedFullAggregateTest.java | 135 ++--
.../{ => part5}/FederatedFullCumulativeTest.java | 151 +++--
.../{ => part5}/FederatedIfelseTest.java | 121 ++--
.../{ => part5}/FederatedMMChainTest.java | 125 ++--
.../FederatedMatrixScalarOperationsTest.java | 2 +-
.../lineage/GPULineageCacheEvictionTest.java | 16 +-
.../paramserv/ParamservLocalNNAveragingTest.java | 23 +-
.../pipelines/BuiltinImageRotateLinTest.java | 116 ++++
.../pipelines/BuiltinImageShearLinTest.java | 122 ++++
.../pipelines/BuiltinImageTransformLinTest.java | 218 +++---
.../expected/ImageTransformLinRotated.csv | 1 +
.../expected/ImageTransformLinTransformed.csv | 1 -
src/test/resources/log4j.properties | 2 +-
.../functions/builtin/image_rotate_linearized.dml | 29 +-
.../builtin/image_sample_pairing_linearized.dml | 37 +-
.../functions/builtin/image_shear_linearized.dml | 30 +-
.../functions/builtin/image_transform_linearized.R | 1 +
.../functions/lineage/GPUCacheEviction6.dml | 746 +++++++++++++++++++++
109 files changed, 5093 insertions(+), 3058 deletions(-)
create mode 100644 scripts/builtin/img_rotate_linearized.dml
copy scripts/builtin/{dist.dml => img_sample_pairing_linearized.dml} (56%)
copy scripts/builtin/{dist.dml => img_shear_linearized.dml} (58%)
copy docker/entrypoint.sh => src/main/cpp/build_BLAS.sh (54%)
copy docker/entrypoint.sh => src/main/cpp/build_HE.sh (61%)
copy docker/entrypoint.sh => src/main/cpp/build_mkl.sh (56%)
create mode 100644
src/main/java/org/apache/sysds/lops/rewrite/RewriteAddGPUEvictLop.java
create mode 100644
src/main/java/org/apache/sysds/runtime/instructions/cp/EvictCPInstruction.java
create mode 100644
src/test/java/org/apache/sysds/test/component/matrix/DenseAndSparseTest.java
create mode 100644
src/test/java/org/apache/sysds/test/functions/builtin/part1/BuiltinImageSamplePairingLinearizedTest.java
delete mode 100644
src/test/java/org/apache/sysds/test/functions/federated/primitives/FederatedCovarianceTest.java
delete mode 100644
src/test/java/org/apache/sysds/test/functions/federated/primitives/FederatedQuantileTest.java
delete mode 100644
src/test/java/org/apache/sysds/test/functions/federated/primitives/FederatedQuantileWeightsTest.java
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedBinaryMatrixTest.java (71%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedBinaryVectorTest.java (71%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedBroadcastTest.java (73%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedCastToFrameTest.java (70%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedCastToMatrixTest.java (72%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedCentralMomentTest.java (58%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedColAggregateTest.java (66%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedConstructionTest.java (78%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedLeftIndexTest.java (65%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part1}/FederatedMisAlignedTest.java (69%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedMultiplyTest.java (71%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedNegativeTest.java (97%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedProdTest.java (60%)
create mode 100644
src/test/java/org/apache/sysds/test/functions/federated/primitives/part2/FederatedQuantileTest.java
create mode 100644
src/test/java/org/apache/sysds/test/functions/federated/primitives/part2/FederatedQuantileWeightsTest.java
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedRCBindTest.java (55%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedRdiagTest.java (58%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedRemoveEmptyTest.java (64%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedReplaceTest.java (62%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedReshapeTest.java (60%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedRevTest.java (63%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedRightIndexTest.java (73%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part2}/FederatedRowIndexTest.java (62%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedSplitTest.java (70%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedStatisticsTest.java (65%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedSumTest.java (62%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedTokenizeTest.java (61%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedTransferLocalDataTest.java (61%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedTriTest.java (61%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedWeightedCrossEntropyTest.java (61%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedWeightedDivMatrixMultTest.java (75%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedWeightedSigmoidTest.java (74%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedWeightedSquaredLossTest.java (72%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part3}/FederatedWeightedUnaryMatrixMultTest.java (75%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part4}/FederatedLogicalTest.java (62%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part4}/FederatedRowAggregateTest.java (70%)
create mode 100644
src/test/java/org/apache/sysds/test/functions/federated/primitives/part5/FederatedCovarianceTest.java
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedCtableTest.java (70%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedFrameMapTest.java (59%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedFullAggregateTest.java (69%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedFullCumulativeTest.java (60%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedIfelseTest.java (70%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedMMChainTest.java (59%)
rename src/test/java/org/apache/sysds/test/functions/federated/primitives/{ =>
part5}/FederatedMatrixScalarOperationsTest.java (99%)
create mode 100644
src/test/java/org/apache/sysds/test/functions/pipelines/BuiltinImageRotateLinTest.java
create mode 100644
src/test/java/org/apache/sysds/test/functions/pipelines/BuiltinImageShearLinTest.java
create mode 100644 src/test/resources/expected/ImageTransformLinRotated.csv
delete mode 100644 src/test/resources/expected/ImageTransformLinTransformed.csv
copy docker/entrypoint.sh =>
src/test/scripts/functions/builtin/image_rotate_linearized.dml (60%)
mode change 100755 => 100644
copy docker/entrypoint.sh =>
src/test/scripts/functions/builtin/image_sample_pairing_linearized.dml (56%)
mode change 100755 => 100644
copy docker/entrypoint.sh =>
src/test/scripts/functions/builtin/image_shear_linearized.dml (60%)
mode change 100755 => 100644
create mode 100644 src/test/scripts/functions/lineage/GPUCacheEviction6.dml
