This is an automated email from the ASF dual-hosted git repository.
guanmingchiu pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/mahout.git
from ac5e58017 fix: parameter validation to detect partially bound
parameters (#792)
new 33f4c1146 [QDP] Initialize QDP, memory management, and DLPack protocol
(#646)
new b631319f1 [QDP] Add Python bindings for QDP using PyO3 (#649)
new 27778fc91 [QDP] Update pre-commit to add license headers for Rust
(#660)
new df97baf7f [QDP] add unit test for qdp-core (#659)
new 2c1192749 [QDP] add NVTX to trace validity (#650)
new 6585ae14a [QDP] add integration test and introduction (#666)
new e3bc0c12e [QDP] improve memory initialization (#668)
new 479449941 [QDP] Add pre-processing module (#673)
new 15049ca31 [QDP] Add tests for pre-processing (#676)
new 4fb7065f0 [QDP] make dataflow async and improve amplitudeEncoder
(#675)
new b0f1977ff [QDP] Update pre-commit to add license headers for Cuda
(#684)
new 7174f3899 [QDP] Integrate Apache Arrow and Parquet for data processing
(#680)
new 5416fce56 [QDP] add vanilla gpu kernel (#677)
new cfb94fb86 [QDP] add DataLoader Test (#687)
new a7a1fd477 [QDP] Gracefully handles OOM (#688)
new 58cf4952a [QDP] Add fidelity test (#694)
new 51f942ada [QDP] improve amplitudeEncoders for less copy memory
allocations (#690)
new d7f524f4d [QDP] update uv usage in qdp-python (#698)
new 230abcd3e [QDP] [Benchmark] Add a End-to-End benchmark to compare with
speed (#697)
new 3175ef931 [QDP] add batch kernel support (#700)
new c27f98e00 [QDP] Add Arrow IPC support (#704)
new 49dc1d9de [QDP] Add data loader benchmark (#701)
new d67cbe902 [QDP] GPU/Cuda kernel Optimizations (#706)
new 213b702be [QDP] improve e2e benchmark (#713)
new cf53ba175 [QDP] Set float32 to Default and leave float64 as Optional
(#712)
new 7cc09800a [QDP] support turing architecture GPU (#731)
new 013972632 [QDP] Choose between different frameworks in benchmark (#741)
new 57f83b020 [QDP] Enable 3060ti 3090ti cuda build sm86 (#737)
new 36b1872d0 [QDP] improve memory management (#708)
new 6b7477b99 [docs][qdp] Add qdp development guide (#732)
new 1a3628fa6 [QDP] fix test bug (#734)
new 89d2a36db [QDP] Fix DLPack device_id hardcoding (#745)
new dab5577cd [QDP] DLPack shape/strides: Support batch 2D tensor (#747)
new c79f06a3b [QDP] Add launch_amplitude_encode_f32 function (#739)
new 4ced8fff3 [QDP][Housekeeping] Clean structure and make commands (#756)
new 3c174ed2f [QDP] Add colab benchmark example (#755)
new a515a8639 [QDP] add Rust Linter to pre-commit hook (#758)
new d7650aa32 [QDP] Update notebooks dir and link (#762)
new 9780d955a [QDP] refactor: introduce 2 traits for flexible io type w/
example (#753)
new 333b3e716 [QDP] Fix Route Error in Makefile (#775)
new 6597d07d7 [QDP] Add Workflow dispatch for manual triggering (#767)
new d317c34c1 [QDP] Rename benchmark throughput script and refresh docs
(#776)
new 4d3fc4b31 [QDP] Pre commit rust followup (#766)
new 4b34a6363 MAHOUT-725: [QDP] PyTorch Tensor Detection and CPU Path
(#763)
new c987a4372 [QDP] Double-buffered pinned I/O pipeline and faster Parquet
decode (#751)
new aeead5a89 [QDP] Numpy IO support (#777)
new 09ca3bf7d [QDP] add scaling test (Latency vs. Qubits) (#778)
new cc326daeb [QDP] [Numpy] add path exist check (#793)
new 509c28f65 [QDP] Fix benchmark to use batch encoding (#794)
new 5b6055a47 [QDP] Update benchmark_throughput to batch encoding (#796)
The 50 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.devcontainer/devcontainer.json | 29 +
.devcontainer/setup.sh | 32 +
.github/workflows/notebook-testing.yml | 1 +
.github/workflows/pre-commit.yml | 5 +-
.github/workflows/python-bindings.yml | 35 +
.github/workflows/python-testing.yml | 1 +
.pre-commit-config.yaml | 36 +-
qdp/Cargo.lock | 1693 ++++++++++++++++++++
qdp/Cargo.toml | 41 +
qdp/DEVELOPMENT.md | 208 +++
qdp/Makefile | 69 +
qdp/docs/observability/NVTX_USAGE.md | 148 ++
qdp/docs/readers/README.md | 220 +++
qdp/docs/test/README.md | 54 +
docs/getting_started.md => qdp/qdp-core/.gitignore | 0
qdp/qdp-core/Cargo.toml | 22 +
qdp/qdp-core/examples/dataloader_throughput.rs | 145 ++
qdp/qdp-core/examples/nvtx_profile.rs | 80 +
qdp/qdp-core/src/dlpack.rs | 188 +++
qdp/qdp-core/src/error.rs | 45 +
qdp/qdp-core/src/gpu/buffer_pool.rs | 151 ++
qdp/qdp-core/src/gpu/cuda_ffi.rs | 50 +
qdp/qdp-core/src/gpu/encodings/amplitude.rs | 473 ++++++
qdp/qdp-core/src/gpu/encodings/angle.rs | 50 +
qdp/qdp-core/src/gpu/encodings/basis.rs | 49 +
qdp/qdp-core/src/gpu/encodings/mod.rs | 84 +
qdp/qdp-core/src/gpu/memory.rs | 536 +++++++
qdp/qdp-core/src/gpu/mod.rs | 33 +
qdp/qdp-core/src/gpu/pipeline.rs | 344 ++++
qdp/qdp-core/src/io.rs | 269 ++++
qdp/qdp-core/src/lib.rs | 495 ++++++
qdp/qdp-core/src/preprocessing.rs | 150 ++
qdp/qdp-core/src/profiling.rs | 77 +
qdp/qdp-core/src/reader.rs | 102 ++
qdp/qdp-core/src/readers/arrow_ipc.rs | 191 +++
qdp/qdp-core/src/readers/mod.rs | 32 +
qdp/qdp-core/src/readers/numpy.rs | 274 ++++
qdp/qdp-core/src/readers/parquet.rs | 537 +++++++
qdp/qdp-core/tests/api_workflow.rs | 295 ++++
qdp/qdp-core/tests/arrow_ipc_io.rs | 226 +++
qdp/qdp-core/tests/common/mod.rs | 21 +
qdp/qdp-core/tests/memory_safety.rs | 201 +++
qdp/qdp-core/tests/numpy.rs | 143 ++
qdp/qdp-core/tests/parquet_io.rs | 161 ++
qdp/qdp-core/tests/preprocessing.rs | 107 ++
qdp/qdp-core/tests/validation.rs | 227 +++
qdp/qdp-kernels/Cargo.toml | 14 +
qdp/qdp-kernels/build.rs | 85 +
qdp/qdp-kernels/src/amplitude.cu | 555 +++++++
qdp/qdp-kernels/src/lib.rs | 200 +++
qdp/qdp-kernels/tests/amplitude_encode.rs | 639 ++++++++
qdp/qdp-python/.gitignore | 72 +
qdp/qdp-python/Cargo.toml | 17 +
qdp/qdp-python/README.md | 65 +
qdp/qdp-python/benchmark/README.md | 108 ++
qdp/qdp-python/benchmark/benchmark_e2e.py | 501 ++++++
qdp/qdp-python/benchmark/benchmark_latency.md | 80 +
qdp/qdp-python/benchmark/benchmark_latency.py | 367 +++++
qdp/qdp-python/benchmark/benchmark_numpy_io.py | 293 ++++
qdp/qdp-python/benchmark/benchmark_throughput.md | 77 +
qdp/qdp-python/benchmark/benchmark_throughput.py | 324 ++++
.../benchmark/notebooks/mahout_benchmark.ipynb | 195 +++
qdp/qdp-python/pyproject.toml | 45 +
qdp/qdp-python/src/lib.rs | 434 +++++
qdp/qdp-python/tests/test_bindings.py | 191 +++
qdp/qdp-python/tests/test_high_fidelity.py | 254 +++
qdp/qdp-python/tests/test_numpy.py | 145 ++
qdp/qdp-python/uv.lock | 1577 ++++++++++++++++++
68 files changed, 14565 insertions(+), 3 deletions(-)
create mode 100644 .devcontainer/devcontainer.json
create mode 100644 .devcontainer/setup.sh
create mode 100644 .github/workflows/python-bindings.yml
create mode 100644 qdp/Cargo.lock
create mode 100644 qdp/Cargo.toml
create mode 100644 qdp/DEVELOPMENT.md
create mode 100644 qdp/Makefile
create mode 100644 qdp/docs/observability/NVTX_USAGE.md
create mode 100644 qdp/docs/readers/README.md
create mode 100644 qdp/docs/test/README.md
copy docs/getting_started.md => qdp/qdp-core/.gitignore (100%)
create mode 100644 qdp/qdp-core/Cargo.toml
create mode 100644 qdp/qdp-core/examples/dataloader_throughput.rs
create mode 100644 qdp/qdp-core/examples/nvtx_profile.rs
create mode 100644 qdp/qdp-core/src/dlpack.rs
create mode 100644 qdp/qdp-core/src/error.rs
create mode 100644 qdp/qdp-core/src/gpu/buffer_pool.rs
create mode 100644 qdp/qdp-core/src/gpu/cuda_ffi.rs
create mode 100644 qdp/qdp-core/src/gpu/encodings/amplitude.rs
create mode 100644 qdp/qdp-core/src/gpu/encodings/angle.rs
create mode 100644 qdp/qdp-core/src/gpu/encodings/basis.rs
create mode 100644 qdp/qdp-core/src/gpu/encodings/mod.rs
create mode 100644 qdp/qdp-core/src/gpu/memory.rs
create mode 100644 qdp/qdp-core/src/gpu/mod.rs
create mode 100644 qdp/qdp-core/src/gpu/pipeline.rs
create mode 100644 qdp/qdp-core/src/io.rs
create mode 100644 qdp/qdp-core/src/lib.rs
create mode 100644 qdp/qdp-core/src/preprocessing.rs
create mode 100644 qdp/qdp-core/src/profiling.rs
create mode 100644 qdp/qdp-core/src/reader.rs
create mode 100644 qdp/qdp-core/src/readers/arrow_ipc.rs
create mode 100644 qdp/qdp-core/src/readers/mod.rs
create mode 100644 qdp/qdp-core/src/readers/numpy.rs
create mode 100644 qdp/qdp-core/src/readers/parquet.rs
create mode 100644 qdp/qdp-core/tests/api_workflow.rs
create mode 100644 qdp/qdp-core/tests/arrow_ipc_io.rs
create mode 100644 qdp/qdp-core/tests/common/mod.rs
create mode 100644 qdp/qdp-core/tests/memory_safety.rs
create mode 100644 qdp/qdp-core/tests/numpy.rs
create mode 100644 qdp/qdp-core/tests/parquet_io.rs
create mode 100644 qdp/qdp-core/tests/preprocessing.rs
create mode 100644 qdp/qdp-core/tests/validation.rs
create mode 100644 qdp/qdp-kernels/Cargo.toml
create mode 100644 qdp/qdp-kernels/build.rs
create mode 100644 qdp/qdp-kernels/src/amplitude.cu
create mode 100644 qdp/qdp-kernels/src/lib.rs
create mode 100644 qdp/qdp-kernels/tests/amplitude_encode.rs
create mode 100644 qdp/qdp-python/.gitignore
create mode 100644 qdp/qdp-python/Cargo.toml
create mode 100644 qdp/qdp-python/README.md
create mode 100644 qdp/qdp-python/benchmark/README.md
create mode 100644 qdp/qdp-python/benchmark/benchmark_e2e.py
create mode 100644 qdp/qdp-python/benchmark/benchmark_latency.md
create mode 100644 qdp/qdp-python/benchmark/benchmark_latency.py
create mode 100644 qdp/qdp-python/benchmark/benchmark_numpy_io.py
create mode 100644 qdp/qdp-python/benchmark/benchmark_throughput.md
create mode 100644 qdp/qdp-python/benchmark/benchmark_throughput.py
create mode 100644 qdp/qdp-python/benchmark/notebooks/mahout_benchmark.ipynb
create mode 100644 qdp/qdp-python/pyproject.toml
create mode 100644 qdp/qdp-python/src/lib.rs
create mode 100644 qdp/qdp-python/tests/test_bindings.py
create mode 100644 qdp/qdp-python/tests/test_high_fidelity.py
create mode 100644 qdp/qdp-python/tests/test_numpy.py
create mode 100644 qdp/qdp-python/uv.lock
