This is an automated email from the ASF dual-hosted git repository.
silver pushed a change to branch gcs-multipart
in repository https://gitbox.apache.org/repos/asf/incubator-opendal.git
omit 39565833e add new ci to test gcs with default storage class
omit 701200e7e fix(services/gcs): migrate to new multipart impl for
gcs_insert_object_request
add b0beb4398 fix(services/sftp): bump openssh-sftp-client to 0.13.9
(#2831)
add 57e6b18b0 feat: ci for php binding (#2830)
add 4b33fcdec feat: Add create_dir, remove, copy and rename API for
oay-webdav (#2832)
add 2c49350a0 fix(oli): oli commands don't work properly for files in CWD
(#2833)
add f17dc099e feat(oli): oli stat should show path as specified by users
(#2842)
add b76664117 fix(oli): oli commands should not accept invalid URI format
(#2845)
add e2f360473 docs: Add an example for PostgreSQL service (#2847)
add dac7910fe docs: improve php binding documentation (#2843)
add d38027533 feat(services/moka, services/mini-moka): Add scan support
(#2850)
add c532a9e27 chore: Make clippy happy (#2851)
add 053b0b9de fix(bindings/c): Fix an example of the C binding (#2854)
add 62d376ac9 fix(doc): Update instructions for building the C binding in
README.md (#2856)
add 57944b763 feat(oay): impl some method for `WebdavMetaData` (#2857)
add bb51cbe24 RFC-2852: Native Capability (#2852)
add f7486c4ce feat: Implement list with metakey for blocking (#2861)
add d280e87c1 docs: Fix missing link for rust example (#2866)
add a31ad051e feat(services/redis): add redis cluster support (#2858)
add 5384936b0 feat(services/dropbox): read support range (#2848)
add b89a2e329 ci: Trying to fix rocksdb build (#2867)
add ccfd536e9 ci: add reproducibility check (#2863)
add 2be8e6551 docs: Add blog on how opendal read data (#2869)
add 64ddcb6ec refactor: Implement backtrace for Error correctly (#2871)
add cfd4da4d1 feat(layers/logging): Allow users to control print backtrace
or not (#2872)
add 0b9d8c00b docs: Fix missing link to the contribution guide for the
Node.js binding (#2876)
add 18c5a1878 ci(services/postgresql): add docker-compose to simplify the
CI (#2877)
add f4cd502f4 feat: add native & full capability (#2874)
add 2048ee99d fix(oay): add some error handle (#2879)
add 8ea4f225e feat: Implement RFC-2758 Merge Append Into Write (#2880)
add 5afec9183 doc: add 0.39.0 release link to download.md (#2882)
add 5033d0440 doc: add missing release step (#2883)
add 876743e0c RFC-2884: Merge Range Read Into Read (#2884)
add 84fcc61d0 feat(binding/ocaml): Add support for operator reader and
metadata (#2881)
add c588676f5 feat(core): replace field `_pin` with `!Unpin` as argument
(#2886)
add b060af98e ci(service/s3): Add docker-compose-minio file to simplify
the CI (#2887)
add 6f74f551f refactor: Move object_store_opendal to integrations (#2888)
add 864918919 feat: Add retry for Writer::sink operation (#2896)
add 4e1c14cb1 refactor(services/gdrive): prepare for CI (#2892)
add ebc579590 chore: add health check for docker-compose minio (#2899)
add 6f4116579 fix: Set default timeouts for HttpClient (#2895)
add b5f343b8d ci(services/hdfs): Load native lib instead (#2900)
add 2934405ac chore(ci): offload healthcheck logic to docker-compose
config (#2901)
add f84715681 ci(services/rocksdb): Make sure rocksdb lib is loaded (#2902)
add ba7e1168f feat: remove operator range_read and range_reader API (#2898)
add 9cf54135c docs: add new committer landing doc (#2905)
add d740edbbe refactor(core): Split buffer logic from underlying storage
operations (#2903)
add ab94da093 feat(core): Add unit test for ChunkedCursor (#2907)
add 581cd9f11 feat(types): remove blocking operation range_read and
range_reader API (#2912)
add 9e15037e9 build(bindings/java): bundle bare binaries in JARs with
classifier (#2910)
add 1161d50b5 fix(website): broken edit link (#2913)
add a39a5264e feat(types): add stat_with API for blocking operator (#2915)
add d101069e0 feat(services/gdrive): credential manage (#2914)
add 29db37227 docs: auto relaese maven artifacts (#2729)
add 1ac9b6edf doc(tests): fix test command (#2920)
add 5c8d9bbfb fix(binding/java): Overwrite default NOTICE file with
correct years (#2918)
add cbac9b842 feat(core): Implement Exact Buf Writer (#2917)
add 1bb977bc7 feat: Add benchmark for buf write (#2922)
add c058ee051 feat(core/raw): Add stream support for multipart (#2923)
add 95fdeae5c fix(services/gcs): migrate to new multipart impl for
gcs_insert_object_request
add 25a5b2391 add new ci to test gcs with default storage class
add 88cfd649d add support for stream
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (39565833e)
\
N -- N -- N refs/heads/gcs-multipart (88cfd649d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.env.example | 10 +-
.github/actions/setup/action.yaml | 14 +-
.github/workflows/bindings_java.yml | 132 +---
.github/workflows/bindings_php.yml | 91 +++
.github/workflows/publish.yml | 2 +-
.github/workflows/service_test_hdfs.yml | 4 +-
.github/workflows/service_test_postgresql.yml | 31 +-
.github/workflows/service_test_redis.yml | 132 ++--
.github/workflows/service_test_rocksdb.yml | 2 +
.github/workflows/service_test_s3.yml | 105 ++--
.typos.toml | 1 +
Cargo.lock | 46 +-
Cargo.toml | 4 +-
bin/oay/src/services/s3/service.rs | 2 +-
bin/oay/src/services/webdav/webdav_dir_entry.rs | 15 +-
bin/oay/src/services/webdav/webdav_file.rs | 22 +-
bin/oay/src/services/webdav/webdav_metadata.rs | 17 +-
bin/oay/src/services/webdav/webdavfs.rs | 99 ++-
bin/oli/src/commands/stat.rs | 2 +-
bin/oli/src/config/mod.rs | 67 +-
bin/oli/tests/cat.rs | 22 +
bin/oli/tests/cp.rs | 21 +
bin/oli/tests/rm.rs | 16 +
bin/oli/tests/stat.rs | 26 +-
bindings/c/README.md | 15 +-
bindings/c/examples/basicrw.c | 6 +-
bindings/c/src/lib.rs | 2 +-
bindings/haskell/src/lib.rs | 4 +-
bindings/haskell/src/logger.rs | 3 +-
bindings/java/.gitignore | 1 +
.../wrapper/maven-wrapper.properties} | 5 +-
bindings/java/README.md | 18 +
bindings/java/mvnw | 308 ++++++++++
bindings/java/mvnw.cmd | 205 +++++++
bindings/java/pom.xml | 4 +-
.../main/java/org/apache/opendal/Environment.java | 19 +-
.../main/java/org/apache/opendal/NativeObject.java | 6 +-
.../java/src/main/resources/META-INF/NOTICE | 2 +-
.../java/src/main/resources/bindings.properties | 1 -
bindings/java/src/operator.rs | 2 +-
bindings/nodejs/README.md | 2 +-
bindings/nodejs/src/lib.rs | 20 +-
bindings/ocaml/build.rs | 4 +
bindings/ocaml/lib/operator.ml | 26 +
bindings/ocaml/lib/operator.mli | 60 ++
bindings/ocaml/src/lib.rs | 7 +-
bindings/ocaml/src/operator.ml | 25 +-
bindings/ocaml/src/operator.mli | 25 +-
.../rm.rs => bindings/ocaml/src/operator/_type.rs | 30 +-
bindings/ocaml/src/operator/metadata.rs | 66 ++
bindings/ocaml/src/operator/mod.rs | 28 +-
bindings/ocaml/src/{lib.rs => operator/reader.rs} | 25 +-
bindings/ocaml/src/seek_from.ml | 27 +
bindings/ocaml/src/seek_from.mli | 27 +
bindings/ocaml/src/seek_from/mod.rs | 58 ++
bindings/ocaml/test/test.ml | 26 +
bindings/php/README.md | 70 ++-
bindings/php/src/lib.rs | 5 +-
bindings/python/src/lib.rs | 10 +-
core/Cargo.toml | 9 +-
core/README.md | 2 +-
core/benches/{ops => oio}/main.rs | 7 +-
.../fs/appender.rs => benches/oio/utils.rs} | 41 +-
core/benches/oio/write.rs | 81 +++
core/benches/ops/read.rs | 8 +-
core/fuzz/fuzz_reader.rs | 9 +-
core/fuzz/fuzz_writer.rs | 12 +-
core/src/docs/rfcs/2852_native_capability.md | 58 ++
.../docs/rfcs/2884_merge_range_read_into_read.md | 80 +++
core/src/docs/rfcs/mod.rs | 3 +
core/src/layers/async_backtrace.rs | 6 -
core/src/layers/await_tree.rs | 8 -
core/src/layers/blocking.rs | 15 +-
core/src/layers/chaos.rs | 5 -
core/src/layers/complete.rs | 122 +---
core/src/layers/concurrent_limit.rs | 26 -
core/src/layers/error_context.rs | 42 --
core/src/layers/immutable_index.rs | 7 +-
core/src/layers/logging.rs | 675 ++++++++-------------
core/src/layers/madsim.rs | 14 +-
core/src/layers/metrics.rs | 5 -
core/src/layers/minitrace.rs | 40 --
core/src/layers/oteltrace.rs | 5 -
core/src/layers/prometheus.rs | 5 -
core/src/layers/retry.rs | 148 ++---
core/src/layers/throttle.rs | 41 --
core/src/layers/timeout.rs | 43 +-
core/src/layers/tracing.rs | 5 -
core/src/layers/type_eraser.rs | 8 -
core/src/raw/accessor.rs | 55 +-
core/src/raw/adapters/kv/api.rs | 2 +-
core/src/raw/adapters/kv/backend.rs | 3 +-
core/src/raw/adapters/typed_kv/backend.rs | 3 +-
core/src/raw/http_util/client.rs | 5 +
core/src/raw/http_util/multipart.rs | 586 ++++++++++++------
core/src/raw/layer.rs | 14 -
core/src/raw/oio/append/api.rs | 108 ----
core/src/raw/oio/append/mod.rs | 19 -
core/src/raw/oio/cursor.rs | 404 +++++++++++-
core/src/raw/oio/mod.rs | 4 +-
core/src/raw/oio/page/into_flat_page.rs | 7 +-
core/src/raw/oio/read/api.rs | 42 +-
.../raw/oio/read/into_seekable_read_by_range.rs | 3 +-
core/src/raw/oio/stream/api.rs | 195 +++++-
core/src/raw/oio/stream/into_stream.rs | 7 +
core/src/raw/oio/stream/into_stream_from_reader.rs | 7 +
core/src/raw/oio/stream/mod.rs | 1 +
core/src/raw/oio/write/append_object_write.rs | 109 ++++
core/src/raw/oio/write/at_least_buf_write.rs | 131 ++++
core/src/raw/oio/write/compose_write.rs | 136 +++++
core/src/raw/oio/write/exact_buf_write.rs | 285 +++++++++
core/src/raw/oio/write/mod.rs | 18 +
core/src/raw/oio/write/multipart_upload_write.rs | 139 +----
.../writer.rs => raw/oio/write/one_shot_write.rs} | 61 +-
core/src/raw/operation.rs | 3 -
core/src/raw/ops.rs | 90 ++-
core/src/raw/rps.rs | 11 -
core/src/services/azblob/appender.rs | 145 -----
core/src/services/azblob/backend.rs | 23 +-
core/src/services/azblob/core.rs | 2 +-
core/src/services/azblob/mod.rs | 1 -
core/src/services/azblob/writer.rs | 127 +++-
core/src/services/azdfs/backend.rs | 3 +-
core/src/services/cos/appender.rs | 162 -----
core/src/services/cos/backend.rs | 73 +--
core/src/services/cos/core.rs | 5 +-
core/src/services/cos/mod.rs | 1 -
core/src/services/cos/writer.rs | 64 +-
core/src/services/dropbox/backend.rs | 10 +-
core/src/services/dropbox/builder.rs | 2 +-
core/src/services/dropbox/core.rs | 44 +-
core/src/services/fs/backend.rs | 37 +-
core/src/services/fs/mod.rs | 1 -
core/src/services/ftp/backend.rs | 4 +-
core/src/services/gcs/backend.rs | 3 +-
core/src/services/gcs/core.rs | 7 +-
core/src/services/gdrive/backend.rs | 181 +++++-
core/src/services/gdrive/builder.rs | 118 +++-
core/src/services/gdrive/core.rs | 452 +++++++++++---
core/src/services/gdrive/writer.rs | 72 ++-
core/src/services/ghac/backend.rs | 3 +-
core/src/services/hdfs/appender.rs | 48 --
core/src/services/hdfs/backend.rs | 55 +-
core/src/services/hdfs/mod.rs | 1 -
core/src/services/http/backend.rs | 3 +-
core/src/services/ipfs/backend.rs | 3 +-
core/src/services/ipmfs/backend.rs | 3 +-
core/src/services/mini_moka/backend.rs | 37 +-
core/src/services/mini_moka/docs.md | 20 +
core/src/services/moka/backend.rs | 32 +-
core/src/services/{dashmap => moka}/docs.md | 4 +-
core/src/services/obs/appender.rs | 141 -----
core/src/services/obs/backend.rs | 75 +--
core/src/services/obs/core.rs | 7 +-
core/src/services/obs/mod.rs | 1 -
core/src/services/obs/writer.rs | 179 +++---
core/src/services/onedrive/backend.rs | 3 +-
core/src/services/oss/appender.rs | 142 -----
core/src/services/oss/backend.rs | 59 +-
core/src/services/oss/core.rs | 5 +-
core/src/services/oss/mod.rs | 1 -
core/src/services/oss/writer.rs | 68 ++-
core/src/services/postgresql/backend.rs | 17 +-
core/src/services/postgresql/docs.md | 25 +
.../postgresql/fixtures/docker-compose.yml | 37 +-
.../src/services/postgresql/fixtures/init.sql | 16 +-
core/src/services/redis/backend.rs | 203 +++++--
core/src/services/redis/docs.md | 1 +
.../fixtures/docker-compose-redis-cluster-tls.yml | 145 +++++
.../fixtures/docker-compose-redis-cluster.yml | 75 +++
.../redis/fixtures/docker-compose-redis-tls.yml | 21 +-
.../redis/fixtures/docker-compose-redis.yml | 11 +-
core/src/services/redis/fixtures/ssl/ca.crt | 21 +
core/src/services/redis/fixtures/ssl/ca.key | 28 +
core/src/services/redis/fixtures/ssl/ca.srl | 1 +
core/src/services/redis/fixtures/ssl/redis.crt | 23 +
core/src/services/redis/fixtures/ssl/redis.key | 28 +
core/src/services/redis/fixtures/ssl/redis.v3.ext | 13 +
core/src/services/s3/backend.rs | 45 +-
core/src/services/s3/core.rs | 1 -
.../services/s3/fixtures/docker-compose-minio.yml | 26 +-
core/src/services/s3/writer.rs | 24 +-
core/src/services/sftp/backend.rs | 31 +-
core/src/services/sftp/writer.rs | 13 -
core/src/services/supabase/backend.rs | 3 +-
core/src/services/vercel_artifacts/backend.rs | 3 +-
core/src/services/wasabi/backend.rs | 3 +-
core/src/services/webdav/backend.rs | 3 +-
core/src/services/webhdfs/backend.rs | 3 +-
core/src/types/appender.rs | 252 --------
core/src/types/capability.rs | 70 +--
core/src/types/entry.rs | 6 +-
core/src/types/error.rs | 87 ++-
core/src/types/mod.rs | 3 -
core/src/types/operator/blocking_operator.rs | 506 +++++++++++----
core/src/types/operator/metadata.rs | 52 +-
core/src/types/operator/operator.rs | 223 +------
core/src/types/operator/operator_functions.rs | 146 +++++
core/src/types/operator/operator_futures.rs | 126 ++--
core/tests/behavior/README.md | 10 +-
core/tests/behavior/append.rs | 41 +-
core/tests/behavior/blocking_copy.rs | 2 +-
core/tests/behavior/blocking_list.rs | 11 +-
core/tests/behavior/blocking_read_only.rs | 8 +-
core/tests/behavior/blocking_rename.rs | 2 +-
core/tests/behavior/blocking_write.rs | 31 +-
core/tests/behavior/copy.rs | 2 +-
core/tests/behavior/fuzz.rs | 4 +-
core/tests/behavior/list.rs | 4 +-
core/tests/behavior/list_only.rs | 2 +-
core/tests/behavior/presign.rs | 2 +-
core/tests/behavior/read_only.rs | 26 +-
core/tests/behavior/rename.rs | 2 +-
core/tests/behavior/utils.rs | 2 +-
core/tests/behavior/write.rs | 98 ++-
integrations/README.md | 7 +
{bindings => integrations}/object_store/.gitignore | 0
{bindings => integrations}/object_store/Cargo.toml | 2 +-
{bindings => integrations}/object_store/README.md | 0
{bindings => integrations}/object_store/src/lib.rs | 3 +-
scripts/merge_local_staging.sh | 38 --
.../blog/2023-08-15-how-opendal-read-data/1.png | Bin 0 -> 135143 bytes
.../blog/2023-08-15-how-opendal-read-data/2.png | Bin 0 -> 136903 bytes
.../blog/2023-08-15-how-opendal-read-data/index.md | 158 +++++
website/docs/contributing/committer.md | 85 +++
website/docs/contributing/gmail-smtp-setting.jpg | Bin 0 -> 149264 bytes
website/docs/contributing/release.md | 66 +-
website/docs/contributing/verify.md | 12 +
website/docusaurus.config.js | 4 +-
website/src/pages/download.md | 1 +
230 files changed, 7062 insertions(+), 4282 deletions(-)
create mode 100644 .github/workflows/bindings_php.yml
create mode 100644 bindings/java/.gitignore
copy bindings/java/{src/main/resources/bindings.properties =>
.mvn/wrapper/maven-wrapper.properties} (77%)
create mode 100755 bindings/java/mvnw
create mode 100644 bindings/java/mvnw.cmd
copy NOTICE => bindings/java/src/main/resources/META-INF/NOTICE (84%)
copy bin/oli/tests/rm.rs => bindings/ocaml/src/operator/_type.rs (62%)
create mode 100644 bindings/ocaml/src/operator/metadata.rs
copy bindings/ocaml/src/{lib.rs => operator/reader.rs} (61%)
create mode 100644 bindings/ocaml/src/seek_from.ml
create mode 100644 bindings/ocaml/src/seek_from.mli
create mode 100644 bindings/ocaml/src/seek_from/mod.rs
copy core/benches/{ops => oio}/main.rs (89%)
rename core/{src/services/fs/appender.rs => benches/oio/utils.rs} (56%)
create mode 100644 core/benches/oio/write.rs
create mode 100644 core/src/docs/rfcs/2852_native_capability.md
create mode 100644 core/src/docs/rfcs/2884_merge_range_read_into_read.md
delete mode 100644 core/src/raw/oio/append/api.rs
delete mode 100644 core/src/raw/oio/append/mod.rs
create mode 100644 core/src/raw/oio/write/append_object_write.rs
create mode 100644 core/src/raw/oio/write/at_least_buf_write.rs
create mode 100644 core/src/raw/oio/write/compose_write.rs
create mode 100644 core/src/raw/oio/write/exact_buf_write.rs
copy core/src/{services/webdav/writer.rs => raw/oio/write/one_shot_write.rs}
(51%)
delete mode 100644 core/src/services/azblob/appender.rs
delete mode 100644 core/src/services/cos/appender.rs
delete mode 100644 core/src/services/hdfs/appender.rs
create mode 100644 core/src/services/mini_moka/docs.md
copy core/src/services/{dashmap => moka}/docs.md (87%)
delete mode 100644 core/src/services/obs/appender.rs
delete mode 100644 core/src/services/oss/appender.rs
copy licenserc.toml =>
core/src/services/postgresql/fixtures/docker-compose.yml (66%)
copy bindings/haskell/test/Spec.hs =>
core/src/services/postgresql/fixtures/init.sql (74%)
create mode 100644
core/src/services/redis/fixtures/docker-compose-redis-cluster-tls.yml
create mode 100644
core/src/services/redis/fixtures/docker-compose-redis-cluster.yml
copy bindings/php/.cargo/config.toml =>
core/src/services/redis/fixtures/docker-compose-redis-tls.yml (64%)
copy bindings/python/python/opendal/__init__.py =>
core/src/services/redis/fixtures/docker-compose-redis.yml (83%)
create mode 100644 core/src/services/redis/fixtures/ssl/ca.crt
create mode 100644 core/src/services/redis/fixtures/ssl/ca.key
create mode 100644 core/src/services/redis/fixtures/ssl/ca.srl
create mode 100644 core/src/services/redis/fixtures/ssl/redis.crt
create mode 100644 core/src/services/redis/fixtures/ssl/redis.key
create mode 100644 core/src/services/redis/fixtures/ssl/redis.v3.ext
copy .github/dependabot.yml =>
core/src/services/s3/fixtures/docker-compose-minio.yml (70%)
delete mode 100644 core/src/types/appender.rs
create mode 100644 integrations/README.md
rename {bindings => integrations}/object_store/.gitignore (100%)
rename {bindings => integrations}/object_store/Cargo.toml (96%)
rename {bindings => integrations}/object_store/README.md (100%)
rename {bindings => integrations}/object_store/src/lib.rs (99%)
delete mode 100644 scripts/merge_local_staging.sh
create mode 100644 website/blog/2023-08-15-how-opendal-read-data/1.png
create mode 100644 website/blog/2023-08-15-how-opendal-read-data/2.png
create mode 100644 website/blog/2023-08-15-how-opendal-read-data/index.md
create mode 100644 website/docs/contributing/committer.md
create mode 100644 website/docs/contributing/gmail-smtp-setting.jpg
