[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883784#comment-16883784
]
Tellier Benoit commented on JAMES-2806:
---
https://github.com/linagora/james-project/pull/2521 contributed the search
contract for BlobStore deleted messages
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882702#comment-16882702
]
Tellier Benoit commented on JAMES-2806:
---
https://github.com/linagora/james-project/pull/2516 contributed some fixes and
plays tests on all possible object storage configurations
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882182#comment-16882182
]
Raphaël Ouazana commented on JAMES-2806:
> I don't understand the proposal, it's context nor how it interacts with the
> work on task management.
The idea would have been to schedule bucket creation, instead of creating it on
the fly (potentially by several James servers at the same time). But we miss a
scheduler for now.
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881227#comment-16881227
]
Tellier Benoit commented on JAMES-2806:
---
> Why not using the (future) brand new task manager for asking rotation /
> creating a new buckets for the next deleted messages? When finished, this
> task would be responsible to update a bucket name somewhere, and it will then
> be used everywhere in the deleted message vault.
I don't understand the proposal, it's context nor how it interacts with the
work on task management.
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881219#comment-16881219
]
Raphaël Ouazana commented on JAMES-2806:
Why not using the (future) brand new task manager for asking rotation /
creating a new buckets for the next deleted messages? When finished, this task
would be responsible to update a bucket name somewhere, and it will then be
used everywhere in the deleted message vault.
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881034#comment-16881034
]
Tellier Benoit commented on JAMES-2806:
---
https://github.com/linagora/james-project/pull/2497 allowed multiple James to
be sharing the same object storage and playing well, while not breaking
retro-compatibility
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880495#comment-16880495
]
Tellier Benoit commented on JAMES-2806:
---
https://github.com/linagora/james-project/pull/2492 contributed the object
storage implementation of this
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879022#comment-16879022
]
Tellier Benoit commented on JAMES-2806:
---
https://github.com/linagora/james-project/pull/2489 is merged and contributed
various preliminary work for bucket implementation (configuration, etc...
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878946#comment-16878946
]
Rene Cordier commented on JAMES-2806:
-
Work for the ObjectStorage implementation has been divided into 3 PRs :
* https://github.com/linagora/james-project/pull/2489 `[JAMES-2806] [Part 1]
ObjectStorageBlobsDAO with BucketName`
* https://github.com/linagora/james-project/pull/2492 `[JAMES-2806] [Part 2]
ObjectStorageBlobsDAO creating bucket when saving fails (Swfit + S3 )`
* https://github.com/linagora/james-project/pull/2497 `[JAMES-2806] [Part 3]
ObjectStorageBlobsDAO BucketPrefix implementation`
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876012#comment-16876012
]
Tellier Benoit commented on JAMES-2806:
---
We currently run into issues with this ticket:
- The ObjectStorage BlobStore expects explicit bucket creation when a blob is
created
- Whereas memory + cassandra such creation is not needed
The issue is that with programatic use of the buckets (akka one per month in
the context of the deletedMessage vault) we will have to handle missing buckets.
We decided to make bucket creation implicit upon blob saving - akka catching
the error, creating the bucket and saving the blob again. Then we can remove
the costly 'attempt default bucket creation upon start'. We will still have to
do some reasonable concurrent testing regarding this.
In the future, we might want to introduce a webadmin endpoint to pre-provision
buckets ahead of time. We discussed the need to relate that to a feature in
order not to require the admin to know about the naming schema. Also a hook
system upon bucket creation can be used to notify and admin / perform
additional configuration (region, etc...). This will not be done as part of
this task.
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875978#comment-16875978
]
Tellier Benoit commented on JAMES-2806:
---
https://github.com/linagora/james-project/pull/2476 is merged
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16873144#comment-16873144
]
Rene Cordier commented on JAMES-2806:
-
Memory + contract PR : https://github.com/linagora/james-project/pull/2478
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore
[
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871985#comment-16871985
]
Rene Cordier commented on JAMES-2806:
-
BlobStore API change PR -> https://github.com/linagora/james-project/pull/2476
> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
> Issue Type: Sub-task
> Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
>
> You should update current BlobStore API to be able to customize the way we
> storeBlob:
> - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation -
> cassandra will be done in a later task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
