Re: [gpfsug-discuss] Spectrum Scale & S3

[Jan-Frode Myklebust]

It has features for both being an Object store for other applications
(running openstack swift/S3), and for migrating/tiering filesystem data to
an object store like Amazon S3, IBM COS, etc...


  -jf

fre. 21. mai 2021 kl. 10:42 skrev David Reynolds :

> When we talk about supported protocols on Scale, S3 is listed but its
> normally prefixed with Object. Does this mean that the S3 connectivity is
> used purely to connect and migrate data in object stores to parallel
> systems (and vice versa) or can you connect to applications/workflows using
> S3?
>
>
>
> David Reynolds
>
> TES Enterprise Solutions
>
> david.reyno...@tes-es.com
>
> 07594 524986
>
>
> ___
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] Spectrum Scale & S3

[Andrew Beattie]

David,

You can use the CES protocol nodes to present a swift Object interface or S3 
Object interface for applications to access data from the filesystem / write 
data into the filesystem over the object protocol. 

Check the openstack Swift -S3 compatabioity matrix to make sure that the 
functionality from swift aligns with your App requirements 

Sent from my iPhone

> On 21 May 2021, at 18:42, David Reynolds  wrote:
> 
> 
> This Message Is From an External Sender
> This message came from outside your organization.
> When we talk about supported protocols on Scale, S3 is listed but its 
> normally prefixed with Object. Does this mean that the S3 connectivity is 
> used purely to connect and migrate data in object stores to parallel systems 
> (and vice versa) or can you connect to applications/workflows using S3?
>  
> David Reynolds
> TES Enterprise Solutions
> david.reyno...@tes-es.com
> 07594 524986
>  

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] GPFS de duplication

[Jonathan Buzzard]

On 20/05/2021 13:58, Dave Bond wrote:

As part of a project I am doing I am looking if there are any de 
duplication options for GPFS?  I see there is no native de dupe for the 
filesystem. The scenario would be user A creates a file or folder and 
user B takes a copy within the same filesystem, though separate 
independent filesets.  The intention would be to store 1 copy.    So I 
was wondering 


1) Is this is planned to be implemented into GPFS in the future?
2) Is anyone is using any other solutions that have good GPFS integration?



Disk space in 2021 is insanely cheap. With an ESS/DSS you can get many 
PB in a single rack. The complexity that dedup introduces is simple not 
worth it IMHO.


Or put another way there is better things the developers at IBM can be 
working on than dedup code.


Historically if you crunched the numbers the licensing for dedup on 
NetApp was similar to just buying more disk unless you where storing 
hundreds of copies of the same data. About the only use case scenario 
would be storing lots of virtual machines. However I refer you back to 
my original point :-)



JAB.

--
Jonathan A. Buzzard Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

[gpfsug-discuss] Spectrum Scale & S3

[David Reynolds]

When we talk about supported protocols on Scale, S3 is listed but its normally 
prefixed with Object. Does this mean that the S3 connectivity is used purely to 
connect and migrate data in object stores to parallel systems (and vice versa) 
or can you connect to applications/workflows using S3?

David Reynolds
TES Enterprise Solutions
david.reyno...@tes-es.com
07594 524986

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss