Re: [fw-general] New Proposal: Zend_S3
Andi Gutmans wrote:
Hi Davey,
You hit on a couple of issues here. First of all re: design and
implementation. This is exactly why we have a proposal process. It is an
opportunity for others in the community to provide valuable feedback
like you are doing and work with the proposal leads to figure out
whether it makes sense to tweak the proposals. Of course there will not
always be agreement but it's the right process and I believe overall the
results have been acceptable. Btw, make sure to also paste your comments
to the Wiki because that's where the discussion gets recorded for
reference.
I agree for the most part too. However, in this case, there is now 3
(though 2 are similar) different
proposals for S3. If I simply reply to the initial or second proposal,
there is a very good chance my
idea will not get equal hearing, it's just a comment at that point. I
see with the other 2 proposals where
a feedback and commentary works, they're just slightly different APIs
for the same idea, but with
mine the implementation is radically different, and I believe is
worthwhile to add as it's own proposal.
How best to make sure that everybody involved see's that this is the
case, even if they're not following the ML?
Re: the direction of the "concrete" Web services. This has been one of
the big goals of framework from day 1. While I agree most of them (if
not all) can be done using other generic interfaces I believe there is a
lot of value in delivering them. This is not real bloat as you don't
have to load it and I am convinced we can generate a lot of interest
from third party vendors who would contribute such connectors (I've
already gotten a lot of interest in fact). I definitely plan on
continuing to pursue this and I don't think it will significantly lessen
the value of generic interfaces.
OK, perhaps the correct term is "perceived" bloat. It would be better to
segregate these somewhat,
in this way, I agree with the Gdata release that was done some while back.
I don't mean to have one package per vendor, but more like, two extra
packages, one with the commonly
used WS stuff in it, and another with the lesser used stuff in it.
There's also no reason we can't provide a "Core" ZF package, and an
"+Extras" package. Though IMO,
this just further highlights the need for a PEAR channel for distribution ;)
It is my opinion that at least one of the existing implementations
should be removed in favor of the generic implementation,
Flickr, which I originally wrote and wholly believe is superfluous now.
We should take this to another thread if discussion is to continue :)
- Davey
Andi
-Original Message-
From: Davey Shafik [mailto:[EMAIL PROTECTED]
Sent: Monday, May 07, 2007 10:15 AM
To: Andi Gutmans
Cc: Stas Malyshev; James Andrews; Zend Framework
Subject: Re: [fw-general] New Proposal: Zend_S3
Andi, James,
I'm very much NOT in agreement regarding these "concrete" web
services APIs goal. This is a great way to bloat the
framework unncessarily. For the most part, the generic web
service components will be enough. I used both the Flickr and
Technorati APIs for testing Zend_Rest_Client, and they were
just as easy to use as using Zend_Service_Flickr. Yes,
something like Amazon has value in being implemented, because
it's such a complex API, but there are a lot of simple,
popular webservices are just pointless.
To that end, I must say that I do NOT agree with Zend_S3 (presumably,
Zend_Service_Amazon_S3 would the correct name) as it
currently stands.
However, in response, I have written a stream wrapper for S3,
this has a lot of merit IMO.
It's not 100% implemented, nor is it 100% perfect, however
all the following works:
file_exists("s3://bucketname"); // Check if a Bucket Exists
file_exists("s3://bucketname/filename"); // Check if a file
exists mkdir("s3://bucketname"); // Create a new bucket
rmdir("s3://bucketname"); // Delete a bucket
file_put_contents("s3://bucketname/file", $content); // Write
data to a file file_get_contents("s3://bucketname/file"); //
Retrieve a file
This means also that the fopen/fread/fwrite/fgets methods
should all work too, but I haven't tested those yet.
Also, I have decided that because S3 does not support
sub-directories, if you try to create or remove a bucket
(mkdir/rmdir) with sub-dir, it will just create the bucket
(first dir). However if you try to create a file using an
arbitrary filename including sub-dirs, it will convert those to .'s
S3 provides enough facilities to make directory listing
possible (though not implemented yet in my code) with this.
rmdir() could be made smart enough that if you try to remove
a sub-directory like s3://bucketname/some/dir/ then it will
remove some.dir.* files.
S
RE: [fw-general] New Proposal: Zend_S3
Hi Davey,
You hit on a couple of issues here. First of all re: design and
implementation. This is exactly why we have a proposal process. It is an
opportunity for others in the community to provide valuable feedback
like you are doing and work with the proposal leads to figure out
whether it makes sense to tweak the proposals. Of course there will not
always be agreement but it's the right process and I believe overall the
results have been acceptable. Btw, make sure to also paste your comments
to the Wiki because that's where the discussion gets recorded for
reference.
Re: the direction of the "concrete" Web services. This has been one of
the big goals of framework from day 1. While I agree most of them (if
not all) can be done using other generic interfaces I believe there is a
lot of value in delivering them. This is not real bloat as you don't
have to load it and I am convinced we can generate a lot of interest
from third party vendors who would contribute such connectors (I've
already gotten a lot of interest in fact). I definitely plan on
continuing to pursue this and I don't think it will significantly lessen
the value of generic interfaces.
Andi
> -Original Message-
> From: Davey Shafik [mailto:[EMAIL PROTECTED]
> Sent: Monday, May 07, 2007 10:15 AM
> To: Andi Gutmans
> Cc: Stas Malyshev; James Andrews; Zend Framework
> Subject: Re: [fw-general] New Proposal: Zend_S3
>
> Andi, James,
>
> I'm very much NOT in agreement regarding these "concrete" web
> services APIs goal. This is a great way to bloat the
> framework unncessarily. For the most part, the generic web
> service components will be enough. I used both the Flickr and
> Technorati APIs for testing Zend_Rest_Client, and they were
> just as easy to use as using Zend_Service_Flickr. Yes,
> something like Amazon has value in being implemented, because
> it's such a complex API, but there are a lot of simple,
> popular webservices are just pointless.
>
> To that end, I must say that I do NOT agree with Zend_S3 (presumably,
> Zend_Service_Amazon_S3 would the correct name) as it
> currently stands.
> However, in response, I have written a stream wrapper for S3,
> this has a lot of merit IMO.
>
> It's not 100% implemented, nor is it 100% perfect, however
> all the following works:
>
> file_exists("s3://bucketname"); // Check if a Bucket Exists
> file_exists("s3://bucketname/filename"); // Check if a file
> exists mkdir("s3://bucketname"); // Create a new bucket
> rmdir("s3://bucketname"); // Delete a bucket
> file_put_contents("s3://bucketname/file", $content); // Write
> data to a file file_get_contents("s3://bucketname/file"); //
> Retrieve a file
>
> This means also that the fopen/fread/fwrite/fgets methods
> should all work too, but I haven't tested those yet.
>
> Also, I have decided that because S3 does not support
> sub-directories, if you try to create or remove a bucket
> (mkdir/rmdir) with sub-dir, it will just create the bucket
> (first dir). However if you try to create a file using an
> arbitrary filename including sub-dirs, it will convert those to .'s
>
> S3 provides enough facilities to make directory listing
> possible (though not implemented yet in my code) with this.
> rmdir() could be made smart enough that if you try to remove
> a sub-directory like s3://bucketname/some/dir/ then it will
> remove some.dir.* files.
>
> S3 auth is supported, and also access to public files, but
> there are still bunches of things I don't support. Checking
> of MD5 sums when sending files, setting access control,
> requesting torrents and creating public key access. But all
> of these things will be (just like auth) implemented using
> stream contexts.
>
> I have no issues with add convenience methods to
> Zend_Service_Amazon_S3 to allow OO calls to it directly, but
> they will use the stream internally, but the stream is the
> best way to do this.
>
> A great feature this allows for, is filters, so you can
> attach an encryption filter if you want all your data encrypted.
> Encryption/decryption would happen transparently and is
> completely outside the wrapper itself. Or maybe compression
> filters. Plenty of options :)
>
> You can find the code attached.
>
> - Davey
>
> Andi Gutmans wrote:
> > Yep that's correct. In general though it's easier for us to push
> > forward these "concrete" Web services APIs then more
> generic framework
> > components esp. during times like now where we are focusing
> on getting
> > a major version out of the door. The reason is that the Web
RE: [fw-general] New Proposal: Zend_S3
Yep that's correct. In general though it's easier for us to push forward these "concrete" Web services APIs then more generic framework components esp. during times like now where we are focusing on getting a major version out of the door. The reason is that the Web Services APIs are very self contained and tend not to be too hard from a design point of view (and in most cases have few inter-dependencies on other componentes). One of our goal is to have as many such "concrete" Web services APIs in the framework as possible. I have also talked to some companies who expose Web Services to have them engage and contribute such APIs. Andi > -Original Message- > From: Stanislav Malyshev [mailto:[EMAIL PROTECTED] > Sent: Saturday, April 28, 2007 11:25 PM > To: James Andrews > Cc: Zend Framework > Subject: Re: [fw-general] New Proposal: Zend_S3 > > > How does the development process work for a "proposal" to > become part > > of the "library"? > > Usually it gets evaluated (as for "how useful it would be for > Framework users", "should we do it", etc.), discussed (as for > "is this the right way to do it") and then confirmed or not > confirmed, if confirmed, then it can be put into the > incubator and then eventually become part of the core > Framework. Here is it described in detail (with a nice picture too > :): > http://framework.zend.com/wiki/display/ZFPROP/Zend+Framework+P > roposal+Process+Detail > > Since right now 1.0 release is very close, I guess it might > take a bit more time until new proposals are reviewed, since > people are concentrating on getting 1.0 out. > > -- > Stanislav Malyshev, Zend Products Engineer [EMAIL PROTECTED] > http://www.zend.com/ > >
Re: [fw-general] New Proposal: Zend_S3
How does the development process work for a "proposal" to become part of the "library"? Usually it gets evaluated (as for "how useful it would be for Framework users", "should we do it", etc.), discussed (as for "is this the right way to do it") and then confirmed or not confirmed, if confirmed, then it can be put into the incubator and then eventually become part of the core Framework. Here is it described in detail (with a nice picture too :): http://framework.zend.com/wiki/display/ZFPROP/Zend+Framework+Proposal+Process+Detail Since right now 1.0 release is very close, I guess it might take a bit more time until new proposals are reviewed, since people are concentrating on getting 1.0 out. -- Stanislav Malyshev, Zend Products Engineer [EMAIL PROTECTED] http://www.zend.com/
Re: [fw-general] New Proposal: Zend_S3
Hey Jan,
S3 is an Amazone Web Service. "Simple Storage Solution". Which
allows you to store files in their storage sollution.
All the docs are available on Amazon's developers site.
http://developer.amazonwebservices.com/connect/index.jspa
James
On Apr 28, 2007, at 4:53 AM, Jan Pieper wrote:
Can you please give us a documentation or something else where S3 ist
descriped because i don't know it and I can't see what this is for.
You
only pasted an example of use.
-- Jan
Hi everyone,
Well I have an idea for an extention to the Zend framework that would
allow you to make use of Amazon's S3 storage technology. This is
kind
of my idea.
The object on construction would take in the key, shared secret,
and the
Amazon S3 Url (which would be optional, in case someone else
mimics S3).
Now one thing I dislike about S3 is that having sub directories in
buckets is not an aspect of S3. I've seen in the past people talk
about
ways to fake it using text and S3 Keys but I am not sure how S3 keys
works yet, as I am just beginning to go down the S3 road, but
think it
would be nice to have something that everyone can use.
Anyway, I want to make it simple for people, I think this is a good
beginning.
// Create the object.
$s3Obj = new Zend_S3($key,$secret);
// Create Bucket.
$s3Obj->createBucket("bucketName);
// Delete Bucket.
$s3Obj->dropBucket("bucketName);
// Upload file
$s3Obj->createFile(bucketName://fileName);
// Get the file
$s3Obj->getFile(bucketName://fileName);
// Delete file
$s3Obj->dropFile(bucketName://fileName);
For a directory path past the bucket I would expect something like
this,
and everything would happen in the background to mimic a directory
path
//Upload file
$s3Obj->createFile(bucketName://path/to/file/fileName);
//Get the file
$s3Obj->getFile(bucketName://path/to/file/fileName);
//Delete file
$s3Obj->dropFile(bucketName://path/to/file/fileName);
I am not sure if there are other aspects of the S3 protocol that I am
not aware of that could be included, but I think this could be a
great
resource.
I think this is a nice way to keep it simple, and is easy to follow.
What do you think?
James
Re: [fw-general] New Proposal: Zend_S3
I am glad that I am not the only one who's thought of this. I guess I need to understand S3 more to understand why it was drafted this way. I actually started working on a S3 class as I need it for work last night based on (but not 100% the same as) what I proposed last night. I'm sure it will change as I go along. How does the development process work for a "proposal" to become part of the "library"? James On Apr 28, 2007, at 7:12 AM, Tomas Markauskas wrote: Hi, have a look at this proposal: http://framework.zend.com/wiki/display/ZFPROP/ Zend_Services_Amazon_S3+-+Travis+Swicegood Tomas James Andrews-2 wrote: Well I have an idea for an extention to the Zend framework that would allow you to make use of Amazon's S3 storage technology. This is kind of my idea. -- View this message in context: http://www.nabble.com/New-Proposal% 3A--Zend_S3-tf3660720s16154.html#a10232007 Sent from the Zend Framework mailing list archive at Nabble.com.
Re: [fw-general] New Proposal: Zend_S3
Hi, have a look at this proposal: http://framework.zend.com/wiki/display/ZFPROP/Zend_Services_Amazon_S3+-+Travis+Swicegood Tomas James Andrews-2 wrote: > > Well I have an idea for an extention to the Zend framework that would > allow you to make use of Amazon's S3 storage technology. This is > kind of my idea. > -- View this message in context: http://www.nabble.com/New-Proposal%3A--Zend_S3-tf3660720s16154.html#a10232007 Sent from the Zend Framework mailing list archive at Nabble.com.
Re: [fw-general] New Proposal: Zend_S3
Can you please give us a documentation or something else where S3 ist
descriped because i don't know it and I can't see what this is for. You
only pasted an example of use.
-- Jan
> Hi everyone,
>
> Well I have an idea for an extention to the Zend framework that would
> allow you to make use of Amazon's S3 storage technology. This is kind
> of my idea.
>
> The object on construction would take in the key, shared secret, and the
> Amazon S3 Url (which would be optional, in case someone else mimics S3).
>
> Now one thing I dislike about S3 is that having sub directories in
> buckets is not an aspect of S3. I've seen in the past people talk about
> ways to fake it using text and S3 Keys but I am not sure how S3 keys
> works yet, as I am just beginning to go down the S3 road, but think it
> would be nice to have something that everyone can use.
>
> Anyway, I want to make it simple for people, I think this is a good
> beginning.
>
> // Create the object.
> $s3Obj = new Zend_S3($key,$secret);
>
> // Create Bucket.
> $s3Obj->createBucket("bucketName);
>
> // Delete Bucket.
> $s3Obj->dropBucket("bucketName);
>
> // Upload file
> $s3Obj->createFile(bucketName://fileName);
>
> // Get the file
> $s3Obj->getFile(bucketName://fileName);
>
> // Delete file
> $s3Obj->dropFile(bucketName://fileName);
>
>
> For a directory path past the bucket I would expect something like this,
> and everything would happen in the background to mimic a directory path
>
> //Upload file
> $s3Obj->createFile(bucketName://path/to/file/fileName);
>
> //Get the file
> $s3Obj->getFile(bucketName://path/to/file/fileName);
>
> //Delete file
> $s3Obj->dropFile(bucketName://path/to/file/fileName);
>
> I am not sure if there are other aspects of the S3 protocol that I am
> not aware of that could be included, but I think this could be a great
> resource.
>
> I think this is a nice way to keep it simple, and is easy to follow.
>
> What do you think?
>
> James
[fw-general] New Proposal: Zend_S3
Hi everyone,
Well I have an idea for an extention to the Zend framework that would
allow you to make use of Amazon's S3 storage technology. This is
kind of my idea.
The object on construction would take in the key, shared secret, and
the Amazon S3 Url (which would be optional, in case someone else
mimics S3).
Now one thing I dislike about S3 is that having sub directories in
buckets is not an aspect of S3. I've seen in the past people talk
about ways to fake it using text and S3 Keys but I am not sure how S3
keys works yet, as I am just beginning to go down the S3 road, but
think it would be nice to have something that everyone can use.
Anyway, I want to make it simple for people, I think this is a good
beginning.
// Create the object.
$s3Obj = new Zend_S3($key,$secret);
// Create Bucket.
$s3Obj->createBucket("bucketName);
// Delete Bucket.
$s3Obj->dropBucket("bucketName);
// Upload file
$s3Obj->createFile(bucketName://fileName);
// Get the file
$s3Obj->getFile(bucketName://fileName);
// Delete file
$s3Obj->dropFile(bucketName://fileName);
For a directory path past the bucket I would expect something like
this, and everything would happen in the background to mimic a
directory path
//Upload file
$s3Obj->createFile(bucketName://path/to/file/fileName);
//Get the file
$s3Obj->getFile(bucketName://path/to/file/fileName);
//Delete file
$s3Obj->dropFile(bucketName://path/to/file/fileName);
I am not sure if there are other aspects of the S3 protocol that I am
not aware of that could be included, but I think this could be a
great resource.
I think this is a nice way to keep it simple, and is easy to follow.
What do you think?
James
