Cesar D. Rodas, web development. Technology news. PHP, MySQL, Apache, C, Bash, ASM

PHP: Amazon S3 Stream Wrapper

PHP since version 4 comes with a great feature that is with the possibility to register new Stream Wrapper. (see this articles are good, this, an this too).

Usually IO functions (fopen, unlink, mkdir and others) are handle by the Operating System or an Deamon(service) that works with the hardware for do the operation. PHP because is an JIT (just in time compiling, a kind of interpreted language) gives a tool to developers that is create a Class that manage a type of IO operations, for example take a look behind:

<?php
//open a file
$fp = fopen(type://file/path,r);
fclose($fp);
//create a dir
mkdir(type://file/path);

//delete a dir
rmdir(type://file/path);

//delete a file
unlink(type://file/path);
?>

The “type://” do not exist and if you try this code will give you an error, this is because PHP is asking to the host OS, and the host OS do not have idea about “type://”. So here came the “PHP magic”. Now lets create a class for handle “type://”

<?phpclass VariableStream {
    //private vars 

     /* fopen */
    function stream_open($path, $mode, $options, &$opened_path)
    {
        return true; //or false
    }
    /* fread */
    function stream_read($count)
    {
        return string of data;
    }
    /* fwrite */
    function stream_write($data)
    {
        return 1; //number of bytes writed
    }
    /*ftell */
    function stream_tell()
    {
        return 0;//actual position 
    }
    /* feof */
    function stream_eof()
    {
        return true;//or false
    }

    /* fseek */
    function stream_seek($offset, $whence)
    {        //$whence: SEEK_SET, SEEK_CUR, SEEK_END

        return true; //or false;

    }

    /* fstats */

    function url_stat (  $path,  $flags )
    {

}    /* opendir */
    function dir_opendir ( $path, $options ) {
    }

/* readdir */

    function dir_readdir() {

}

/* rewinddir */

    function dir_rewinddir() {

    }

    /* close dir*/

    function dir_closedir (  ) {

    }

}

//register type be handler for the class “VariableStream”

stream_wrapper_register(type, VariableStream)

    or die(Failed to register protocol);

?>

So every time that we call IO functions with “type://” as scheme PHP internaly will instance the class “VariableStream” and that class will handle the functions. For further details visit this page.

This great tool open an infinity of possibility for write apps. Here are a few of apps that implements a Stream Wrapper:

  • filesnap : Stream wrapper to write and read snapshot files
  • gFeed : Stream wrapper to read and write RSS feeds
  • gHttp : Stream wrapper to send HTTP POST and GET requests
  • MS-Excel Stream Handler : Stream wrapper to read and write MS Excel file.
  • POP3 e-mail client : Access to e-mail mailboxes using the POP3 protocol.
  • smb4php: Stream wrapper to access Windows shared files.

In the past week I’ve wrote an Stream Wrapper for access Amazon S3 Files. Supported functions are fopen, fread, fwrite, fclose, stat, mkdir, rmdir, unlink, opendir, readdir, rewinddir, closedir.

dirs are amazon “buckets”, here is an example.

<?php
set_time_limit(0);
include(gs3.php);
/* fake keys!, please put yours */
define(S3_KEY, DA5S4D5A6S4D);
define(S3_PRIVATE,adsadasd); 

/* create a folder,and every one can read*/
mkdir(s3://cesar_gs3,_PUBLIC_READ);
/* create a file and any one can read*/
/*
 * r = read,
 * w= write, private
 * w+r = write, public read,
 * w+w=write, public write
 */
$f = fopen(s3://cesar_gs3/file.txt, w+r);
        for($i=0; $i <= 1000; $i++)
            fwrite($f, Line #$in);
        rewind($f);
        fwrite($f, Line #5n);
fclose($f); 

/* displain the files contains in the folder */
$e = opendir(s3://cesar_gs3);
while($f = readdir($e) ) {
	echo $f.<br/>;
}
closedir($e);
/* deleting file */
unlink(s3://cesar_gs3/file.txt);
/* deleting foler */
rmdir(s3://cesar_gs3);
?>

DOWNLOAD The class.

Here is the online documentation is here and the project could be download from here

The project is protected under a BSD license, so you are free to use and modify for free.

Free domain registration packages are offered by the hosting service providers to attract extensive range of clients immediately. The internet marketing tools are applied by the internet marketing managers to boost up development of specific business. The webmaster may get benefit of variety of webmaster tools, available in the marketplace. The techniques of ppc are used by the webmasters to enhance their revenue.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages. Digg del.icio.us StumbleUpon Technorati BlinkList Furl NewsVine Reddit

6 Responses to “PHP: Amazon S3 Stream Wrapper”

  1. James Says:

    Great idea Caesar, I’ll be sure to look at it more closely when I have a chance to play with S3 a little more. Are you using this for any live projects or production sites? How stable is it?

  2. meneame.net Says:

    Servidores S3 de Amazon desde PHP [ENG]…

    Un stream wrapper (que se puede manejar como un filesystem local) para manejar los servidores S3 de Amazon (Amazon Simple Storage Service). S3 es un servicio de almacenamiento escalabe de bajo coste para ser utilizado por aplicaciones. Permite hasta ci…

  3. www.programame.net Says:

    Servidores S3 de Amazon desde PHP [ENG]…

    Un stream wrapper (que se puede manejar como un filesystem local) para manejar los servidores S3 de Amazon (Amazon Simple Storage Service). S3 es un servicio de almacenamiento escalabe de bajo coste para ser utilizado por aplicaciones. Permite hasta ci…

  4. Tolan Says:

    Hi, I’d love a copy of your class if possible.

    Cheers!

    Tolan.

  5. Alan Says:

    This is great Caesar. I would like to use your code in my project, however, I am only able to use GNU licensed code. I am fully willing to give you full credit for it and will provide you with a link to the project.

    Would you be willing to dual license it under BSD & GNU?

    Alan

  6. jim sloan Says:

    Thanks for a very practical demonstration of PHP streams. It was very easy to setup; within 10 minutes I was able to use it to do some mass deletes in an S3 bucket that I was cleaning up.

    I did run into an issue with special characters in the S3 filenames(Keys). The xml_set_character_data_handler() will fire off multiple times if the data contains certain characters. There is some discussion of it in the PHP manual. see: http://us3.php.net/xml_set_character_data_handler

    I followed the suggestion to append data until “_dirEnd” is called then add it to the array and unset the data. So now the callback functions look like this:

    /**
    * Use this to collect the data when parsing XML
    * @access private
    * @var string
    */
    var $dataXML;

    /**
    * Handle end of XML tags
    *
    */
    function _dirEnd(&$parser,&$name){
    if ($this->actualTag==”KEY”) $this->dirList[] = this->dataXML;
    unset($this->dataXML);
    unset($this->actualTag);
    }
    /**
    * Handle data of XML tags
    *
    * Save in an array when TAG == “KEY”
    */
    function _dirData(&$parser,&$data){
    if ($this->actualTag==”KEY”) $this->dataXML .= $data;
    }

Leave a Reply


Fatal error: Call to undefined function display_cryptographp() in /home/.schmuckie/saddor/cesarodas.com/wp-content/themes/first-spring-10/comments.php on line 94