The TaleCrafter’s Scribbles

February 19, 2009

Accessing AWS SimpleDB from PHP

Filed under: Distributed System Design — Tags: , , , , — Andy @ 7:13 pm

This week, as I built part of my App Server for Distributed Systems Design, I hit another stumbling block. The library that Amazon provides in PHP for accessing SimpleDB requires PHP 5.2. I should have known that I need to use the latest version.

Not only did Amazon’s library not work for me, but it was huge and complicated. I found another library at: Google Code, but as fate would have it, that library didn’t work either. The code was pretty ugly imho, but at least it was straighforward enough for me to understand how accessing SimpleDB worked, which led me to make my own SimpleDB client.

The script will work with any PHP 5, and doesn’t depend on anything that isn’t built in by default. I hope it is helpful to someone else. It would be really easy to add the SimpleDB requests I haven’t implemented yet.

<?php
/**
 * AWS_SimpleDB_Client v0.1 by Andy VanWagoner, distributed under the ISC licence.
 * Provides simple access to Amazon's SimpleDB from PHP 5.
 *
 * Copyright (c) 2009, Andy VanWagoner
 *
 * Permission to use, copy, modify, and/or distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
 * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
 * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
 * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */

class AWS_SimpleDB_Client {

	// AWS SimpleDB API Constants
	private static $service_endpoint	= "sdb.amazonaws.com";
	private static $api_version			= "2007-11-07";
	private static $timestamp_format	= "Y-m-d\TH:i:s.\\\\\Z";
	private static $signature_version	= 1;

	private static $user_agent = "AWS_SimpleDB_Client 0.1 - Andy VanWagoner";

	/**
	* Constructor
	*
	* @param string $access			// your AWS "Access Key ID"
	* @param string $secret			// your AWS "Seceret Access Key"
	*/
	function AWS_SimpleDB_Client($access, $secret) {
		$this->access_key = $access;
		$this->secret_key = $secret;
	}

	/**
	* AWS SimpleDB API - CreateDomain
	* NOTE: This call will take a while (AWS says 10 seconds)
	*
	* @param string $domain			// the domain to create
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>)
	*/
	function create_domain($domain) {
		$params = array(
			'Action' => 'CreateDomain',
			'DomainName' => $domain
		);

		return $this->post($params);
	}

	/**
	* AWS SimpleDB API - DeleteDomain
	* NOTE: This call will take a while (AWS says 10 seconds)
	*
	* @param string $domain			// the domain to delete
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>)
	*/
	function delete_domain($domain) {
		$params = array(
			'Action' => 'DeleteDomain',
			'DomainName' => $domain
		);

		return $this->post($params);
	}

	/**
	* AWS SimpleDB API - ListDomains
	*
	* @param string $next = ''		// Optional - Sent as NextToken parameter
	* @param string $max = 100		// Optional - Sent as MaxNumberOfDomains
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>,
	* 				'DomainName'=>array('...', ...) [, 'NextToken'=>])
	*/
	function list_domains($next = '', $max = 0) {
		$params = array('Action' => 'ListDomains');

		if ($max > 0 && $max post($params);
	}

	/**
	* AWS SimpleDB API - PutAttributes
	*
	* @param string $domain			// The domain the item is in
	* @param string $item			// The name of the item
	* @param array  $attributes		// array(array('Name'=>, 'Value'=> [, 'Replace'=>]), ...)
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>)
	*/
	function put_attributes($domain, $item, $attributes) {
		$params = array(
			'Action' => 'PutAttributes',
			'DomainName' => $domain,
			'ItemName' => $item
		);

		foreach($attributes as $i => $value) {
			$params["Attribute.$i.Name"] = $value['Name'];
			$params["Attribute.$i.Value"] = $value['Value'];
			if (isset($value['Replace']))
				$params["Attribute.$i.Replace"] = $value['Replace'];
		}

		return $this->post($params);
	}

	/**
	* AWS SimpleDB API - DeleteAttributes
	*
	* @param string $domain			// The domain the item is in
	* @param string $item			// The name of the item
	* @param array  $attributes		// array(array('Name'=>, 'Value'=>), ...)
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>)
	*/
	function delete_attributes($domain, $item, $attributes) {
		$params = array(
			'Action' => 'DeleteAttributes',
			'DomainName' => $domain,
			'ItemName' => $item
		);

		foreach($attributes as $i => $value) {
			$params["Attribute.$i.Name"] = $value['Name'];
			$params["Attribute.$i.Value"] = $value['Value'];
		}

		return $this->post($params);
	}

	/**
	* AWS SimpleDB API - GetAttributes
	*
	* @param string $domain			// the domain name
	* @param string $item			// the item's name
	* @param string $attribute		// Optional - If specified, only this attribute's values are retrieved.
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>,
	* 				'Attribute'=>array(array('Name'=>,'Value'=>), ...))
	*/
	function get_attributes($domain, $item, $attribute = '') {
		$params = array(
			'Action' => 'GetAttributes',
			'DomainName' => $domain,
			'ItemName' => $item
		);

		if ($attribute)
			$params['AttributeName'] = $attribute;

		return $this->post($params);
	}

	/**
	* AWS SimpleDB API - Query
	*
	* @param string  $domain		// The domain name
	* @param string  $query			// The query to run on this domain
	* @param string  $next = ''		// OPTIONAL - token supplied on last paged call
	* @param integer $max = 100		// OPTIONAL - max items you want returned 1-250, default = 100
	*
	* @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>,
	* 				'ItemName'=>array('...', ...))
	*/
	function query($domain, $query, $next = '', $max = 0) {
		$params = array(
			'Action' => 'Query',
			'DomainName' => $domain,
			'QueryExpression' => $query
		);

		if ($max > 250) $max = 250;
		if ($max > 0)
			$params['MaxNumberOfItems'] = $max;
		if ($next)
			$params['NextToken'] = $next;

		return $this->post($params);
	}

	/**
	 * Sign the parameters, following AWS version 1 signing
	 *
	 * @param array $params			// array of all (except for the signiture) params to be passed to amazon
	 *
	 * @return string				// signature string
	 */
	private function sign($params) {
		uksort($params, 'strnatcasecmp');

		$data = '';
		foreach ($params as $key=>$value) {
			$data .= $key . $value;
		}

		return base64_encode (	pack("H*", sha1((str_pad($this->secret_key, 64, chr(0x00)) ^ (str_repeat(chr(0x5c), 64))) .
								pack("H*", sha1((str_pad($this->secret_key, 64, chr(0x00)) ^ (str_repeat(chr(0x36), 64))) .
								$data)))) );
	}

	/**
	 * POST to AWS SimpleDB and then parse the response.
	 *
	 * @param array $params			// all params to pass on the post
	 *
	 * @return array('status'=>array('code'=>, 'message'=>), 'RequestId'=>, 'BoxUsage'=>, ...)
	 */
	private function post($params) {

		// Add all of the common parameters needed by AWS SimpleDB
		$params['AWSAccessKeyId']	= $this->access_key;
		$params['Timestamp'] 		= gmdate(self::$timestamp_format, time());
		$params['Version'] 			= self::$api_version;
		$params['SignatureVersion']	= self::$signature_version;
		$params['Signature'] 		= $this->sign($params);

		// Generate the POST request
		$content = http_build_query($params);

		$post  = 'POST / HTTP/1.0'															. "\r\n";
		$post .= 'Host: ' 			. self::$service_endpoint 								. "\r\n";
		$post .= 'Content-Type: ' 	. 'application/x-www-form-urlencoded; charset=utf-8'	. "\r\n";
		$post .= 'Content-Length: ' . strlen($content)										. "\r\n";
		$post .= 'User-Agent: ' 	. self::$user_agent 									. "\r\n";
		$post .= 																			  "\r\n";
		$post .= $content;

		$socket = @fsockopen(self::$service_endpoint, 80, $errno, $errstr, 10);
  		if ($socket) {
			fwrite($socket, $post);

			$response = stream_get_contents($socket);
			fclose($socket);

			// Parse the response
			return $this->format_result($response);
		}

		// Return a fail result
		return array('status' => array('code' => 404, 'message' => 'Not Found'),
			'Error' => array('Code' => $errno, 'Message' =>
				'Could not connect to ' . $this->$service_endpoint . " ($errstr)"
			)
		);
	}

	/**
	 * Take the XML document returned by AWS SimpleDB, and transform it into a hash
	 *
	 * @param string $result		// the full http response string from SimpleDB
	 */
	private function format_result($result) {
		list($http_headers, $content) = explode("\r\n\r\n", $result, 2);
		$header_lines = explode("\r\n", $http_headers);
		list($protocol, $code, $message) = explode(" ", $header_lines[0], 3);

		// record the http status
		$formatted = array('status' => array('code' => $code, 'message' => $message));

		$xml = simplexml_load_string($content);

		// Look for Errors
		if (isset($xml->Errors)) {
			$formatted['RequestId'] = (string)$xml->RequestId;
			$formatted['Error'] = array();
			foreach($xml->Errors->Error as $error) {
				array_push($formatted['Error'], array(
					'Code' => (string)$error->Code,
					'Message' => (string)$error->Message
				));
			}
			return $formatted;
		}

		// Get the metadata for this request
		$metadata = $xml->ResponseMetadata;
		$formatted['RequestId'] = (string)$metadata->RequestId;
		$formatted['BoxUsage'] = (string)$metadata->BoxUsage;

		// GetAttributes Response
		if (isset($xml->GetAttributesResult)) {
			$formatted['Attribute'] = array();
			foreach($xml->GetAttributesResult->Attribute as $attribute) {
				array_push($formatted['Attribute'], array(
					'Name' => (string)$attribute->Name,
					'Value' => (string)$attribute->Value
				));
			}
		}

		// ListDomains Response
		if (isset($xml->ListDomainsResult)) {
			$formatted['DomainName'] = array();
			foreach($xml->ListDomainsResult->DomainName as $domain) {
				array_push($formatted['DomainName'], (string)$domain);
			}
			if (isset($xml->ListDomainsResult->NextToken)) {
				$formatted['NextToken'] = (string)$xml->ListDomainsResult->NextToken;
			}
		}

		// Query Response
		if (isset($xml->QueryResult)) {
			$formatted['ItemName'] = array();
			foreach($xml->QueryResult->ItemName as $item) {
				array_push($formatted['ItemName'], (string)$item);
			}
			if (isset($xml->QueryResult->NextToken)) {
				$formatted['NextToken'] = (string)$xml->QueryResult->NextToken;
			}
		}

		return $formatted;
	}
}

?>

February 4, 2009

JSON and POST in PHP

Filed under: Distributed System Design — Tags: , , , , , — Andy @ 10:25 pm

As I’ve been trying to do Lab 2 without having to modify my ami or change my apache configuration, I’ve found some nice helpers.

First, trying to encode and decode JSON in PHP 5.2 is easy… you just use the built in functions json_encode() and json_decode(). However, my Fedora ami is only running PHP 5.03. So, how do use JSON without recompiling my php installation, or downloading 5 billion files? Michal Migurski created a php-json library that is now a part of PEAR, but he still has a copy of his original encoder/decoder at http://mike.teczno.com/JSON/JSON.phps. It’s licenced BSD-style so have at it.

Next, I wanted to sent http requests by POST, including file uploads, again without downloading 5 billion files or messing with my ami. My solution was to actually learn the ‘application/x-www-form-urlencoded’ format and ‘multipart/form-data’ format and send the HTTP request across a socket.

A resource that helped me with the ‘application/x-www-form-urlencoded’ format is on http://www.wellho.net/resources/ex.php4?item=h110/getpost.php. For the ‘multipart/form-data’ format http://chxo.com/be2/20050724_93bf.html was very helpful. One gotcha to remember though is that PHP heredoc strings usually use \n line endings. While this may not cause any problems, to be safe and consistent with HTTP, you should use \r\n line endings.

Putting the two together into one function gave me the following:

function http_post($host, $path, $data_hash, $file = '', $file_param_name = '') {
	$boundary = md5(uniqid());
	if ($file && $file_param_name) {
		$binary = file_get_contents($file['tmp_name']);

		$content_type = "multipart/form-data; boundary=$boundary";

		$items = array();
		foreach (array_keys($data_hash) as $key) {
			array_push($items, "--$boundary\r\nContent-Disposition: form-data; name=\"$key\"\r\n\r\n{$data_hash[$key]}\r\n");
		}
		array_push($items, "--$boundary\r\nContent-Disposition: form-data; name=\"$file_param_name\"; filename=\"{$file['name']}\"\r\n");
		array_push($items, "Content-Type: {$file['type']}\r\nContent-Transfer-Encoding: binary\r\n\r\n$binary\r\n--$boundary--\r\n");
		$data = implode('', $items);
	} else {
		$content_type = 'application/x-www-form-urlencoded; charset=UTF-8';

		$items = array();
		foreach (array_keys($data_hash) as $key) {
			array_push($items, urlencode($key) . '=' . urlencode($data_hash[$key]));
		}
		$data = implode('&', $items);
	}

	$content_length = strlen($data);
	$fp = fsockopen($host, 80);
	fputs($fp, "POST $path HTTP/1.1\r\n");
	fputs($fp, "Host: $host\r\n");
	fputs($fp, "Content-Type: $content_type\r\n");
	fputs($fp, "Content-Length: $content_length\r\n");
	fputs($fp, "Connection: close\r\n\r\n");
	fputs($fp, $data, $content_length);

	$http_response = stream_get_contents($fp);
	fclose($fp);

	list($headers, $body) = explode("\r\n\r\n", $http_response, 2);
	return $body;
}

Note that the $file parameter would be $_FILES['your-form-input-name'], and $file_param_name would be ‘your-form-input-name’. $data_hash, I assume would be obvious. It’s an array with key => value pairs to send. The upload file would not appear in $data_hash.

January 19, 2009

Distributed System Design

Filed under: Distributed System Design — Andy @ 11:38 am

This is my last semester at BYU, and I am taking Distributes System Design. As I go through the preocess of creating (with my classmates) a distributed web application, I will be logging my progress and experiences here.

P.S. – Happy Civil Rights Day! I’m using this holiday to catch up on my homework.

Blog at WordPress.com.