Amazon SimpleDB Developer Guide

Table of Contents

Amazon SimpleDB Developer Guide

Credits

Foreword

About the Authors

About the Reviewers

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Errata

Piracy

Questions

1. Getting to Know SimpleDB

Experimenting with SimpleDB

How does SimpleDB work?

Domain

Item

Attributes

Values

How do I interact with SimpleDB?

How is SimpleDB priced?

Free tier

Why should I use SimpleDB?

Make your applications simpler to architect

Build flexibility into your applications

Create high-performance web applications

Take advantage of lower costs

Scale your applications on demand

Architect for the cloud

Summary

2. Getting Started with SimpleDB

Creating an AWS account

Enabling SimpleDB service for AWS account

Query API and authentication

SimpleDB libraries

SDBtool — Firefox plugin

Sample outline — performing basic operations

Basic operations with Java

Exploring SimpleDB with Java

Creating a domain with Java

Listing domains with Java

Manipulating items with Java

Deleting a domain with Java

Basic operations with PHP

Exploring SimpleDB with PHP

Creating a domain with PHP

Listing domains with PHP

Manipulating items with PHP

Deleting a domain with PHP

Basic operations with Python

Introducing boto

Exploring SimpleDB with Python

Creating a domain with Python

Retrieving a domain with Python

Creating items with Python

Summary

3. SimpleDB versus RDBMS

No normalization

No joins

No schemas

Simpler SQL

Only strings

Eventual consistency

Flash: February 24, 2010 — consistent read added

Scalability

Low maintenance

Advantages of the SimpleDB model

Disadvantages of the SimpleDB model

Summary

4. The SimpleDB Data Model

Domains

Domains with Java

Domains with PHP

Domains with Python

Exploring the metadata for a domain and costs

Retrieving domain metadata

Retrieving domain metadata with Java

Retrieving domain metadata with PHP

Retrieving domain metadata with Python

Components of a domain's metadata

SimpleDB domain constraints

Items

Adding attributes to an item with Java

Adding attributes to an item with PHP

Adding attributes to an item with Python

Constraints on SimpleDB items

Attributes

Attributes with Java

Attributes with PHP

Attributes with Python

Constraints on SimpleDB item attributes

Values

Storing multiple values in a single attribute with Java

Storing multiple values in a single attribute with PHP

Storing multiple values in a single attribute with Python

Constraints on values of a SimpleDB item

Consistent Read and Conditional Put / Delete announcement

ConsistentRead = true for getAttributes and Select

Conditional Put / Delete

Summary

5. Data Types

What is lexicographical comparison?

Storing numeric values

Storing numeric values with Java

Storing numeric values with PHP

Storing numeric values with Python

Storing date values

Storing date values with Java

Storing date values with PHP

Storing date values with Python

Storing Boolean values

Storing Boolean values with Java

Storing Boolean values with PHP

Storing Boolean values with Python

XML-restricted characters

Using base64 values with Java

Using base64 values with PHP

Using base64 values with Python

Summary

6. Querying

Sample data

Loading the sample data

Importing the sample data with Java

Importing the sample data with PHP

Importing the sample data with Python

Using Select

Simple select with Java

Simple select with PHP

Simple select with Python

Quoting values in Select expressions

Predicates in Select expressions

Simple predicate query with Java

Simple predicate query with PHP

Simple predicate query with Python

Comparison operators

Queries with Select

Comparison operators

Greater than

LIKE

NOT LIKE

BETWEEN

IN

IS NULL

IS NOT NULL

EVERY()

Combining expressions

INTERSECTION

NOT

OR

Sorting

Counting the results

COUNT()

Queries on multi-valued attributes

Querying for multi-valued attributes

getAttribute

getAttributes in Java

getAttributes in PHP

getAttributes in Python

Summary

7. Storing Data on S3

Amazon S3

Buckets

Objects

Keys

Pricing for S3

Plan of action

Creating an S3 bucket

Creating an S3 bucket with Java

Creating an S3 bucket with PHP

Creating an S3 bucket with Python

Creating additional metadata

Creating additional metadata with Java

Creating additional metadata with PHP

Creating additional metadata with Python

Uploading the songs to S3

Uploading the songs to S3 with Java

Uploading the songs to S3 with PHP

Uploading the songs to S3 with Python

Retrieving the files from S3

Retrieving the files from S3 with Java

Retrieving the files from S3 with PHP

Retrieving the files from S3 with Python

Summary

8. Tuning and Usage Costs

BoxUsage

Computing costs using BoxUsage

Usage reports

BoxUsage — Java

Cost of Select

BoxUsage — PHP

Cost of NextToken

Cost of Select

Cost of creating a domain

Cost of creating items

BoxUsage — Python

Partitioning

Summary

9. Caching

Caching

Memcached

Memcached and security

Installing memcached

Installing memcached client

Memcached client in Java

Memcached client in PHP

Memcached client in Python

Storing and retrieving data from memcached

Storing and retrieving data from memcached in Java

Storing and retrieving data from memcached in PHP

Storing and retrieving data from memcached in Python

Cache_Lite

Logic flow for using caching with SimpleDB

Using memcached with SimpleDB in Java

Using Cache_Lite with SimpleDB in PHP

Using memcached with SimpleDB in Python

Summary

10. Parallel Processing

BatchPutAttributes

BatchPutAttributes in Java

BatchPutAttributes in PHP

BatchPutAttributes in Python

Serialized SimpleDB requests

Parallelizing SimpleDB requests

Parallelizing SimpleDB requests in Java

Parallelizing SimpleDB requests in PHP

Parallelizing SimpleDB requests in Python

Simple threading

Threading with queues

Threading with workerpool

Concurrency and SimpleDB

Summary

Index

Amazon SimpleDB Developer Guide

Prabhakar Chaganti

Rich Helms

Amazon SimpleDB Developer Guide

Copyright © 2010 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: June 2010

Production Reference: 2240510

Published by Packt Publishing Ltd.

32 Lincoln Road

Olton

Birmingham, B27 6PA, UK.

ISBN 978-1-847197-34-4

www.packtpub.com

Cover Image by Tina Negus (<tina_manthorpe@sky.com>)

Credits

Authors

Prabhakar Chaganti

Rich Helms

Reviewers

Deepak Anupalli

Anders Samuelsson

Ashley Tate

Acquisition Editor

James Lumsden

Development Editors

Dhwani Devater

Reshma Sundaresan

Technical Editor

Ishita Dhabalia

Indexer

Monica Ajmera Mehta

Editorial Team Leader

Gagandeep Singh

Project Team Leader

Lata Basantani

Project Coordinator

Joel Goveya

Proofreader

Lynda Silwoski

Graphics

Nilesh Mohite

Production Coordinator

Adline Swetha Jesuthas

Cover Work

Adline Swetha Jesuthas

Foreword

Most software developers who work on the Internet love change. Change presents a new challenge, a new paradigm, and new technologies to learn. To realize this, all you have to do is look at the evolution of computers. During the 70s, we worked in a world of mainframes and raised floors. Only special people got to touch the computer, while others had to be content watching from outside of the fishbowl.

The 80s brought the mini-computer with dedicated CRT terminals. You could show data on the screen in any color as long as it was green, but the computer was down the hall in the back room. The 80s also introduced the personal computer. As PC power grew, the mini was replaced with the LAN-connected PC.

The 90s saw the advent of the Internet, and people dialed in, and in the early 2000s, the Internet went viral. As high-speed connections became common, the Internet replaced corporate networks. Computers went from rooms to luggables to in my briefcase to in my pocket.

In 2010, we are seeing the growth of cloud computing. Selecting a brand and model of server computer is being replaced with renting a virtual server at a hosting service like Amazon. The purchaser of these virtual servers doesn't have to select a hardware brand. I no more care about the brand of computer than I would care about what brand of pipe the water utility used to connect to my house. All I am buying is cycles and reliability.

This move to virtual servers also changes the capital required to propose the next viral application. I don't need to buy a large database cluster, hoping for the acceptance to fill it. I am billed for usage, not capacity. SimpleDB is one of those virtual offerings and the topic of this book.

Rich Helms

About the Authors

Prabhakar Chaganti is the founder and CTO of Ylastic, a startup that is building a single unified interface to architect, manage, and monitor a user's entire AWS Cloud computing environment: EC2, S3, RDS, AutoScaling, ELB, Cloudwatch, SQS, and SimpleDB. He is the author of Xen Virtualization and GWT Java AJAX Programming, both by Packt Publishing, and is also the winner of the community choice award for the most innovative virtual appliance in the VMware Global Virtual Appliance Challenge. He hangs out on Twitter as @pchaganti.

It's never been done is a call to action for Rich Helms. He has built a career on breaking new ground in the computer field. He developed CARES (Computer Assisted Recovery Enhancement System) for the Metropolitan Toronto Police in Canada. CARES was the first computer system in the world for aging missing children. CARES has been internationally recognized as pioneering work in child aging. Rich has also created several generations of e-learning platforms including Learn it script and most recently Educate Press.

Rich can be reached at http://webmasterinresidence.ca.

Rich is a seasoned software developer with over 30 years of experience. He spent 22 years in various positions at IBM including Chief Image Technology Architect. His credentials range from deep technical work (five patents in hardware and software) to running multinational R&D.

About the Reviewers

Deepak Anupalli is Architect for the Server Engineering group at Pramati Technologies. He has deep insight into various Java/J2EE technologies. He represents Pramati on the EJB and JPA expert groups and has led the Java EE 5 certification effort of Pramati Server. He is currently leading the effort to build a standards-based web-scale Application server. He is a visiting faculty member with IIIT-Hyderabad for a course on middleware and also speaks at various technology conferences. He holds a graduate degree in Computer Science and Engineering from National Institute of Technology (NIT Warangal, India).

Anders Samuelsson has over 25 years of experience in the computing industry. The main focus during this time has been with computer security. He currently works for Amazon.com with Amazon Web Services.

I'd like to thank my wife Malena and my son Daniel and daughter Ida, for always standing by me and allowing me to spend time helping out with this book. I love you forever.

Ashley Tate is the founder of Coditate Software and the creator of Simple Savant, an advanced C# interface to SimpleDB. He is currently working on GridRoom, an application for collaborative sports-video review built on several Amazon Web Services, including SimpleDB. He lives near Atlanta with his wife and four children. You can find him online at http://blog.coditate.com.

Preface

SimpleDB is a highly scalable, simple-to-use, and inexpensive database in the cloud from Amazon Web Services. But in order to use SimpleDB, you really have to change your mindset. This isn't a traditional relational database; in fact it's not relational at all. For developers who have experience working with relational databases, this may lead to misconceptions as to how SimpleDB works.

This practical book aims to address your preconceptions on how SimpleDB will work for you. You will be led quickly through the differences between relational databases and SimpleDB, and the implications of using SimpleDB. Throughout this book, there is an emphasis on demonstrating key concepts with practical examples for Java, PHP, and Python developers.

You will be introduced to this massively scalable schema less key-value data store: what it is, how it works, and why it is such a game changer. You will then explore the basic functionality offered by SimpleDB including querying, code samples, and a lot more. This book will help you deploy services outside the Amazon cloud and access them from any web host.

You will see how SimpleDB gives you the freedom to focus on application development. As you work through this book you will be able to optimize the performance of your applications using parallel operations, caching with memcache, asynchronous operations, and more.

Gain in-depth understanding of Amazon SimpleDB with PHP, Java, and Python examples, and run optimized database-backed applications on Amazon's Web Services cloud.

What this book covers

Chapter 1, Getting to Know SimpleDB, explores SimpleDB and the advantages of utilizing it to build web-scale applications.

Chapter 2, Getting Started with SimpleDB, moves on to set up an AWS account, enable SimpleDB service for the account, and install and set up libraries for Java, PHP, and Python. It also illustrates several SimpleDB operations using these libraries.

Chapter 3, SimpleDB versus RDBMS, sheds light on the differences between SimpleDB and a traditional RDBMS, as well as the pros and cons of using SimpleDB as the storage engine in your application.

Chapter 4, The SimpleDB Data Model, takes a detailed look at the SimpleDB data model and different methods for interacting with a domain, its items, and their attributes. It further talks about the domain metadata and reviews the various constraints imposed by SimpleDB on domains, items, and attributes.

Chapter 5, Data Types, discusses the techniques needed for storing different data types in SimpleDB, and explores a technique for storing numbers, Boolean values, and dates. It also teaches you about XML-restricted characters and encoding them using base64 encoding.

Chapter 6, Querying, describes the Select syntax for retrieving results from SimpleDB, and looks at the various operators and how to create predicates that allow you to get back the information you need.

Chapter