Category: Cloud

In my last post, I took an initial look at the Amazon Web Services (AWS) SimpleDB product and compared it to the Microsoft Windows Azure Table storage.  I showed that both solutions are relatively similar in that they embrace a loosely typed, flexible storage strategy and both provide a bit of developer tooling.  In that post, I walked through a demonstration of SimpleDB using the AWS SDK for .NET.

In this post, I’ll perform a quick demonstration of the Windows Azure Table storage product and then conclude with a few thoughts on the two solution offerings.  Let’s get started.

Windows Azure Table Storage

First, I’m going to define a .NET object that represents the entity being stored in the Azure Table storage.  Remember that, as pointed out in the previous post, the Azure Table storage is schema-less so this new .NET object is just a representation used for creating and querying the Azure Table.   It has no bearing on the underlying Azure Table structure. However, accessing the Table through a typed object differs from the AWS SimpleDB which has a fully type-less .NET API model.

I’ve built a new WinForm .NET project that will interact with the Azure Table.  My Azure Table will hold details about different conferences that are available for attendance.  My “conference record” object inherits from TableServiceEntity.

public class ConferenceRecord: TableServiceEntity
    {
        public ConferenceRecord()
        {
            PartitionKey = "SeroterPartition1";
            RowKey = System.Guid.NewGuid().ToString();

        }

        public string ConferenceName { get; set; }
        public DateTime ConferenceStartDate { get; set; }
        public string ConferenceCategory { get; set; }
    }

Notice that I have both a partition key and row key value.  The PartitionKey attribute is used to identify and organize data entities.  Entities with the same PartitionKey are physically co-located which in turn, helps performance.  The RowKey attribute uniquely defines a row within a given partition.  The PartitionKey + RowKey must be a unique combination.

Next up, I built a table context class which is used to perform operations on the Azure Table.  This class inherits from TableServiceContext and has operations to get, add and update ConferenceRecord objects from the Azure Table.

public class ConferenceRecordDataContext : TableServiceContext
    {
        public ConferenceRecordDataContext(string baseAddress, StorageCredentials credentials)
            : base(baseAddress, credentials)
        {}

        public IQueryable<ConferenceRecord> ConferenceRecords
        {
            get
            {
                return this.CreateQuery<ConferenceRecord>("ConferenceRecords");
            }
        }

        public void AddConferenceRecord(ConferenceRecord confRecord)
        {
            this.AddObject("ConferenceRecords", confRecord);
            this.SaveChanges();
        }

        public void UpdateConferenceRecord(ConferenceRecord confRecord)
        {
            this.UpdateObject(confRecord);
            this.SaveChanges();
        }
    }

In my WinForm code, I have a class variable of type CloudStorageAccount which is used to interact with the Azure account.  When the “connect” button is clicked on my WinForm, I establish a connection to the Azure cloud.  This is where Microsoft’s tooling is pretty cool.  I have a local “fabric” that represents the various Azure storage options (table, blob, queue) and can leverage this fabric without ever provisioning a live cloud account.

Connecting to my development storage through the CloudStorageAccount looks like this:

string connString = "UseDevelopmentStorage=true";

storageAcct = CloudStorageAccount.Parse(connString);

After connecting to the local (or cloud) storage, I can create a new table using the ConferenceRecord type definition, URI of the table, and my cloud credentials.

 CloudTableClient.CreateTablesFromModel(
                typeof(ConferenceRecordDataContext),
                storageAcct.TableEndpoint.AbsoluteUri,
                storageAcct.Credentials);

Now I instantiate my table context object which will add new entities to my table.

string confName = txtConfName.Text;
string confType = cbConfType.Text;
DateTime confDate = dtStartDate.Value;

var context = new ConferenceRecordDataContext(
      storageAcct.TableEndpoint.ToString(),
      storageAcct.Credentials);

ConferenceRecord rec = new ConferenceRecord
 {
       ConferenceName = confName,
       ConferenceCategory = confType,
       ConferenceStartDate = confDate,
  };

context.AddConferenceRecord(rec);

Another nice tool built into Visual Studio 2010 (with the Azure extensions) is the Azure viewer in the Server Explorer window.  Here I can connect to either the local fabric or the cloud account.  Before I run my application for the first time, we can see that my Table list is empty.

If I start up my application and add a few rows, I can see my new Table.

I can do more than just see that my table exists.  I can right-click that table and choose to View Table, which pulls up all the entities within the table.

Performing a lookup from my Azure Table via code is fairly simple and I can either loop through all the entities via a “foreach” and conditional, or, I can use LINQ.  Here I grab all conference records whose ConferenceCategory is equal to “Technology”.

var val = from c in context.ConferenceRecords
            where c.ConferenceCategory == "Technology"
            select c;

Now, let’s prove that the underlying storage is indeed schema-less.  I’ll go ahead and add a new attribute to the ConferenceRecord object type and populate it’s value in the WinForm UI.  A ConferenceAttendeeLimit of type int was added to the class and then assigned a random value in the UI.  Sure enough, my underlying table was updated with the new “column’” and data value.

I can also update my LINQ query to look for all conferences where the attendee limit is greater than 100, and only my latest column is returned.

Summary of Part II

In this second post of the series, we’ve seen that the Windows Azure Table storage product is relatively straightforward to work with.  I find the AWS SimpleDB documentation to be better (and more current) than the Windows Azure storage documentation, but the Visual Studio-integrated tooling for Azure storage is really handy.  AWS has a lower cost of entry as many AWS products don’t charge you a dime until you reach certain usage thresholds.  This differs from Windows Azure where you pretty much pay from day one for any type of usage.

All in all, both of these products are useful for high-performing, flexible data repositories.  I’d definitely recommend getting more familiar with both solutions.

We have a multitude of options for storing data in the cloud.  If you are looking for a storage mechanism for fast access to non-relational data, then both the Amazon Web Service (AWS) SimpleDB product and the Microsoft Windows Azure Table storage are viable choices.  In this post, I’m going to do a quick comparison of these two products, including how to leverage the .NET API provided by both.

First, let’s do a comparison of these two.

	Amazon SimpleDB	Windows Azure Table
Feature
Storage Metaphor	Domains are like worksheets, items are rows, attributes are column headers, items are each cell	Tables, properties are columns, entities are rows
Schema	None enforced	None enforced
“Table” size	Domain up to 10GB, 256 attributes per item, 1 billion attributes per domain	255 properties per entity, 1MB per entity, 100TB per table
Cost (excluding transfer)	Free up to 1GB, 25 machine hours (time used for interactions); $0.15 GB/month up to 10TB, $0.14 per machine hour	0.15 GB/month
Transactions	Conditional put/delete for attributes on single item	Batch transactions in same table and partition group
Interface mechanism	REST, SOAP	REST
Development tooling	AWS SDK for .NET	Visual Studio.NET, Development Fabric

These platforms are relatively similar in features and functions, with each platform also leveraging aspects of their sister products (e.g. AWS EC2 for SimpleDB), so that could sway your choice as well.

Both products provide a toolkit for .NET developers and here is a brief demonstration of each.

Amazon Simple DB using AWS SDK for .NET

You can download the AWS SDK for .NET from the AWS website.  You get some assemblies in the GAC, and also some Visual Studio.NET project templates.

In my case, I just built a simple Windows Forms application that creates a domain, adds attributes and items and then adds new attributes and new items.

After adding a reference to the AWSSDK.dll in my .NET project, I added the following “using” statements in my code:

using Amazon;
using Amazon.SimpleDB;
using Amazon.SimpleDB.Model;

Then I defined a few variables which will hold my SimpleDB domain name, AWS credentials and SimpleDB web service container object.

NameValueCollection appConfig;
AmazonSimpleDB simpleDb = null;
string domainName = "ConferenceDomain";

I next read my AWS credentials from a configuration file and pass them into the AmazonSimpleDB object.

appConfig = ConfigurationManager.AppSettings;
simpleDb = AWSClientFactory.CreateAmazonSimpleDBClient(appConfig["AWSAccessKey"],
                appConfig["AWSSecretKey"]);

Now I can create a SimpleDB domain (table) with a simple command.

CreateDomainRequest domainreq = (new CreateDomainRequest()).WithDomainName(domainName);
simpleDb.CreateDomain(domainreq);

Deleting domains looks like this:

DeleteDomainRequest deletereq = new DeleteDomainRequest().WithDomainName(domainName);
simpleDb.DeleteDomain(deletereq);

And listing all the domains under an account can be done like this:

string results = string.Empty;
ListDomainsResponse sdbListDomainsResponse = simpleDb.ListDomains(new ListDomainsRequest());
if (sdbListDomainsResponse.IsSetListDomainsResult())
{
   ListDomainsResult listDomainsResult = sdbListDomainsResponse.ListDomainsResult;
   
   foreach (string domain in listDomainsResult.DomainName)
   {
        results += domain + "\n";
    }
 }

To create attributes and items, we use a PutAttributeRequest object.  Here, I’m creating two items, adding attributes to them, and setting the value of the attributes.  Notice that we use a very loosely typed process and don’t work with typed objects representing the underlying items.

//first item
string itemName1 = "Conference_PDC2010";
PutAttributesRequest putreq1 = 
     new PutAttributesRequest().WithDomainName(domainName).WithItemName(itemName1);
List<ReplaceableAttribute> item1Attributes = putreq1.Attribute;
item1Attributes.Add(new ReplaceableAttribute().WithName("ConferenceName").WithValue("PDC 2010"));
item1Attributes.Add(new ReplaceableAttribute().WithName("ConferenceType").WithValue("Technology"));
item1Attributes.Add(new ReplaceableAttribute().WithName("ConferenceDates").WithValue("09/25/2010"));
simpleDb.PutAttributes(putreq1);

//second item
string itemName2 = "Conference_PandP";
PutAttributesRequest putreq2 = 
    new PutAttributesRequest().WithDomainName(domainName).WithItemName(itemName2);
List<ReplaceableAttribute> item2Attributes = putreq2.Attribute;
item2Attributes.Add(new ReplaceableAttribute().WithName("ConferenceName").
     WithValue("Patterns and Practices Conference"));
item2Attributes.Add(new ReplaceableAttribute().WithName("ConferenceType").WithValue("Technology"));
item2Attributes.Add(new ReplaceableAttribute().WithName("ConferenceDates").WithValue("11/10/2010"));
simpleDb.PutAttributes(putreq2);

If we want to update an item in the domain, we can do another PutAttributeRequest and specify which item we wish to update, and with which new attribute/value.

//replace conference date in item 2
ReplaceableAttribute repAttr = 
    new ReplaceableAttribute().WithName("ConferenceDates").WithValue("11/11/2010").WithReplace(true);
PutAttributesRequest putReq = 
    new PutAttributesRequest().WithDomainName(domainName).WithItemName("Conference_PandP").
    WithAttribute(repAttr);
simpleDb.PutAttributes(putReq);

Querying the domain is done with familiar T-SQL syntax.  In this case, I’m asking for all items in the domain where the ConferenceType attribute equals ‘Technology.”

string query = "SELECT * FROM ConferenceDomain WHERE ConferenceType='Technology'";
SelectRequest selreq = new SelectRequest().WithSelectExpression(query);
SelectResponse selresp = simpleDb.Select(selreq);

Summary of Part I

Easy stuff, eh?  Because of the non-existent domain schema, I can add a new attribute to an existing item (or new one) with no impact on the rest of the data in the domain.  If you’re looking for fast, highly flexible data storage with high redundancy and no need for the rigor of a relational database, then AWS SimpleDB is a nice choice.  In part two of this post, we’ll do a similar investigation of the Windows Azure Table storage option.