3 Rarely Discussed, But Valuable, Uses for Cloud Object Storage

I’ve got object storage on the brain. I’m finishing up a new Pluralsight course on distributed systems in AWS that uses Amazon S3 in a few places, and my employer Tier 3 just shipped a new Object Storage service based on Riak CS Enterprise. While many of the most touted uses for cloud-based object storage focus on archived data, backups, media files and the like, there are actually 3 more really helpful uses for cloud-based object storage.

1. Provided a Degraded “Emergency Mode” Website

For a while, AWS has supported running static websites in S3. What this means is that customers can serve simple static HTML sites out of S3 buckets. Why might you want to do this? A cool blog post last week pointed out the benefits of having a “hot spare” website running in S3 for when the primary site is flooded with traffic. The corresponding discussion on Hacker News called out a bit more of the logistics. Basically, you can use the AWS Route 53 DNS service to mark the S3-hosted website as a failover that is only used when health checks are failing on the primary site. For cases when a website is overloaded because it gets linked from a high-profile social site, or gets flooded with orders from a popular discount promotion, it’s handy to use a scalable, rock solid object storage platform to host the degraded, simple version of a website.

2. Partner file transfer

Last year I wrote about using Amazon S3 or Windows Azure Blob Storage for managed file transfer. While these are no substitute for enterprise-class MFT products, they are also a heck of a lot cheaper. Why use cloud-based object storage to transfer files between business partners? Simplicity, accessibility, and cost. For plenty of companies, those three words do not describe their existing B2B services that rely on old FTP infrastructure. I’d bet that plenty of rogue/creative employees are leveraging services like Dropbox or Skydrive to transfer files that are too big for email and too urgent to wait for enterprise IT staff to configure FTP. Using something like Amazon S3, you have access to ultra-cheap storage that has extreme high availability and is (securely) accessible by anyone with an internet connection.

I’ve spent time recently looking at the ecosystem of tools for Amazon S3, and it’s robust! You’ll find free, freemium, and paid software options that let you use a GUI tool (much like an FTP browser) or even mount S3 object storage as a virtual disk on your computer. Check out the really nice solutions from S3 Browser, Cloud Berry, DragonDisk, Bucket Explorer, Cross FTP, Cyberduck, ExpanDrive, and more. And because products like Riak CS support the S3 API, most of these tools “just work” with any S3-compliant service. For instance, I wrote up a Tier 3 knowledge base article on how to use S3 Browser and ExpanDrive with our own Tier 3 Object Storage service.

3. Bootstrap server builds

You have many choices when deciding how to deploy cloud servers. You could create templates (or “AMIs” in the AWS world) that have all the software and configurations built in, or you could build up the server on the fly with software and configuration scripts stored elsewhere.

By using cloud-based object storage as a repository for software and scripts, you don’t have to embed them in the templates and have to maintain them. Instead, you can pass in arguments to the cloud server build process and pull the latest bits from a common repository. Given that you shouldn’t ever embed credentials in a cloud VM (because they can change, among other reasons), you can use this process (and built in identity management integration) to have a cloud server request sensitive content – such as ASP.NET web.config with database connection strings – from object storage and load it onto the machine. This could be part of the provisioning process itself (see example of doing it with AWS EMR clusters) or as a startup script that runs on the server. Either way,  consider using object storage as a centrally accessible source for cloud deployments and upgrades!

Summary

Cloud-based object storage has lots of uses besides just stashing database backups and giant video files. The easy access and low cost makes it a viable option for the reasons I’ve outlined here. Any other ways you can imagine using it?