Databases aren’t a solved problem. Dozens of new options have sprouted up over the past five years as developers look for ways to effectively handle emerging application patterns. But how do you choose which one to use, especially when your data demands might span the competencies of an individual database engine?
NoSQL choices abound. Do you need something that stores key-value information for quick access? How about stashing JSON documents that represent fluid data structures? What should you do for Internet-of-Things scenarios or fast moving log data where “time” is a first class citizen? How should you handle dynamic relationships that require a social graph approach? Where should you store and use geo-spatial data? And don’t forget about the need to search all that data! If you need all of the aforementioned characteristics, then you typically need to stand up and independently manage multiple database technologies and weave them together at the app layer. Orchestrate.io does it differently, and yesterday, CenturyLink acquired them.
What does Orchestrate do? It’s a fast, hosted, managed, multi-model database fabric that is accessible through a single REST API. Developers only pay for API calls, and have access to a flexible key-value store that works with time-ordered events, geospatial data, and graph relationships. It runs an impressive tech stack under the covers, but all that the developers have to deal with is the web interface and API layer. In this blog post, I’ll walk through a simple sample app that I built in Node.js.
First off, go sign up for an Orchestrate account that comes with a nice free tier. Do it, now, I’ll wait.
Once in the Orchestrate management dashboard, I created an application. This is really just a container for data collections for a given deployment region. In my case, I chose one of the four brand new CenturyLink regions that we lit up last night. Psst, it’s faster on CenturyLink Cloud than on AWS.
For a given app, I get an API key used to authenticate myself in code. Let’s get to work (note that you can find my full project in this GitHub repo). I built a Node.js app, but you can use any one of their SDKs (Node, Ruby, Python, Java and Go with community-built options for .NET and PHP) or, their native API. To use the Node SDK, I added the package via npm.
In this app, I’ll store basketball player profiles, relationships between players, and a time-ordered game log. I spun up a Node Express project (using Visual Studio, which I’m getting used to), and added some code to “warm up” my collection by adding some records, some event data, and some relationships. You can also query/add/update/delete via the Orchestrate management dashboard, but this was a more repeatable choice.
var express = require('express'); var router = express.Router(); //Orchestrate API token var token = "<key>"; var orchestrate = require('orchestrate'); //location reference orchestrate.ApiEndPoint = "<endpoint>"; var db = orchestrate(token); /* Warmup the collection. */ router.get('/', function (req, res) { //create key/value records db.put('Players', "10001", { "name": "Blake Griffin", "team": "Los Angeles Clippers", "position": "power forward", "birthdate": "03/16/89", "college": "Oklahoma", "careerppg": "21.5", "careerrpg": "9.7" }) .then(function (result1) { console.log("record added"); }); //create key/value records db.put('Players', "10002", { "name": "DeAndre Jordan", "team": "Los Angeles Clippers", "position": "center", "birthdate": "07/21/88", "college": "Texas A&M", "careerppg": "8.0", "careerrpg": "9.0" }) .then(function (result2) { console.log("record added"); }); //create key/value records db.put('Players', "10003", { "name": "Matt Barnes", "team": "Los Angeles Clippers", "position": "strong forward", "birthdate": "03/09/80", "college": "UCLA", "careerppg": "8.1", "careerrpg": "4.5", "teams": [ "Los Angeles Clippers", "Sacramento Kings", "Golden State Warriors" ] }) .then(function (result3) { console.log("record added"); }); //create event db.newEventBuilder() .from('Players', '10001') .type('gamelog') .time(1429531200) .data({ "opponent": "San Antonio Spurs", "minutes": "43", "points": "26", "rebounds": "12" }) .create() .then(function (result4) { console.log("event added"); }); //create event db.newEventBuilder() .from('Players', '10001') .type('gamelog') .time(1429012800) .data({ "opponent": "Phoenix Suns", "minutes": "29", "points": "20", "rebounds": "8" }) .create() .then(function (result5) { console.log("event added"); }); //create graph relationship db.newGraphBuilder() .create() .from('Players', '10001') .related('currentteammate') .to('Players', '10002') .then(function (result6) { console.log("graph item added"); }); //create graph relationship db.newGraphBuilder() .create() .from('Players', '10001') .related('currentteammate') .to('Players', '10003') .then(function (result7) { console.log("graph item added"); }); res.send('warmed up'); }); module.exports = router;
After running the app and hitting the endpoint, I saw a new collection (“Players”) created in Orchestrate.
It’s always nice to confirm that things worked, so I switched to the Orchestrate UI where I queried my key/value store for one of the keys I added above. Sure enough, the item for Los Angeles Clippers superstar Blake Griffin comes back. Success.
Querying data items from code is super easy. Retrieve all the players in the collection? Couple lines of code.
/* GET all player listing. */ router.get('/', function (req, res) { db.list("Players") .then(function (result) { playerlist = result.body.results; //console.log(playerlist); res.render('players', { title: 'Players List', plist: playerlist }); }) .fail(function (err) { console.log(err); }); });
When the app runs, I get a list of players. Thanks to my Jade template, I’ve also got a hyperlink to the “details” page.
The player details include their full record info, any relationships to other players, and a log of games played. As you can see in the code below, it’s extremely straightforward to pull each of these entity types.
/* GET one player profile. */ router.get('/:playerid', function (req, res) { console.log(req.params.playerid); //get player object db.get("Players", req.params.playerid) .then(function (result) { player = result.body; //console.log(player); //get graph of relationships db.newGraphReader() .get() .from('Players', req.params.playerid) .related('currentteammate') .then(function (relres) { playerlist = relres.body; //console.log(playerlist); //get time-series events db.newEventReader() .from('Players', req.params.playerid) .type('gamelog') .list() .then(function (evtres) { gamelog = evtres.body; //console.log(gamelog); res.render('player', { title: 'Player Profile', profile: player, plist: playerlist, elist: gamelog }); }); }); }) .fail(function (err) { console.log(err); }); });
I didn’t bake in any true “search” capability into my app, but it’s one of the most powerful parts of the database service. Devs search through collections with a Lucene-powered engine with a robust set of operators. Do fuzzy search, deep full-text search of nested objects, grouping, proximity search, geo queries, aggregations, complex sorting, and more. Which player ever played for the Sacramento Kings? It’s easy to search nested objects.
Summary
I don’t usually flog my own company’s technology on this blog, but this is fun, important stuff. Developers are faced with a growing set of technologies they need to know in order to build efficient mobile experiences, data-intensive systems, and scalable cloud apps. Something like Orchestrate flips the script by giving developers a quick on-ramp to a suite of database engines that solve multiple problems.
Take it for a spin, and give me feedback for where we should go with it next!