Leveraging and Managing the StreamInsight Standalone Host

In my recent post that addressed the key things that you should know about Microsoft StreamInsight, I mentioned the multiple hosting options that are at your disposal.  Most StreamInsight examples (and documentation) that you find demonstrate the “embedded” server option where the custom application that you build hosts the StreamInsight engine in-process.  In this post, I’m going to dig into how you take advantage of the out-of-process standalone server for StreamInsight.  I’m also going to give you a little application I created that fills the gaps in the visual tooling for StreamInsight.

If you chose to leverage the embedded server model, your code would probably start off something like this:

//create embedded server
using (Server server = Server.Create("RSEROTER"))
{

//create application in the embedded server
var myApp = server.CreateApplication("SampleEvents");

// .. create query, start query

}

This type of solution is perfectly acceptable and provides the developer with plenty of control over the way the queries are managed.  However, you don’t get the high availability and reuse that the standalone server offers.

Creating the Host

So how do we use the remote, standalone host?  When you install StreamInsight, you are given the option to create a server host instance.

Above, you can see that I created an instance named RSEROTER.  When the installation is completed, a folder is created in the StreamInsight directory.

A Windows Service is also created for this instance, and it uses a configuration file from folder created above.

Configuring the Host

To be able to start this Windows Service, you’ll have to make sure that the endpoint address referenced in the service’s configuration file matches a registered endpoint for the server.  The configuration file for this StreamInsight host looks like this:

The endpoint address for the StreamInsight Management Service needs to be one of the addresses in my server’s reserved list.  Go to a command prompt and type netsh http show urlacl to see reserved endpoints and associated accounts.  Mine looks like this:

If your addresses and permissions line up, your service will start just fine. If your StreamInsight Windows Service uses a logon account that doesn’t have rights to the reserved endpoint, then the Windows Service won’t start. If the values in the configuration file and the registered endpoint list differ, the service won’t start. If you plan on using both an embedded and standalone server model concurrently, you will want to register a different URL and port for the embedded endpoints.

In my case, I changed the user account associated with my registered endpoint so that the StreamInsight Windows Service could open the endpoint. First I deleted the existing registered entry by using netsh http delete urlacl url=http://localhost:80/StreamInsight/RSEROTER/ and then added a new entry back with the right account (Network Service in my case) via netsh http add urlacl url=http://localhost:80/StreamInsight/RSEROTER user=”Network Service”. The StreamInsight installation guide has more details on setting up the right user accounts to prevent “access is denied” errors when connecting the debugger or trying to create/read server applications.

Considerations for Standalone Host Model

Now that you have a StreamInsight server instance started up, what should you know? Unlike the “embedded” StreamInsight hosting model where your application starts up and runs the StreamInsight engine in process, the standalone model uses a remote connection-based strategy.  The other thing to remember is that because you are using an out-of-process service, you also have to strong-name and GAC the assemblies containing your event payload definitions and adapters. Note that if you forget to start the Windows Service, you’ll get a warning that the WCF endpoint is in a faulted state.  Finally, be aware that you can only explicitly create a management endpoint in code if you have an embedded server.

Before I show you how to deploy queries to this standalone host, I should tell you about the management activities you CANNOT do via the only graphical tool that StreamInsight provides, the StreamInsight Event Flow Debugger.  The Debugger allows you to view existing applications, show queries included in applications, and both start and stop queries.  What you CANNOT do graphically is create applications, delete applications and delete queries.  So, I’ve built a tool that lets you do this.

The New StreamInsight Server Manager

Prior to writing code that connects to the StreamInsight server and deploys queries, I want to create the application container on the server.  I open up my StreamInsight Server Manager, connect to my endpoint (value read from my application’s configuration file) and choose to Create a new server application.

Once you have an application, you can right-click it and choose to either Delete the application or view any queries associated with it.

Coding to and Using the Standalone Server Instance

Let’s write some code!  I’ve built a console application that creates or starts a StreamInsight query.  First off, I use a “connect” operation to link to my standalone server host.

//connect to standalone server
using(Server server = Server.Connect(new System.ServiceModel.EndpointAddress(@"http://localhost/StreamInsight/RSEROTER")))
{

}

I then find the application that I created earlier.

Application myApp;
//get reference to existing application
myApp = server.Applications["CallCenterEvents"];

If my query is already on the server, than this application will just start it up.  Note that I could have also used my StreamInsight Server Manager or the Event Flow Debugger to simply start a server query.  I don’t need a custom application for that if I have a standalone server model.  But, this is what starting the query in code looks like:

//if query already exists, just start it
if (myApp.Queries.ContainsKey("All Events"))
{
Query eventQuery = myApp.Queries["All Events"];
eventQuery.Start();

//wait for keystroke to end
Console.ReadLine();

eventQuery.Stop();

}

If my query does NOT exist, then I create the query and start it up.  When I start my custom application, I can see from the StreamInsight Event Flow Debugger that my query is running.

If I flip to my StreamInsight Server Manager application, I can also see the query (and it’s status).

Unlike the Event Flow Debugger, this application also lets you delete queries.

Because I’m using the standalone server host option, I could choose to stop my custom application and my query is still available on the server.  I can now start and stop this query using the Event Flow Debugger or my StreamInsight Server Manager.

Summary

I expect that we’ll soon see more from Microsoft on building highly available StreamInsight solutions by using the standalone instance model.  This model is a great way to get reuse out of adapters and queries and get metadata durability in a central server host.  When using the standalone instance model you just have to remember the few things I pointed out above (e.g. using the GAC, getting the management endpoint set up right).

You can grab the executable and source code for the StreamInsight Server Manager here.  As you can expect from me in these situations, this is hardly production code.  But, it works fairly well and solves a problem.  It also may prove a decent example of how to access and loop through StreamInsight applications and queries.  Enjoy.