Leveraging and Managing the StreamInsight Standalone Host

In my recent post that addressed the key things that you should know about Microsoft StreamInsight, I mentioned the multiple hosting options that are at your disposal.  Most StreamInsight examples (and documentation) that you find demonstrate the “embedded” server option where the custom application that you build hosts the StreamInsight engine in-process.  In this post, I’m going to dig into how you take advantage of the out-of-process standalone server for StreamInsight.  I’m also going to give you a little application I created that fills the gaps in the visual tooling for StreamInsight.

If you chose to leverage the embedded server model, your code would probably start off something like this:

//create embedded server
using (Server server = Server.Create("RSEROTER"))
{

//create application in the embedded server
var myApp = server.CreateApplication("SampleEvents");

// .. create query, start query

}

This type of solution is perfectly acceptable and provides the developer with plenty of control over the way the queries are managed.  However, you don’t get the high availability and reuse that the standalone server offers.

Creating the Host

So how do we use the remote, standalone host?  When you install StreamInsight, you are given the option to create a server host instance.

2010.06.27si05

Above, you can see that I created an instance named RSEROTER.  When the installation is completed, a folder is created in the StreamInsight directory.

2010.06.27si01

A Windows Service is also created for this instance, and it uses a configuration file from folder created above.

2010.06.27si02

Configuring the Host

To be able to start this Windows Service, you’ll have to make sure that the endpoint address referenced in the service’s configuration file matches a registered endpoint for the server.  The configuration file for this StreamInsight host looks like this:

2010.06.27si03

The endpoint address for the StreamInsight Management Service needs to be one of the addresses in my server’s reserved list.  Go to a command prompt and type netsh http show urlacl to see reserved endpoints and associated accounts.  Mine looks like this:

2010.06.27si04

If your addresses and permissions line up, your service will start just fine. If your StreamInsight Windows Service uses a logon account that doesn’t have rights to the reserved endpoint, then the Windows Service won’t start. If the values in the configuration file and the registered endpoint list differ, the service won’t start. If you plan on using both an embedded and standalone server model concurrently, you will want to register a different URL and port for the embedded endpoints.

In my case, I changed the user account associated with my registered endpoint so that the StreamInsight Windows Service could open the endpoint. First I deleted the existing registered entry by using netsh http delete urlacl url=http://localhost:80/StreamInsight/RSEROTER/ and then added a new entry back with the right account (Network Service in my case) via netsh http add urlacl url=http://localhost:80/StreamInsight/RSEROTER user=”Network Service”. The StreamInsight installation guide has more details on setting up the right user accounts to prevent “access is denied” errors when connecting the debugger or trying to create/read server applications.

Considerations for Standalone Host Model

Now that you have a StreamInsight server instance started up, what should you know? Unlike the “embedded” StreamInsight hosting model where your application starts up and runs the StreamInsight engine in process, the standalone model uses a remote connection-based strategy.  The other thing to remember is that because you are using an out-of-process service, you also have to strong-name and GAC the assemblies containing your event payload definitions and adapters. Note that if you forget to start the Windows Service, you’ll get a warning that the WCF endpoint is in a faulted state.  Finally, be aware that you can only explicitly create a management endpoint in code if you have an embedded server.

Before I show you how to deploy queries to this standalone host, I should tell you about the management activities you CANNOT do via the only graphical tool that StreamInsight provides, the StreamInsight Event Flow Debugger.  The Debugger allows you to view existing applications, show queries included in applications, and both start and stop queries.  What you CANNOT do graphically is create applications, delete applications and delete queries.  So, I’ve built a tool that lets you do this.

The New StreamInsight Server Manager

Prior to writing code that connects to the StreamInsight server and deploys queries, I want to create the application container on the server.  I open up my StreamInsight Server Manager, connect to my endpoint (value read from my application’s configuration file) and choose to Create a new server application.

2010.06.27si06

Once you have an application, you can right-click it and choose to either Delete the application or view any queries associated with it.

2010.06.27si07

Coding to and Using the Standalone Server Instance

Let’s write some code!  I’ve built a console application that creates or starts a StreamInsight query.  First off, I use a “connect” operation to link to my standalone server host.

//connect to standalone server
using(Server server = Server.Connect(new System.ServiceModel.EndpointAddress(@"http://localhost/StreamInsight/RSEROTER")))
{

}

I then find the application that I created earlier.

Application myApp;
//get reference to existing application
myApp = server.Applications["CallCenterEvents"];

If my query is already on the server, than this application will just start it up.  Note that I could have also used my StreamInsight Server Manager or the Event Flow Debugger to simply start a server query.  I don’t need a custom application for that if I have a standalone server model.  But, this is what starting the query in code looks like:

//if query already exists, just start it
if (myApp.Queries.ContainsKey("All Events"))
{
Query eventQuery = myApp.Queries["All Events"];
eventQuery.Start();

//wait for keystroke to end
Console.ReadLine();

eventQuery.Stop();

}

If my query does NOT exist, then I create the query and start it up.  When I start my custom application, I can see from the StreamInsight Event Flow Debugger that my query is running.

2010.06.27si08

If I flip to my StreamInsight Server Manager application, I can also see the query (and it’s status).

2010.06.27si09

Unlike the Event Flow Debugger, this application also lets you delete queries.

2010.06.27si10

Because I’m using the standalone server host option, I could choose to stop my custom application and my query is still available on the server.  I can now start and stop this query using the Event Flow Debugger or my StreamInsight Server Manager.

2010.06.27si11

Summary

I expect that we’ll soon see more from Microsoft on building highly available StreamInsight solutions by using the standalone instance model.  This model is a great way to get reuse out of adapters and queries and get metadata durability in a central server host.  When using the standalone instance model you just have to remember the few things I pointed out above (e.g. using the GAC, getting the management endpoint set up right).

You can grab the executable and source code for the StreamInsight Server Manager here.  As you can expect from me in these situations, this is hardly production code.  But, it works fairly well and solves a problem.  It also may prove a decent example of how to access and loop through StreamInsight applications and queries.  Enjoy.

Share

Author: Richard Seroter

Richard Seroter is Director of Developer Relations and Outbound Product Management at Google Cloud. He’s also an instructor at Pluralsight, a frequent public speaker, the author of multiple books on software design and development, and a former InfoQ.com editor plus former 12-time Microsoft MVP for cloud. As Director of Developer Relations and Outbound Product Management, Richard leads an organization of Google Cloud developer advocates, engineers, platform builders, and outbound product managers that help customers find success in their cloud journey. Richard maintains a regularly updated blog on topics of architecture and solution design and can be found on Twitter as @rseroter.

11 thoughts

  1. When you say ” If you plan on using both an embedded and standalone server model concurrently, you will want to register a different URL and port for the embedded endpoints”, what do you mean by that?

    I have installed StreamInsight, and added a service called “StreamInsight”. Whether or not this service is started, demo code that does “CreateApplication(“Default”)” does not work. It works when I change “Default” to “StreamInsight”, but then I am unable to connect to the instance through the Event Flow Debugger.

    How do I resolve this?

    Thanks,
    Dave

    1. Hi Dave. The service name IS what you use when Creating or Connecting to an application, so yes, you need to use the “StreamInsight” name that you set up.

      What I was referring to was the *management endpoint* which uses registered URL endpoints that the Event Flow Debugger looks at. You can choose a URL for this when doing the embedded model, but the standalone model uses the URL in the streaminsight.exec.config file.

      That help?

      1. I found this code snippet:

        WSHttpBinding binding = new WSHttpBinding(SecurityMode.Message);
        binding.HostNameComparisonMode = HostNameComparisonMode.Exact;
        host.AddServiceEndpoint(typeof(IManagementService),
        binding,
        “http://localhost:8090/MyStreamInsightServer”);

        But when i add that in and try to connect to that endpoint URL through Event Flow Debugger and click “Applications” I get the following error: “The given key was not present in the dictionary”

        Have you ever seen that before?

    1. When I install a StreamInsight instance, with or without the accompanying service, I am not able to connect to an embedded host running in Visual Studio using the Event Flow Debugger, unless I explicitly specify a WSHttpBinding like in the above code sample. When I add that code along with host.Open() and host.Close(), etc., I am able to give it any endpoint URL, regardless of the instance name. I finally got past the “given key was not present” error I mentioned above (I was using v1.2 of the Event Flow Debugger and trying to connect to a v1.1 instance).

      Without this code, I would expect the default endpoint url to correspond to the default name for that instance, but Event Flow Debugger doesn’t allow me to connect to it the way I would expect when running an embedded version from within Visual Studio.

      1. I see. Yes, for an embedded StreamInsight server, you have to explicitly open up that WCF endpoint for the Event Flow Debugger to “see” your app. When you use the standalone service, the URL in the StreamInsight.exe.config file gets used so you don’t explicitly open an endpoint.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.