Category: BizTalk

  • Setting “KeepAlive” Value in BizTalk Web Service Calls

    A few months back I posted about getting “canceled web requests” when calling a service on WebLogic from a BizTalk Server. Now, there appears to be a Microsoft hotfix that can address this.

    While looking for another hotfix, I located this …

    The cause given states “This problem occurs because you cannot set the HTTP header KeepAlive property to false when you use the HTTP adapter to send a message.”

    There’s a non-hotfix workaround offered (which isn’t great), and then a description on how to set the “KeepAlive” to “false” after applying the hotfix. It’s a bit humorous, however, that the installation instructions include this little tidbit … “We do not recommend that you deploy this schema because future BizTalk Server updates may include an HTTP schema to set the KeepAlive property.” I’d prefer you not offer it as an option then! It’s recommended that you do NOT actually build out the property schema, but instead set the KeepAlive value in the pipeline.

    Setting KeepAlive to false isn’t a great thing to do, but if you’re desperate, you now have a means to do it.

    Technorati Tags:

  • XML, Web Services and Special Characters

    If you’ve worked with XML technologies for any reasonable amount of time, you’re aware of the considerations when dealing with “special” characters. This recently came up at work, so I thought I’d share a few quick thoughts.

    One of the developers was doing an HTTP post of XML content to a .NET web service. However, we discovered that a few of the records coming across had invalid characters.

    Now you probably know that the following message is considered invalid XML:

    <Person>
    	<Name>Richard</Name>
    	<Nickname>Thunder & Lightning</Nickname>
    </Person>
    

    The ampersand (“&”) isn’t allowed within a node’s text. Neither are “<“, “>” and a few others. Now if you call a web service by first doing an “Add Web Reference” in Visual Studio.NET, you are using a proxy class that covers up all the XML/SOAP stuff going on underneath. The proxy class (Reference.cs) inherits System.Web.Services.Protocols.SoapHttpClientProtocol, which you can see (using Reflector) takes care of proper serialization using the XmlWriter object. So setting my web service parameters like so …

    When this actually goes across the wire to my web service, the payload has been appropriate encoded and the ampersand has been replaced …

    However, if I decided to do my own HTTP post to the service and bypass a proxy, this is NOT the way to do it ..

    HttpWebRequest webRequest = 
       (HttpWebRequest)HttpWebRequest.Create("http://localhost/bl/sv.asmx");
    webRequest.Method = "POST";
    webRequest.ContentType = "text/xml";
    
    using (Stream reqStream = webRequest.GetRequestStream())
    {
    
      string body = "<soap:Envelope xmlns:soap="+
      "\"http://schemas.xmlsoap.org/soap/envelope/\">"+
      "<soap:Body><Operation_1 xmlns=\"http://tempuri.org/\">" +
      "<ns0:Person xmlns:ns0=\"http://testnamespace\">" +
      "<ns0:Name>Richard & Amy</ns0:Name>" +
      "<ns0:Age>10</ns0:Age>" +
       "<ns0:Address>411 Broad Street</ns0:Address>" +
      "</ns0:Person>" +
      "</Operation_1></soap:Body></soap:Envelope>";
    
        byte[] bodyBytes = Encoding.UTF8.GetBytes(body);
        reqStream.Write(bodyBytes, 0, bodyBytes.Length);
    
    }
    HttpWebResponse webResponse = 
       (HttpWebResponse)webRequest.GetResponse();
    MessageBox.Show("submitted, " + webResponse.StatusCode);
    
    webResponse.Close();
    

    Why is this bad? This may work for most scenarios, but in the case above, I have a special character (“&”) that is about to go unmolested across the wire …

    Instead, the code above should be augmented to use an XmlTextWriter to build up the XML payload. These types of errors are such a freakin’ pain to debug since no errors actually get thrown when the receiving service fails to serialize the bad XML into a .NET object. In a BizTalk world, this means no SOAP exception to the caller, no suspended message, no error in the Event Log. Virtually no trace (outside of the IIS logs). Not good.

    BizTalk itself doesn’t like poorly constructed XML either. The XmlReceive pipeline, in addition to “typing” the message (http://namespace#root) also parses the message. So while everyone says that the default XmlReceive pipeline doesn’t validate the structure (meaning XSD structure) of the message, it DOES validate the XML structure of the message. Keep that in mind. If I try to pass an invalid XML document (special characters, unclosed tags) that WILL bomb out in the pipeline layer.

    If you try to cheat, and do pass-through pipelines and use XmlDocument as your initial orchestration message (thus bypassing any peeking at the message by BizTalk), you will still receive errors when you try to interact with the message later on. If you set the XmlDocument to the actual message variable in the orchestration, the message gets parsed at that time and fails if the structure is invalid.

    So, this is probably elementary for you smart people, but it’s one of those little things that you might forget about. Be careful about generating XML content via string building and instead consider using XmlDocuments or XmlWriters to make sure that your content passes XML parsing rules.

    Technorati Tags: ,

  • Adventures With WCF and BizTalk

    After my mini-rant on WCF last week, I figured that my only course of action was to spend a bit of my free time actually re-learning WCF (+ BizTalk) and building out the scenarios that most interest me.

    In my effort to move my WCF skill set from “able to talk about it” to “somewhat dangerous”, I built each of the following scenarios:

    Scenario Comments
    Service hosted in Windows Form (HTTP) Pretty simple to build the service contract, and use operations made up of simple types and complex types (using [DataContract]). Fairly straightforward to modify the app.config used by the WinForm host to hold the Http endpoint (and provide metadata support). Screwed around with various metadata options for a while, and found this blog post on metadata publication options quite useful during my adventures. To consume the service, I used svcutil.exe to build the message, client objects and sample configuration file. Decided to call the service using the client vs. going directly at the ChannelFactory.
    Service hosted in Windows Form (TCP) Liked that the ServiceHost class automatically loads up all the endpoints in the host configuration. No need to explicitly “start” each one. Don’t love that by default, the generated configuration file (from svcutil.exe) uses the same identifier for the bindingConfiguration and name values. This mixed me up for a second, so I’ve taken to changing the name value to something very specific.
    Service hosted in IIS I don’t learn well by “copy/paste” scenarios, but I DO like having a reference model to compare against. That said, this post on hosting WCF services in IIS is quite useful to use as a guide. Deploying to IIS was easier than I expected. My previous opinion that setting up WCF services takes too many steps must have been a result of getting burned by an early build of Indigo.
    Service generated by BizTalk (WSHttp) and hosted in IIS BizTalk WCF Wizard is fairly solid. Deployed a new WSHttp service to IIS, used svcutil.exe to build the necessary consuming components, and ripped out the bits from the generated configuration file and added them to my existing “WCF Consumer” application. See the steps below which I followed to get my BizTalk-generated service ready to run.
    Service Generated By BizTalk (TCP) and hosted in BizTalk I added a receive location to the receive port generated by the WCF Wizard in the scenario above. I then walked through the WCF Wizard again, this time creating a MEX endpoint in IIS to provide the contract/channel information for the service consumer. As expected (but still neat to see), the endpoint in the app.config generated by svcutil.exe had the actual TCP endpoint stored, not the MEX endpoint in IIS. Of course that’s how it’s supposed to work, but I’m easily amused. I was able to call this service using identical code (except for the endpoint configuration name) as the WSHttp BizTalk service.
    Service Generated by BizTalk (WSHttp) and hosted in BizTalk This excites me a bit. Hosting my web service in process without needing to use IIS. I plan on exploring this scenario much more to identify how handling is different on an in-process hosted web service vs. an IIS hosted on (how exceptions are handled, security configuration, load balancing). To make this work, I created yet another receive location on the above created receive port, set the adapter as WCF-Custom and chose the WS-Http binding. I also added a metadata behavior in case I wanted to generate any bits using svcutil.exe. Instead of generating any new bits, I simply added an endpoint to my configuration file while reusing the same binding, bindingConfiguration and contract as my other WsHttp service. After switching my code to use this new endpoint configuration, everything processed successfully.
    Consuming basicHttp WCF service via classic “add web reference” This was my “backwards compatible” test. Could I build a fancy WCF service that my non-WCF clients could consume easily? If I charge forward with WCF, do I risk screwing up the plethora of systems that use SOAP Basic Profile 1.1 as their web interface? My WCF service provided a basicHttp binding in addition to more robust binding options. In Visual Studio.NET I did an “add web reference” and attempted to use this WCF service as I would a “classic” SOAP service. And … it worked perfectly. So it shouldn’t matter if a sizable part of my organization can’t utilize WS* features in the near future. I can still “downgrade” services for their consumption, while providing next-level capabilities to clients that support it.

    I’ve got a few more scenarios queued up (UriTemplates, security configurations, transactions and reliable sessions), but so far, things are looking good. My wall of skepticism is slowly crumbling.

    That said, I still had a bit of work to first get all this running. First off, I got the dreaded plain text shows up when browsing the svc file issue. I reinstalled .NET Framework 3.0 and reassociated it with IIS and it appears that this cleared things up. However, after first walking through the BizTalk WCF Publishing Wizard, I got the following page upon browsing the generated IIS-hosted web service:

    Ok, next step was to add <customErrors mode=”Off”/> to the web.config file. This now resulted in this error:

    Once again, SharePoint screws me up. If you’ve got SharePoint on the box, you need to add <trust level=”Full” originUrl=”” /> to your web.config file. In fairness, this is mentioned in the BizTalk walkthrough as a “note”. After adding this setting, I now got this message:

    That’s cool. The WSHttpWebServiceHostFactory used by the service is in tune with the BizTalk configuration, so it knows the receive location is currently disabled. Once I enable the receive location, I get this:

    All in all, a nice experience. A bit of trial and error to get things right, but that’s the best way to learn, right?

    Technorati Tags: ,

  • SoCal BizTalk [and WCF/WF] User Groups Started Up

    BizTalk Server has always benefitted by a strong community of contributors. One might argue that the PRIMARY reason that BizTalk took hold with so many shops is the availability of information in newsgroups, blogs, user groups, open source projects, and discussion boards. For the longest time, the official Microsoft documentation was a bit thin, so the community provided the depth of information that developers needed. Clearly Microsoft has done a significantly better job explaining the guts of BizTalk and providing solid samples and tools, but, the BizTalk community is where I still look to for creative ideas and innovative solutions.

    All that said, I’m glad to see that Southern California is finally getting BizTalk (and WCF/WF) user groups set up. Group discussion and debate is often where the best ideas originate. In SoCal we now have …

    Southern California has dozens of BizTalk customers, ranging in scale from 70+ processors to one processor. Each organization has unique use cases, but there’s a wide cross-section of common challenges and best practices. We also have some of the brightest and most forward-thinking implementation partners, so I’ll be jazzed to hear what those folks have to say as well. I’m looking forward to hanging out with the LA UG crowd.

    Technorati Tags:

  • New Whitepaper on BizTalk + WCF

    Just finished reading the excellent new whitepaper from Aaron Skonnard (hat tip: Jesus) entitled Windows Communication Foundation Adapters in Microsoft BizTalk Server 2006 R2. Very well written and it provides an exceptionally useful dissection of the BizTalk 2006 R2 usage of WCF. Can’t recommend it enough.

    That said, I still have yet to entirely “jump into the pool” on WCF yet. It’s like a delicious, plump steak (WCF) when all I’m really want is a hamburger (SOAP Basic Profile). My shop is very SOAP-over-HTTP focused for services, so the choice of channel bindings is a non-starter for me. Security for us is handled by SOA Software, so I really don’t need an elaborate services security scheme. I like the transaction and reliability support, so that may be where the lightbulb really goes on for me. I probably need to look harder for overall use cases inside my company, but for me, that’s often an indicator that I have a solution with no problem. Or, that I’m a narrow-minded idiot who has to consider more options when architecting a solution. Of course with the direction that BizTalk is heading, and all this Oslo stuff, I understand perfectly that WCF needs to be a beefy part of my repetoire moving forward.

    In the spirit of discussing services, I also just finished the book RESTful Web Services and found it an extremely useful, and well-written, explanation of RESTful design and Resource Oriented Architecture. The authors provided a detailed description of how to identify and effectively expose resources, while still getting their digs at “Big Web Services” and the challenges with WSDL and SOAP. As others have stated, it seems to me that a RESTful design works great with CRUD operations on defined resources, but within enterprise applications (which aren’t discussed AT ALL in this book), I like having a strong contract, implementation flexibility (on hazier or aggregate resources) and access to WS* aspects when I need them. For me, the book did a bit of disservice only focusing on Amazon S3 and Flickr (and like services) without identifying how this sort of design holds up for the many enterprise applications that developers build web services integration for. On a day to day basis, aren’t significantly more developers building services to integrate with SAP/Oracle/custom app then the internet-facing services used as the examples in the book?

    All of this is fairly irrelevant to me since WCF has pleasant support for both URI-based services (through UriTemplate) and RPC-style services and developers can simply choose the right design for each situation. Having a readable URI is smart whether you’re doing RFC-style SOAP calls using only HTTP POST, or doing the academically friendly RESTful manner. The REST vs. WS* debate reminds me of a statement by my co-worker a few weeks back (and probably lifted from elsewhere): “The reason that debates in academia are so intense is because the stakes are so small.” Does it really matter which service design style your developers go with, assuming they are built well? Seems like a lot of digital ink has been spent on a topic that shouldn’t cause anyone to lose sleep.

    Speaking of losing sleep, it’s time for me to change and feed my new boy. As you were.

    Technorati Tags:

  • Painful Oracle Connectivity Problems

    I’ve spent the better part of this week wrestling with Oracle connectivity issues, and figured I’d share a few things I’ve discovered.

    A recent BizTalk application deployment included an orchestration that does a simple update to an Oracle table. Instead of using the Oracle adapter, I used .NET component and the objects in the System.Data.OracleClient .NET framework namespace. As usual, everything worked fine in the development and test environments.

    Upon moving to production, all of a sudden I was seeing the following error with some frequency:

    Logging failure … System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
    at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)

    Yowza. The most common reason for this occuring is failing to properly close/dispose a database connection. After scouring the code, I was positive that this wasn’t the case. After a bit of research, I came across the following two Microsoft .NET Framework hotfixes:

    So in a nutshell, bad database connections are by default, returned to the connection pool. Nice. I went ahead and applied this hotfix in production, but still saw intermittent (but less frequent) occurences of the error above.

    Next, I decided to turn on the SQL/Oracle performance counters so that I could actually see the pooling going on. There are a few counters that are “off” by default (including NumberOfActiveConnections and NumberOfFreeConnections) and require a flag in the application configuration file. To add these counters, go to the BTSNTSvc.exe.config file, and add the following section …

    <system.diagnostics>
        <switches>
          <add name="ConnectionPoolPerformanceCounterDetail"
               value="4"/>
        </switches>
      </system.diagnostics>
    

    Now, on my BizTalk server, I can add performance counters for the .NET Data Provider for Oracle and see exactly what’s going on.

    For my error above, the most important counter to initially review is NumberofReclaimedConnections which indicates how many database connections were cleaned up by the .NET Garbage Collector and not closed properly. If this number was greater than 0, or increasing over time, then clearly I’d have a connection leak problem. In my case, even under intense load, this value stayed at 0.

    When reviewing the NumberOfFreeConnections counter, I noticed that this was usually 0. Because my database connection string didn’t include any pooling details, I wasn’t sure how many connections the pool allocated automatically. As desperation set in, I decided to tweak my connection string to explicitly set pooling conditions (new part in bold):

    User Id=useracct1;Password=secretpassword;
       Data Source=prod_system.company.com;
       Pooling=yes;Max Pool Size=100;Min Pool Size=5;
    

    Once I did this, my counters looked like my picture above, with a minimum of 5 connections available in the pool. As I type this (2 days after applying this “fix”), the problem has yet to resurface. I’m not declaring victory yet since it’s too small of a sample size.

    However, given the grief that this has caused me, I’m tempted to switch from the System.Data.OracleClient to the System.Data.Odbc objects where I’ve had previous success and never seen this error in production. My other choice is give up my dream of using the API altogether and use the BizTalk Oracle adapter instead. Thoughts?

    To add insult to my week of Oracle connectivity hell, I’ve noticed that the Oracle adapter for a DIFFERENT application has been spitting this message out with greater occasion …


    Failed to send notification : System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

    Naturally the message in the Event Log doesn’t tell me which send/receive port this is associated with because that would make troubleshooting less exciting. Anyone else see this rascal when using the Microsoft Biztalk Adapters for Enterprise Applications? I’ve also see it on occasion with my .NET code solution.

    All of this is the reason I missed the Los Angeles BizTalk Server 2006 R2 launch event this week. I’m still bitter. However, I’m told that bets were made at the event as to whether I’d blog more or less while out on paternity leave in a week or two, so it’s nice to know they were thinking of me! Stay tuned.

    Technorati Tags: ,

  • How to Distinguish BizTalk Schema Record Nodes

    I recently came across a newsgroup post discussing distinguishing fields in an auto-generated SQL Adapter schema, and after a bit of investigation, came up with a way to easily distinguish schema records.

    Now Jan Eliasen gave a perfectly good response to the newsgroup post, and helpfully pointed to his blog post on how to flip the default “records” to “elements” for easier manipulation.

    This however got me thinking as to whether the restriction on distinguishing record types was a tool limitation, or compiler/engine related. If you try to distinguish a record type, the “Promoted Properties” window doesn’t enable the “Add” button. Given that a “record” is really just an XSD element, and that often auto-generated schemas build all the nodes as records, this limitation sometimes screws you. So, I opened my XSD schema in the VS.NET XML Editor instead of the BizTalk Editor.

    I then manually added a new “distinguished field” to the “properties” collection of the schema. After saving, and then opening the schema once more in the BizTalk Editor, voila, it now shows up as a distinguished field in the “Promoted Properties” window.


    To prove that this isn’t some sort of trickery, I then processed a message through the BizTalk engine, stopped by send port, and observed the context properties of my message. Sure enough, my “record” was properly distinguished and accessible.

    I got a little frisky and wondered if I could also solve the age-old problem of distinguishing repeating nodes. The Editor tool prevents this activity because there’s no way to designate which index of the repeating node you want. The standard solution is to promote/distinguish in an inbound pipeline instead. However, what if you KNEW that you only wanted the first repeating node as the distinguished value? Could you also manually add this distinguished field to the schema?

    Alas, despite numerous varieties of syntax, I couldn’t get the compiler to approve of this. I consistently got the compile time error saying The promoted property field or one of its parents has Max Occurs greater than 1. Only nodes that are guaranteed to be unique can be promoted as property fields.. I tried using “position()=1” or a “[1]” indexer, and either way, I struck out.

    But, at least now I have a simple way to distinguish records, so it’s not a total loss.

    Technorati Tags:

  • Issue When Serializing BizTalk Auto-Generated Schemas To .NET Objects

    Yesterday a co-worker of mine was having issues serializing an auto-generated BizTalk schema into a .NET object. We found an obscure fix that solved the problem.

    In Darren’s Professional BizTalk Server 2006 book, he’s a proponent of working with serializable classes (instead of messages) where possible. In our case, my buddy Prashant was doing some mass Oracle table updates using data retrieved from the BizTalk Siebel adapter. Instead of having countless “Oracle Insert” messages, we discussed simply turning the Siebel messages into .NET objects and using a helper class to do one big transactional insert.

    So, he took the Siebel adapter schemas, ran them through xsd.exe, and ended up with a nice .NET object representing all the nodes in the schema. However, upon doing the XLANGMessage “RetrieveAs” operation, he got a gnarly error (actual type names removed) stating:

    Cannot use XLANGMessage.RetrieveAs to convert message part part with type [SampleNamespace].[TypeName]+QueryEx2Response to type QueryEx2Response.”

    Exception type: InvalidCastException
    Source: Microsoft.XLANGs.Engine
    Target Site: System.Object RetrieveAs(System.Type)

    Unable to generate a temporary class (result=1).
    error CS0030:
    Cannot convert type ‘Customer_Complaint_Case_BCResultRecord[]’ to
    ‘Customer_Complaint_Case_BCResultRecord’
    error CS0029:
    Cannot implicitly convert type
    ‘Customer_Complaint_Case_BCResultRecord’ to
    ‘Customer_Complaint_Case_BCResultRecord[]’

    Ouch. Well from reading that, clearly there looks like a problem serializing that “BCResultRecord” array. After doing a quick web search, I came across a newsgroup post discussing the same serialization problem we hit. The solution? Add a temporary “attribute” to the unbounded item to force the xsd.exe tool to properly deal with array types. So, before the change, my offending piece of the Siebel-generated XSD looked like this:

    <xsd:complexType name="Customer_Complaint_Case_BCResultRecordSet">
        <xsd:sequence>
          <xsd:element minOccurs="0" maxOccurs="unbounded" 
    	  name="Customer_Complaint_Case_BCResultRecord" 
    	  type="BizObj:Customer_Complaint_Case_BCResultRecord" />
        </xsd:sequence>
      </xsd:complexType>
      

    When running xsd.exe, the generated type looked like this …

    public partial class QueryEx2Response {
        
        private Customer_Complaint_Case_BCResultRecord[][] 
    	    Customer_Complaint_Case_BCResultRecordSetField;
        
        [System.Xml.Serialization.XmlArrayItemAttribute
    	(typeof(Customer_Complaint_Case_BCResultRecord),
    	 Namespace="http://schemas.microsoft.com/Business_Objects",
    	  IsNullable=false)]
        public Customer_Complaint_Case_BCResultRecord[][] 
                       Customer_Complaint_Case_BCResultRecordSet {
            get {
             return this.Customer_Complaint_Case_BCResultRecordSetField;
            }
            set {
             this.Customer_Complaint_Case_BCResultRecordSetField = value;
            }
        }
    }
    

    Here’s where the problem was. So, I *temporarily* tweaked the schema to add the temporary attribute …

    <xsd:complexType name="Customer_Complaint_Case_BCResultRecordSet">
        <xsd:sequence>
          <xsd:element minOccurs="0" maxOccurs="unbounded" 
    	  name="Customer_Complaint_Case_BCResultRecord" 
    	  type="BizObj:Customer_Complaint_Case_BCResultRecord" />
        </xsd:sequence>
        <xsd:attribute name="temp" type="xsd:string" />
      </xsd:complexType>
      

    NOW, after re-running xsd.exe, my generated type looked like this …

    public partial class QueryEx2Response {
        
        private Customer_Complaint_Case_BCResultRecordSet[] 
    	Customer_Complaint_Case_BCResultRecordSetField;
        
        [System.Xml.Serialization.XmlElementAttribute
    	("Customer_Complaint_Case_BCResultRecordSet")]
        public Customer_Complaint_Case_BCResultRecordSet[] 
                      Customer_Complaint_Case_BCResultRecordSet {
            get {
             return this.Customer_Complaint_Case_BCResultRecordSetField;
            }
            set {
             this.Customer_Complaint_Case_BCResultRecordSetField = value;
            }
        }
    }
    

    You can see how the generated class now recognizes the “BCResultRecordSet” object as an array, vs. using a double-array of type “BCResultRecord.” Also, the metadata about accessor changed from being a XmlArrayItemAttribute to a XmlElementAttribute. Once this change was made, everything worked perfectly.

    I was able to successfully switch the schema back to it’s original form (sans “temporary attribute”), and the serialization still worked fine. The key was adding that temporary attribute for the creation of the serializable class only. You don’t need to keep this temporarily attribute in the schema after that.

    I suspect that this situation would arise for many of the auto-generated schemas from the BizTalk adapters (Siebel, Oracle, Peoplesoft, SQL Server etc). It’s quite nice to deal with these messages as pure .NET objects, but watch out for tricky serialization issues.

    Technorati Tags:

  • New Microsoft Whitepaper on BizTalk Ordered Delivery

    Interesting new white paper from Microsoft on maintaining ordered delivery across concurrent orchestrations (read online or download here).

    Specifically, this paper identifies an architecture where you receive messages in order, stamp them with a sequence number in a receive pipeline, process them through many parallel orchestration instances, and then ensure resequencing prior to final transmission. The singleton “Gatekeeper” orchestration does the resequencing by keeping track of the most recent sequence number, and then temporarily storing out-of-sequence messages (in memory) until their time is right for delivery.

    One thing that’s wisely highlighted here is the considerations around XLANG/s message lifetime management. Because orchestration messages are being stored (temporarily) in an external .NET object, you need to make sure the XLANG engine treats them appropriately.

    Good paper. Check it out.

    Technorati Tags:

  • Securely Storing Passwords for Accessing SOA Software Managed Services

    One tricky aspect of consuming a web service managed by SOA Software is that the credentials used in calling the service must be explicitly identified in the calling code. So, I came up with a solution to securely and efficiently manage many credentials using a single password stored in Enterprise Single Sign On

    A web service managed by SOA Software may have many different policies attached. There are options for authentication, authorization, encryption, monitoring and much more. To ease the confusion on the developers calling such services, SOA Software provides a clean API that abstracts away the underlying policy requirements. This API speaks to the Gateway, which attaches all the headers needed to comply with the policy and then forwards the call to the service itself. The code that a service client would implement might look like this …

    Credential soaCredential = 
        new Credential("soa user", "soa password");
    
    //Bridge is not required if we are not load balancing
    SDKBridgeLBHAMgr lbhamgr = new SDKBridgeLBHAMgr();
    lbhamgr.AddAddress("http://server:9999");
    
    //pass in credential and boolean indicating whether to 
    //encrypt content being passed to Gateway
    WSClient wscl = new WSClient(soaCredential, false);
    WSClientRequest wsreq = wscl.CreateRequest();
    
    //This credential is for requesting (domain) user. 
    Credential requestCredential = 
        new Credential("DOMAIN\user", "domain password");
    
    wsreq.BindToServiceAutoConfigureNoHALB("unique service key", 
        WSClientConstants.QOS_HTTP, requestCredential);
    

    The “Credential” object here doesn’t accept a Principal object or anything similar, but rather, needs specific values entered. Hence my problem. Clearly, I’m not going to store clear text values here. Given that I will have dozens of these service consumers, I hesitate to use Single Sign On to store all of these individual sets of credentials (even though my tool makes it much simpler to do so).

    My solution? I decided to generate a single key (and salt) that will be used to hash the username and password values. We originally were going to store these hashed values in the code base, but realized that the credentials kept changing between environments. So, I’ve created a database that stores the secure values. At no point are the credentials stored in clear text in the database, configuration files, or source code.

    Let’s walk through each component of the solution.

    Step #1

    Create an SSO application to store the single password and salt used to encrypt/decrypt all the individual credential components. I used the SSO Configuration Store Application Manager tool to whip something up. Then upon instantiation of my “CryptoManager”, I retrieve those values from SSO and cache them in the singleton (thus saving the SSO roundtrip upon each service call).

    Step #2

    I need a strong encryption mechanism to take the SOA Software service passwords and turn them into gibberish to the snooping eye. So, I built a class that encrypts a string (for design time), and then decrypts the string (for runtime). You’ll notice my usage of the ssoPassword and ssoSalt values retrieved from SSO. The encryption operation looks like this …

    /// <summary>
    /// Symmetric encryption algorithm which uses a single key and salt 
    /// securely stored in Enterprise Single Sign On.  There are four 
    /// possible symmetric algorithms available in the .NET Framework 
    /// (including DES, Triple-DES, RC2, Rijndael/AES). Rijndael offers 
    /// the greatest key length of .NET encryption algorithms (256 bit) 
    /// and is currently the most secure encryption method.  
    /// For more on the Rijndael algorithm, see 
    /// http://en.wikipedia.org/wiki/Rijndael
    /// </summary>
    /// <param name="clearString"></param>
    /// <returns></returns>
    public string EncryptStringValue(string clearString)
    {
        //create instance of Rijndael class
        RijndaelManaged RijnadaelCipher = new RijndaelManaged();
        //let add padding to ensure no problems with encrypted data 
        //not being an even multiple of block size
        //ISO10126 adds random padding bytes, vs. PKCS7 which does an 
        //identical sequence of bytes
        RijnadaelCipher.Padding = PaddingMode.ISO10126;
    
        //convert input string to a byte array
        byte[] inputBytes = Encoding.Unicode.GetBytes(clearString);
    
        //using a salt makes it harder to guess the password.
        byte[] saltBytes = Encoding.Unicode.GetBytes(ssoSalt);
    
        //Derives a key from a password
        PasswordDeriveBytes secretKey = 
    	    new PasswordDeriveBytes(ssoPassword, saltBytes);
    
        //create encryptor which converts blocks of text to cipher value 
        //use 32 bytes for secret key
        //and 16 bytes for initialization vector (IV)
        ICryptoTransform Encryptor = 
    	    RijnadaelCipher.CreateEncryptor(secretKey.GetBytes(32), 
                     secretKey.GetBytes(16));
    
        //stream to hold the response of the encryption process
        MemoryStream ms = new MemoryStream();
    
        //process data through CryptoStream and fill MemoryStream
        CryptoStream cryptoStream = 
    	    new CryptoStream(ms, Encryptor, CryptoStreamMode.Write);
        cryptoStream.Write(inputBytes, 0, inputBytes.Length);
    
        //flush encrypted bytes
        cryptoStream.FlushFinalBlock();
    
        //convert value into byte array from MemoryStream
        byte[] cipherByte = ms.ToArray();
    
        //cleanup
        //technically closing the CryptoStream also flushes
        cryptoStream.Close();
        cryptoStream.Dispose();
        ms.Close();
        ms.Dispose();
    
        //put value into base64 encoded string
        string encryptedValue = 
            System.Convert.ToBase64String(cipherByte);
    
        //return string to caller
        return encryptedValue;
    }
    

    For decryption, it looks pretty similar to the encryption operation …

    public string DecryptStringValue(string hashString)
    {
        //create instance of Rijndael class
        RijndaelManaged RijnadaelCipher = new RijndaelManaged();
        RijnadaelCipher.Padding = PaddingMode.ISO10126;
    
        //convert input (hashed) string to a byte array
        byte[] encryptedBytes = Convert.FromBase64String(hashString);
    
        //convert salt value to byte array
        byte[] saltBytes = Encoding.Unicode.GetBytes(ssoSalt);
    
        //Derives a key from a password
        PasswordDeriveBytes secretKey = 
    	    new PasswordDeriveBytes(ssoPassword, saltBytes);
    
        //create decryptor which converts blocks of text to cipher value
    	//use 32 bytes for secret key
        //and 16 bytes for initialization vector (IV)
        ICryptoTransform Decryptor = 
    	    RijnadaelCipher.CreateDecryptor(secretKey.GetBytes(32), 
                     secretKey.GetBytes(16));
    
        MemoryStream ms = new MemoryStream(encryptedBytes);
    
        //process data through CryptoStream and fill MemoryStream
        CryptoStream cryptoStream = 
    	    new CryptoStream(ms, Decryptor, CryptoStreamMode.Read);
    
        //leave enough room for plain text byte array by using length of 
    	//encrypted value (which won't ever be longer than clear text)
        byte[] plainText = new byte[encryptedBytes.Length];
    
        //do decryption
        int decryptedCount = 
            cryptoStream.Read(plainText, 0, plainText.Length);
    
        //cleanup
        ms.Close();
        ms.Dispose();
        cryptoStream.Close();
        cryptoStream.Dispose();
    
        //convert byte array of characters back to Unicode string
        string decryptedValue = 
            Encoding.Unicode.GetString(plainText, 0, decryptedCount);
    
        //return plain text value to caller
        return decryptedValue;
    }
    

    Step #3

    All right. Now I have an object that BizTalk will call to decrypt credentials at runtime. However, I don’t want these (hashed) credentials stored in the source code itself. This would force the team to rebuild the components for each deployment environment. So, I created a small database (SOAServiceUserDb) that stores the service destination URL (as the primary key) and credentials for each service.

    Step #4

    Now I built a “DatabaseManager” singleton object which upon instantiation, queries my SOAServiceUserDb database for all the web service entries, and loads them into a member Dictionary object. The “value” of my dictionary’s name/value pair is a ServiceUser object that stores the two sets of credentials that SOA Software needs.

    Finally, I have my actual implementation object that ties it all together. The web service proxy class first talks to the DatabaseManager to get back a loaded “ServiceUser” object containing the hashed credentials for the service endpoint about to be called.

    //read the URL used in the web service proxy; call DatabaseManager
    ServiceUser svcUser = 
        DatabaseManager.Instance.GetServiceUserAccountByUrl(this.Url);
    

    I then call into my CrytoManager class to take these hashed member values and convert them back to clear text.

    string bridgeUser = 
        CryptoManager.Instance.DecryptStringValue(svcUser.BridgeUserHash);
    string bridgePw = 
        CryptoManager.Instance.DecryptStringValue(svcUser.BridgePwHash);
    string reqUser = 
        CryptoManager.Instance.DecryptStringValue(svcUser.RequestUserHash);
    string reqPw = 
        CryptoManager.Instance.DecryptStringValue(svcUser.RequestPwHash);
    

    Now the SOA Software gateway API uses these variables instead of hard coded text.

    So, when a new service comes online, we take the required credentials and pass them through my encryption algorithm to get a hash value, then add a record in the SOAServiceUserDb to store the hash value, and that’s about it. As we migrate between environments, we simply have to keep our database in sync. Given that my only real risk in this solution is using a single password/salt to hash all my values, I feel much better knowing that the critical password is securely stored in Single Sign On.

    I would think that this strategy stretches well beyond my use case here. Thoughts as to how this could apply in other “single password” scenarios?

    Technorati Tags: