Richard Seroter's Architecture Musings

Author: Richard Seroter

  • Oracle ODBC Driver Problems After Database Upgrade

    Given that my posts on BizTalk + Oracle integration are consistently my most popular, I thought I’d add another one to the mix.

    The first BizTalk application that my company deployed required BizTalk to poll an Oracle 8i database.   When BizTalk was first installed here, we followed the documented instructions and installed the Oracle 9.2 client, and specifically, the Oracle 9.2.0.54 version of the ODBC driver.  For the most part, everything has been smooth sailing.

    Recently, we took that application down for to fix a bug in the Oracle view, and while that work was being done, the underlying Oracle database was upgraded from 8i to 9i.   No big deal, right?  Well, after the bug had been fixed in the Oracle view, and we turned the application back on in our “test” environment, we noticed some very bizarre behavior.

    After some initial investigation, I saw that the value (of type varchar2) being pulled from the Oracle database appeared to be truncated!  Only seven characters (out of eight) were being pulled, and, every other value came in empty.  Yowza. 

    So where to begin troubleshooting this?  The first place we looked was the database itself.  Had anything changed?  Oracle 9i has significantly more Unicode support, so we checked to see if data was improperly converted from 7-bit ASCII to Unicode.  The data appeared to be intact and unmolested. 

    Given that no changes had occurred on the BizTalk application itself, it seemed unlikely that BizTalk was the culprit.  We checked a few basic things such as the “maxLength” value (identical in both BizTalk schema and Oracle view), and also confirmed that the BizTalk binding configuration had not changed.

    So, next we looked at the connection to Oracle itself.  Was the ODBC connection the troublemaker here?  I decided to open Visual Studio.NET, and using the Server Explorer, create a connection to my Oracle database by reusing the existing Oracle ODBC DSN.  Sure enough, when I viewed the Oracle view in question, I saw truncated (and missing) values!  I knew positively that the underlying data was right, so the connection was the piece that was distorting my data.

    I confirmed this by going to a different environment (with the Oracle 10g client installed) and using Visual Studio.NET to browse the Oracle database table.  Using the 10g client, the data appeared correctly.

    So, in our development environment, our administrators tried downgrading our Oracle ODBC driver to 9.2.0.0.0.  When we did that, the data also appeared correctly.  However, I wasn’t comfortable downgrading, and suggested moving one Oracle ODBC version past the Microsoft-recommended 9.2.0.54.  We installed the 9.2.0.65 version of the ODBC driver, and once again, everything looked perfect.  After that, one of the DBAs suggested skipping 9.2.0.65 (due to a few noteworthy bugs) and jump to 9.2.0.8 (with the latest DST patch).

    So what the heck caused this?  I still don’t have a great answer.  The only change to this system was the database upgrade, but the DBAs were fairly confident that existing connections should not have been affected.  The set of tubes known as the internet turned up very few results.  There was some information on Oracle ODBC and Unicode, and some other gripes with the 9.2.0.54 driver, but nothing that  explained why the ODBC connection to the database mangled my result set.

    Moral of the story?  When troubleshooting these sorts of scenarios, check off each area that could have caused the problem, and use multiple environments (with different configurations) to isolate the problem.  Then when all else fails, upgrade Oracle drivers!

    Technorati Tags: ,

  • Changing Roles

    It’s been just about a year since I left Microsoft to take my current job, and after some success in my initial role, I’m switching teams and responsibilities.

    I was brought into this company to help establish our BizTalk practice.  After participating in 10 projects, teaching 8 classes (to 110+ colleagues), creating a few tools and frameworks, and writing up some best practices and checklists, it was time for new challenges.

    So, I’ve moved into our Solutions Architecture team as a Solutions Architect.  My job now is to execute architecture reviews, help define and guide architectural principles, design and model new systems, research and introduce new technologies, and more.   While I’ll remain the “BizTalk guy” here for a while, that’s no longer my central technology each and every day.

    Blog-wise, expect to continue seeing various tidbits and demonstrations of BizTalk concepts (because it’s still fun for me), but I’ll probably start peppering in other architectural topics that are on my mind.

    Technorati Tags: , ,

  • Microsoft Architecture Journal Reader Tool

    Yesterday I was checking out the MSDN Architecture Center and noticed a reader application for the Architecture Journal.

    This interface makes the Journal easy to read, and demonstrates a few nice UI concepts.

    This tool is auto-updating, so it should automatically pull down the latest versions of the Journal.  You can add interesting articles to a “reading list” for later reference.   For each particular article, you have the option of saving it to the desktop, emailing it, or copying the summary to the Windows clipboard.

    Another nice feature is the ability to add notes to each article.  Once a note is added to a particular passage or section, that area is highlighted with a different color.  I was hoping that my notes would show up in the search results, but alas, I’d have to remember where I had added notes vs. finding them later via broad search.

    I find this more interesting as a UI design then for actually reading the Journal.  There probably aren’t enough issues to justify reading list and search functionality (I’d like to see this tool for MSDN Magazine!), but, it does offer a few interesting ideas about an internet-updated thick client.

    Technorati Tags: ,

  • InfoPath 2007 Scenarios With BizTalk Server 2006 R2

    I’m a fan of InfoPath, but one barrier to entry has been the need to install the client software on user machines.  We have one deployed solution that uses it (as part of ESB Guidance), but I wanted to explore the new Forms Services capability and see how I can use that to simplify BizTalk workflow use cases.  In this post, I will examine a few common use cases, and demonstrate how I built them.  The theme of the solution is the workflow around system support incident management.

    InfoPath Setup

    To build this solution, first I needed schemas with which to generate the necessary InfoPath forms.  So, within a new BizTalk project I created an “Incident” schema that looked like this:

    Next, I have a “Survey” schema which the system owner will fill out after the incident has been successfully resolved.

    After deploying the BizTalk solution, I went ahead and built a Web Service using the BizTalk Web Services Publishing Wizard so that incidents can be sent from InfoPath directly back to the running BizTalk workflow process.

    Now, I can go ahead and build the necessary InfoPath forms.  When designing the form, I’ve chosen to support both the InfoPath rich client AND, InfoPath Forms Services.

    The form itself is fairly basic.  It simply uses the XSD schema as a data source and allows for capture of incident data.

    The first tricky part was getting the “Submit” action to work.  On my first iteration building this, the rich client could submit to the web service just fine, but the Forms Services version kept giving me “an error occurred accessing the data source.”  So, I had to learn all about UDC files and SharePoint data connections (thanks to the InfoPath Team Blog and Mark Bower‘s posts).  So, my InfoPath form’s “submit action” now points to a SharePoint-managed data connection.

    I then deployed this Incident form to my SharePoint server.  When deploying forms in InfoPath 2007, you’ll see that Forms Services is mentioned.

    Once deployed, I can go to the SharePoint document library’s Advanced Settings and set the form to open in the browser by default.

    Next I built and deployed the “Survey” form which will be saved directly to the SharePoint library, so no extra submit action is needed.

    BizTalk Setup

    On the BizTalk side, I built a simple workflow to demonstrate the following use cases:

    • Emailing a link to a InfoPath form existing in SharePoint
    • Receiving submitted feedback from InfoPath back into BizTalk
    • Emailing a link to a “new” document for someone to fill out and save in SharePoint

    Also, to deal with the ridiculous InfoPath/SharePoint namespace bug, I decided to build a Jeff Lynch-style map so that now my promoted columns show up in the SharePoint document library.

    To send the first email (asking the user to fill out the Incident Report), I need the correct URL to embed in the email message.  Since I clearly need a Incident to refer to in this hyperlink, I first send the Incident to the SharePoint library.  Because I dynamically set the file name, I have that value in my orchestration, and can use it for my email link.  The link looks like:

    http://myserver:89/sites/Richard/Incident%20Reporting/”+wssMessageName+”?OpenIn=Browser

    The next hyperlink I need is for the “Survey” so that I can ask someone to create an entirely new form (that doesn’t already exist in the SharePoint document library).  What does that look like?

    http://myserver:89/sites/Richard/Satisfaction%20Survey/Forms/template.xsn?
    SaveLocation=http://myserver:89/sites/Richard/Satisfaction%20Survey&
    Source=http://myserver:89/sites/Richard/Satisfaction%20Survey&
    OpenIn=Browser

    Running the Scenario

    Ok, so let’s kick this off and see how it looks.  When I drop a file (to signify a system sending an incident report), I expect to see a message sent to SharePoint, AND, an email with a link to the same document.  In my SharePoint library I see …

    Notice that a form can be viewed either in the rich client or browser.  In my email box I have a link to the web version of my Incident form.  Clicking that link brings me to the form served up by InfoPath Forms Services.

    Notice that I have both “save” and “submit” buttons available on the web form.  The “submit” button will trigger the default submit action, which in my case, calls my BizTalk-generated web service.

    Once the form submits, I then get a receipt (via my orchestration), and, a request to fill out a satisfaction survey.  The email link creates a new empty form, that when saved, will appear in my SharePoint document library.  Notice that I turned off “submit” in this form, since there is no submit action.

    Summary

    So, unlike with InfoPath 2003, InfoPath 2007 makes it very easy to design a form once, and have it surfaced up via the rich client, web browser, or mobile browser with no additional effort.  From a BizTalk perspective, instead of emailing forms around and trying to keep track of them, we can now send links to web forms and be confident that any user, regardless of platform or software install, can participate in our workflow.   This should make it much more compelling to use InfoPath + SharePoint in workflow solutions instead of doing custom development.

    Technorati Tags: , ,

  • BizTalk Atomic Transaction Rollback of Variables

    This week I’m delivering a BizTalk development class for my colleagues, and during my preparation of demonstrations, I spent a few extra minutes confirming the behavior of members variables and atomic transactions.

    Atomic transactions in BizTalk don’t necessarily behave like some folks expect.  For instance, if a BizTalk orchestration calls a component with ADO.NET code which updates a database, and then raises an error later in the atomic scope, this will NOT cause that database operation to be rolled back.  In another example, if you place a “send” shape inside an atomic scope, and a later operation fails, BizTalk doesn’t roll back the send operation.  It’s standard design to use BizTalk “compensation” to execute logic to roll back any operations performed in the atomic scope.

    Darren’s excellent BizTalk book states that there are realistically three scenarios where a developer will use atomic transactions in a BizTalk orchestration:

    • Using a class not marked as serializable
    • Calling a COM+ component that’s participating in the orchestration transaction
    • Minimizing persistence points

    However, don’t assume that atomic transactions in BizTalk are completely neutered.   There are cases that you would want to use an atomic transaction as a part of your actual process.

    In this scenario, my orchestration has two variables.  One references a custom .NET data object (Employee), and the other variable is a simple string.  After receiving a message into the orchestration, I instantiate those variables with default values and print them out.

    Next, I have an atomic transaction which changes the value of both variables, prints them, and then throws an error.  Immediately after the atomic scope, I print the variable values once more.  As you can see here, I’ve also wrapped the atomic scope inside a long-running scope so that I can swallow the error and move on gracefully.

    What do you expect to happen?  Does the orchestration reset the member variable only, or also the custom .NET object’s member value?

    As you can see, both variables start off with values of “started up”, inside the atomic scope the value switches to “changed status”, and after the exception is thrown within the atomic scope, the values get reset to their pre-atomic scope values.  It makes perfect sense since changes occurring within the atomic scope don’t commit until the transaction is complete, but, it’s always useful to test it.

    Now, this scope behavior does not extend much beyond this.  If I add a function to my “Employee” class which writes a file to disk, and I call that operation within the atomic scope, the exception does NOT cause that operation to roll back.  The file stays on my drive even after the orchestration has completed.  That’s a scenario where compensation would be needed.

    Technorati Tags:

  • Presentations Available Online for Microsoft SOA/BPM Conference

    If you missed the recent SOA & BPM Conference from Microsoft, you can now review nearly all of the presentation decks via the conference website.

    Visit the presentation download page to grab PDF versions of material.

    Technorati Tags:

  • BizTalk In-Process Hosting Of WCF Http Services

    After my post of various WCF scenarios, I received a couple questions about using the in-process host to receive WCF HTTP requests, so I thought I’d briefly show my configuration setup for making this work.

    First off, I had created a “regular” IIS-hosted WCF web service and auto-generated a receive port and location. I decided to reuse that receive port, and created a new receive location for my in-process HTTP receive. I used the WCF-Custom adapter, which as you can see, runs only within an in-process host.

    The first adapter configuration tab is where you identify the endpoint URL. This value is completely made-up. I chose an unused port (8910), and then created my desired URL.

    Next, on the Binding tab, I set the wsHttpBinding as the desired type.

    Next, I added a behavior for “serviceMetadata” to allow for easy discovery of my service contract.

    That’s it for the receive location configuration. I need to enable the receive location in order to instantiate the WCF service host. If I try to browse to my service URL while the location is disabled, I get a “page cannot be displayed” error. Once I enable the location, and hit my made-up URL in the browser, I can see the service description. Note that if I had not created the serviceMetadata behavior, I would have received a “Metadata publishing for this service is currently disabled.” message when viewing my service in the browser.

    So, now I can generate the necessary client-side objects and configuration to call this service. My client application’s configuration file has the following endpoint entry:

    <endpoint 
   address="http://localhost:8910/incidentreporting/incident.svc"
   binding="wsHttpBinding" 
   bindingConfiguration="WSHttpBinding_ITwoWayAsyncVoid"
   contract="Service1" name="IncidentInProcSvc">
   <identity>
       <userPrincipalName value="myserver\user123" />
   </identity>
</endpoint>

    You’ll notice my endpoint address matches the value in the receive location, and an “identity” node exists because my service configuration (in the receive location) identified clientCredentialType as “Windows” for message/transport security.

    There you go. Pretty easy to “build” a service that is hosted within the BizTalk process, completely bypassing IIS, and leave the service consumer none the wiser.

    UPDATE: You may notice that nowhere above did I build a contract into the service itself. I reused a contract in my client endpoint, but how would the service consumer know what to send to my service? This is probably where you’d decide to create a MEX endpoint. You’d point at the WS-Custom receive location in the WCF Publishing Wizard, and choose a schema(s) to represent the contract. Then users would point to the MEX service to generate their strongly-typed client components.

    Technorati Tags: ,

  • New Co-Worker Blog

    One of my brightest co-workers decided to set up a blog this week, and I encourage you to check him out.

    The first post for Victor Fehlberg’s Tech Postings (about Victor) goes over the process of setting up Terminal Services access to a shared BizTalk environment. Besides being a newly minted BizTalk guru, Victor’s our resident expert on DataFlux and is also a rock star on Java, RUP and UML, so expect a nice variety of interesting topics.

    Welcome aboard, Victor.

    Technorati Tags:

  • Setting “KeepAlive” Value in BizTalk Web Service Calls

    A few months back I posted about getting “canceled web requests” when calling a service on WebLogic from a BizTalk Server. Now, there appears to be a Microsoft hotfix that can address this.

    While looking for another hotfix, I located this …

    The cause given states “This problem occurs because you cannot set the HTTP header KeepAlive property to false when you use the HTTP adapter to send a message.”

    There’s a non-hotfix workaround offered (which isn’t great), and then a description on how to set the “KeepAlive” to “false” after applying the hotfix. It’s a bit humorous, however, that the installation instructions include this little tidbit … “We do not recommend that you deploy this schema because future BizTalk Server updates may include an HTTP schema to set the KeepAlive property.” I’d prefer you not offer it as an option then! It’s recommended that you do NOT actually build out the property schema, but instead set the KeepAlive value in the pipeline.

    Setting KeepAlive to false isn’t a great thing to do, but if you’re desperate, you now have a means to do it.

    Technorati Tags:

  • XML, Web Services and Special Characters

    If you’ve worked with XML technologies for any reasonable amount of time, you’re aware of the considerations when dealing with “special” characters. This recently came up at work, so I thought I’d share a few quick thoughts.

    One of the developers was doing an HTTP post of XML content to a .NET web service. However, we discovered that a few of the records coming across had invalid characters.

    Now you probably know that the following message is considered invalid XML:

    <Person>
	<Name>Richard</Name>
	<Nickname>Thunder & Lightning</Nickname>
</Person>

    The ampersand (“&”) isn’t allowed within a node’s text. Neither are “<“, “>” and a few others. Now if you call a web service by first doing an “Add Web Reference” in Visual Studio.NET, you are using a proxy class that covers up all the XML/SOAP stuff going on underneath. The proxy class (Reference.cs) inherits System.Web.Services.Protocols.SoapHttpClientProtocol, which you can see (using Reflector) takes care of proper serialization using the XmlWriter object. So setting my web service parameters like so …

    When this actually goes across the wire to my web service, the payload has been appropriate encoded and the ampersand has been replaced …

    However, if I decided to do my own HTTP post to the service and bypass a proxy, this is NOT the way to do it ..

    HttpWebRequest webRequest = 
   (HttpWebRequest)HttpWebRequest.Create("http://localhost/bl/sv.asmx");
webRequest.Method = "POST";
webRequest.ContentType = "text/xml";

using (Stream reqStream = webRequest.GetRequestStream())
{

  string body = "<soap:Envelope xmlns:soap="+
  "\"http://schemas.xmlsoap.org/soap/envelope/\">"+
  "<soap:Body><Operation_1 xmlns=\"http://tempuri.org/\">" +
  "<ns0:Person xmlns:ns0=\"http://testnamespace\">" +
  "<ns0:Name>Richard & Amy</ns0:Name>" +
  "<ns0:Age>10</ns0:Age>" +
   "<ns0:Address>411 Broad Street</ns0:Address>" +
  "</ns0:Person>" +
  "</Operation_1></soap:Body></soap:Envelope>";

    byte[] bodyBytes = Encoding.UTF8.GetBytes(body);
    reqStream.Write(bodyBytes, 0, bodyBytes.Length);

}
HttpWebResponse webResponse = 
   (HttpWebResponse)webRequest.GetResponse();
MessageBox.Show("submitted, " + webResponse.StatusCode);

webResponse.Close();

    Why is this bad? This may work for most scenarios, but in the case above, I have a special character (“&”) that is about to go unmolested across the wire …

    Instead, the code above should be augmented to use an XmlTextWriter to build up the XML payload. These types of errors are such a freakin’ pain to debug since no errors actually get thrown when the receiving service fails to serialize the bad XML into a .NET object. In a BizTalk world, this means no SOAP exception to the caller, no suspended message, no error in the Event Log. Virtually no trace (outside of the IIS logs). Not good.

    BizTalk itself doesn’t like poorly constructed XML either. The XmlReceive pipeline, in addition to “typing” the message (http://namespace#root) also parses the message. So while everyone says that the default XmlReceive pipeline doesn’t validate the structure (meaning XSD structure) of the message, it DOES validate the XML structure of the message. Keep that in mind. If I try to pass an invalid XML document (special characters, unclosed tags) that WILL bomb out in the pipeline layer.

    If you try to cheat, and do pass-through pipelines and use XmlDocument as your initial orchestration message (thus bypassing any peeking at the message by BizTalk), you will still receive errors when you try to interact with the message later on. If you set the XmlDocument to the actual message variable in the orchestration, the message gets parsed at that time and fails if the structure is invalid.

    So, this is probably elementary for you smart people, but it’s one of those little things that you might forget about. Be careful about generating XML content via string building and instead consider using XmlDocuments or XmlWriters to make sure that your content passes XML parsing rules.

    Technorati Tags: ,