Author: Richard Seroter

  • New Collection of WCF Security Task Guidance

    If you’re looking to dig a bit deeper into WCF security, and specifically how to perform particular tasks, go check out J.D. Meier’s blog for a link list of new “how tos” and videos recently posted to CodePlex.

    Specifically, you’ll find some useful demonstrations for creating temporary certificates for message-based security, some interesting looks at impersonation, using SQL Role Providers for authentication, and more.

    Technorati Tags: 

  • BizTalk Environment Migration Checklist

    My company’s “standard operating procedure” for BizTalk Server doesn’t call out the specific requirements to deploy among environments (development to test, test to production, etc), so I’m trying to help the team get those articulated.  Here’s my first stab at a checklist that should be followed for BizTalk application migration between environments.   I don’t want the list to be abusive, but want it to be as comprehensible as possible.  Any feedback is appreciated.

    Migrate from Local Development to Shared Development
      Task Comments
    1. Code review executed by BizTalk team Refer to BizTalk Code Review guidelines
    2. Consistent system artifact naming in place System DSNs, application configuration settings
    3. Server access request approved BizTalk Administrator rights ok
    4. User has taken BizTalk developer training, and optionally, BizTalk administrator training  
    5, BizTalk application and host usage defined Are standard hosts used, or are additional ones defined? Does this project belong in an existing BizTalk Application, or a new one?
    6, Reusable assets are factored into common applications and assemblies Schemas, web services, business rules
    7. Application security requirements are clearly identified Authentication and authorization

     

    Migrate from Shared Development to Test
      Task Comments
    1. Code successfully builds and runs all core use cases  
    2. Code review is re-executed if significant deviations discovered during initial code review  
    3. Performance test plan is in place Test expected load + 25%; test LOB adapters to confirm downstream system load acceptance
    4. Repeatable release management from development environment is set up Binding files created, MSI built, scripts constructed; helper components and web services added to MSI
    5. Exception handling strategy is confirmed System retries, exception logging, resuming suspended messages
    6. Application is ready for multi-box deployment All hard-coded file system references considered
    7. List prepared of all encountered application and system exceptions, and resolution strategy for each Used for testing and eventual administration purposes
    8. Need for new host instances or changes to throttling settings considered  
    9. All web services are managed by governance platform [SOA Software]  

     

    Migrate from Test to Production
      Task Comments
    1. Application monitoring requested and configured [Microsoft Operations Manager] Identify events to monitor and notification recipients
    2. Appointed application administrator has BizTalk Operator rights approved  
    3. All web services are managed by governance platform [SOA Software]  
    4. Debug statements removed and Event Log trace statements removed  

    Technorati Tags: 

  • Gracefully Uploading to SharePoint 2007 From BizTalk Server 2006 R1

    Scenario: I want to allow BizTalk Server 2006 (R1) to send XML (InfoPath forms) to a MOSS 2007 document library without resorting to hacks.

    Resolution: I can’t use the out-of-the-box BizTalk SharePoint adapter (only BizTalk Server 2006 R2 works natively with MOSS 2007) so I decided to utilize the available SharePoint web services to upload my file.  I wrote a wrapper web service to (a) encapsulate some additional logic (b) shield the BizTalk developer from understanding SharePoint services and (c) require no usage of the SharePoint 2007 object model.

    What did this solution look like?  I decided to use the CopyIntoItems method available on the SharePoint Copy web service.  This allows you to send a byte array of data to a document library and have it appear as a new document.  To hit the WSDL for this service, you’d go to:

    http://<your sharepoint base url>/sites/<site name>/_vti_bin/Copy.asmx

    Here you’ll see the CopyIntoItems operation.  My wrapper service starts with a couple “using” statements …

    using System.Net;   //for NetworkCredentials object
    using System.Text;  //for encoding bytes
    using System.IO;    //for stringwriter
    using System.Xml;

    Next I have my wrapper operation …

    [WebMethod]
    public string UploadXmlToSharePoint(string docToUpload,
    string siteRoot,
    string docLibrary,
    string fileName,
    string userName,
    string password,
    string domain)

    I’m taking the XML document input as a string in order to make the schema easier in BizTalk, and, ensure I don’t lose my InfoPath processing instructions when transporting over the wire (which seemed to be happening when I used an XmlDocument type input parameter).  Also note that I’m taking in a user/password/domain combo.  This is to allow for reuse later down the line.  The account used to call the SharePoint service MUST be a site administrator, so I’m making it an explicit parameter. The first thing I do inside my operation is build up the destination Uri based on the input parameters.

    //build full destination Url
    string destinationPath = siteRoot + "/" + docLibrary + "/" + fileName;

    Next I have to take the input string and convert it to the byte array required by the MOSS web service …

    //convert string to byte array
    byte[] fileIn = ConvertDocToBytes(docToUpload);
    ...
    private byte[] ConvertDocToBytes(string xmlString)
        {
            ASCIIEncoding encoding = new ASCIIEncoding();
    
            return encoding.GetBytes(xmlString);
        }

    Now I need to instantiate some values needed by the MOSS service. First we have the “result” object which conveys the state of the copy transaction. Then I have a “FieldInformation” array which can be used to pass in specific field values. Note that you CANNOT pass in a null value here, or else you get a cryptic error when calling the service. You can make it blank, but don’t use a null parameter in its place. Finally, I create a destination Uri array.

    //holds MOSS service response values
    SharePointSvc.CopyResult[] results;
    
    //required fieldinformation array
    SharePointSvc.FieldInformation fieldInfo =
        new SharePointSvc.FieldInformation();
    SharePointSvc.FieldInformation[] fieldInfoArray = { fieldInfo };
    
    //destination url (notice that it's an array, meaning
    //multiple sites COULD be targeted
    string[] destUri = { destinationPath };

    Now I can actually call this puppy. After instantiating the web service proxy class (generated by the Add Web Reference command), I need to provide explicit credentials.

    //create instance of web service proxy
    SharePointSvc.Copy copy = new SharePointSvc.Copy();
    
    //pass valid credentials
    copy.Credentials = new NetworkCredential(userName, password, domain);
    
    //call primary operation;  sourceUri, doesn't matter here
    copy.CopyIntoItems(
         "http://none",
         destUri,
         fieldInfoArray,
         fileIn,
         out results);

    The last step is to actually check the “result” object for errors and return any errors back to the caller.

    //check for error and return final result
    if (results[0].ErrorMessage != null)
      {
        return "Error: " + results[0].ErrorMessage;
      }
      else
      {
        return "Success";
      }

    Sweet.  After building and deploying this service, I can call it from any client, BizTalk (2004/06) included.  I’ll obviously want to securely store my credentials (using Enterprise Single Sign On) and not embed those in my client directly.

    So if I call my service, and pass in a plain old XML file, it shows up in my document library as expected.

    Now, if I send an InfoPath document to a Forms Library set up with an InfoPath template, the result is this …

    Nice! The document is recognized as an InfoPath document (see the icon), and, the promoted columns are properly loaded.

    So, if you’re interested in a fairly easy way to programmatically upload documents to a MOSS 2007 library, without having to use the SharePoint object model or BizTalk adapter, this web service might just work for you.

    Technorati Tags: ,

  • How Do You Know If You’ve Reached Practical SOA Maturity?

    I’d like to think that my company is fairly far along the SOA path, but we’ll be spending time this year looking to refine our SOA reference architecture and map our progress on a SOA maturity model.  Most of the maturity models out there have some sort of scale which ranges from “starting with basic services in pilot projects” and graduates to “enterprise services and process orchestration” all the way to “agile, event-driven environments.”  If you’re unfamiliar with concept, check out what Sonic has to say.

    All that said, I was thinking about some of the lower-level, tangible ways that I could observe whether we were indeed becoming more advanced in our SOA journey.  So, I put together a short list of 10 criteria that would indicate to me that we were doing more than paying lip service to the architectural principles of SOA.  Let it be noted that I don’t think my company does many of these, but you can’t know what you’re shooting for if you don’t throw up a target.

    1. Organizational agreement on software validation processes and procedures.  Working in an FDA-regulated environment means that many of our systems require vigorous software validation processes.  Now considering that the nirvana of an SOA environment is that business and system processes can be constructed using the farm of existing reusable services, how do you properly validate this new system?   That is, must you revalidate each of the reused services that make up the new system in order to validate the entire new system, or do the software validation processes take into account composite applications?  The thinking of some is that a key criteria for this is risk.  Assess the risk points of the services working together, and then focus the validation effort on only those areas called out in the risk analysis.  I tend to like this approach since it’s progressive enough that we aren’t clumsily and coarsely retesting existing services unnecessarily, and instead focusing on what the FDA cares about the most, risk.
    2. SOA implementation is not platform-specific.   I don’t want our SOA infrastructure to built solely upon Microsoft technology.  I don’t want it solely built with Java technologies either.  The services that we expose should all be supportive of cross-platform standards (and data types) and be consumable by any of the (many) technology platforms we have in place.   I should be able to call, with no issue, any exposed service from either a Windows, Unix, Linux environment and/or Microsoft or Java client.  This touches on the next point.
    3. Service testing criteria should be defined and rigorous.   We shouldn’t simply be unit testing our service with blinders on that limit our scope to our target system.  If the goal is to maintain enterprise services, then each service should be challenged with cross platform invocation, and be proven to work as part of a composite service or process.  It’d be great for us to have a short, but comprehensive checklist that any service, regardless of situation, is tested against.
    4. Common set of enterprise objects is available.  We’ve done this with our SAP implementation, where many of the core SAP objects are represented via XSD schemas and it is assumed that if you receive data from SAP for these functional areas (e.g. employee data, vendor information), it will conform to the enterprise schema.  Now, it’s impractical that every existing system and data source will be evaluated and a complete set of enterprise objects will be defined.   That doesn’t make sense.  But, we would probably benefit by having a “customer” object that all our various system services would accommodate.   Once you’ve defined these objects, the next critical piece is publishing them for others to discover.
    5. Service repository contains services AND metadata.  We use SOA Software for our service directory, but simply discovering and listing services isn’t enough.  We also need clear guidance around how to best *describe* the service.  What is the metadata description needed for someone to not only discover this service, but know if it fits their needs, and how to consume it?  The first step is to find and register the services, but maturity will only be reached once we’ve properly cataloged and described them.
    6. Service lifecycle is well defined.  Specifically, what is the change management, release management, and service ownership story?  Does a single group *own* all the services, or is there distributed ownership and management?  When changes are needed, how do you ensure that the service is appropriately versioned, and that breaks to the contract do not cause issues for existing consumers?
    7. Metrics are captured and actionable.  While SOA Software provides us the means to observe all sorts of details on service usage, it’s arguably even more important to define how to act upon this data.  Do you defined a BAM dashboard?  What happens if specified SLAs are failing?  What’s the policy here?
    8. Service developers incorporate common service patterns.  I’ll specifically call out communication patterns.  Can our developers articulate why they would choose a request/response pattern, asynchronous one-way pattern, or publish/subscribe pattern, and know how or when to use callbacks?  Many services are well served by an asynchronous processing model, but I wonder how many developers are comfortable with that.  Also, how many of our services support MULTIPLE communication patterns?  In many cases, it makes lots of sense to support not only a request/response channel, but also a one-way channel.
    9. Developers build services that can ACTUALLY be reused.  Some folks may say that reuse is critical to an SOA architecture, but plenty of others will say that reuse isn’t a requirement and can actually hamper, not help, your design.  However, developers should be considering reuse while modeling their services, even if that only means that they carefully evaluate the service contract and exposed operations.  Likewise, can the service you’re building be detached into smaller, more modular components?   Maybe the broader application-specific service you’re writing has no possibility of reuse, but potentially a smaller part of the operation does include a repeatable process.  Sufficiently considering enterprise reuse is often one of the hardest things to do on a project.  How can you guess what someone MIGHT do with your service months from now?   But, taking a small slice of time to debate both “who” and “how” your service might be reused may yield design decisions otherwise not considered.
    10. New architects/developers can quickly understand our SOA.   Can we grab a new college-hire or a seasoned developer and have them build services that comply with our SOA vision?  If our SOA principles and design criteria aren’t clearly articulated, we will quickly find an excessive amount of deviations to our master plan.  We don’t need a 400 page manifesto, but it is important to call out our standards for envisioning, designing, testing, security, deploying, consuming and managing enterprise services if we’re truly committed to building a vibrant service-oriented infrastructure.

    Thoughts?  Do you find that you or your company already do many of these?  Or do you think any of these criteria are unnecessary or incomplete?

    Technorati Tags:

  • BizTalk Database Architecture Poster Now Available

    The BizTalk product team keeps pumping out posters, so I guess that I’ll keep highlighting them.  This particular one focuses on the core BizTalk databases, their functions, and how they relate to each other.  Specifically, you’ll find each BizTalk database alongside it’s role, functional events, data movement events and more.  Given that SQL Server databases form the heart and soul of BizTalk Server, it’s fairly important that any self-respecting BizTalk person knows this information.

    I can’t imagine that there are many more poster ideas left for the BizTalk team to produce.  I mean, what’s left?  Security roles?  Functoids?  Maybe they’ll mail it in and produce a poster of the product team itself playing a spirited game of freeze tag.

    Technorati Tags:

  • Wally the Architect

    After reading today’s Dilbert comic, I’m not sure Wally is the standard-bearer for architects that I want to get behind …

    Dilbert.com

  • SSO Config Data Store Tool, BizTalk/WCF Scenario Source Code Available

    I’ve finally gotten around to publishing my source code for my SSO Configuration Data Store Tool.  You can download both the runtime application and underlying source code from here.  If you want to mess around with it, great, but please keep my name on there.

    Although I’m not finished with my “BizTalk + WCF” series for TopXML.com yet, I have finished the first part of the series on consuming WCF services.  So, I’ve put the source code I used for all those demonstrations right here.  The binding file is in there as well.  Once I finish the last part of the series, I’ll post that source code too.

    Enjoy.

    Technorati Tags: ,

  • You’ve Just Bought BizTalk Server. Congrats. Now What?

    It’s just over a year ago that my company switched from their previous EAI platform to BizTalk Server 2006.  Over that time, we’ve learned some best practices for setting up a brand new BizTalk program, and I thought I’d share my “Top 10” recommendations for organizations getting started with BizTalk.   The below recommendations aren’t in any particular priority-based ordering …

    1. The first project should be a quick win.  I’m sure that BizTalk Server was brought into your organization with a particular project in mind.  However, unless you have all the resources lined up to help with a complex project (see tips #2 and #3), I suggest identifying a “quick win” project that can be built with relative ease and minimal complexity in a short amount of time.  This helps establish the platform within the organization and builds confidence among the team.  If you bite off a monster project right off the bat, you greatly increase the risk of failure or disenchantment with your purchase.
    2. Get an expert.  If your plan for getting BizTalk Server up and running is to hand a BizTalk Server 2006 book to your best developer and saying “read this over the weekend and be ready to lead a project on Monday”, you’re hosed.  You really need to hire (a) a full-time employee or (b) consultant to help you get the platform established in your company.   Why?  Integration solutions are a different beast than standard custom applications that developers are familiar with building.  There are different paradigms, patterns and gotchas that only an expert can help with.  Now, the tricky part is, there aren’t a lot of experts out there.  In my experience, significantly less than 50% of all folks who put “BizTalk” on their resume are qualified to lead your team.  I suggest seeking a BizTalk expert in your area to help with interviews.  Ping your local Microsoft specialist, or reach out to an MVP.  You want the best person possible brought onboard, and you should be confident that you’re getting someone who knows what they are doing.
    3. Invest in training and building a “Center of Excellence.”  Now, a “Center of Excellence” for BizTalk may mean 20 people, or 2 people.  Either way, you want a team of people who are well trained in BizTalk and can provide advice on upcoming projects.   My first year here, I taught 8 BizTalk classes to 100+ folks, and now I can have intelligent conversations about BizTalk with project managers, business analysts, developers, architects and system administrators.  An in-house team of dedicated resources (i.e. not weekend BizTalk warriors) that can evangelize the product and guide your success is critical to stabilizing your platform.
    4. Commit to, and enforce naming standards.  This is so important.  As the amount of deployed projects increase, the more critical it is that all developers and administrators use common names for artifacts and configuration items.  Any developer should be able to pick up any other developer’s BizTalk solution and be able to troubleshoot it without the aid of the Rosetta Stone.  It’s just as easy to type a good artifact name as a bad one.  Your “Center of Excellence” leaders should have the authority to halt the deployment of a project if the naming standards have been ignored.
    5. Agree upon code review items and processes.  Before any project is deployed, you should require that a full BizTalk code review is executed.  Even the smartest developer benefits by having a second (or third/fourth/fifth) set of eyes on their solution.  Maybe they missed a pattern, or built something that won’t scale.  A good code review will catch that.  I posted our code review guidelines on my blog last year.
    6. Set up a standard release management plan.  Let’s assume you have a BizTalk development, test and production environment.  You want clear guidelines as to how artifacts move between each set of servers.  Who does installations?  Where do we drop the installation packages?  How are differences between environments (passwords, server names) managed?  A new build of a deployed solution should have a very clear path to production that any administrator can execute.
    7. Identify your disaster recovery / archive  / purge strategy.  Don’t wait until after projects get deployed to commit to platform maintenance.  Your disaster recovery plan needs to be set up right after BizTalk Server is first installed.  Are you using log shipping?  Or doing a more local backup and restore?  What is your Recovery Point Objective (RPO)?  No more than 15 minutes of data lost?  The archiving/purging jobs on the BizTalk tracking database are NOT enabled by default.  You need to consider how long to keep tracking data and get those jobs enabled before projects get deployed.  Otherwise, you end up with an increasingly slower system.
    8. Define source code and server access control.  Put BizTalk artifacts in source control.  You don’t want critical project source control sitting in a zip file on a shared drive.  Figure out who needs access to BizTalk source control (which may contain bindings with passwords) and lock that group down.  Also, determine a process for adding folks to the production “BizTalk Operators” group.  At my company, we committed to distributed ownership where a central group “owns” the platform, but, divisions with BizTalk projects are set up as “Operators” and can perform basic maintenance on their own appliations.  This way, the central group only deals with critical issues, while local divisions can resume suspended messages, or turn off their ports for maintenance.
    9. Establish error reporting and monitoring solutions.  My company uses Microsoft MOM for BizTalk monitoring.  If you don’t have MOM in house, then use whatever is available, but use SOMETHING.  You don’t want to find out that a particular host has been offline for 2 weeks because no one was alerted that a problem occurred.  Also, consider establishing a standard “error logging” framework for your developers.  Do you write to the Event Log with a specific string pattern?  How about a database logging system using something like Log4Net?  If you don’t do this, I’ll guarantee that you’ll end up with countless different ways that each deployed solution records their issues.
    10. Stay actively in tune to hotfixes and upcoming releases.  Microsoft regularly releases hotfixes for BizTalk, and many of them address important areas.  We’ve found a number of hotfixes that solve development or environmental issues that we had been banging our heads against for days.  If web searches for your problems don’t turn up valid results, consider browsing the Microsoft BizTalk Solution Center.  Also, keep an eye on product announcements, toolkits or whitepapers that may help your team continue increase their efficiencies with the product.

    So there you go.  My company learned some of those lessons above the hard way, so hopefully I can save someone else a little heartache.

    For those of you who have helped establish BizTalk for your own company (or as a consultant), are there other recommendations you’d like to share?

     

    Technorati Tags:

  • New BizTalk Server 2006 Hotfixes

    A few interesting Microsoft Knowledge Base hotfixes for BizTalk Server 2006 were recently added and worthy of sharing.

     

    Technorati Tags: