Category: WCF/WF

  • Publishing XML Content From SQL Server 2008 to BizTalk Server 2009

    I’m looking at the XML capabilities of SQL Server a bit this week, and it reminded me to take another look at how the new BizTalk Server 2009 SQL Adapter (WCF-based) interacts with XML content stored in SQL Server.

    I’ve shown in the past (in my book, and available as a free read here) that the new adapter can indeed read/write to SQL Server’s XML data type, but it does so in a bit of a neutered way.  That is, the XML content is stuffed into a string element instead of a structured node, or even an “any” node.  That said, I want to see how to take XML data from SQL Server and have it directly published to BizTalk for routing.

    First things first, I need to create a table in SQL Server with an XML data type.  I wanted to “type” this column (just for the heck of it), so I built a valid XSD schema using the BizTalk Editor in Visual Studio.

    I then opened the SQL Server 2008 Management Studio and defined a new XML Schema Collection.  The definition of the XML structure consists of the XSD schema we just created in Visual Studio.

    Next, I created a new table and made one of the columns (“DetailsXml”) use the xml data type.  Then, I set the XML Type Specification’s “Schema Collection” property equal to our recently defined “OrderDetailsSchema” XML definition.

    To test this configuration, I ran a quick SQL statement to make sure that an insert consisting of a schema-compliant XML fragment would successfully process.

    Lookin’ good.  Now I have a row in that new table.  Ok, next, I went back to my BizTalk project in Visual Studio and walked through the Consume Adapter Service wizard to generate SQL adapter-compliant bits.  Specifically, in my “connection” I had to set the client credentials, InboundId (because we’re polling here), initial catalog, server, inbound operation type (typed polling), polled data available (“SELECT COUNT([OrderID]) FROM [BlogDemo]”) and polling statement (“SELECT [OrderID] ,[DetailsXml] FROM [BlogDemo]”).   Once those connection properties were set, I was able to connect to my local SQL Server 2008 instance.  I then switched to a “service” contract type (since we’re polling, not pushing) and picked the “typed polling” contract.

    As with all the WCF adapters, you end up with XSD files and binding files after the Consume Adapter Service wizard completes.  My schema shows that the “DetailsXml” node is typed as a xsd:string.  So whether you “type” the XML column in SQL Server or not, the adapter will not ever give you a structured message schema.

    After deploying the BizTalk project, and importing the wizard-generated binding into my BizTalk application, I have a valid receive location that can poll my database table.  I built a quick send port that subscribed on the receive port name.  What’s the output when I turn the receive location on?  Take a look:

    We have the “typedpolling” root node, and our lovely XML content is slapped into a CDATA blob inside the string node.  That’s not very nice.  Now, I have two options as to what to do next: First, I could take this message, pull it into an orchestration and leech out the desired XML blob and republish it to the bus.  This is a decent option IF you also need other data points from the SQL Server message.  However, if ALL you want is the XML blob, then we want option #2.  Here, I muck with the generated receive location and tell it to pull out the XML node from the inbound message and only publish THAT to the bus.

    I do this by going to the “Messages” tab of the adapter configuration and switching the source from “body” (which is the default) to “path” which let’s me set a forward-only Xpath statement.

    Note that the encoding is string.  I wasn’t sure this would work right, but when I turned my receive location back on after making this update, this is the message my send port distributed:

    Well hello my lady.  Nice to see you.  To go for the home run here, I switched the receive location’s pipeline to XmlReceive (to force message typing) and set the send port’s subscription to the BTS.MessageType.  I wanted to confirm that there were no other shenanigans going on, and that I was indeed getting a typed XML message going through, not a message of type “string.”  Sure enough, I can see from the context that I have a valid message type, and it came from my SQL adapter.

    So, I’m glad this capability (extract and type the nested XML) is here, or else the BizTalk Server 2009 promise of “SQL Server XML data type compatibility” would have been a bit of a sham.   Has anyone tried accessing the data from an orchestration instead?  I’m assuming the orchestration xpath function could be used to get at the nested XML.  Feel free to share experiences.

    Technorati Tags: ,

  • TechEd 2009: Day 1 Session Notes

    Good first day.  Keynote was relatively interesting (even though I don’t fully understand why the presenters use fluffy “CEO friendly” slides and language in a room of techies) and had a few announcements.  The one that caught my eye was the public announcement of the complex event processing (CEP) engine being embedded in SQL Server 2008 R2.  In my book I talk about CEP and apply the principles to a BizTalk solution.  However, I’m much happier that Microsoft is going to put a real effort into this type of solution instead of the relative hack that I put together.  The session at TechEd on this topic is Tuesday.  Expect a write up from me.

    Below are some of the session notes from what I attended today.  I’m trying to balance sessions that interest me intellectually, and sessions that help me actually do my job better.  In the event of a tie, I choose the latter.

    Data Governance: A Solution to Privacy Issues

    This session interested me because I work for a healthcare organization and we have all sorts of rules and regulations that direct how we collect, store and use data.  Key Takeaway: New website from Microsoft on data governance at http://www.microsoft.com/datagovernance

    • Low cost of storage and needs to extend offerings with new business models have led to unprecedented volume of data stored about individuals
    • You need security to achieve privacy, but security is not a guarantee of privacy
    • Privacy, like security, has to be embedded into application lifecycle (not a checkbox to “turn on” at the end)
    • Concerns
      • Data breach …
      • Data retention
        • 66% of data breaches in 2008 involved data that was not known to reside on the affected system at the time of incident
    • Statutory and Regulatory Landscape
      • In EU, privacy is a fundamental right
        • Defined in 95/46/EC
          • Defines rules for transfer of personal data across member states’ borders
        • Data cannot be transported outside of EU unless citizen gives consent or legal framework, like Safe Harbor, is in place
          • Switzerland, Canada and Argentina have legal framework
          • US has “Safe Harbor” where agreement is signed with US Dept of Commerce which says we will comply with EU data directives
        • Even data that may individually not identify you, but if aggregated, might lead you to identify an individual; can’t do this as still considered “personal data”
      • In US, privacy is not a fundamental right
        • Unlike EU, in US you have patchwork of federal laws specific to industries, or specific to a given law (like data breach notification)
        • Personally identifiable information (PII) – info which can be used to distinguish or trace an individual’s identity
          • Like SSN, or drivers license #
      • In Latin America, some countries have adopted EU-style data protection legislation
      • In Asia, there are increased calls for unified legislation
    • How to cope with complexity?
      • Standards
        • ISO/IEC CD 29100 information technology – security techniques – privacy framework
          • How to incorp. best practices and how to make apps with privacy in mind
        • NIST SP 800-122 (Draft) – guidelines for gov’t orgs to identify PII that they might have and provides guidelines for how to secure that information and plan for data breach incident
      • Standards tell you WHAT to do, but not HOW
    • Data governance
      • Exercise of decision making and authority for data related matters (encompasses people, process and IT required for consistent and proper handling across the enterprise)
      • Why DG?
        • Maximize benefits from data assets
          • Improve quality, reliability and availability
          • Establish common data definitions
          • Establish accountability for information quality
        • Compliance
          • Meet obligations
          • Ensure quality of compliance related data
          • Provide flexibility to respond to new compliance requirements
        • Risk Management
          • Protection of data assets and IP
          • Establish appropriate personal data use to optimally balance ROI and risk exposure
      • DG and privacy
        • Look at compliance data requirements (that comes from regulation) and business data requirements
        • Feeds the strategy made up of documented policies and procedure
        • ONLY COLLECT DATA REQUIRED TO DO BUSINESS
          • Consider what info you ask of customers and make sure it has a specific business use
    • Three questions
      • Collecting right data aligned with business goals? Getting proper consent from users?
      • Managing data risk by protecting privacy if storing personal information
      • Handling data within compliance of rules and regulations that apply
    • Think about info lifecycle
      • How is data collected, processed and shared and who has access to it at each stage?
        • Who can update? How know about access/quality of attribute?
        • What sort of processing will take place, and who is allowed to execute those processes?
        • What about deletion? How does removal of data at master source cascade?
        • New stage: TRANSFER
          • Starts whole new lifecycle
          • Move from one biz unit to another, between organizations, or out of data center and onto user laptop
    • Data Governance and Technology Framework
      • Secure infrastructure – safeguard against malware, unauthorized access
      • Identity and access control
      • Information protection – while at risk, or while in transit; protecting both structured and unstructured data
      • Auditing and reporting – monitoring
    • Action plan
      • Remember that technology is only part of the solution
      • Must catalog the sensitive info
      • Catalog it (what is the org impact)
      • Plan the technical controls
        • Can do a matrix with stages on left (collect/update/process/delete/transfer/storage) and categories at top (infrastructure, identity and lifecycle, info protection, auditing and reporting)
        • For collection, answers across may be “secure both client and web”, “authN/authZ” and “encrypt traffic”
          • Authentication and authorization
        • For update, may log user during auditing and reporting
        • For process, may secure host (infra) and “log reason” in audit/reporting
    • Other tools
      • IT Compliance Management Guide
        • Compliance Planning Guide (Word)
        • Compliance Workbook (Excel)

    Programming Microsoft .NET Services

    I hope to spend a sizeable amount of time this year getting smarter on this topic, so Aaron’s session was a no-brainer today.  Of course I’ll be much happier if I can actually call the damn services from the office (TCP ports blocked).  Must spend time applying the HTTP ONLY calling technique. Key Takeaway: Dig into queues and routers and options in their respective policies and read the new whitepapers updated for the recent CTP release.

    • Initial focus of the offering is on three key developer challenges
      • Application integration and connectivity
        • Communication between cloud and on-premises apps
        • Clearly we’ve solved this problem in some apps (IM, file sharing), but lots of plumbing we don’t want to write
      • Access control (federation)
        • How can our app understand the various security tokens and schemes present in our environment and elsewhere?
      • Message orchestration
        • Coordinate activities happening across locations centrally
    • .NET Service Bus
      • What’s the challenge?
        • Give external users secure access to my apps
        • Unknown scale of integration or usage
        • Services may be running behind firewalls not typically accessible from the outside
      • Approach
        • High scale, high availability bus that supports open Internet protocols
      • Gives us global naming system in the cloud and don’t have to deal with lack of IP v4 available addresses
      • Service registry provides mapping from URIs to service
        • Can use ATOM pub interface to programmatically push endpoint entries to the cloud
      • Connectivity through relay or direct connect
        • Relay means that you actually go through the relay service in the bus
        • For direct, the relay helps negotiate a direct connection between the parties
      • The NetOneWayRelayBinding and NetEventRelayBinding don’t have a OOB WCF binding comparison, but both are set up for the most aggressive network traversal of the new bindings
      • For standard (one way) relay, need TCP 828 open on the receiver side (one way messages through TCP tunnel)
      • Q: Do relay bindings encrypt username/pw credentials sent to the bus? Must be through ACS.
      • Create specific binding config for binding in order to set connection mode
      • Have new “connectionstatechangedevent” so that client can respond to event after connection switches from relay to direct connection as result of relay negotiations based on “direct” binding config value
        • Similar thing happens with IM when exchanging files; some clients are smart enough to negotiate direct connections after the session is established
      • Did quick demo showing performance of around 900 messages per second until the auto switch to direct when all of sudden we saw 2600+ messages per second
      • For multi-cast binding (netEventRelayBinding), need same TCP ports open on receivers
      • How deal with durability for unavailable subscribers? Answer: queues
      • Now can create queue in SB account, and clients can send messages and listeners pull, even if online at different times
        • Can set how long queue lives using queue policy
        • Also have routers using router policy; now you can set how you want to route messages to listeners OR queues; sets a distribution policy and say distribute to “all” or “one” through a round-robin
        • Routers can feed queues or even other routers
    • .NET Access Control Service
      • Challenges
        • Support many identities, tokens and such without your app having to know them all
      • Approach
        • Automate federation through hosted STS (token service)
        • Model access control as rules
      • Trust established between STS and my app and NOT between my app and YOUR app
      • STS must transform into a claim consumable by your app (it really just does authentication (now) and transform claims)
      • Rules are set via web site or new management APIs
        • Define scopes, rules, claim types and keys
      • When on solution within management portal, manage scopes; set your solution; if pick workflow, can manage in additional interface;
        • E.g. For send rule, anytime there is a username token with X (and auth) then produce output claim with value of “Send”
        • Service bus is looking at “send” and “listen” rules
      • Note that you CAN do unauthenticated senders
    • .NET Workflow Service
      • Challenge
        • Describe long-running processes
      • Approach
        • Small layer of messaging orchestration through the service bus
      • APIs that allow you to deploy, manage and run workflows in the cloud
      • Have reliable, scalable, off-premises host for workflows focused specifically on message orchestration
      • Not a generic WF host; the WF has to be written for the cloud through use of specific activities
  • My Book "SOA Patterns with BizTalk Server 2009" is Up for Pre-order

    Well look at that.  I just happen to check my publisher’s website today and noticed that they put my book description online and made it available for pre-order.

    The book, entitled SOA Patterns with BizTalk Server 2009, is due out by May 2009 (hopefully sooner).  I’m still in the process of editing the book and responding to technical review comments, but we’re nearing completion of that phase.  I have three fantastic technical reviewers (Charles Young, Zach Bonham, Ewan Fairweather) who have added a ton of great comments that I’m almost done wading through and addressing.

    This publisher (Packt Publishing) specializes in short, targeted technical books that address specific topics.   I tried to be “short” but still went over my suggested page count by nearly double.  Oh well, hopefully that’s better for the readers. 

    The high level chapter summaries are …

    • Chapter 1 – Building BizTalk Server 2009 Solutions – A brief walkthrough of creating a new BizTalk project from scratch
    • Chapter 2 – Windows Communication Foundation Primer – An overview of WCF with examples of developing, hosting, and consuming WCF services
    • Chapter 3 – Using WCF Services in BizTalk Server 2009 – An explanation of the marriage between BizTalk and WCF and how to both expose and consume WCF services within BizTalk Server
    • Chapter 4 – Planning Service Oriented BizTalk Solutions – A discussion of the core aspects of SOA, the types of services one can construct, available message exchange patterns, and how service-oriented principles apply to BizTalk solutions
    • Chapter 5 – Schema and Endpoint Patterns – How to build effective schemas and endpoints depending on the type of service constructed
    • Chapter 6 – Asynchronous Communication Patterns – A look at options for asynchronous communication and how to exploit this pattern in BizTalk solutions
    • Chapter 7 – Orchestration Patterns – We analyze the role of orchestration in service solutions and see how to build loosely coupled workflow processes
    • Chapter 8 – Versioning Patterns – We see here how to effectively version our SOA solutions and cleanly introduce updates to our BizTalk components
    • Chapter 9 – New SOA Capabilities in BizTalk Server 2009 – A first look at the latest additions to the BizTalk product. This includes the new UDDI v3 registry, WCF SQL Server Adapter, and ESB Guidance 2.0

    So there you go.  Once the book actually hits the shelves, I’ll make another mention of it on the blog.

    Technorati Tags: , ,

  • Presentations from 2009 Microsoft SOA/BPM Conference Available Online

    Hadn’t noticed this before, but found the complete collection of videos and presentations from this year’s SOA & BPM Conference.  I didn’t make it up to Redmond for this, so it’ll be nice to peruse the content which covers topics such as:

    • Customer case studies
    • Designing services for “Dublin”
    • Using BAM for Service and SLA monitoring
    • Supporting WS*, REST and POX simultaneously with WCF
    • SOA patterns from the field
    • Lap around “Oslo”

    Check it out.

    Technorati Tags: , ,

  • New "WCF Adapter FAQ" Whitepaper From Microsoft

    The folks over at the BizTalk Adapter Development blog pointed out their new paper which answers a number of questions about the WCF adapters for BizTalk Server 2006 R2.

    This collection of FAQs address topics such as:

    • Deciding which WCF adapter to use
    • When to use the two custom adapters
    • How do messages flow in from WCF adapters and flow back out
    • How to preserve the entire inbound message arriving via a WCF adapter
    • Accessing WCF message properties within a BizTalk solution
    • Decision criteria when choosing the WCF LOB Adapter SDK vs. the old BizTalk Adapter Framework
    • How BizTalk uses the WCF adapters
    • The difference between WCF behaviors and BizTalk pipelines

    There are plenty more besides these.  If you are doing WCF development alongside your BizTalk solution and need to integrate the two, you really should check this out.  The document is well written and nicely explains the concepts.

    Technorati Tags: ,

  • Interview Series: Four Questions With … Jesus Rodriguez

    I took a hiatus last month with the interview, but we’re back now.  We are continuing my series of interviews with CSD thought leaders and this month we are having a little chat with Jesus Rodriguez.  Jesus is a Microsoft MVP, blogger, Oracle ACE, chief architect at Tellago, and a prolific speaker.  If you follow Jesus’ blog, then you know that he always seems to be ahead of the curve with technology and can be counted on for thoughtful insight. 

    Let’s see how he handles the wackiness of Seroter’s Four Questions.

    Q: You recently published a remarkably extensive paper on BAM.  Did you learn anything new during the creation of this paper, and what do you think about the future of BAM from Microsoft?

    A:  Writing an extensive paper is always a different experience. I am sure you are familiar with that feeling given that these days you are really busy authoring a book. A particular characteristic of our BAM whitepaper is the diversity of the target audience. For instance, while there are sections that are targeting the typical BizTalk audience, others are more intended to a developer that is really deep with WCF-WF and yet others sections are completely centered on Business Intelligence topics. I think I learned a lot in terms of how to structure content that targets a largely diverse audience without confusing everybody. I am not sure we accomplish that goal but we certainly tried šŸ˜‰

    I think BAM is one of the most appealing technologies of the BizTalk Server family. In my opinion, in the next releases, we should expect BAM to evolve beyond being a BizTalk-centric technology to become a mainstream infrastructure for tracking and representing near real time business information. Certainly the WCF-WF BAM interceptors in BizTalk R2 were a step on that direction but there are a lot of other things that need to be done. Specifically, BAM should gravitate towards a more integrated model with the different Microsoft’s Business Intelligence technologies such as the upcoming Gemini. Also, having interoperable and consistent APIs is a key requirement to extend the use of BAM to non Microsoft technologies. That’s why the last chapter of our paper proposes a BAM RESTful API that I believe could be one of the channels for enhancing the interoperability of BAM solutions.

    Q: You spoke at SOA World late last year and talked about WS* and REST in the enterprise.    What sorts of enterprise applications/scenarios are strong candidates for REST services as opposed to WS*/SOAP services and why?

    A: Theoretically, everything that can be modeled as a resource-oriented operation is a great candidate for a RESTful model. In that category we can include scenarios like exposing data from databases or line of business systems. Now, practically speaking, I would use a RESTful model over a SOAP/WS-* alternative for almost every SOA scenario in particular those that require high levels of scalability, performance and interoperability. WS-* still has a strong play for implementing capabilities such as security, specifically for trust and federation scenarios, but even there I think we are going to see RESTful alternatives that leverages standards like OpenID, OAuth and SAML in the upcoming months. Other WS-* protocols such as WS-Discovery are still very relevant for smart device interfaces.

    In the upcoming years, we should expect to see a stronger adoption of REST especially after the release the JSR 311 (http://jcp.org/en/jsr/detail?id=311 ) which is going to be fully embraced by some of the top J2EE vendors such as Sun, IBM and Oracle.

    Q: What is an example of a “connected system” technology (e.g. BizTalk/WCF/WF) where a provided GUI or configuration abstraction shields developers from learning a technology concept that might actually prove beneficial?

    A:  There are good examples of configuration abstractions in these three technologies (BizTalk, WCF and WF). Given the diversity of its feature set, WCF hides a lot of things behind its configuration that could be very useful on some situations. For instance, each time we configure a specific binding on a service endpoint we are indicating the WCF runtime the configuration of ten or twelve components such as encoders, filters, formatters or inspectors that are required in order to process a message. Knowing those components and how to customize them allows developers to optimize the behavior of the WCF runtime to specific scenarios.

    Q [stupid question]: Many of us have just traveled to Seattle for the Microsoft MVP conference.  This year they highly encouraged us to grab a roommate instead of residing in separate rooms.  I’ve been told that one way to avoid conference roommates is to announce during registration some undesirable characteristic that makes you an lousy roommate choice.  For instance, I could say that I have a split personality and that my alter ego is a nocturnal, sexually-confused 15th century sea pirate with a shocking disregard for the personal space of others.  Bam, single room.  Give us a (hopefully fictitious) characteristic that could guarantee you a room all to yourself.

    A:  My imaginary friend is a great Opera singer šŸ™‚ We normally practice signing duets after midnight and sometimes we spend all night rehearsing one or two songs. We are really looking have our MVP roommate as our audience and, who knows, maybe we can even try a three-voice song.

    Seriously now, given work reasons I had to cancel my attendance to the MVP summit but I am sure you guys (BizTalk MVP gang) had a great time and drove your respective roommates crazy šŸ˜‰

    As always, I had fun with this one.  Hopefully Jesus can say the same.

    Technorati Tags: , ,

  • First Stab at Using New Tool to Migrate SQL Server Adapter Solutions

    If you saw the recent announcement about the Adapter Pack 2.0 release, you may have seen the short reference to a tool that migrates “classic” SQL Adapter solutions to the new WCF SQL Adapter.Ā  This tool claims to:

    • Works on the source project files
    • Generates equivalent schema definitions for the operations used in the existing project
    • Generates new maps to convert messages from older formats to the new format
    • Modifies any existing maps to work with the new schemas
    • A new project is created upon completion, which uses the SQL adapters from the BizTalk Adapter Pack v2

    Given how much “migration pain” can be a big reason that old projects never get upgraded, I thought I’d run a quick test and see what happens.

    The SQL Server part of my solution consists of a pair of tables and pair of stored procedures.Ā  In my solution, I poll for new customer complaint records, and receive that data into an orchestration where I take the ID of the customer and query a different database for the full record of that customer.

    In my BizTalk Server 2006 R2 environment, I walked through the “Add Generated Items” wizard in Visual Studio.NET and pointed at the classic SQL Adapter in order to generate the schemas necessary to receive and query data.Ā  As you would expect, the message arriving from the SQL Adapter polling port has a single node representing the customer complaint.

    The schema generated by the wizard for the patient record query has nodes for both the stored procedure request and result.

    My orchestration is very straightforward as it receives the polled message, constructs the patient query using a map, executes its query, and broadcasts the result.

    Great.Ā  After deploying this solution, I still need the messaging ports required to absorb and transmit the necessary data.Ā  My classic SQL Adapter receive location has the necessary settings to poll my database.

    After adding two send ports (one using the classic SQL adapter to send my patient query, and another to drop my result to a FILE location), I started everything up and it worked perfectly.Ā  Now the fun part, migrating this bad boy.

    Because this SQL adapter migration tool claims to work on the “project files” and not configuration bindings, I figured that I could package up the working Visual Studio.NET project and perform the migration in my BizTalk Server 2009 environment (which also had the Adapter Pack 2.0 beta installed).

    When you install the Adapter Pack 2.0 beta, you get the SQL Migration tool.Ā Ā  My SQL Server instance uses integrated authentication, so while I had to specify a “username” and “password” in the command line entry, I left them blank.

    MigrationTool Sqlsource=”Blog.BizTalk.SqlMigrationDemo.btproj”

    -dest=”C:\SQL_Mig\Blog.BizTalk.SqlMigrationDemoConverted” –uri=mssql://<server>//DemoDb? –username= –password=

    Once this query completes, you get a notice and a brand new project.

    The new project also contains a conversion report showing what was added and changed in the solution.Ā  I can see two new files added, and two files that the migration tool says it can reuse with the new adapter.

    If I open the actual project that the migration tool built, I can see new folders and a couple new files.Ā  The SqlBindingProcedure.dbo.xsd schema is also new.

    Notice that I have a new (WCF SQL Adapter) binding file for my “send” transmission that looks up patient details.Ā  A note: the BizTalk project system in 2006 R2 is different than the new one in BizTalk 2009.Ā  So, because I transferred my R2 project to my 2009 environment and THEN ran the wizard, my new project is still in the R2 format.Ā  I had to manually create a new 2009 project and include all the files generated by the wizard instead of just opening up the generated btproj file.

    The verdict?Ā  Well, pretty mixed.Ā  The schema it generated to replace my “query” schema is a mess.Ā  I get an untyped result set now.

    And the map that the migration tool created simply took my original “patient query” format and mapped it to their new one.Ā  I guess I’m supposed to apply that at the send port and keep my existing “complaint to patient” map that’s in my orchestration?

    Also, because the migration tool doesn’t appear to look at the runtime application configuration, I still have to manually create the receive location, which also seems like I have to manually recreate my inbound schema that can comply with the new WCF SQL Adapter format.Ā  I haven’t done all that yet because I’m not that motivated to do so.

    So, there could be a few reasons for my non-seamless experience.Ā  First, I used stored procedures on all sides, so maybe that part isn’t fully baked yet.Ā  I also switched environments and took a working BizTalk 2006 R2 solution and ran the conversion tool on a BizTalk 2009 box.Ā  Finally, there’s a good chance this falls under the “Seroter Corollary” which states that when all else fails, let’s assume I’m an idiot.

    Anyone else run this migration tool yet on an existing project?Ā  Any obvious things I may have missed that made my migration more work that rebuilding the project from scratch?

    Technorati Tags: ,

  • Service Security Guide on MSDN

    The Improving Web Services Security: Scenarios and Implementation Guidance for WCF project on CodePlex now has its results in an online browsable from within the MSDN site.    I linked to this project last year, but it’s great that everything has been made available on MSDN as well.

    Even if you aren’t using WCF, this set of deliverables has some very insightful components.  For example, the Security Fundamentals for Web Services chapter barely even mentions WCF but rather focuses on defining services, overarching security principles, as well as a set of security patterns that address topics such as authentication, data confidentiality and message validation.

    Chapter 2, Threats and Countermeasures for Web Services, is also technology-neutral and identifies a set of security threats, vulnerabilities, and countermeasures.

    Of course it is a WCF guide, so expect to find a wealth of information about WCF security options and trade-offs as well as 20+ “how to” walkthroughs that range from hosting services, to impersonation to using certificate-based authentication.

    Finally, if you’re not a “read tons of pages about security” kind of fella, then at least peruse the WCF Security Checklist (which can provide a good development checkpoint prior to service release), the summary of WCF Security Practices at a Glance (which provides a clean list of categories and related articles) and the very important Q&A section that contains dozens of realistic questions with straightforward answers.

    Great job on this.  Thanks J.D. and team.

    Technorati Tags: ,

  • Interview Series: Four Questions With … Jon Flanders

    You’re probably surprised that I’ve kept this up, aren’t you.  Here we are, five interviews into this series and still going strong.  This month, we chat with the one Flanders that Homer Simpson actually appreciates: Jon Flanders.  Jon is a blogger, MVP, thought leader in the SOA space, and is comfortable wearing a skirt. Jon has recently released his book RESTful .NET to critical acclaim and has taken a break from his whirlwind book tour (and the thousands of screaming ladies) to engage in a little Q&A with us.

    Q: Tell us why a developer who has always built SOAP-based web services should care about REST. Why is it worth it to them to learn a different paradigm and what benefit does this paradigm offer to enterprise services that typically are built in a SOAP/RPC fashion?

    A:  What I typically tell people here is two things.

    1) REST has some significant advantages over traditional RPC styles (which most SOAP-based services are). GET results can be cached, REST services are *more* interoperable than SOAP and WS-*, and the statelessness constraint encourages more scalable implementations, and the uniform interface (GET, POST, PUT, DELETE) make building and using services much simpler than custom APIs (which SOAP-based services are because each one is a custom interface). If you use all of the constraints of REST (specifically the hypermedia constraint), you also get a highly decoupled implementation.

    2) Because of these advantages, most of the non-Microsoft parts of the computer industry have moved towards a RESTful approach already, and Microsoft is currently moving that way. When you look at ADO.NET Data Services, Windows Azure, you see a lot of Microsoft’s effort going into building RESTful services. Because of this, even if you aren’t planning on implementing all your services using REST, you probably will be consuming one or more RESTful services in the near future.

    In the end, I don’t advocate moving away from SOAP/WS-* where it makes sense or is necessary (for things like transactional calls between .NET and Java for example), but I think more services than people think could benefit from using a RESTful approach.

    Q: Outside of the plethora of WCF related things you inevitably learned during the writing of your latest book, what more general “service design” concepts/principles/pitfalls have you picked up as a result of authoring this book?

    A: Nothing really new. The concept/principle I believe in most is Keep it Simple Stupid (KISS).

    Q: In addition to being an author, blogger, instructor, and part-time samurai, you also do consulting work. Tell us about the most complicated BizTalk Server project you ever worked on and how you solved the business problem.

    A:  Honestly, I’ve never been involved in a “real” BizTalk Server project (what do they say “those who can’t teach” ;-)). I have built a number of fairly complex demos for Microsoft using BizTalk, probably the most complicated demo involved using BizTalk Server with BizTalk Services (now .NET Services).

    Q [stupid question]: You regularly make the rounds in the conference circuit and naturally meet folks who only know you by your online presence. What’s the oddest thing someone has remarked to you upon meeting you in person for the first time? For me, on multiple occasions, I got a “oh, I thought you were taller.” Apparently I have the writing style of a 7 footer.

    A:  Where’s the kilt?

    Hope you all are enjoying this series, and if you have interest in being added to my “interview queue”, do let me know.

    Technorati Tags: , ,

  • Interview Series: Four Questions With … Yossi Dahan

    We continue our monthly look at thought leaders in the “connected systems” space by interviewing Yossi Dahan.  Yossi is a great technologist,  prolific blogger, Microsoft MVP as well as a good tipper.  Yossi recently attending Microsoft’s PDC conference in Los Angeles, and I wanted to get some insight from him as to what he saw there.

    Yossi provides some great insight into technology, and also tests the PG-13 limits of my blog with his answer to this month’s “stupid question.”  Enjoy.

    Q: At the just-completed Microsoft PDC conference, we saw a wide range of new technologies announced including Azure, Oslo, and Dublin. Given that you have a real job and can’t play with technology all day, which of the just-announced products/frameworks do you expect to spend the most time with, and why?

    A:  I will undoubtedly try to spend as much as I can looking at all of the above as I sincerely believe they are all pieces of a big change that is coming to how software is developed and run; of course, you are quite right, and it is rather unlikely that anyone with a day job will be able to spend enough time learning all of these, and so I think I will probably focus initially at the Azure platform and the services built on top of that.

    The main reason is that out of the various technologies announced during PDC and the weeks leading to it, I believe that the Azure platform is the one with the highest impact on how software is architected and designed; also, if my understanding is correct (and there are not concrete statements on this one yet) it is the .net services and bit of the Azure platform that will be the first ā€œout of the doorā€ while there is still some time before we could consider using Dublin or Oslo in a production environment.

    If I have a little bit more time left (or maybe ā€œofflineā€ time) to spend on anything else Oslo’s ā€œMā€ would be what I’d spend it on. I find this (defining modeling and textual dsls) a fascinating area and I really want to look deeper into this; it kind of doing my head in at the moment, just trying to grasp the concepts and potential they carry, but I have a feeling that for some of us this can make a big difference in how we work (and help others work).

    Last I would add that I’m already looking at some of the Geneva aspects, mostly the Geneva Framework (formerly known as ā€œZermattā€) and think this will also become a very common component in the enterprise environment.

    Q: You and I were recently chatting about a PDC pre-conference session that you attending where Juval Lowy was trying to convince the audience that ā€œeverything should be a service.ā€ Explain what he meant by that, and whether or not you agree.

    A:  It would be pretentious of me to try to explain Juval’s ideas, so let’s just say I’ll try to convey some of the points I’ve taken from his talk…

    Basically Juval argues that WCF is a lot more than just a ā€œframework for developing servicesā€ much like .net is more than just a ā€œframework for developing web servicesā€ as it was once presented; he argues that WCF services have so much ā€œgoodnessā€ that it would be silly not to want to use them for every class developed and he goes on to give quite a few examples, here are a couple of examples (he must have had over half a dozen)– Take the timeout default behavior in WCF for example – with WCF every call to a service operation has built in support for timeout, so if the method’s implementation takes forever (because of a deadlock situation for example, or simply an endless loop) the caller would receive a timeout exception after the configured time; this is a great feature, and to implement it in custom code, while possible, will take some effort (on the client side); to implement it around every method call seems unthinkable, let alone in every client out there.

    Another example that Juval goes through is tracing – with WCF you get built in tracking for each method call, including correlation of multiple logs (client and server for example etc) and the trace viewer provided with the framework; how much effort would it take you to build that into your code? with WCF you simply get it for free through configuration; quite neat.

    Juval goes on to list many such benefits like Fault tolerance , built-in performance counters, security, reliability, transactions, versioning tolerance etc. I will not repeat all of it here, but I hope you get the point; Juval actually goes as far as suggesting that every class should be a service – including type once known as primitive types such as String and Integer (they are already classes in .net, and now Juval suggests they could benefit from being a service)

    That was pretty much Juval’s point of view as I understand it; as for my perspective – do I like his idea? I certainly think it’s a great a food-for-thought exercise; do I agree? Not completely. It is true that WCF incorporates a lot of goodies, and I love it, but – and there’s a big but – it comes with a cost; it comes with a performance cost, which Juval tries to play down, but I think he’s taking a rather convenient stand; it comes with a complexity cost – WCF is not simple, especially when you start to combine things like security, reliability, transactions, instance management; do we want/need all that complexity in every class we write? I doubt it.

    Many of the benefits Juval lists really only apply once you’ve decided you want to use services; if I’m not using services – do I need reliable messaging? Do I need security? It’s easy to argue for WCF once you’ve decided that you need to run everything as a service, which I guess is Juval’s starting point, but if you’re not in that thinking mode (yet?), and I am certainly not – then you might think he has gone just a little bit too far šŸ™‚

    Now – I was never interested in looking too far into the future, I’m pretty much a now-and-there-and-around-the-corner type of guy who argues that it’s important to know where things are going but in my day to day job I need to give my client’s solid advice on what they can (and should) do now. Looking into the future performance is certainly going to be less of an issue, and I’m sure WCF management will improve significantly (Dublin is already a great step in the right direction) so we might end up very close; but that’s not present tense.

    It is worth noting that I do not at all disagree that we will be seeing a lot more services; we’ve already seeing a lot of enterprises and ISV’s adopt SOA architecture of one flavor or another, and the cloud services/platforms will only add more capabilities in that space, so I don’t want to play down the role of services and WCF in enterprise IT, I just think this will still be, for the foreseen future at least, another tool in the toolbox, albeit a major one.

    Q: As we now know that BizTalk Server has a new lease on life (i.e. releases planned after 2009), what types of engine-level changes would you like to see? In your opinion, what would make BizTalk messaging even more robust?

    A:  I should probably start by saying that I truly believe that BizTalk is, and has been for a while now, a very complete and mature product, and while there are clearly a few quirks and rough edges, the good definitely out-weighs the bad… I suspect it was not by chance that you have asked me to focus on engine-level changes – most of the stuff I have ā€œon my listā€ is related to user experience – both developer and administrator, there are less things that I believe need changing around the engine, but here are a few examples –

    One thing I would have like to see is the management database thinned a little bit – I don’t think, for example, that the entire schema is needed in the database (which makes deployment of updates harder); I would imagine that this could have been reduced in scope to store only xpaths related to promoted/distinguished fields etc.

    I also think, as both me and Mike Stephenson talked about in the past, that it would be a good idea to get rid of the compiled-pipeline concept and instead make it a configuration artifact, such as send ports for example; at the end of the day all a pipeline is just a set of components and their properties, represented as xml; sounds familiar? Doesn’t it feel suspiciously like a binding file element?

    While I don’t know if you would consider the above as engine-level changes (I think they could be considered as such), the next one certainly is –

    Better support for low latency scenario; several people have mentioned this in the past – BizTalk is great (no! really!) but it seems to be positioned a little bit in the middle – it’s not the best tool for large batch files processing (ETL is the technology of choice there), but with the latency introduced by multiple message box hops it is hard to position it in low latency scenarios; I know that Dublin is getting into that space, but I think Microsoft will do well to add in-memory pub-sub support to BizTalk to better support low latency scenarios.

    Others on the list – Somebody clever (not mentioning names!) once suggested giving better control over (orchestration) instance throttling, I completely second that. Also nice to have would be the ability to run a map on a typeless message (XmlDocument) – let my xslt figure out which template to run .

    Not much to ask, is it!?

    Q [stupid question]: If you work in the same office for quite a while, you may tend to let your guard down and ask questions or make comments that you wouldn’t typically say to strangers. Everyone’s had those awkward moments such as congratulating a woman on her pregnancy when no such congratulations were in order. Or, my new personal favorite, someone walking into your office and saying ā€œLast night I had a wildly vivid, erotic dream and you were in it!ā€ What is your example of a terribly awkward ā€œofficeā€ conversation?

    A:  Unfortunately finding embarrassing moments is not very hard, here’s one from the far history , I just hope I can correctly paint the scene –

    Quite a few years ago, let’s just say – before BizTalk was invented – I did a relatively small project in Sydney, Australia. The client was a lingerie company wishing to build a web application to compete with Victoria’s Secret very successful ecommerce web site, and I was called to the flag to build that.

    The owners of the company, if my memory serves me right, were a couple of playboy type guys (with most of the staff seem to be either ex-models or models-to-be) and once or twice a week they would come over to our dev shop, accompanied by one or two such assistants, to discuss the current status and any open issues around the development and design.

    I can’t remember what it was now, but there was this one thing they kept asking for time after time which made absolutely no sense – not from a visual design or usability perspective, not from an architecture perspective, and, as these things often go, it was also very hard to achieve technically; and so we constantly had debates in those meetings about whether and how we should implement this requirement. In one of those meetings they kept going on and on about this thing, while me and my Australian colleagues (yes – worth stating that was not at all alone in my reluctance to implement this) were trying to explain why it was so difficult to implement, but mostly, why it simply does not make sense as a feature on the web site. Eventually, being quite young and inexperienced (and Israeli, some would say) I got into a slightly too heated debate about it and eventually lost my cool and said, rather loudly, something like – ā€œI only have two words to say– I can’tā€.

    On its own – it’s not too bad (although now I know that such discussions are often doomed to failure from the beginning, but I had much less experience back then :)), but, and here’s the hard thing to explain perhaps, stupidly, I was trying at the time, with a fair bit of effort, to assume an Australian accent. Being Israeli, brought up on American television and having been in Australia for just about 3 weeks at the time, it did not go too well as you can imagine, and mostly it screwed up any chance I could have to be understandable, and that’s when not in a way-too-heated- debates; and so what I said and what they heard were two completely different things (I’m sure you can guess what they had in mind). Being the playboy types that they were they were certainly not going to let this one slip and so I they were having a laugh at my expense for the rest of that meeting (and the rest of that week in fact); much to my embarrassment.

    At least it made me stop trying to assume any accents, and with me working all over Europe, then landing in the north of England and now living just outside London I would say – good thing that I did, it’s all messed up as it is!

    Great job Yossi.  You are an engrossing storyteller.

    Technorati Tags: , ,