Author: Richard Seroter

  • Grab the BizTalk Server 2009 Beta

    Microsoft announced the public BizTalk 2009 beta today.  You can snag it from the Microsoft Connect site.  I’ve been running BizTalk 2009 on a VM since late summer, and while you don’t notice a whole lot of differences, there are some subtle things that you’ll notice.  Recall that the big ticket features are platform specific such as support for VS.NET 2008, Windows 2008 and SQL 2008.  However, you’ll also find the new UDDI services, better Visual Studio integration (e.g. Mapper debugging!), and some new queries in the Admin Console (i.e. historical data).  I admittedly haven’t spent too much time on the newest features, so I’ll look forward to what the community at large discovers.

    Update: Don’t forget to read up on what SteveM has to say about the release.  Specifically, note that you can now go grab the latest version of the ESB Guidance.

    Technorati Tags:

  • Attended Microsoft Cloud Computing Briefing

    Yesterday I was hanging out at the Microsoft office in Los Angeles for an “Architecture Council” meeting on the new Cloud Computing platform.  You can find the slide materials on the Strategic Architecture Council blog.

    Overall, a worthwhile session that you should attend if you get the chance in your city.  My favorite local evangelists were there, including presentations from Woody Pewitt, Kevin Boyle and the delightful David Chou

    We discussed what Microsoft considers cloud computing, the trade-offs between “on-premises” and cloud apps (e.g. on-premises apps have more control, customizations vs. cloud apps that are cheaper but sacrifice control and customization).  Also liked the point that we shouldn’t see cloud apps as simply current in-house apps that we shove to the cloud (ala people throwing SOAP interfaces on existing APIs and claiming to be “service oriented”).  We should be identifying a new class of apps that exploit the cloud in a safe, efficient manner.

    Woody presented Live Mesh which I’m aware of, but haven’t spent time actually playing with it.  As with many demos for these new MS tools, the focus was a bit more consumer-oriented.  I’m still trying to get my head around any use cases for a tool like this in enterprise software.

    David presented on .NET Services (formerly BizTalk Services) and once again inspired me to install the latest bits and walk through some scenarios.  Damn him for distracting me from other things.  Finally, we covered Azure and the types of web based apps we can now host in the cloud. 

    David discussed a few use cases for cloud services that got me thinking about how I’d use this in enterprise scenarios.  For instance, it makes a lot of sense to me to expose my organizational’s reference data (product list, sites, etc) as cloud-based services that folks can use in their own apps that collect data points about us.  Why should they maintain their own tables with copies of our reference data if we can provide it in a public internet service? By putting this (data+service) in the cloud, I have one less hole to punch in our external facing infrastructure.

    Clearly lots of this stuff will change prior to the formal release, so for me, these technologies fall into the bucket of “have enough knowledge to be dangerous” and no more.  I need to be able to talk about these technologies and hack up small demonstrations, but most importantly, I want to be able to know when I should consider these technologies in future projects for my company

    Technorati Tags: , ,

  • Interview Series: Four Questions With … Jon Flanders

    You’re probably surprised that I’ve kept this up, aren’t you.  Here we are, five interviews into this series and still going strong.  This month, we chat with the one Flanders that Homer Simpson actually appreciates: Jon Flanders.  Jon is a blogger, MVP, thought leader in the SOA space, and is comfortable wearing a skirt. Jon has recently released his book RESTful .NET to critical acclaim and has taken a break from his whirlwind book tour (and the thousands of screaming ladies) to engage in a little Q&A with us.

    Q: Tell us why a developer who has always built SOAP-based web services should care about REST. Why is it worth it to them to learn a different paradigm and what benefit does this paradigm offer to enterprise services that typically are built in a SOAP/RPC fashion?

    A:  What I typically tell people here is two things.

    1) REST has some significant advantages over traditional RPC styles (which most SOAP-based services are). GET results can be cached, REST services are *more* interoperable than SOAP and WS-*, and the statelessness constraint encourages more scalable implementations, and the uniform interface (GET, POST, PUT, DELETE) make building and using services much simpler than custom APIs (which SOAP-based services are because each one is a custom interface). If you use all of the constraints of REST (specifically the hypermedia constraint), you also get a highly decoupled implementation.

    2) Because of these advantages, most of the non-Microsoft parts of the computer industry have moved towards a RESTful approach already, and Microsoft is currently moving that way. When you look at ADO.NET Data Services, Windows Azure, you see a lot of Microsoft’s effort going into building RESTful services. Because of this, even if you aren’t planning on implementing all your services using REST, you probably will be consuming one or more RESTful services in the near future.

    In the end, I don’t advocate moving away from SOAP/WS-* where it makes sense or is necessary (for things like transactional calls between .NET and Java for example), but I think more services than people think could benefit from using a RESTful approach.

    Q: Outside of the plethora of WCF related things you inevitably learned during the writing of your latest book, what more general “service design” concepts/principles/pitfalls have you picked up as a result of authoring this book?

    A: Nothing really new. The concept/principle I believe in most is Keep it Simple Stupid (KISS).

    Q: In addition to being an author, blogger, instructor, and part-time samurai, you also do consulting work. Tell us about the most complicated BizTalk Server project you ever worked on and how you solved the business problem.

    A:  Honestly, I’ve never been involved in a “real” BizTalk Server project (what do they say “those who can’t teach” ;-)). I have built a number of fairly complex demos for Microsoft using BizTalk, probably the most complicated demo involved using BizTalk Server with BizTalk Services (now .NET Services).

    Q [stupid question]: You regularly make the rounds in the conference circuit and naturally meet folks who only know you by your online presence. What’s the oddest thing someone has remarked to you upon meeting you in person for the first time? For me, on multiple occasions, I got a “oh, I thought you were taller.” Apparently I have the writing style of a 7 footer.

    A:  Where’s the kilt?

    Hope you all are enjoying this series, and if you have interest in being added to my “interview queue”, do let me know.

    Technorati Tags: , ,

  • Application Architecture "Pocket Guides" From Microsoft

    Was trolling CodePlex today and noticed that those Patterns and Practices cats just released a neat little set of short architecture guides that cover a few specific architectural areas including the Agile architecture method, web application architecturemobile architecture, rich Internet application (RIA) architecturerich client architecture, and finally, service architecture.

    The service guide is a fairly useful little document that spells out a few important things.  It covers different considerations for SOA services, data services and workflow services.  I like the sections which cover items such as considerations for each design area (e.g. exception management, message endpoints, authentication, etc), as well as the considerations for each service layer.  There are brief descriptions of common patterns, and a handy checklist for evaluating your service architecture. 

    Overall, a pleasant read.  May not be breaking new ground for some of you, but if anything, this set of guides should encourage you to read about a particular architecture topic you aren’t that familiar with.

    Technorati Tags:

  • Source Code For BizTalk Rules Authorization Manager

    Back on the old Microsoft blog, I built and demonstrated a tool that exploits the built-in (but hidden) BizTalk Business Rules Engine API for securing access to business rules.

    I have a link to this Rules Authorization Manager on the downloads page of this blog, but up until now, I had only included the executable.  After a few requests for the source code during the past couple months, I finally got inspired to dig out the VM it was built on and extract the source code files.  So, now the downloaded zip file for the RAM tool has the source code which includes all my questionable coding practices for the world to see.

    Technorati Tags:

  • Interview Series: Four Questions With … Yossi Dahan

    We continue our monthly look at thought leaders in the “connected systems” space by interviewing Yossi Dahan.  Yossi is a great technologist,  prolific blogger, Microsoft MVP as well as a good tipper.  Yossi recently attending Microsoft’s PDC conference in Los Angeles, and I wanted to get some insight from him as to what he saw there.

    Yossi provides some great insight into technology, and also tests the PG-13 limits of my blog with his answer to this month’s “stupid question.”  Enjoy.

    Q: At the just-completed Microsoft PDC conference, we saw a wide range of new technologies announced including Azure, Oslo, and Dublin. Given that you have a real job and can’t play with technology all day, which of the just-announced products/frameworks do you expect to spend the most time with, and why?

    A:  I will undoubtedly try to spend as much as I can looking at all of the above as I sincerely believe they are all pieces of a big change that is coming to how software is developed and run; of course, you are quite right, and it is rather unlikely that anyone with a day job will be able to spend enough time learning all of these, and so I think I will probably focus initially at the Azure platform and the services built on top of that.

    The main reason is that out of the various technologies announced during PDC and the weeks leading to it, I believe that the Azure platform is the one with the highest impact on how software is architected and designed; also, if my understanding is correct (and there are not concrete statements on this one yet) it is the .net services and bit of the Azure platform that will be the first “out of the door” while there is still some time before we could consider using Dublin or Oslo in a production environment.

    If I have a little bit more time left (or maybe “offline” time) to spend on anything else Oslo’s “M” would be what I’d spend it on. I find this (defining modeling and textual dsls) a fascinating area and I really want to look deeper into this; it kind of doing my head in at the moment, just trying to grasp the concepts and potential they carry, but I have a feeling that for some of us this can make a big difference in how we work (and help others work).

    Last I would add that I’m already looking at some of the Geneva aspects, mostly the Geneva Framework (formerly known as “Zermatt”) and think this will also become a very common component in the enterprise environment.

    Q: You and I were recently chatting about a PDC pre-conference session that you attending where Juval Lowy was trying to convince the audience that “everything should be a service.” Explain what he meant by that, and whether or not you agree.

    A:  It would be pretentious of me to try to explain Juval’s ideas, so let’s just say I’ll try to convey some of the points I’ve taken from his talk…

    Basically Juval argues that WCF is a lot more than just a “framework for developing services” much like .net is more than just a “framework for developing web services” as it was once presented; he argues that WCF services have so much “goodness” that it would be silly not to want to use them for every class developed and he goes on to give quite a few examples, here are a couple of examples (he must have had over half a dozen)– Take the timeout default behavior in WCF for example – with WCF every call to a service operation has built in support for timeout, so if the method’s implementation takes forever (because of a deadlock situation for example, or simply an endless loop) the caller would receive a timeout exception after the configured time; this is a great feature, and to implement it in custom code, while possible, will take some effort (on the client side); to implement it around every method call seems unthinkable, let alone in every client out there.

    Another example that Juval goes through is tracing – with WCF you get built in tracking for each method call, including correlation of multiple logs (client and server for example etc) and the trace viewer provided with the framework; how much effort would it take you to build that into your code? with WCF you simply get it for free through configuration; quite neat.

    Juval goes on to list many such benefits like Fault tolerance , built-in performance counters, security, reliability, transactions, versioning tolerance etc. I will not repeat all of it here, but I hope you get the point; Juval actually goes as far as suggesting that every class should be a service – including type once known as primitive types such as String and Integer (they are already classes in .net, and now Juval suggests they could benefit from being a service)

    That was pretty much Juval’s point of view as I understand it; as for my perspective – do I like his idea? I certainly think it’s a great a food-for-thought exercise; do I agree? Not completely. It is true that WCF incorporates a lot of goodies, and I love it, but – and there’s a big but – it comes with a cost; it comes with a performance cost, which Juval tries to play down, but I think he’s taking a rather convenient stand; it comes with a complexity cost – WCF is not simple, especially when you start to combine things like security, reliability, transactions, instance management; do we want/need all that complexity in every class we write? I doubt it.

    Many of the benefits Juval lists really only apply once you’ve decided you want to use services; if I’m not using services – do I need reliable messaging? Do I need security? It’s easy to argue for WCF once you’ve decided that you need to run everything as a service, which I guess is Juval’s starting point, but if you’re not in that thinking mode (yet?), and I am certainly not – then you might think he has gone just a little bit too far 🙂

    Now – I was never interested in looking too far into the future, I’m pretty much a now-and-there-and-around-the-corner type of guy who argues that it’s important to know where things are going but in my day to day job I need to give my client’s solid advice on what they can (and should) do now. Looking into the future performance is certainly going to be less of an issue, and I’m sure WCF management will improve significantly (Dublin is already a great step in the right direction) so we might end up very close; but that’s not present tense.

    It is worth noting that I do not at all disagree that we will be seeing a lot more services; we’ve already seeing a lot of enterprises and ISV’s adopt SOA architecture of one flavor or another, and the cloud services/platforms will only add more capabilities in that space, so I don’t want to play down the role of services and WCF in enterprise IT, I just think this will still be, for the foreseen future at least, another tool in the toolbox, albeit a major one.

    Q: As we now know that BizTalk Server has a new lease on life (i.e. releases planned after 2009), what types of engine-level changes would you like to see? In your opinion, what would make BizTalk messaging even more robust?

    A:  I should probably start by saying that I truly believe that BizTalk is, and has been for a while now, a very complete and mature product, and while there are clearly a few quirks and rough edges, the good definitely out-weighs the bad… I suspect it was not by chance that you have asked me to focus on engine-level changes – most of the stuff I have “on my list” is related to user experience – both developer and administrator, there are less things that I believe need changing around the engine, but here are a few examples –

    One thing I would have like to see is the management database thinned a little bit – I don’t think, for example, that the entire schema is needed in the database (which makes deployment of updates harder); I would imagine that this could have been reduced in scope to store only xpaths related to promoted/distinguished fields etc.

    I also think, as both me and Mike Stephenson talked about in the past, that it would be a good idea to get rid of the compiled-pipeline concept and instead make it a configuration artifact, such as send ports for example; at the end of the day all a pipeline is just a set of components and their properties, represented as xml; sounds familiar? Doesn’t it feel suspiciously like a binding file element?

    While I don’t know if you would consider the above as engine-level changes (I think they could be considered as such), the next one certainly is –

    Better support for low latency scenario; several people have mentioned this in the past – BizTalk is great (no! really!) but it seems to be positioned a little bit in the middle – it’s not the best tool for large batch files processing (ETL is the technology of choice there), but with the latency introduced by multiple message box hops it is hard to position it in low latency scenarios; I know that Dublin is getting into that space, but I think Microsoft will do well to add in-memory pub-sub support to BizTalk to better support low latency scenarios.

    Others on the list – Somebody clever (not mentioning names!) once suggested giving better control over (orchestration) instance throttling, I completely second that. Also nice to have would be the ability to run a map on a typeless message (XmlDocument) – let my xslt figure out which template to run .

    Not much to ask, is it!?

    Q [stupid question]: If you work in the same office for quite a while, you may tend to let your guard down and ask questions or make comments that you wouldn’t typically say to strangers. Everyone’s had those awkward moments such as congratulating a woman on her pregnancy when no such congratulations were in order. Or, my new personal favorite, someone walking into your office and saying “Last night I had a wildly vivid, erotic dream and you were in it!” What is your example of a terribly awkward “office” conversation?

    A:  Unfortunately finding embarrassing moments is not very hard, here’s one from the far history , I just hope I can correctly paint the scene –

    Quite a few years ago, let’s just say – before BizTalk was invented – I did a relatively small project in Sydney, Australia. The client was a lingerie company wishing to build a web application to compete with Victoria’s Secret very successful ecommerce web site, and I was called to the flag to build that.

    The owners of the company, if my memory serves me right, were a couple of playboy type guys (with most of the staff seem to be either ex-models or models-to-be) and once or twice a week they would come over to our dev shop, accompanied by one or two such assistants, to discuss the current status and any open issues around the development and design.

    I can’t remember what it was now, but there was this one thing they kept asking for time after time which made absolutely no sense – not from a visual design or usability perspective, not from an architecture perspective, and, as these things often go, it was also very hard to achieve technically; and so we constantly had debates in those meetings about whether and how we should implement this requirement. In one of those meetings they kept going on and on about this thing, while me and my Australian colleagues (yes – worth stating that was not at all alone in my reluctance to implement this) were trying to explain why it was so difficult to implement, but mostly, why it simply does not make sense as a feature on the web site. Eventually, being quite young and inexperienced (and Israeli, some would say) I got into a slightly too heated debate about it and eventually lost my cool and said, rather loudly, something like – “I only have two words to say– I can’t”.

    On its own – it’s not too bad (although now I know that such discussions are often doomed to failure from the beginning, but I had much less experience back then :)), but, and here’s the hard thing to explain perhaps, stupidly, I was trying at the time, with a fair bit of effort, to assume an Australian accent. Being Israeli, brought up on American television and having been in Australia for just about 3 weeks at the time, it did not go too well as you can imagine, and mostly it screwed up any chance I could have to be understandable, and that’s when not in a way-too-heated- debates; and so what I said and what they heard were two completely different things (I’m sure you can guess what they had in mind). Being the playboy types that they were they were certainly not going to let this one slip and so I they were having a laugh at my expense for the rest of that meeting (and the rest of that week in fact); much to my embarrassment.

    At least it made me stop trying to assume any accents, and with me working all over Europe, then landing in the north of England and now living just outside London I would say – good thing that I did, it’s all messed up as it is!

    Great job Yossi.  You are an engrossing storyteller.

    Technorati Tags: , ,

  • I, For One, Welcome our New Cloud Overlords

    I’m trying really hard to not pay attention to PDC today, but, damn them and their interesting announcements!  The “Cloud OS” turned out to be Azure.  Good stuff there.  “BizTalk Services” are dead, long live .NET Services.    Neat that you have both Java and Ruby SDKs for .NET Services.

    Also, we got a full release of the Microsoft Federation Gateway (whitepaper here) and a preview of the Microsoft Service Connector (announcement here).  For companies tackling B2B scenarios with a myriad of partners, these technologies may offer a simplified route.

    Ok, back to real work.  Stop distracting me with your sexy cloud products.

    Technorati Tags: ,

  • Reason #207 Why the BizTalk WCF Adapter is Better Than the SOAP Adapter

    In writing my book, I’ve had a chance to compare the two BizTalk service generation wizards, and I now remember why the BizTalk Web Services Publishing Wizard (ASMX) drove me nuts.

    Let’s look at how the WCF Wizard and ASMX Wizard take the same schema, and expose it as a service.  I’ve purposely included some complexity in the schema to demonstrate the capabilities (or lack thereof) of each Wizard.  Here is my schema, with notations indicating the node properties that I added.

    Now, I’ve run both the BizTalk Web Services Publishing Wizard (ASMX) and the BizTalk WCF Service Publishing Wizard (WCF) on this schema and pulled up WSDL of each.   First of all, let’s look at the ASMX WSDL.  Here is the start of the schema definition.  Notice that the “Person” element was switched back to “sequence” from my XSD definition of “all.”  Secondly, see that my regular expression no longer exists in the “ID” node.

    We continue this depressing journey by reviewing the rest of the ASMX schema.  Here you can see that a new schema type was created for my repeating “address” node, but I lost my occurrence boundaries.  The “minOccurs” is now 0, and the “maxOccurs” is unbounded.  Sweet.  Also notice that my “Status” field has no default value, and the “City” node doesn’t have a field restriction.

    So, not a good story there.  If you’ve thoughtfully designed a schema to include a bit of validation logic, you’re S.O.L.  Does the WCF WSDL look any better, or will I be forced to cry out in anger and shake my monitor in frustration?  Lucky for me (and my monitor), the WCF wizard keeps the ENTIRE schema intact when publishing the service endpoint.

    There you go.  WCF Wizard respects your schema, while the ASMX Wizard punches your schema in the face.  I think it’s now time to take the ASMX Wizard to the backyard, tie it to a tree, and shoot it.  Then, tell your son it “ran away but you got a brand NEW Wizard!”

    Technorati Tags:

  • That’s "MR" Skeptical Blogger to You!

    A couple weeks back I answered some interview questions regarding my impressions of Microsoft’s Oslo.  One of my answers found its way into an article you can read on ComputerWorld entitled Microsoft eyes Oslo as game-changer for application development.   It’s a good article with some interesting perspective.  You’ll find my quote on page three.  I wouldn’t say that I’m skeptical of the intentions of Oslo, but I AM just curious to see how they pull it off.  So, maybe I’m skeptical of their implementation!  The point I was trying to make is that expansive models aren’t easy to either design or maintain, and attempts that I’ve seen so far to synchronize the model with the implementation have been lacking.  We’ll see if the Oslo repository and “M” language are actually game changers.  Here’s hoping.

    I’m not making it to PDC this year, but since I live close by, I hope to pop in on some extracurricular events.

  • In-Memory BizTalk Resequencer Pattern

    I was asked a couple days ago whether it was possible to receive a related but disjointed set of files into BizTalk Server and both aggregate and reorder them prior to passing the result to a web service.  Below is small sample I put together to demonstrate that it was indeed possible.

    You can find some other resequencer patterns (most notably, in the Pro BizTalk Server 2006 book), but I was looking for something fairly simple and straightforward.  My related messages all come into BizTalk at roughly the same time, and, there are no more than 20 in a related batch.

    Let’s first take a look at a simplified version of the schema I’m working with.

    I’ve highlighted a few header values.  I know the unique ID of the batch of related records (which is a promoted value), how many items are in the batch, and the position of this individual message in the batch sequence.  These are crucial for creating the singleton, and being able to reorder the messages later on.  The message payload is a description of a document.  This same schema is used for the “aggregate” message because the “Document” node has an unbounded occurrence limit.

    I need a helper component which stores, sorts and combines my batch messages.  My class starts out like this:

    Notice that I’m using a SortedDictionary class which is going to take the integer-based sequence number as the “key” and an XML document as the “value.”  The SortedDictionary is pretty cool in that it will automatically sort my list based on the key.  No extra work needed on my part.  I’ve also got a couple member variables that hold values universal to the entire batch of records.  I accept those values in the constructor.

    Next, I have an operation to take an inbound XML document and add it to the list.

    You can see that I yank out the document-specific “SequenceID” and use that value as the “key” in the SortedDictionary.

    Next I created an “aggregation” function which drains the SortedDictionary and creates a single XML message that jams all the “Document” nodes into a repeating collection.

    As you can see, I extract values from the dictionary using a “for-each” loop and a KeyValuePair object.  I then create a new “Document” node, and suck out the guts of the dictionary value and slap it in there.

    Now I can build my BizTalk singleton.  Because we promoted the “BatchID” value, I can create a correlation set based on it.  My initial receive shape takes in a “BatchRecord” message and initializes the correlation set.  In the “Set Variables” Expression Shape, I instantiate my loop counters (index at 1 and maximum based on the “BatchCount” distinguished field), and the helper class by passing in the “BatchID” and “BatchCount” to the constructor.  In the “AddDocToBatch” Expression Shape, I set my message equal to a variable of type “XmlDocument”, and pass that variable to the “AddDocumentToDictionary” method of my helper class.

    Next, I have a loop where I receive the (following correlation) “BatchRecord” message, once again call “AddDocumentToDictionary”, and finally increment my loop counter.

    Finally, I create the “BatchResult” message (same message type as the “BatchRecord”) by setting it equal to the result of the “GetAggregateDocument” method of the helper class.  Then, I send the message out of the orchestration.

    So, if I drop in 5 messages at different times and completely out of order (e.g. sequence 3, 5, 4, 2, 1), I get the following XML output from the BizTalk process:

    As you can see, all the documents show up in the correct order.

    Some parting thoughts: this pattern clearly doesn’t scale as the number of items in a batch increases.  Because the batch aggregate is kept in memory, you will run into issues if either (a) the batch messages come in over a long period of time or (b) there are lots of messages in a batch.  If either case is true, you would want to consider stashing the batch records in an external storage (e.g. database) and doing the sorting and mashing at that layer.

    Any other thoughts you wish to share?

    Technorati Tags: