Updating the Retry Activity - adding timed interval

by Matt Milner 28. November 2007 14:19

Updated 11/29 - see below

I recently created a retry activity that I've been wanting to write for some time now, and had a few features that I wanted to add. I was teaching a class a couple of weeks ago and took the opportunity to add a retry interval to the activity.  The activity is designed as a composite activity which can retry its child activities if it detects that a fault/exception has occurred.  Rather than retrying immediately, I wanted the consumer of my activity to be able to specify a TimeSpan indicating how long to wait between retry attempts.

The simplest part was adding a property that allowed people to specify the interval. 

private TimeSpan retryInt;

public TimeSpan RetryInterval 
{
    get { return retryInt; }
    set { retryInt = value; }
}

 

The next step was to actually use this information.  When the child activity closes, my Retry activity checks if it faulted and then needs to reschedule the child activity for execution.  With the interval, I need a way to wait for a duration and then schedule the activity for execution.  Since my workflow could be persisted during this interval, I have to use a mechanism that will both support passivation, and allow for the workflow to be resumed automatically when the duration has been reached.  To support this I used the same mechanism that the delay activity uses, which is to create  timer subscription and then wait for an event signaling the time has expired. 

In my event handler for closing child activities, if I determine that I should retry the child activity, I use the following code to 1) create a new workflowqueue where the timer can signal completion, 2)register for data arriving on the queue, 3) create a TimerEventSubscription with my duration and my queue information and 4) add my timer subscription the collection on the workflow. 

 

DateTime expires = DateTime.UtcNow.Add(RetryInterval);
SubscriptionID = Guid.NewGuid();
thisContext.GetService<WorkflowQueuingService>().CreateWorkflowQueue(SubscriptionID, false).QueueItemAvailable += new EventHandler<QueueEventArgs>(RetryActivity_QueueItemAvailable);
TimerEventSubscription subscription = new TimerEventSubscription(SubscriptionID, WorkflowInstanceId, expires);
TimerEventSubscriptionCollection timers = GetTimerSubscriptionCollection();
timers.Add(subscription);

 

The GetTimerSubscriptionCollection simply grabs the collection from the parent workflow by recursively climbing the parent hierarchy. 

 

private TimerEventSubscriptionCollection GetTimerSubscriptionCollection()
{
    Activity parent = this;
    while (parent.Parent != null)
    {
        parent = parent.Parent;
    }
    TimerEventSubscriptionCollection timers = (TimerEventSubscriptionCollection)parent.GetValue(TimerEventSubscriptionCollection.TimerCollectionProperty);
    return timers;
}

 

Finally, I have the event handler for when data arrives on the queue for the timer.  I simply empty and delete the queue, then begin my next iteration on the child activity. 

 

void RetryActivity_QueueItemAvailable(object sender, QueueEventArgs e)
{
    ActivityExecutionContext ctx = sender as ActivityExecutionContext;
    ctx.GetService<WorkflowQueuingService>().GetWorkflowQueue(e.QueueName).Dequeue();
    ctx.GetService<WorkflowQueuingService>().DeleteWorkflowQueue(e.QueueName);

    BeginIteration(ctx);
}

 

So adding a retry interval turned out to be pretty simple and I was also able to provide a good example of how you can put your own delay into your activities.  In a later post, I'm going to talk about the other issue I had to manage which is how to deal with exceptions and control whether or not my Retry activity got put into the faulting state. 

 

BEGIN UPDATE

Someone posted a question on the original activity indicating some concern over the fact that I hadn't overridden the Cancel method.  In the simpler scenario I should have still done that, but with the addition of my retry interval it became even more important as I had more to clean up.  So I added the Cancel method and do two things in it.  First, I had to see if there were any spawned execution contexts that were currently executing indicating that I had an iteration in progress and cancel the root activity if I did.  I also had to add some code to remove the timersubscription in both the expected case and the exception case.  In the process, I factored some code into a helper method to remove the timer subscription and the workflow queue when I no longer needed them.  The cancel method now looks like this:

protected override ActivityExecutionStatus Cancel(ActivityExecutionContext executionContext)
{
    bool canClose = true;
    ActivityExecutionContext ctx = executionContext;
    //cancel any execution iteration
    if (ctx.ExecutionContextManager.ExecutionContexts.Count > 0)
    {
        ctx.ExecutionContextManager.ExecutionContexts[0].CancelActivity(ctx.ExecutionContextManager.ExecutionContexts[0].Activity);
        canClose = false;
    }
    //remove the timer and related queue if there is a subscription
    CleanupSubscription(ctx);
    return canClose ? ActivityExecutionStatus.Closed : ActivityExecutionStatus.Canceling;
}
Notice that I can only close the activity if the child is not currently executing.  If I had to cancel the child activity, then I can't yet close the retry, that will happen in the usual way when the child activity closes.  The other important step I added was to check the current ExecutionStatus of the retry activity before scheduling a new iteration.  If the activity has moved into the Canceling state, I shouldn't start a new iteration regardless of how many times I am supposed to retry. 

If you have any other feedback or suggested improvements, keep them coming. 

END UPDATE

The updated retry activity can be found here, and has been updated for Visual Studio 2008.  If you want the updates for 2005, just download the original, and then copy the class file for the retry activity from the new project.  I didn't use any new features of 2008, just updated the project and solution files, so it should work fine. 

Tags:

Windows Workflow Foundation

Visual Studio 2008 released!

by Matt Milner 19. November 2007 11:25

Hot off the presses, you can get it now from MSDN subscriber downloads or the trial version.  See Soma's blog for more details on the new features, etc. 

Surprisingly, being a WF and BizTalk guy, I'm least excited about the WF/WCF integration.  I'll talk more about what I dislike later, but this is a happy post!  I'm jazzed about the new web models in WCF if only because it provides a nice affirmation of the extensibility of WCF.  I might not like all of the details of how they have accomplished things, but the CSD team has certainly shown that WCF is a solid platform with all the extensibility hooks you need. 

I'm also excited to spend some time with LINQ now that I'll have the bits on my main PC (I have only put the beta on VMs or my secondary machine in my office).  Like everyone else, I've done the simple examples to get some data out of a database and bind it to a UI, but I really want to dig in and see how it works after I have some code written and want to change something in the database. I want to see what the visual modeler/designer can and cannot do, and how using XML files might benefit me more than using the simple path that is there in V1.  

The main thing that really sells this, and some folks don't know this, is that VS 2008 and .NET 3.5, build on the existing .NET 2.0 framework.  So, it should be close to a no-brainer for organizations to upgrade because they can still build their 2.0 applications without having to worry about using 3.5 assemblies if they use the multi-targeting correctly in VS / MS Build.  That's right, there are only additive changes and bug fixes in 3.5 and it uses the same 2.0 runtime that you are using today.  So there should be no issues with breaking changes to core things like datasets and remoting like there were moving from 1.1 to 2.0.  I'm upgrading all my boxes to take advantage of the new features and perf improvements.  The only reason I could see for not updating is if you have other tools like BizTalk or Commerce Server that depend on the 2005 version.  Then you can run both side-by-side, and maybe strip out some of the tools from 2005 that you no longer used to free up some space on your hard drive.   

So, download, enjoy, and happy coding! 

Tags:

General Musings | Windows Workflow Foundation | Windows Communication Foundation

Demos from SOA conference (including RetryActivity)

by Matt Milner 31. October 2007 14:48

If you attended my talk at the SOA conference on building composite activities, then you'll find the demos here.  I really liked doing this talk and it gave me a reason to work on a composite activity that I wanted to write for a while: the RetryActivity.  I started wanting to build this activity because when I teach people about WF and we talk about security on incoming data, there is always a question about how to handle failed authorization.  When the user fails authz on the built-in activities today, an exception is raised which I can catch and handle in the workflow.  The problem is that once I have done that catch, I close out the composite activity that caught the exception and I'm no longer listening for data.  Well, if Bob tries to submit data and fails because he isn't supposed to, I don't want to stop listening for data, I want to keep listening and let Alice (who is authorized) submit data and interact with my workflow.   The retry activity lets me do that by re-executing the child activities for a certain number of tries as long as an exception gets raised. 

The way I implemented this was to spawn a new execution context for each iteration much like the while activity.  In my case, what I use to determine if I'll iterate again is a combination of whether the child activity failed to succeeded, along with a retry count/max pair that lets me know if I should quit trying.  Naturally, if there are no failures, I just close and finish.  This allows me to use a single purpose activity for retry behavior instead of continually having use the while activity and check whether an exception was raised or not.  It also makes it easier to manage the errors raised by the child activities and nulling out the CurrentException property so I can decide when errors should propagate beyond my activity. 

Now, what I'd really like, related to security, is to move the authorization out of the workflow, so that authorization failures don't have to be modeled in my workflow unless I want them to be.  That way my workflow doesn't even get invoked if you don't have rights.  Something where I can declare the security in the workflow, and then generally be able to apply security to the queues that are used to communicate with the workflow.   The "Silver" or WCF activities in .NET 3.5 do a little of this by reading the validation from the workflow definition and invoking it before sending the data to my workflow. 

 

I'll be working on some other features of this activity to allow things like retry only on certain exception types, and the ability to put an interval between retries.  If there are other features you'd like to see, let me know and I'll put them on my plan. 

Tags:

Windows Workflow Foundation

Microsoft announces next big wave of products &amp;quot;Oslo&amp;quot;

by Matt Milner 31. October 2007 14:00

I attended the Microsoft SOA and BP conference this week in Redmond where they announced their plans for future of connected systems (the division responsible for BizTalk, WCF, WF, AD, and much more).  You can read more about the announcement and what impacted products are on the Microsoft SOA site.  What I found most interesting is that this effort is not just about the Connected Systems Division (CSD) but also includes the Developer Division with the next release of Visual Studio and the .NET Framework, as well as the System Center products.  The close alliance of those product groups is something we don't often see from Microsoft and can only be better for customers as we'll get a better story about using those products together. 

Now being a BizTalk/WCF/WF guy, I'm pretty interested in how all of those technologies get impacted by the modeling and management tools as well as how the integration of these technologies continues to improve.  It's only going to get better, but like all technology, we'll continue to have things to learn and understand about these technologies and how they get leveraged in different environments. 

Most interesting to me is that the keynote included information stating that CTPs and Betas would be available in 2008. That makes this stuff much more real for the near term than say a 5 year timeline.  Microsoft obviously won't be able to create the entire vision in a single release of each product, but the next phase is pretty exciting and being able to get your hands on the technology and see what they are doing and thinking can be critical as you plan for your future projects.  It's also a great way to give feedback and help direct the product team about what is important to you as a customer. They really do listen, even if you don't always get exactly what you want, significant changes happen based on customer feedback.  

We also had our two day pre-conference training on the new features in BizTalk R2 and .NET 3.5.  In addition we had some fun talking about "Astoria" and BizTalk Services which are emerging technologies built on WCF and other CSD technologies that provide some really compelling options for messaging and data access.  The class went really well and we're thinking about making it available as a seminar course for people who want to get a good deep dive into the broad range of technologies.  Let us know if you are interested in the class for your company. 

Tags:

BizTalk Server | General Musings | Windows Workflow Foundation | Windows Communication Foundation

BizTalk Mapper/XSLT - Creating sequence number based on destination tree

by Matt Milner 25. October 2007 10:48

When creating maps in BizTalk server, or generally creating XSLT transforms, you might run into a situation, as I did, that calls for you to generate a sequence number in your destination document.  If the source document and destination have a similar repeating structure, you can always use the Iteration Functoid or the position() function to create a sequence based on the source tree.  I recently ran into a situation where I needed to create a sequence number for the output records, but I was conditionally mapping the input records.  For example, I had a structure like the following for my source:

<Invoices>

  <Invoice>

    <InvoiceDetail>

       <SomeFlag>true</SomeFlag>

    </InvoiceDetail>

    <InvoiceDetail>

       <SomeFlag>false</SomeFlag>

    </InvoiceDetail>

</Invoice>

  <Invoice>

    <InvoiceDetail>

       <SomeFlag>false</SomeFlag>

    </InvoiceDetail>

    <InvoiceDetail>

       <SomeFlag>true</SomeFlag>

    </InvoiceDetail>

</Invoice>

</Invoices>

In my output, I actually had a similar structure, but I was only creating output records for the items where the "someFlag" element was true.  So in each case in the example, I'd only have one detail record for each invoice rather than two.  In the first case, creating a sequence number based on the position() function or Iteration functoid would work just fine.  However, in the second Invoice, my sequence would start with 2 since that is what the position() function would return.  This really gets fun when you have multiple details and your sequence starts skipping positions.  I was getting sequences like 1,4,10,13. 

What I needed was a way to generate the number based on the destination tree, not the source; enter xsl:number.  Using xsl:number you can provide a value and do some formatting, etc., but I was more interested in using it without submitting a value which causes a new number to be generated. If you look at the MSDN documentation for xsl:number you'll see that you can specify more attributes to control how the number gets generated.  Specifically the "count" and "from" attributes allow you to scope the generation of the numbers and where to start, respectively.  These attributes allow you to generate the numbers as you want them in the output.  A lot of the examples I found on the web were for formatting numbering in lists or table of contents formats to number things like 1.1  or 1.A.  The "format" attribute really helps here, but I just wanted a raw number. 

I added a scripting functoid to my map and set the type to inline XSLT.  I then used the xsl:number like so (based on the sample document above). 

<xsl:number count="*[./s0:SomeFlag='true']" />

That got me all I needed. My sequence numbers in my output were only generated when an output detail was created and the sequences were unique to each invoice (i.e. they restarted for each invoice which is what I wanted).  In my case I didn't need to use any other controls on xsl:number but you might find that you need to further restrict it using the "from" attribute.  I'd start simple and then work from there. 

A couple of things to note.  First, the XSLT expression that I have in the "count" attribute points to all elements where the SomeFlag is true.  You may need to play around with this to get the right expression, just make sure you are referencing the right context (based on your XSLT) and filter the records appropriately.  Also notice that I used a namespace prefix on the SomeFlag element.  In the BizTalk mapper I needed to add this since BizTalk uses this prefix for the primary/first namespace of my source document.  Without this prefix, I always got empty output, so watch for this in your own code and use the validation map functionality in BizTalk to view the XSLT and see what prefixes you might need.  If you want to make sure things are working at a base level, then just use * for the "count" attribute and you should get a similar output as if you used the position() function.  Then apply filters from there to get the results you want. 

I couldn't find anything to really help me figure this out without some work, so hopefully this will save someone some time (or maybe you are just better at searching the web than I am). 

Tags:

BizTalk Server

Demos from Heartland Developers Conference

by Matt Milner 21. October 2007 14:45

I had a great time again this year at the Heartland Developers Conference (HDC) talking about extending WCF.  The guys did a great job of putting on a top notch 2 day conference.  It just gets better each year.  If you attended my session, then you will find the source for the demos here

I'm headed out to Redmond next week for Pluralsight's 2 day pre-conference event on all the great technologies coming out of the Connected Systems Division at Microsoft and then speaking on best practices for building composite activities at the SOA and BP conference on the Microsoft campus. 

Tags:

Pre-conference training at the upcoming SOA BPM conference

by Matt Milner 7. September 2007 07:58

Jon Flanders, Aaron Skonnard and I are going to be doing a two-day workshop just before Microsoft's SOA and BPM conference at the end of October.  This will be two days chock full of training content on the latest tools coming out of the Connected Systems Division at Microsoft including BizTalk Services, WCF/WF integration, and BizTalk Server R2.  Check out our official page and sign up if you'll be attending the conference or if you are in the area.  This will be a great way to start your week and maximize your travel. 

Tags:

Upcoming talk at the Heartland Developer's Conference - October 19

by Matt Milner 24. August 2007 17:42
I've got an upcoming talk at the Heartland Developer's Conference coming up in Omaha in November.  If you are a developer in the Midwest, you should definitely check out this event.  These guys put on a nice two day conference with some great technical content and  fun atmosphere.  Despite what you might think, downtown Omaha can be fun after all the technical sessions.  :)   I love doing these smaller regional events because the people are so much fun and put so much into making sure the show is an awesome experience for the attendees. 
 
I'll be talking about extending the WCF runtime so come check it out if that sounds interesting. 
 
 

Tags:

General Musings

Downloads for my presentations at the Minnesota Developer's Conference

by Matt Milner 24. August 2007 17:37
If you joined me for the two presentations I did Wednesday here in Minneapolis, then you'll find the code downloads for those sessions here(WCF/WF) and here (SharePoint and WF).  If you weren't there, then never mind. 

Tags:

General Musings

Workflow Communications article posted

by Matt Milner 10. August 2007 03:39
My MSDN article on communications in workflow has been posted.  In this article, I really wanted to try and focus on the low level communication so that people would understand what was going on under the covers.  I don't both with the "local communications" pieces as they provide a simple abstraction on top of this underlying architecture. 
 
I hope you find this useful.  Any feedback is always appreciated, and if you have ideas for things you'd like to see in future columns, leave a comment here. 
 
 

Tags:

Windows Workflow Foundation