Recently, Eric asked me to speak at one of our company meetings about the future of the OpenSpan platform. He didn’t want me to speak about releases or features, but rather about the platform as a whole. What is our vision for the OpenSpan platform?

This is actually a topic I think a lot about. For every feature we design and every release we plan, I ask the question: “How will this push the platform forward?” One of the things I think we’ve done very well at OpenSpan is to stay true to the principles that led us to create OpenSpan in the first place. When Stephen and I first started planning the platform we each brought two sets of distinct experiences to the table.

Stephen had spent the past several years creating visual programming environments. He believed strongly that the barrier that prevented more business analysts from programming was not the concepts of programming but rather the syntax of programming. Ultimately, many of the concepts programmers use everyday, conditions and loops in particular, are familiar to every advanced Excel or Access user. Other concepts, such as types, casts, threads and objects are either unnecessary for many tasks or presented in such a way as to be impenetrable to the lay person. Stephen was committed to providing an environment where business users could develop or participate in the development of automations.

I had spent the past few years creating custom desktop automation solutions for enterprise customers with diverse application requirements. I had become painstakingly familiar with the available APIs: COM libraries for Lotus Notes, Internet Explorer, Office, Siebel and others; MSAA and windows functions for Win32 applications; HLLAPI, EHLLAPI, etc. for mainframe emulators. I had also become painfully familiar with the limitations of these APIs: narrow or incomplete APIs; little or no support for events; frustratingly inconsistent implementations of supposedly standard APIs. My experiences had convinced me that there was a better way to automate these applications.

In our very first face to face conversation, Stephen and I decided two things: OpenSpan would be accessible to both developers and non-developers and OpenSpan would be completely reliable. If you clicked a button with OpenSpan, it would click every time.

I would be lying if I claimed that we knew that these principles would lead us to our current automation engine and injection library. Some ideas like controls, targets and match rules evolved quickly. Other ideas like asynchronous links and keys evolved in response to customer requirements. Whatever happened, though, we stayed true to our principles. When we reached the limits of message hooks and accessibility interfaces, we scrapped them and wrote an injection and hooking library from the ground up. When we encountered a control we didn’t support, we reverse engineered the platform to get at the real object. When a public interface didn’t supply the events we needed, we made our own with some well placed hooks.

Over time, some new principles evolved:

1. A button is a button is a button. In other words, hide the platform implementation details. Every control in OpenSpan should act the same regardless of the actual platform.
2. Never hide functionality. If a platform provides functionality that another platform does not provide, expose it. A great example of this is HTML controls. They follow the same pattern as other OpenSpan controls, but add extra properties to manipulate the inner HTML, set attributes, etc.
3. Always talk to the object. Pretty quickly it became apparent that every time we tried to take a shortcut, it burned us. GDI hooking. Sucks. MSAA. Sucks. Message hooks. Suck. There’s always something to tweak or modify or re-implement for each environment. The only API you can rely on is the one the original developer of the application used. So what if it’s not exposed? Sure, it takes more time initially, but we only need to do it once and then it works every time. I’m very proud of the fact that OpenSpan implementations never need engineers on site to develop integrations. If there is a control we don’t support, we write a translator and publish it for all of our customers. One and done.
4. Know everything about the object. It’s not enough to talk to the object. We know when an object is created. We know when an object is destroyed. We intercept events before the object receives them. ‘Nuff said.
5. There is no such thing as too advanced. Early on we had a debate about threading. Should we hide threading from our users? Should the platform make threading decisions for them? Ultimately, we decided not to hide threading because doing so would inevitably limit our users. We made threading accessible by making it easy to visualize. Do people screw themselves up with threading? Sure, but people burn down their house with turkey fryers every year too. That doesn’t mean fried turkey doesn’t taste good.
6. Developers do not always want to drag and drop. This one is pretty obvious. Even though we want the platform to be accessible to non-developers, we want it to be just as accessible to developers. For developers, sometimes it’s just easier to write code. And when it is easier, there’s nothing more frustrating than not being able to. From the very beginning, we made sure that developers could write C# and VB .NET scripts and use their own .NET components in automations.

So what is our vision for the OpenSpan platform?
OpenSpan was built from the ground up to be a accessible and reliable desktop automation platform. Those two simple words, accessible and reliable, have directly guided the evolution of our most valuable IP: automation and injection. Our vision is to extend those two technologies into new areas. We are actively working on new ways to expose them to developers including our upcoming Visual Studio plug-in, Lotus Expeditor container and translator SDK. We are also working to apply these technologies in new environments. In the future, we will move beyond the desktop to provide automation and injection on the server. When we do, you can be sure it will be just as accessible and reliable as our current platform.

It’s that time again! As our 4.1 release nears completion, it’s time to start posting about our new features. One of the new features we’re testing currently is a new automation stack implementation. As our platform has evolved, we have begun to use our automation engine in new and more complicated environments. Previously, most of our environments were single-threaded automations of applications on the user desktop. Now, our platform is being used on the server to automate back-office requests, process messages and expose applications as web services. In this new environment, it quickly became clearly that our automation engine needed a proper stack to ensure that our automations remained thread safe.

If you’re not familiar with OpenSpan, one of your first questions might be how the heck don’t you already have a stack? To explain that, I need to go under the hood for a little bit. Looking at the automation your first instinct would be to assume that OpenSpan is a visual programming language that generates source code. For example, the automation below could be expressed in a C# class as:

private string varString;private void Setup(){   btnSubmit.Click += delegate   {      varString = txtPhone.Text;      webBrowser.Navigate(txtUrl.Text);   };}

Pretty straightforward, right? How about this automation? In this case we’ve made one of the links asynchronous indicating that the next step should be performed on a new thread.

private string varString;private void Setup(){   btnSubmit.Click += delegate   {      varString = txtPhone.Text;      NavigateDelegate navDel = delegate(string url)      {         webBrowser.Navigate(url);      };      navDel.BeginInvoke(txtUrl.Text, null, null);   };}

Now it’s starting to get more complicated. Every time we need to execute an asynchronous link we would need to generate a delegate type. I’m using anonymous methods which makes it easier, but it’s still pretty difficult. How about this automation? We have two execution threads from different events converging on a single block. Easy to express in OpenSpan, but we would need to use another method or delegate in C#.

private string varString;private void Setup(){   btnSubmit.Click += delegate { NextStep(); };   webForm.Submitting += delegate(object sender, CancelEventArgs e)   {      NextStep();   };}

private void NextStep(){   varString = txtPhone.Text;   NavigateDelegate navDel = delegate(string url)   {      webBrowser.Navigate(url);   };   navDel.BeginInvoke(txtUrl.Text, null, null);}

Even more complicated, and I haven’t even added entry points to the automation. So what can we conclude from this exercise? As you might have guessed, OpenSpan does not generate source code. Instead, we model execution flow using a serialized object graph. This gives us greater flexibility to express logical concepts such as branches, joins, etc. without the limitations of traditional syntax. However, because we do not generate code we are unable to rely on the existing thread stack to manage our local variables for us. Thus, prior to 4.1, we simply didn’t have a stack at all.

So, how does the new stack work? For the most part, it works like the normal thread stack we all know and love. When an automation is entered, all of the local variables and components are allocated and pushed onto the automation stack. So far so good, right? But what about those pesky asynchronous links? In OpenSpan it’s perfectly legal to have two links on different threads in the same automation updating variables.

So what should we do here? In a traditional thread stack model, each asynchronous link gets it’s own stack. OpenSpan, however, makes creating new threads absurdly easy. Thus, it’s easy to create a number of parallel threads to accomplish tasks such as updating three or four application simultaneously. Should we allocate new variables and components every time we execute a new asynchronous link? Should we initialize them to their original defaults or should we initialize them to their current values?

As you probably guessed, the correct answer is none of the above. The new automation stack is not a thread stack. Local components and variables are allocated on a per automation basis not on a per thread basis. Thus, each of those asynchronous links in the example above are actually updating the same variable. This approach preserves the ease of threading OpenSpan traditionally provides, while adding the additional safety of stack-based variables.

Now that I’ve explained how the new automation stack works, I’ll cover some of the common questions I’ve been asked about this feature.

What if I want my variables to be available to all automations?
No problem. OpenSpan automations now have a global and a local tray. Components in the global tray behave just like they did in previous versions of OpenSpan. They can be updated from any automation at any time. Components in the local tray can only be used on that automation and are not exposed to other automations.

When are local and global components created?
Global components are created when the project is started and can be used in any automation. Local components are only created when an automation is run. Local components can only be used in the automation that contains them.

When are local and global components destroyed?
Global components are destroyed when the project is stopped. Local components are destroyed when all the threads in the automation have completed.

When should I use a local component?
Local components are appropriate when the component in question does not need to be accessed from other automations or when an automation can be run simultaneously on multiple threads. Local components are recreated every time an automation is run and thus will not maintain state across multiple runs. For instance, you could not create a local variable to keep track of the number of times an automation is run. Every time the automation runs the counter variable would be recreated and initialized to zero.

What happens when I upgrade my old automations?
Variables and components from old automations will automatically be placed in the global tray during the upgrade process. Thus, your old automations will continue to function as they did before. To take advantage of the automation stack, you can easily make any global components local by right-click and selected the “Make Local” menu item.

Over the next few weeks I’ll be writing some articles to spotlight new features in our 3.2 release. For many of our users, the most exciting new feature is debugging. For the first time, you now have the ability to debug OpenSpan automations in real time using paradigms familiar to users of Visual Studio, Eclipse and other IDEs such as breakpoints, step functions, and watch, call stack and thread windows.

Breakpoints

The essence of debugging is giving the developer the ability to stop execution and examine the state of an application. Stopping execution is referred to as breaking. By breaking execution a developer can find flaws in his logic, scenarios he did not anticipate, or invalid data. The most convenient way to break execution is a breakpoint. A breakpoint is simply a way of telling an application to “stop when you get here.” When the application is run, the debugger will break execution when the breakpoint is reached.

To set a breakpoint in an OpenSpan automation, simply highlight a link, right click and select “Toggle Breakpoint.” A red dot will now appear on the link indicating that a breakpoint has been set. If you right click on an existing breakpoint, you can select “Disable” to temporarily prevent the debugger from breaking when the breakpoint is reached, or “Delete” to remove the breakpoint entirely.

When a breakpoint is reached within an automation, the executing link will become a dashed, animated line. The blocks on either side of the link will also be outlined. The executed block will have a solid blue outline whereas the executing block will have a dotted blue outline.
You can view all of the breakpoints you have set within a project in the “Breakpoints” window. The Breakpoints window lists each breakpoint by automation. You can double-click on a breakpoint to go to that automation. You can also enable or disable a breakpoint using the checkbox next to it. If you right-click on an automation, you can also enable, disable or delete all breakpoints within that automation.

Step Functions

Once a breakpoint has been reached, you can begin “stepping” through an automation. By using the step functions, “Step Over” or “Step Into”, you can walk through your logic at your own pace. The “Step Over” function moves the automation to the next yellow execution link. The “Step Into” function moves the automation to the next blue data link, or if there are no blue data links, to the next yellow execution link. Once you have finished stepping through your automation, you can select “Continue” to resume execution until your next breakpoint is reached. The “Step Over”, “Step Into” and “Continue” options are available within the “Debug” menu and toolbar, but most developers simply use the F10, F11 and F5 keys respectively.

Watch Windows

When debugging, you can view the value of any property by hovering over its data port in the automation. If you want to “watch” how a property value changes while you step through your automation, you can right-click on the property and select “Add Watch.” The property will now be displayed in the “Watch Window.” Anytime the property changes, the watch window will update to reflect the new value. The watch window can display properties utilized in any automation, even ones that are not open or executing. OpenSpan studio also provides a special type of watch window called the “Locals” window. The “Locals” window will automatically display all of the properties in the currently executing automation.

Call Stack and Threads Windows

In addition to the “Watch” and “Locals” windows, OpenSpan Studio provides the “Call Stack” and “Threads” windows to help debug automations. The “Call Stack” window shows the previously executed blocks within an automation. This allows you to see what path was executed within your automation to get to your breakpoint. The “Threads” window shows all of the currently executing automation threads. This allows you to debug complicated multi-thread scenarios within your project.

Future Directions

With the 3.2 release, OpenSpan has for the first time provided automation developers with integrated debugging. We are already discussing additional debugging features for our future releases such as runtime match rule debugging. Let me know what other debugging features you would like to see.

One of the questions we frequently get asked is “When should I use client-side integration instead of server-side integration?” Typically, we provide a list of scenarios where client-side integration is the only viable option:

1. You’re planning to deprecate the application and don’t want to spend money on it anymore.
2. You didn’t write the application, but it doesn’t have an integration layer or it’s integration layer doesn’t provide access to the functionality you need.
3. You wrote the application, but you don’t have anyone with the skillset to modify it or you don’t have the source code.
4. The application doesn’t have a backend. It’s client-side only.

Today, however, I’d like to make an argument I don’t think we make enough. For many scenarios, client-side integration is the best option, even when viable server-side options exist.

When?

When you’re trying to make your users more productive.

Why?

Because you’re spending your money making what they do easier.

You’re not spending your money exposing integration points for user oriented tasks. You’re not spending your money writing a new user interface that wraps or embeds older functionality. You are automating tasks. You are eliminating errors. You are making your user experience better.

In our industry we are accustomed to thinking that there is only one way to solve user productivity problems: a new user interface. Typically, this means jamming as much functionality as possible into a single window using mashups, portlets, frames, tabs, etc. Now you’re users don’t have to switch between different windows to do their job.

But was that really the problem? Instead of multiple windows, they now have everything in one place, either smashed together or separated into tabs or frames. They now have a new user interface to learn rather than the one they were accustomed to. Moreover, you’re application developers are now spending their time creating portlets, web services, etc. rather than adding new functionality.

But what if the user interface you already have is the best one for that particular business function. It was designed for that specific function. It has been through multiple improvement cycles. It does what you need it to do. Were you really unhappy with the user interface or were you unhappy that you have to switch back and forth when you copy and paste? Or that your workflow requires you to look up the user in three different applications? Or that it takes too long to place an order because the wizard was designed for the public website?

Why are there still so many host systems in the enterprise when windows alternatives have existed for thirty years?

Because their faster than windows systems so your most productive, expert users don’t want to let go of them.

Why are there so many windows systems in the enterprise when web alternatives have existed for 10 years?

Because their user interfaces are more rich than web systems so your most productive, expert users don’t want to let go of them.

When you’re trying to make your users more productive, server-side integration is a sledgehammer when you just need a hammer. Your server-side architects and developers should focus on the big integration problems: synchronizing data, backend transactions, exposing web services for your partners. User productivity is a different kind of problem that requires a different kind of solution.

OpenSpan allows you to improve user productivity without changing all of your applications. It allows your users to be more productive without having to be trained on an entirely new system. It allows your users to do what they always did, but faster and with fewer errors. It allows your application developers to focus on new features instead of integration. It not just the only solution, it’s the best solution.

When we first started OpenSpan we didn’t really have any marketing. When it came time to produce some glossies, we all sat around the table trying to come up with a good catch phrase. We kicked around things like “Empowering Your Enterprise Applications” and “A Remote Control for Your Enterprise Applications.” Sometime during that discussion I suggested “Do It on the Desktop.” It was short, funny and memorable. Plus, it would make a great trade show t-shirt. Mostly, however, it summed up our philosophy really well.

Unlike most of our competitors, our technology started on the desktop and was designed for the desktop. From the beginning we did more than just automate applications. We added functionality. We modified behavior. We responded to events. We saw the desktop as the forgotten foundation of the enterprise. Every knowledge worker spends most of their day in front of a screen, moving windows around, entering data, cutting and pasting. We knew we could make those users more productive if we just made their applications work together.

In the end, we didn’t make “Do It on the Desktop” our catch phrase. Since then we’ve gotten a marketing department and they’ve come up with some great catch phrases like “The Last Mile of SOA” and “The New Enterprise Desktop.” For the technology organization, however, our catch phrase will always be “Do It on the Desktop.” I’m still hoping for some t-shirts.