Using Microsoft Robotics Developer Studio with Lego Mindstorms NXT Robot

A good way to learn about Robotics is to build a Lego Mindstorms NXT robot and to control it using Visual Studio and Microsoft Robotics Developer Studio. Although the results will hardly do the washing up for you, you'll gain valuable insights into thread synchronisation and coordination in a multiprocess, multi-code event-driven environment. Yes, and it is great fun too!

In this article I would like to share my experience of using Microsoft Robotics Studio with a Lego Mindstorms NXT robot. I want to present my view on the topic, not as an expert in robotics, but rather from the perspective of a software developer with an interest in robotics and of positive technology in general. Maybe my enthusiasm could encourage some of you to get your hands dirty and try writing some code yourself, because I believe that writing code, facing, and overcoming real problems, is the best way to learn.

1112-secondCapture.JPG

 Most software written today is for web, desktop or server applications and only a few dare to enter the field of robotics, and even they in most cases are primarily involved with a low level code, very close to the hardware. So what about all of us who have great experience with a high level managed code? Do we have any chance to do something in this area? Apparently, and fortunately, the answer to this question is ‘Yes, we do’. But would we want to? Wouldn’t that be a waste of our time?

Many times the field of robotics has been compared to the early days of the information technology revolution, 30 years ago gigs were primarily working with electronics and only a few considered writing software as a cool and inspiring. And yet those that dared to enter this new realm became pioneers in a whole new page in the history of the human technology. We are now in the same situation with the emerging field of robotics.

1112-fourthCapture.JPGBut wait a minute, wasn’t the robotic revolution been promised by science fiction writers and futurists, so many times, for so long, and somehow it is always ‘just coming’? Well it took us longer than we thought. In fact in 50s and 60s people envisioned it as a purely hardware task. Now we know: you can’t have intelligence without a more or less sophisticated software. Later when we had the software we were lacking the computational power. But now it is all slowly but surely coming together. We are just entering the multi-core era, the price for hardware becomes cheaper and cheaper almost every day and the number of developers being capable of writing good software is as never high. And most of all we do have real robot in this very moment, roaming the surface of the planet Mars, exploring the depths of our oceans, building cars and airplanes and even vacuuming our homes. Yes, so much more can be said about this, but because we are developers, we know that a line of code worth more that a thousand word, so let’s just see how we could apply our skills in building real robots.

Microsoft Robotics Developer Studio is a tool that will help you to start writing real code without having to do all the low level tasks by yourself. Lego Mindstorms NXT on the other hand is a relatively cheap robot, with a lot of flexibility that is already very well integrated with Microsoft Robotics Studio. My explanations will be focused in using them together, because this is a good and affordable choice, but please note that this is just one of many options currently existing on the market, and you could very well choose to start with something else. I have setup a sample project on the codeplex site and you can also check my demo videos on YouTube here and here.

What is required to start?

For this particular solution you will need Visual Studio, the free express version is enough, and Microsoft Robotics Studio which is also free.

1112-FirstCapture.JPGIn terms of hardware, you will need Lego Mindstorms NXT. My first version uses two motors, ultrasonic sensor, two bumpers and a compass, and the second uses two motors, ultrasonic sensor, bumper, accelerometer, and a compass. But you could choose your own setup as adding and configuring sensors is relatively easy and quick. One thing that you will probably want to have as well is a Bluetooth Connector because having a robot with an USB cord is no fun.

But before jumping to the steps of building a real robot, we have to at least briefly cover some background tools that underlie Microsoft Robotics Studio. In particular those are CCR (Concurrency and Coordination Runtime) and DSS (Decentralized Software Services).

Concurrency and Coordination Runtime (CCR)

CCR is a .NET library that abstracts many common problems that you will face in a multi-threaded, multi-core environment. Writing your own thread synchronization and coordination code can often be a tedious task that takes your focus away from the critical problems that you want to solve and you waste a big part of your time in what we often call plumbing tasks. Code that normally spreads on many pages will be just a few lines long with the CCR. Let’s review some of the most common uses of CCR that you may benefit from.

For many of the examples that I will mention you can look at my service class code.

First of all let’s see how you would send data across threads. This is done by the use of generic Ports. You can think of a Port as a first-in-first-out queue of objects of the generic type you chose. For example If we want to post a string that will be handled by another method in another thread, we would use:

This also is what happens when when a sensor sends data to the service. An object with the information from the sensor is sent to a Port and received by your handler. We will see how to subscribe to events coming from sensors in a bit.

One very common application of CCR is in the use of Iterators. When you want to execute a sequence of actions in a non-blocking way, you would normally have to spawn a thread and have a separate method for each action. With CCR you can write your code in a sequential way within a single method and have it being executed as a sequence of background tasks. You can see that in the Initialize method of my service. A simplified version follows below:

The method is executed in steps separated by yield return statements and treated as separate methods. However you write it as a sequence of actions. To start this iterator you call SpawnIterator method passing your iterator method as a parameter. You have the SpawnIterator method available because the DSS service inherits from from Microsoft.Ccr.Core.CcrServiceBase where it is defined. If you want to use the same approach for a class that does not inherit from CcrServiceBase your code could look like this:

Here we create a thread pool with number of threads equal to the number of processors indicated by the zero in the Dispatcher constructor. We could have specified a specific number of threads instead. Then we create dispatcher queue that we will use for the activation of the concurrent tasks. As you can see CcrServiceBase abstracts for you the creation of a Dispatcher and DispatcherQueue.

At the bottom of the Initialize method you can see how I use the Activate method also inherited from the CcrServiceBase class to subscribe to asynchronous callbacks coming from the sensors.

You could read the first line of this code as follows: Expect to receive SonarSensorUpdate object (meaning an object posted on a Port<sonarsensor.SonarSensorUpdate>) from the UltrasonicSensorOperations object (here named ultrasonicNotify) and handle it with method UltrasonicHandler. UltrasonicHandler method must accept SonarSensorUpdate parameter. The other lines follow the same logic. The flag true indicates that the subscription is going to be persistent. If we set this value to false, only the first data received will be handled.

Choice arbiter is the other CCR tool that worth mentioning. For example you can see how I use it to check for error when setting the current power for the motors of the robot.

Here I set the power and the response can either be success, in which case I do nothing, or failure, in which case I issue an error warning. The methods SetDrivePower returns PortSet of success or failure messages. PortSet is a set of Ports, in this case the Port<DefaultResponseUpdateType> and Port<Fault>. Two handlers are passed for whatever message comes first.

I will wrap up my CCR introduction here, as this is not the primary topic of this article but I strongly encourage you to read the MSDN article introducing CCR available here and to look at the source code attached to the article. Special points on interest are the use of Arbiter.MultipleItemReceive method that allows you to process data in batches and Arbiter.Interleave method that would allow you to define tasks that should be executes concurrently, exclusively or as a clean up task ending all other handlers.

Decentralized Software Services (DSS)

DSS is build on top of CCR and represent a restful service oriented environment. Each sensor or actuator (for example motor) is represented as a separate service. When you start your service two applications will open: a command prompt and a browser. The browser will point to a view of all your services, by default running on port 50000. There you could browse each registered service to view their properties and current state.

The main class that will implement the connection to the sensors and actuators of your robot is a DSS Service. The service runs within the Robotics Studio Installation Directory/bin/DssHost.exe process. Some of the features that DSS gives you are implementation of publisher-subscriber model, lifetime management, security management, logging, service composition and others. For more information on the topic I would advise looking at MSDN introduction to the DSS available here.

Putting it all together

The project that I have setup is available on Codeplex it represents a LEGO NXT robot, that navigates it’s way and avoids obstacles. With it there is a WPF application that draws the current position of the robot and the state of it’s sensors.

But let’s start with the basic setup of the project. In order to understand how the DSS host process is wired to the service we have to look at the debug start actions and command line arguments of the DSS Service project. The start actions point to the DssHost.exe file and the command line arguments are as follow:

Make sure that the start action points to the DssHost.exe file. By executing the start action we run the DssHost.exe process while passing along our manifest file LegoBotPrimaService.manifest.xml located in the Config folder of the LegoBotPrimaService project. All the sensors and actuators are defined in this manifest file. Each one is identified by unique namespace. In addition to that, the Service tag of each sensor or actuator points to a separate XML configuration file.

Before trying to execute the service you should make sure you have the Bluetooth connection established properly. Connect your Bluetooth to an USB port. Find the Bluetooth icon in the system trey at the lower right corner of your Windows. Right click on the icon and choose Show Bluetooth Devices. Then choose Add Device and you will have to confirm the password on the LEGO side. Once you have the connection established, right click that device and look at the Hardware tab. Under the Name column there you will find the number of the COM port. This is the number you need to specify in the SerialPort node of the LegoBotPrimaService/Config/LEGO.NXT.Brick.Config.xml file.

My current architecture of the solution is not that important as this is just one of many possible ways of assembling things together. However I would like to stress a few points that I find important:

1112-FifthCapture.JPG

  • Be kind to your robot and write your tests first.
    It will be difficult to debug a problem or behaviour if you rely on running it on the real robot each time. You need a way to easily turn a real problematic behaviour in a unit test. The way I tried to achieve this is by having my logger to generate real C# code calls that I can copy, paste and with only a few modifications turn into unit tests. They will not only be important to solve your current problems, but also they will ensure that future modifications will not alter the expected behaviour.

  • Do not forget that you are working in a multi-threaded environment.
    Each time you collect a data coming from the sensors, remember that it may be accessed by different threads. Make sure you have the right locks in place.
  • Make your setting easily configurable
    Quite often you will need to adjust settings for a different environment, or after altering the expected behaviour. Remember that App.config file of your service can not be used as a configuration because the real process is DssHost.exe, unless of course you load a separate application domain, but the easiest option is to use DssHost.exe.config file located in the Robotics Studio Installation Directory/bin folder. Make sure you prefix your configurations to avoid interference of the configurations of different robots.

Tips for overcoming common problems

Here I would like to briefly mention some common problems that you may encounter:

  • Make sure your project is located in the root of your Robotics Studio Installation folder. For example if your user is called john, by default, the path will be C:\Users\john\Microsoft Robotics Dev Studio 2008 R3\LegoBotPrima
  • Make sure that LEGO brick configuration file holds the right COM port number, matching what you would see in the Bluetooth Device Properties – Hardware Tab – Name Column
  • Make sure that the sensor or actuator namespace used in the manifest file matches the one specified in the Identifier attribute of the Contract class of your sensor. Use Reflector to disassemble the sensor assembly and look at the literal for the contract identifier. For example if you want to see it for the Compass sensor you will reflect bin\HiTechnic.Y2007.M07.proxy.dll assembly under Microsoft.Robotics.Services.Sample.HiTechnic.Compass.Proxy namespace, Contract class and Identifier attribute.
  • Make sure that the root node name of your sensor XML file matches the name of the sensor state class, as well as all the property names. For example my LEGO.NXT.v2.touchsensor.config.xml file has as root TouchSensorState node that matches TouchSensorState class name defined in bin\NxtBrick.Y2007.M07.proxy.dll assembly Microsoft.Robotics.Services.Sample.Lego.Nxt.TouchSensor.Proxy namespace.
  • Make sure that the SerialPort for each sensor matches the slot that you connected this sensor on the LEGO brick. For example if you have connected the compass sensor on the third sensor input your HiTechnicCompassSensor.Config.xml SensorPort node should look like <SensorPort>Sensor3</SensorPort> The same way MotorPort node in the LEGO.NXT.v2.drive.config.xml file, should hold corresponding MotorA, MotorB or MotorC values.

Conclusion

Microsoft Robotics Studio combined with LEGO Mindstorms NXT robot can offer affordable and relatively easy way for any .NET developer to start writing real robotic applications. Yes this is just a basic app and so much more can be done but if you haven’t tried writing robotic apps yet, I hope this article can encourage you to try. I am sure you will be surprised how easy it is and also how much fun is to see something so much resembling a living creature coming from your own code.

Tags: , , , ,

  • 35532 views

  • Rate
    [Total: 36    Average: 4.2/5]
  • Anonymous

    Great Post!
    Really exciting article! Thank you! I’ve never developed on .NET, have some Java and C++ experience. Is important to be professional in .NET or it can be learn easily, to use in MRDS?

  • Gokulnath

    Very Informative
    Thanks a lot for this wonderful post !! Hope to see many more projects from you using LEGO NXT.

  • vladibo

    Re:
    Thank you. If you are good with C++, I am sure you’d find C# very easy. Download the code and try playing with it. Try running the tests if you don’t have lego yet.