Moving to .NET 7 MSMQ Alternatives

Enough is enough.

You've been sitting on the sidelines long enough, and you've decided now is the time to move to the new .NET. It only took seven versions to get you to make the big leap. There's only one thing holding you back though, your legacy app has a lot of dependencies on MSMQ. You've been able to move it all the way up to .Net Framework 4.8, but you know you can't go further unless you resolve that dependency.

The question is, how?

Before we go on, for the uninitiated, here's a brief description of MSMQ:

MSMQ

Microsoft's MessageQueuing is a messaging infrastructure and platform available on windows to allow the creation of loosely-coupled messaging applications. Such applications can use MSMQ to communicate across machines and networks. It provides a range of services including guaranteed message delivery, routing, security, priority-based messaging and transaction support.

With a whole host of functions, replacing all of them is a tall order. And often it calls for some degree of application redesign, depending on how complex the messaging solution was and how much it depended on MSMQ to deliver that value.

For all those benefits, with no MSMQ available in the new versions of .NET, if it's time to move, this article presents a few considerations on what that transition can look like. It will explore some of the available options. By the end of it, you should see that even though it may be tough, there is a pathway forward.

Making it real

Before we dive into options, let's use a simple sample project to highlight the issue:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Messaging;


namespace QueueWriterOne
{
    public struct Payment
    {
        public string Payor, Payee;
        public int Amount;
        public string DueDate;

        public override string ToString() {
            return $"{Payor} owes {Payee} amt: {Amount} from {DueDate}";
        }

    }

    internal class Program
    {
        static void Main(string[] args)
        {
            Payment myPayment;
            myPayment.Payor = "Pa-or1";
            myPayment.Payee ="Pa-ee";
            myPayment.Amount = 100;
            myPayment.DueDate = DateTime.Now.ToString();
            System.Messaging.Message msg = new System.Messaging.Message();
            msg.Body = myPayment;
            var path = ".\\Private$\\billpay";
            MessageQueue msgQ = new MessageQueue(path);
            if (MessageQueue.Exists(path) != true)
                MessageQueue.Create(path);

            System.Type[] arrTypes = new System.Type[1];
            arrTypes[0] = myPayment.GetType();
            msgQ.Formatter = new XmlMessageFormatter(arrTypes);

            msgQ.Send(msg);
            Console.WriteLine("Press to receive.");`
            Console.Read();
            var recPayment = ((Payment)msgQ.Receive().Body);
            Console.WriteLine(recPayment.ToString());
            Console.WriteLine("Press to out.");
            Console.ReadKey();

        }
    }
}

That code runs fine in .Net Framwork 4.8. As a first step, I ran this in the .NET upgrade assistant. It converted the project well enough, but it couldn't run, because, well, there's no support for System.Messaging and the like in .NET. But let's look at how this might be addressed for those who want to move.

Here are some of the options:

1. Don't worry, it's coming? (It's not).  
2. Create a wrapper.
3. Try an alternative: RabbitMQ
4. Go with the cloud: Azure Service Bus.

There are many other ways of treating with the need for a lot of the functionality that MSMQ provided, but for now, let's look at those options in the context of the sample above:

1. It's coming?

Well, the ask has been on the table since 2020. Because of the work of the team behind CoreWCF, there has been some discussion about MSMQ support in future versions of .NET. That thread leads to the work by weloytty in this repo. When applied to the code above, the changes were indeed minimal.

1. Install [the NuGet package](https://www.nuget.org/packages/MSMQ.Messaging/).
2. Change from 'using System.Messaging' to 'using MSMQ.Messaging' and remove references to System.Messaging in the code.
3. Run

When those steps were performed, the code sample above ran flawlessly. Mind you, that's toy code. The concerns for many an enterprise that runs MSMQ may be diverse and too weighty to drop on the lone maintainer of the MSMQ.Messaging product. He's been responsive on the one open pr at time of writing, but this may not give much comfort to the teams that may come to depend on this library. Essentially, any team going in this direction should be prepared to jump in and fix issues that occur themselves. This may not be a bad thing in the short term, if a firm is minded to get on to the latest version of .NET as soon as possible to then make upgrades using one or more of the options below.

2. Create a wrapper.

Martin Fowler described the Strangler Fig pattern as a way of dealing with implementing upgrades to legacy solutions. When using it, a developer is going after gradually upgrading a system by replacing it piece by piece.

This option mandates extracting the queue code out of everywhere, putting it into an API, and using web calls to and from the API to maintain the work of the queue. Depending on the implementation details, this may not be that significant a change. It certainly feels heavy, and the example code above will go through a more complex modification.

The abstraction of the web api can be a simple wrapper that lets the queue functions be accessed via web calls

The code remains pretty simple on the client-side:

static void Main(string[] args)
        {
            Payment myPayment = new Payment();
            myPayment.Payor = "Pa-or1";
            myPayment.Payee ="Pa-ee";
            myPayment.Amount = 100;
            myPayment.DueDate = DateTime.Now.ToString();
            var msgQ = new WebMessageQueue();

            msgQ.Send(myPayment);
            Console.WriteLine("Press to receive.");
            Console.Read();
            var result = msgQ.Receive();
            Console.WriteLine("Received: "+result);
            Console.WriteLine("Press to out.");
            Console.ReadKey();

        }

Only including the main method of the Program.cs class for brevity.

public class WebMessageQueue
    {
        public WebMessageQueue()
        {

        }

        public void Send(object message)
        {
            string baseAddress = "http://localhost:9000/";
            var jsonContent = JsonContent.Create(message, message.GetType());
            var client = new HttpClient();
            HttpResponseMessage response = 
                client.PostAsync(baseAddress + "/api/queue", jsonContent).Result;
            Trace.WriteLine(response);
        }

        public object Receive()
        {
            string baseAddress = "http://localhost:9000/";
            var client = new HttpClient();
            HttpResponseMessage response =
                client.GetAsync(baseAddress + "/api/queue").Result;
            var result = response.Content.ReadAsStringAsync().Result;
            //use jsonserializer to deserialize result
            var obj = JsonSerializer.Deserialize(result, typeof(Payment));
            return obj;
        }
    }

The new WebMessageQueue class abstracts out of the Main method, the calls to the queue - which is now hosted in a web API.

And now, observe the QueueController, set up to send messages and allow a simple polling mechanism to receive queue messages.

public class QueueController : ApiController
    {
        public Payment Get()
        {
            MessageQueue msgQ = GetQueue();

            var recPayment = ((Payment)msgQ.Receive(TimeSpan.FromSeconds(1)).Body);
            return recPayment;
        }

        public void Post([FromBody] Payment value)
        {
            MessageQueue msgQ = GetQueue();
            var queueMessage = new Message(value);
            msgQ.Send(value);
            Console.WriteLine($"{value}  was RECD + SENT to: {msgQ.Path}");

        }

        private static MessageQueue GetQueue()
        {
            var path = ".\\Private$\\sample";
            var msgQ = new MessageQueue(path);
            if (MessageQueue.Exists(path) != true)
                MessageQueue.Create(path);

            System.Type[] arrTypes = new System.Type[1];
            arrTypes[0] = typeof(Payment);
            msgQ.Formatter = new XmlMessageFormatter(arrTypes);
            return msgQ;
        }
    }

This is a deliberately simplistic approach to the code of this pattern. Instead of polling to see if there's anything to be received, a websocket could be implemented or even the use of web hooks.

The idea of putting the MSMQ needs behind an API call lines up well with recommendations from firms like Gartner as it relates to digital transformation of legacy assets and systems. One can start off simple and grow the approach.

3. Try an alternative: RabbitMQ

A great definition from the RabbitMQ site is this:

Among its many strong points, includes a diverse range of supported clients, as well as .NET. To use it, it must either be installed natively, run from a container or an online service can be provisioned. Since, I'm only experimenting with it, I'm going with the [community Docker image](https://www.rabbitmq.com/download.html).

# latest RabbitMQ 3.10
docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.10-management

Following the .NET getting started tutorial on RabbitMQ is straightforward. Since we already have code from the console app we've been using, we'll continue to modify that in-place.

 static void Main(string[] args)
        {
            Payment myPayment = new Payment();
            myPayment.Payor = "Pa-or1";
            myPayment.Payee ="Pa-ee";
            myPayment.Amount = 100;
            myPayment.DueDate = DateTime.Now.ToString();
            var msgQ = new RabbitMessageQueue();

            msgQ.Send(myPayment);
            Console.WriteLine("Press to receive.");
            Console.Read();
            var result = msgQ.Receive();
            Console.WriteLine("Received: "+result);
            Console.WriteLine("Press to out.");
            Console.ReadKey();

        }

As you can see, only the type of queue being instantiated was changed in main.

public class RabbitMessageQueue
    {
        private IModel channel;
        public RabbitMessageQueue()
        {
            var factory = new ConnectionFactory() { HostName = "localhost" };
            var connection = factory.CreateConnection();
            channel = connection.CreateModel();
            channel.QueueDeclare(queue: "hello",
                                     durable: false,
                                     exclusive: false,
                                     autoDelete: false,
                                     arguments: null);
        }

        public void Send(Payment payment)
        {
                var message = JsonSerializer.Serialize(payment);
                var body = Encoding.UTF8.GetBytes(message);

                channel.BasicPublish(exchange: "",
                                     routingKey: "hello",
                                     basicProperties: null,
                                     body: body);
        }

        public object Receive()
        {
            Payment result = null;
            var consumer = new EventingBasicConsumer(channel);
            bool received = false;
            var receiveHandler = new EventHandler((model, ea) =>
            {
                var body = ea.Body.ToArray();
                var message = Encoding.UTF8.GetString(body);
                result = JsonSerializer.Deserialize(message);
                received = true;

            });
            consumer.Received += receiveHandler;
            channel.BasicConsume(queue: "hello",
                                autoAck: true,
                                consumer: consumer);
            while(!received)
            {
                Thread.Sleep(1000);
            }
            consumer.Received -= receiveHandler;
            return result; 
        }
    }

In the RabbitMQ example, both Send and Receive are very familiar, if you've been around these kinds of messaging semantics for any meaningful time. There are a host of customization strategies that are beyond the scope of the article but which should be explored to ensure the most value is extracted from using this approach.

4. Go with the cloud: Azure Service Bus.

The previous examples all are implementable on premise. The services may span multiple hosts, but all can be kept managed locally. If a team is willing to explore message broker solutions using the cloud, there are many options, from running a managed Apache Kafka instance, to AWS's SQS as well as Azure's Service Bus. Let's take a look at how using Azure Service Bus would change things.

On Azure Service Bus

There's a great overview of both message broker concepts as well as Azure Service Bus in general that can bring one up to speed on those areas really well.
If you're coming from the world of MSMQ most of these concepts should be familiar:

• Messaging via queues
• Decoupling of producers and consumers
• Load balancing
• Transactions

Azure service bus supports those functions and many more. Thus, with a greater appreciation for how Azure service bus may help, how would the sample code in the example change? Moving to the cloud for this part of the examples dictates a bit of setup in the Azure portal. The actions that need to be done at the portal are:

1. Create a Service Bus namespace
2. Get the Service Bus connection string
3. Create a Service Bus queue
4. Note the name of the queue created

Details on how the steps above are performed can be seen on the Azure Service Bus "Getting Started" quick start. Now, on to the good stuff. Sending messages with Azure Service Bus is made easy through the many libraries available to use to get it done.

In the example project, perform the following steps:

a. Install the Azure.Messaging.ServiceBus NuGet package.

b. Modify the main method to simply call a different type of message queue

 static void Main(string[] args)
        {
            Payment myPayment = new Payment();
            myPayment.Payor = "Pa-or1";
            myPayment.Payee ="Pa-ee";
            myPayment.Amount = 100;
            myPayment.DueDate = DateTime.Now.ToString();
            var msgQ = new ServiceBusMessageQueue();

            msgQ.Send(myPayment);
            Console.WriteLine("Press to receive.");
            Console.Read();
            var result = msgQ.Receive();
            Console.WriteLine("Received: "+result);
            Console.WriteLine("Press to out.");
            Console.ReadKey();

        }

c. Add the following class that abstracts the Service Bus calls.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Azure.Messaging.ServiceBus;

namespace QueueWriterOne
{
    public class ServiceBusMessageQueue
    {
        // connection string to your Service Bus namespace
        static string connectionString = "";

        // name of your Service Bus queue
        static string queueName = "";

        // the client that owns the connection and can be used to create senders and receivers
        static ServiceBusClient client;

        // the sender used to publish messages to the queue
        static ServiceBusSender sender;


        public ServiceBusMessageQueue()
        {
            var clientOptions = new ServiceBusClientOptions() { TransportType = ServiceBusTransportType.AmqpWebSockets };
            client = new ServiceBusClient(connectionString, clientOptions);
            sender = client.CreateSender(queueName);
        }

        public object Receive()
        {
            object result = null;
            bool stillWaiting = true;
            // create a processor that we can use to process the messages
            var processor = client.CreateProcessor(queueName, new ServiceBusProcessorOptions());
            Func func = async (ProcessMessageEventArgs args) =>
            {
                result = args.Message.Body.ToString();
                await args.CompleteMessageAsync(args.Message);
                stillWaiting = false;
            };
            processor.ProcessMessageAsync += func;
            processor.ProcessErrorAsync += Processor_ProcessErrorAsync;
            processor.StartProcessingAsync().Wait();

            while (stillWaiting)
                Thread.Sleep(100);
            processor.StopProcessingAsync().Wait();
            return result;
        }

        private async Task Processor_ProcessErrorAsync(ProcessErrorEventArgs arg)
        {
            Console.WriteLine("ERROR: "+arg.Exception.ToString());
        }

        internal void Send(Payment payment)
        {
            var message = JsonSerializer.Serialize(payment);
            var svcBusMsg = new ServiceBusMessage(message);
            sender.SendMessageAsync(svcBusMsg).Wait();
        }


        // handle any errors when receiving messages
        Task ErrorHandler(ProcessErrorEventArgs args)
        {
            Console.WriteLine(args.Exception.ToString());
            return Task.CompletedTask;
        }
    }
}

Now, before the design pattern kings and queens come for me, I know, that's pretty horrible message queueing code. Any code that uses a queue in today's world that features a while (true) should be taken out in the back and summarily deleted. So, as you approach real life, your code should reflect much better mechanisms, including event orientation and asynchronicity.

Conclusion

My, what an adventure we've been on.
From starting with how to stay in the land of MSMQ, to changing local providers to using a cloud-based message broker, like Azure Service Bus. The larger concern is that with the continual updates to .NET, the framework and the tools all around, finding ways to become unstuck and moving forward is essential.