What exactly is middleware? Some might say, “It is the layer of clothing worn between the underwear and the outerwear.” And whereas those people are technically right, it isn’t what I’ll be discussing today.
In projects where there is an integration between two or more disconnected systems, usually there is one or more applications sitting between them facilitating their communications (if there isn’t, there probably should be – I’ll explain in a minute). These applications stand to be a middleman (hence the name middleware), so that each system can stay disconnected.
Let us begin…
Middleware is important, as I just mentioned, for keeping disconnected systems disconnected. Depending on the systems being integrated, a skilled developer could easily devise a way to integrate two systems and completely make them interdependent.
I’m pretty sure I just heard someone ask, “Why is that a bad thing? Middlemen just complicate processes.”
Being an opinionated kind of guy, I’ll tell you why. Middleware only make things complicated when they go against good practices when implementing them. When done properly, your systems keep their independence. You can swap out your e-commerce package or your ERP solution without affecting the other side of the system. The middleware would possibly need to be modified to meet the demands of the system swap, but that is relatively simple compared with customizing an off-the-shelf platform like an e-commerce package or ERP solution. To complicate it further, add a third system. You now have a tangled mess of interdependency, a brittle and fickle system that is impossible to maintain.
Think of it like Lego blocks. You have a blue one (your ERP) and a red one (your e-commerce platform). You need to connect them, so what do you do? You grab a big green one and snap them all together. The structure is complete. Complete, however, until you want the yellow block instead of the blue block. Because the Lego blocks have a common interface – the “bump” as I call it – you just yank the blue one off and slap the yellow one on. To confront the complication of changing systems, I introduced the third system. If you didn’t have that green “hub” piece, you would have a hard time connecting all the blocks together. Add a fourth system and it would be impossible to connect all the “bumps.”
Doing it right
When designing middleware, ideally there are two types, and all middleware will fall completely into one of these categorizations: the “Know It All’s” and the “Know Nothings.” As soon as you start mixing the two, you now have a “thinks it knows it all, but really knows nothing” middleware. All fun and wordplay aside, when you mix them, that is when your system becomes harder to maintain and more complicated.
Know It All’s
You’ve probably figured it out already. The “Know It All’s” attempt to know everything about both systems. Any business logic or data translations happen inside the middleware. The communication to the connecting systems would be clean and clear with no need for those systems to manipulate the data. A good example of this would be an e-commerce package and an ERP solution that both have an API, such as a web service, that can be used to transfer customer order information. To avoid having to customize either of the APIs, the data coming from the e-commerce package or the ERP solution has to be translated and massaged into the contract the other needs. The middleware would know everything about this translation, all the way down to possible duplicate order detection, maybe even inventory level checks and so forth.
Again, you’ve probably nailed this one too. The “Know Nothings” try to know as little about each connecting system as possible. These are my personal favorites, because they are the absolute easiest to implement. The mechanisms for this type of system are usually scheduled file drops. The ERP solution or e-commerce package has some form of CSV or XML report that it can generate or customize, and the other side has some form of import process that it uses. The middleware then becomes a simple scheduled data shovel, much like the awesome cartoon from my youth: http://www.youtube.com/watch?v=U91VjRy9ZEI
Pick up the file, move it to the other. Blissful ignorance.
One or the other
Keeping to one type or the other has recently caused me pain and suffering. The issues with the middleware being in both camps will arise if there are issues with the integration (big surprise). Unless all parties involved with the project – developers, coordinators, business owners, etc. – know all the moving pieces, are aware what the middleware does and doesn’t do, and the middleware is well documented and the documentation is up to date and easy to find, confusion is sure to follow. If the role of the middleware isn’t clearly defined, a simple issue will involve the e-commerce package developers, the ERP solution administrators or developers and the middleware developers. One connecting system thinks the middleware does something it doesn’t, the other doesn’t know what it actually does, and the middleware developers are stuck in meetings explaining what it does and doesn’t do almost constantly. It is a giant sinkhole of time.
The trick is knowing which to choose. As I eluded to in the definitions above, you can choose which type of middleware to utilize by the capabilities of the systems being integrated, and what type of resources are available to facilitate change in the systems being integrated. If you have developers on both sides and a nice customizable ERP solution and e-commerce package, then go with a “Know Nothing”. If you have a smaller obscure ERP solution or a black-box-type e-commerce package, you may be forced into a “Know It All“ middleware.
Wrapping it up
Keep it simple. Keep it maintainable. Following those two principles when applied together can make for some really rock-solid integration.