At Microserve we're always ambitious about the solutions that we design and develop from scratch, but we're also conscious that there's no point in 'reinventing the wheel' when perfectly good solutions already exist. Our clients usually have third-party systems that they rely on for all kinds of business-critical services like CRM, marketing automation, authentication, recruitment and lots more. It's our job as technical architects to understand where those systems end and where the system that we’re developing begins. Crucially, we need to plan how data flows between those systems to get them working seamlessly together. In other words: how to integrate the systems.
We’ve carried out dozens of integration projects over the years, from complex API integrations between multiple systems to simple integrations using off-the-shelf 'widgets' from Twitter or Eventbrite. What we've learned is that every integration project is different and the complex ones always have a number of moving parts which aren't always under our control. In short, they are predictably unpredictable. We've also learned that a rock-solid process is the only way to bring these factors under control and make these projects a success. Below I've shared some of the key principles that have informed our process: my '5 steps for a successful integration':
1. Understand the ‘why’
As someone much smarter than me famously said, ‘start with why’. Understanding why we are doing something provides the foundation for a successful project. It aligns the tasks that we need to complete with our clients’ objectives and it makes us feel like we are all on the same team.
Having a really clear spec and understanding what we’re doing is very important too, of course, but if we don’t know why then we can't fully apply our insight to decide whether this is the best solution from a wider perspective.
This is particularly important when it comes to integrations that support marketing initiatives. A really solid understanding of the client's objectives can upgrade the perception of the task from "moving some arbitrary data from A to B" to "enhancing visibility and understanding of customer behaviour". The latter increases synergy between client and developer, and ultimately delivers a more effective solution.
2. Plan, plan, plan
Anticipating what is going to happen during a project is imperative for getting it over the line on time and on budget. Especially so with complex integrations, since it requires coordination of disparate systems and teams to make things work.
Before we start any integration work we undergo a thorough discovery phase where the focus is on documentation. Assessments are made of the requirements and these are systematically turned into well-defined pieces of work: or 'tickets' as we call them. We also put together ‘data-journey’ documentation and detailed specifications of the integration endpoints.
3. Identify knowledge gaps
Every integration project has (at least) two systems in play. For us that usually means a website that we're developing and a third-party system that's managed by another company or team. For a complex integration to run smoothly we find it's imperative to have at least one (preferably more) person with good technical knowledge on the opposite side of the integration. Before we start we always make an assessment of whether there is a suitable level of skill across the wider team, and whether we'll be able to get a reasonable level of access to the key knowledge-holders.
Highlighting any gaps early is really helpful. We're always confident in delivering our part of the integration work, but integrations are never done in a silo and the importance of getting the wider team working together can’t be underestimated. Acknowledging this is crucial to accurately anticipate the efficiency of a project.
4. Mind your language
Because there are various parties interested in the flow of data at different stages, all with different perspectives and priorities, there’s a risk that terminology is used inconsistently. For example, the word ‘user’ could be referring to a ‘Drupal user’, or to a ‘Lead’ or ‘Contact’ in the client's CRM system. Stakeholders will use these terms in their day-to-day work and be broadly understood, but questions like “was a user created?” become ambiguous when talking about overlapping systems.
The solution to this is just a question of mindfulness. I always try to think "does what I’m saying mean the same thing to this other person?". If the answer is no, I make a note of it and think about how to minimise any ambiguity. We've found it helpful on some projects to maintain a shared glossary of these terms that everyone can refer to.
5. Don’t be afraid to talk about risks
In fact doing so is positively encouraged! Being pessimistic is good. This includes thinking through any security considerations, how to debug things when they don’t work, and what to do when data channels fail. Good development anticipates the negative outcomes as well as the desired one. If you can anticipate what might go wrong, then you’ll also have a plan of action should that scenario actually happen.
Hopefully these five key points shed some light on our broad approach to integration work, and how we get it right first time. Check out our Case Studies if you’d like to find out more about the projects where we’ve put these principles into action.