Documentation - the blind spot of integration and API development

The agile manifesto prefers a running system over extensive documentation, but having integrations documented is unavoidable. On the other hand, documentation is part of the definition of done in agile development. And yet documentation is often created on a best effort basis, which translates to minimal effort.

Integration development is challenging because of several factors:

• Dependencies and moving parts
• Stakeholders with various levels of technological understanding
• Business criticality
• Complex testing
• Access management and security
• Multitude of technologies

The documentation of integrations comes with similar challenges such as dependencies and complexity thus producing it is not straightforward either.

The many benefits of documentation

So, why should we invest time and money in the creation and maintenance of documentation?

There are many reasons. Up to date information will decrease the time and also cost needed for knowledge transfer, development, testing and incident resolution. In the long term, it will also help to avoid vendor locks by increasing overall visibility.

As highlighted in the Public Administration API Principles, integration and API documentation increases cooperation, reusability, interoperability, information security, data protection and quality. It is useful for both technical roles such as development, testing, support and IT security and business-oriented roles, such as operation, product owner and analyst.

How, what and where do we document?

The main challenge to do documentation is the nature of integrations – they are the glue in the middle – the part which connects different systems. And ultimately the "grey area" where things get unclear.

In my experience, the following points need to be considered when planning and implementing integrations and their documentation.

How and what do we document?

• The purpose, service level and use of the integration, plus instructions for testing
• Relevant technical details
• Relationship between integrations and APIs
• The list and format of data transferred between systems
• References to source systems / backends
• Can we generate the documentation? Can we import it?
• How do we keep it up to date?

Where do we document?

• In the technical Integration environment?
• In a central system like Confluence?
• In a visualization tool?

How do we find it?

• Search capabilities
• Keywords
• Categories
• Links
• Indices (can they be generated?)

And finally, some suggestions which will help on the way to useful documentation:

1. Have one place for Integration and API documentation
2. Invest time in the documentation structure
3. Make use of links and tags/labels/categories, to enable different views on the same documents
4. Have templates and standardize the use of tags/labels/categories
5. Create and update the documentation as part of the development and not only when there is time
6. Review the documentation and check for completeness in regular intervals
7. Use exports/imports from technical development environments to generate documentation
8. Plan the access to and the publishing of the documentation

Read more:

Manifesto for Agile Software Development

Ministry of Finance, Finland: Public Administration API Principles

Do you see the documentation of integrations and APIs as a challenge? We would like to hear about your experiences so do not hesitate to reach out! And stay tuned for more articles where me and my colleagues will delve into topics such as iPaaS platforms, API monetization, and overcoming the typical integration challenges that I mentioned in the start of this post.