Live electrical wires overhead at Longbridge Station.
PHOTO: Elliott Brown

IT departments that want to help their businesses digitally transform and master data management still struggle with integration. If you’re in IT, that’s not news to you. Nonetheless, recent industry reports still confirm the integration dilemma.

For example, most data teams (96%) are still facing challenges like data and privacy and harmonizing data to ensure high quality data for analytics and machine learning/artificial intelligence tools, according to a Forrester/Liaison Technologies report last month on integration (download required).

MuleSoft this year reported that while 97 percent of organizations are currently undertaking or planning to undertake digital transformation initiatives, integration challenges are hindering efforts for 84 percent of organizations, according to IT leaders. Almost half said more than 1,000 applications are being used across their business, but only 29 percent are currently integrated together, trapping valuable data in silos.

Integration Problem Confirmed

OK, so each of the vendors in the aforementioned reports have a clear stake in reporting integration challenges because they offer services that help. Even still, there is proof beyond those vendors that the biggest of companies are dealing with integration challenges.

Gartner reported last year that through 2020, integration work will account for 50% of the time and cost of building a digital platform. “Moreover, the complex challenges posed by digital business transformation require a radical change in the integration technology platform and in the way organizations deal with integration,” Massimo Pezzini, research vice president and Gartner fellow, said in that report.

Related Article: Manage Your APIs Right and You'll Crack the Integration Challenge 

Hybrid Approach Recommended

Gartner recommends a hybrid integration platform approach to solving these woes. It will provide a “home” for all functionalities around integration. And, further, an IT-controlled centralized integration team with its “integration factory” model will need to shift toward an approach that supports HIP-enabled, self-service integration by lines of business, subsidiaries, application development teams and eventually business users, according to Gartner.

Hybrid integration platforms should include:

  • Personas (constituents): Integration specialists, ad hoc integrators, citizen integrators and digital integrators
  • Integration domains: Application, data, B2B and process
  • Endpoints: On-premises devices, the cloud, mobile devices and IoT devices
  • Deployment models: Cloud (potentially across multiple environments), on-premises, hybrid (cloud and on-premises) and embedded in IoT devices

Related Article: Dismantling Data Silos Isn't Just a Tech Challenge, It's About Meeting Customer Needs

Is iPaaS the Answer?

Many organizations turn to an integration strategy called Integration Platform as a Service (iPaaS). Gartner defines iPaaS as a “suite of cloud services enabling development, execution and governance of integration flows connecting any combination of on premises and cloud-based processes, services, applications and data within individual or across multiple organizations.” It could be the answer for some. 

Gartner, however, issued a warning in a report last month: be careful where you invest your iPaaS dollars. Gartner predicts that by 2023, up to two-thirds of existing iPaaS vendors will merge, be acquired or exit the market. That’s not good news for those in the iPaaS market for integration fixes, according to Bindi Bhullar, senior research director at Gartner. It’s a risk that platform services will be “discontinued due to the vendor exiting the market or being acquired,” and buyers, Bhullar said in a press release, should “minimize exposure to vendor risk by adopting platforms that can deliver short-term payoffs, so that the cost of any eventual replacement can be more easily justified.”

Simple iPaaS Tools Are Weak

Forrester and Liasion Technologies found in their report last month that there are multiple tools that can enable integration, from the simplest IFTTT to the most complex and completely hybrid integration platforms. That’s where iPaaS enters the picture for many organizations. And it has some fans

However, nearly 70 percent report that these tools increase data security or compliance risks, direct their resources to less strategic tasks (63%), create resource strains (60%), and don’t meet the needs of complex integration (52%). (It should be noted report co-sponsor Liaison Technologies itself provides iPaaS offerings.)

More and More Apps to be Integrated

The integration problem for many? More apps. More data. More than half of data integration decision makers highlight a growing number of applications as one of the top challenges they experience with their integration strategies, according to the Forrester/Liasion Technologies study.

Ani Pandit, director for solution engineering within the America’s Office of the CTO at MuleSoft, summarizes the integration challenges for IT by calling them a “convergence of digital forces.” In an interview with CMSWire, he said mobility, SaaS, cloud, big data, IoT, AI and even social are creating “massive disruptions in the market.” Every aspect of a customer’s interaction with a brand touches, he said, on various different systems and various different applications and data and devices across businesses. 

“What ends up happening is IT has a severe backlog of projects,” Pandit said. “They're trying to do order to cash [technologies], deliver scalable solutions, building an AI chatbot for service requests. … And all these systems need to communicate with each other. … Every quarter, they are bringing in four new SaaS applications and they need to have the ability to integrate all these systems together to automate their business processes fast enough.”

Related Article: 6 Digital Transformation Challenges Enterprises Need to Overcome 

Resources and Skills Still an Issue

More than half of IT leaders say their organization struggles to find resources with integration skills, according to Forrester/Liaison Technologies. Implementing integration technologies requires knowledge of technical (security, scalability, data at rest and in motion), business (business processes extensions or moment within customer journeys) and integration (canonical formats for reuse, APIs) to correctly develop interfaces, according to researchers. “Integration skills should be in every data science and development project team,” they reported.

It Comes Down to Strategy, Vision

Beyond tools and talent, the first step of dealing with integration woes is coming out with the right strategy and vision, according to Pratik Parikh, director of product marketing at OpenText, which acquired Liaison Technologies. “Once you build a strategy and vision on how you want to go overcome these challenges implementing data integration, you might have to set up some kind of a lab integration,” Parikh said. Work on different integration patterns and look at how those patterns and how they fit into your organization. ... Data integration and data transformation very much needs to be in your DNA. You need to make sure that all departments and and disciplines are understanding of the importance of that.”

Integration should be a part of the “top down culture,” he added, understanding how your data transformation and data integration is playing out into digital transformation and how it’s all connected in your organization.