The problem: where exactly is the data flowing to and from
When we started developing COSMIC, we knew we needed a solution that would:
Offer seamless connectivity between multiple systems. This was the single biggest driver for us. As our data-driven culture continues to evolve, the movement and use of data is of paramount importance. In our old, disjointed system, it was impossible to see across all components. Delays occurred, and errors were common. COSMIC communicates seamlessly across multiple platforms, including portals that Microsoft customers and partners—companies that provide Microsoft products or services—use to submit issues or check the status of a case. The system offers real-time agent and management dashboards, and supplies all data components for self-service reporting, including Power BI, Azure cloud repositories, and document libraries within Dynamics 365 and SharePoint
Harness the power of the cloud. Our old system was on-premises. COSMIC takes advantage of flexible Azure Blob storage, advanced Azure Machine Learning, and artificial intelligence capabilities that give us instant text translation services and sentiment analysis. This means less work for our agents, and instant feedback that makes it easy for us to assess the quality of work.
To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is public knowledge and can be found online at Microsoft's IT Showcase.
The COSMIC solution can be broken down into three logical areas. They are the partner and customer experience (which is external to Microsoft), the agent experience within COSMIC, and the Microsoft management and business intelligence experience. Figure 1 shows COSMIC topology and shows how data flows through and is extracted from many different systems.
Figure 1. The COSMIC topology
On the left, partners and customers interact with COSMIC through external systems like our Partner and Customer portals, and their input becomes source data in COSMIC. They don’t have access to COSMIC itself, which is used by agents. For example, a partner submits a ticket through Order Central. The data flows through the Online Call Logging Service, and then through the COSMIC API. All external feeds are processed through the COSMIC API. This API allows COSMIC to seamlessly integrate with other applications to share case information. For example, COSMIC integrates with the CSE case management tool—updates made in either updates both of them. Our master data systems are also external to COSMIC.
The center area represents Dynamics 365 and the agent experience in COSMIC. Data management, security, and configured workflows are all managed within the Dynamics 365 experience. Agents work on a variety of devices and sign in through Active Directory and Office 365 security. Agent work instructions are hosted through SharePoint. Azure hosts integration services, like our bot framework, and the Computer Vision API. Data feeds into Power BI for business intelligence services such as dashboards, reporting, and telemetry.
The right side of the graphic represents Microsoft management overseeing the system and pushing data back to the business. The COSMIC Data Hub is an offline data store for intelligence, and also serves downstream needs like integrations. An archive stores transaction information.
To help stakeholders, vendors, engineers, managers and business intelligence understand the data synergy between systems by working with a systems designer to create a data flow diagram.
What we learned
In order to successfully configure the data flow diagram I had to interview numerous people. The process was so complex that most engineers only had experience with one or two of the components.