Enterprise Architecture Discovery at Runtime

Motivation

Business is in the middle of an unfolding era of disruption, driven by digital transformation, which challenges how IT is organized and managed today. The role of IT in the business is elevated from being a support function to an enabler to drive innovation, enhance competitive advantage, boost productivity, and reduce cost by applying new innovative technology. However, new technologies, such as big data, cloud, IOT and blockchain, can only provide value to the business if these can be properly implemented and managed. Hence, within the last years, large IT organizations experience a culture shift that encourages collaboration for improving the implementation of those new technologies while being able to develop them more quickly and reliably. This shift yields new software development methodologies such as agile practices, DevOps and continuous deployment of containerized applications, which have significant influences on the further development of Enterprise Architectures (EA). These include a re-prioritization of conflicting goals, such as product-oriented vs. process-oriented IT organizations, a continuous evolution of the application landscape vs. long-lived stable products, and small microservices vs. large monolithic applications.

EA management (EAM) has been established as an important instrument for managing the complexity of the application landscape and enabling enterprise-wide transparency. One of the tasks of EAM is to document and analyze the status-quo of the current enterprise architecture (EA) in order to define requirements and plans for transformations to an architecture that optimally supports the business strategy. Hereby, it aims to visualize the relationships among regulations, business processes, software and the underlying infrastructure.

The collection and aggregation of required information from several sources support the EA management function by obtaining an overview of the status-quo and the derivation of planned states that outline EA transformations. The analysis of the current state of an EA is commonly documented in the course of EA model maintenance endeavors and serves as a basis for decision-making for EA management and its stakeholders. Thus, it is of high importance that the documentation of the current EA reflects the reality in terms of desired granularity, correctness, actuality, and completeness, which is often not the case.

Problem Statement

IT-landscape modelling, as a sub-area of EAM, tries to discover and assess the IT-landscape of an organization. EAM tools support the modelling of the IT-landscape, however, the creation of these models is a time-consuming task. Enterprise architects struggle to collect EA relevant information in order to unveil the as-is IT-environment. The EA models are often out-of-date which leads to decisions made on wrong data or bad data quality. The reasons derive from the approach of how the data is collected which is still performed mainly manually. In addition, cloud environments, new software development methodologies like agile practices, as well as the delegation of architectural decisions to DevOps teams, makes IT-landscape modelling suffer more than ever from the problem of capturing the full essence of an ever-changing landscape. As a consequence, many EA management endeavours lack the ability to show an early Return on Investment (ROI) or an ROI at all; a situation which often ends in stakeholder dissatisfaction and a badmouthed EA initiative that is very likely to fail.

The aforementioned challenges have been becoming a bigger issue in the last years due to the rise of agile methods which brings unprecedented changes to software development practice by strongly emphasizing change tolerance and continuous delivery that aims to significantly shorten software release cycles. In addition, the popularity of microservice-based architectures is increasing in many organizations. Even though microservices have several advantages in contrast to monolithic systems, this architecture style also introduces a high level of complexity with regard to architecture documentation. Due to agile practices new microservices can be introduced very quickly into the current infrastructure or removed when they are no longer needed. In these scenarios, it is crucial to keep track of the current IT landscape, which further raise the documentation overhead.

Solution Approach

In this project, we investigate on how to discover the as-is EA by leveraging runtime instrumentation that provides information about 1) the technology layer of an EA that encompasses all technological-related aspects like hardware, network and other physical components and 2) the application layer that defines software components, information systems, services that run on the technology layer, and their communication structure. In addition, we present an approach how to link those runtime data with further relevant information that belong to the particular IT artefacts and cannot be extracted out of runtime data, as it resides in the various collaboration tools along the IT value chain. The collection and mapping of all mentioned information sources contribute to discovering the alignment of IT models with business-specific artefacts; solving the challenge of automating EA model maintenance.

Research Questions

1) What are the building blocks of modern cloud-, microservice-based IT landscapes?

2) Which microservice-based IT landscape models need to be managed from an EAM perspective and which models can be discovered automatically via runtime instrumentation?

3) How to enhance microservice-based IT landscape models with further EA relevant information that cannot be extracted from runtime data?

4) How can a system and a process design look like that continuously discovers architecture changes in microservice-based IT landscapes?

5) What is a suitable integrated UI for visualizing the discovered microservice-based IT landscape and what is the experience of real stakeholders using this system? What challenges do they face?

Use Cases

1. EA Model Maintenance
   • Automated creation and updates of particular EA models
   • Enhancing of EA models with additional data that is difficult to gather manually (TCO, communication dependencies, etc.)
   • Enhancing EA models with runtime data (comparable to LeanIX and txture.io)
2. Architecture Analysis
   • Analysis of coupling degree
   • Analysis of missing incoming and outgoing connections
   • Communication hot spots
   • Failure impact analysis
   • Automated accountability analysis
3. Cloud-ready Applications

Tool Support

Visualization of business domains including assigned products and applications as well dependencies between those models

Application landscape visualization including communication paths. Click on a particular application opens a sidebar with information about this application.

The whole discovered EA can be queried via GraphQL.

Micoservice architecture visualization including communication paths.

Publications