Getting the Most Out of AI Agents? How Companies Should Think About Architecture | Insights

As many companies have moved forward with the use of AI in recent years, AI agents have come to the fore as a way of further automating and enhancing operational processes. Autonomously performing specific tasks or carrying out real-time decision making through the use of AI agents can help streamline and automate not only rote tasks, but also core operations. We believe, however, that there are many companies that have attempted to introduce AI agents, around which there are growing expectations, before abandoning such attempts when they could not get the outcomes they had expected. What tends to be lacking in such failures is system architecture thinking for the whole of the company introducing AI agents.
This Insight describes how the emergence of AI agents is changing information systems at companies, and what roles AI agents should be fulfilling within those systems. It goes on to offer three prescriptions necessary for making use of AI agents, and presents how ABeam Consulting can support clients in implementing those prescriptions.

About the Author

Kohei Kimura

Manager
Shunsuke Kobayashi

Senior Expert
Kazuki Iida

Senior Manager

Background: The Growing Expectations Around AI Agents

Some two and a half years have passed since the release of the chat-bot-style generative AI, ChatGPT. Since then, the use of generative AI as an operational assistant has only increased. However, AI’s present scope of use is limited to supporting specific operations such as summarizing or surveying texts. Amidst this context, there are high hopes for AI agents that can further enhance and streamline operational processes as a new solution for a Japan facing a declining working-age population.
An AI agent is an artificially intelligent system that can autonomously perform specific tasks and make real-time decisions. It is hoped that AI agents will be able to streamline and automate not only simple and repetitive rote tasks, but also core operations that require judgment. By effectively incorporating AI agents into a company’s corporate activities, it may be possible for the company to not only improve its labor productivity, but to also improve its competitiveness through better decision making and faster business promotion.

Challenges: What Should Companies Limit in Making Use of AI Agents?

The fact that AI agents are effective in improving labor productivity and business competitiveness is already being pointed out by many experts, so there is hardly room for doubt left on that point. However, in practice, there are decidedly few companies that have produced significant results with this technology.
At many companies, there has been a tendency for the introduction of AI technology to become the goal in itself. As a result of there being a greater focus on ease of implementation rather than outcomes, they have wound up simply applying the technology to their operations and achieving little more than past RPA and full-text search engines did. Furthermore, due to the siloing of data and knowledge within companies, they have failed to secure the data necessary for AI to perform adequate judgment, so issues remain in terms of it not being able to express its original capabilities. Additionally, we often see cases where development of personnel and establishment of systems capable of understanding and operating AI technology has not kept up, and where operation of AI agents has begun without sufficient coordination between departments. We thus see that such issues have led to hopes for AI agents turning to disappointment, and companies stalling in implementing the technology.

What the companies encountering such issues all lack in common is systems architecture thinking for the whole of the company. To get more out of the technology, it is essential to view the challenge in terms of what position to give AI agents within overall information systems and how to operate them in that context, rather than simply introducing them as a mere tool.
To fix such situations, companies first need to understand how the emergence of AI agents will change their information systems, and to understand these trends through the movements of companies that provide system packages in particular. Companies then need to understand what sorts of roles and tasks AI agents should be taking on when using AI agents internally, and how to design each agent’s role when having multiple agents execute operations cooperatively.

Changes AI Agents Bring to a Company’s Systems

The use of generative AI over the past two years has advanced dramatically. However, this has, at most, gone no further than people using both their company’s IT systems and generative AI to perform work, with the two remaining separate.
Now, however, the integration of generative AI into core systems is making headway. For example, Germany’s SAP has announced a partnership with US company Perplexity as a means of improving its AI assistant “Joule” feature, while US company Salesforce has announced that it has improved “Agent Force” while also acquiring Informatica. As systems that have AI-native services become more common, the AI agents embedded in these systems are becoming more able to autonomously support non-rote work and decision making.
In addition to the implementation of AI agents in various systems, the standardization of coordination technologies such as the Agent2Agent Protocol (A2A) or the Model Context Protocol (MCP) is also advancing. Going forward, we can envisage a world in which AI agents coordinate with one another or across systems, and work together across the boundaries of systems, to be able to perform work in whole rather than in part. Furthermore, it is likely that we will see the emergence of companies that secure new competitive edges by having AI agents work together across the boundaries of different companies.

In such a world, some work done by humans will remain, in phases involving responsibility for tasks such as confirming orders or face-to-face negotiations, but it is possible that the bulk of work will be able to be done by AI agents in our stead. For this reason, the use of AI agents will become an important element in the coming competitive environment, and companies that are not able to make use of the technology will fall behind the competition. Consequently, companies will need to update themselves to fit in a world premised on the performance of operations in partnership with AI agents (see Figure 1).

The Roles to Be Fulfilled by AI Agents, Looking Ahead to Such Changes

So, what sorts of features would it be appropriate to deploy in corporate systems to get the most out of AI agents across the board. It is well known that Large Language Models (LLM) can misunderstand the original intent of a query or run the risk of returning logically inconsistent outputs when given prompts processing multiple aims or conditions simultaneously. With a view to future extensibility, companies should also avoid having a single AI agent cover all domains and having limited independence per feature.
For this reason, we believe that multiple AI agents with a division of labor working in harmony will, for the first time, truly demonstrate the value of the technology. Assigning appropriate roles to each layer, up to and including integrators achieving coordination and optimization among multiple domains in pursuit of achieving some goal, experts leveraging insight into specific domains and even on-the-ground workers accurately carrying out each action, will allow each model to work in a way that is focused on the domain that it is good at.
Thus, rather than having these AI agents work individually, they will coordinate with one another, with organizer agents overseeing tasks as a whole, coordinating between domain agents that have insight into specific operational domains, while application agents make use of the systems. Incorporating such cooperation among AI agents into corporate systems is likely to help companies establish management platforms for the AI era and to contribute to solving increasingly complex operational challenges (see Figure 2).

Prescriptions for Securing a Management Platform for the AI Era

Such a future, in which AI agents with multiple different roles work together, is something that we believe will become necessary for future management platforms seeking to improve the competitive strength of companies. With that said, it is not realistic for companies to establish such idealized company-wide coordination structures for AI agents and attain high impacts from them all in one go. Beginning by improving the quality of each AI agent role in stages will likely allow companies to operate overall operational processes smoothly.
We will outline actions needed as prescriptions for getting the most out of the power of AI agents in each role (see Figure 3). We will also explain measures based in future system architecture to help companies avoid getting bogged down in provisional measures.

(1) “Organizational Know-How” That Improves the Quality of Process Inference

For organizer agents to fulfill the aims of users, they need a strong capacity for inference in order to be able to properly plan necessary processes, and to rebuild their plans autonomously in line with the circumstances of their tasks.
In terms of inference, while attention tends to focus on the capabilities of LLM, in practice, many organizations make use of managed models provided by the likes of OpenAI, Anthropic, Google, AWS and Meta, so the reality is that it is hard to create any differentiation there.
What is really needed is documentation that makes explicit the organizational know-how involved - that is, the processes needed to get to producing outputs. Through such organizational know-how, organizer agents can deeply understand not just the surface-level of the words of user requests, but also their context and company-specific operational background, enabling them to make inferences according to correct processes.
The key to improving the quality of such organizational knowledge is to make explicit the implicit knowledge built up within organizations. On the ground, there is a great deal of knowledge that is not made explicit such as standards for decision making and tricks used by veteran employees. Companies need to make this knowledge explicit and have AI agents learn from it.
Making such implicit knowledge explicit, however, is no mean feat. Not only are on-the-ground managers under limitations of time, but it is by no means easy for third parties to extract and systematize thought processes that seem obvious to the person in question.
As a first step in these situations, it is useful to have generative AI aggregate and organize information. By extracting and structuring shared or repeatable patterns from the data or operational records located within the company, we can generate an initial “rough draft” of its organizational know-how. Of course, we do not expect to be able to create a highly complete set of knowledge from a one-off process, but, compared to manually creating something like this from scratch, we can vastly more efficiently get to the start line.
In the second stage, companies need to apply AI agents within operational processes and refine the content of this knowledge base while continuously incorporating feedback from people on the ground. Here we can envisage a process in which generative AI produces amendment proposals based on feedback that is frequently brought up, with humans then carefully checking them before they are confirmed as organizational know-how.
Through this sort of two-step approach, companies can effectively formalize implicit knowledge and consolidate it in their organizations as practical knowledge.

(2) Data Management for Improving Input Quality

The biggest impacts on domain agent performance and operational consistency come from data management. This is an area that many organizations have so far recognized as important to tackle, and some efforts have been made in that direction. Going forward, as the use of AI goes into full gear, the importance of such efforts will increase still further. In environments where data is not properly organized, domain agents may make mistaken judgments and then issue orders to application agents. Far from making work more efficient, this even runs the risk of creating disorder. For example, suppose a product master is not properly integrated. Despite being a single product, an item could be treated as multiple separate products, potentially giving rise to issues such as duplicate orders in procurement or mistakes in sales analysis.
It is important to push the following three points in order to reduce such risks and secure proper inputs for domain agents.

Integrating master data in core operations
Integrating master data that is divided up between departments and systems, and building platforms that can be used company-wide.
Organizing metadata
Stating explanations that let AI agents properly understand the meaning, structure and usage of all data.
Building data governance structures
Clarifying who the manager for the given data is, and creating systems for continuing to generate data that is properly in line with quality standards.
Through such efforts, companies can build platforms that allow AI agents to accurately and efficiently make use of data.

(3) Selecting AI Native Systems for Seamless Execution

At present, companies use SaaS systems, or systems built on-premise or in the cloud, as appropriate for each operational domain. What matters here is how companies can make effective use of the application agents provided by the SaaS provider or by the package company providing the system in question. To now, many companies have employed package or SaaS standards under the “Fit to Standard” idea. However, to make the most effective use of AI agents, companies need to even more fully adopt such attitudes. Naturally, even in cases where companies are adding features themselves, it may be possible to have AI agents learn these additionally, but, in such cases, companies need to balance the costs of such learning with the benefits brought about by the features they want to add.
AI-native services often also have in place “Agent2Agent Protocol (A2A)” features, which connect AI agents to AI agents, and “Model Context Protocol (MCP)” features, which connect AI agents to systems, rather than standard implementations of AI agents in company systems. Anticipating a future in which multiple AI agents will coordinate with one another, we believe that whether or not a service provides an AI-native service that anticipates future AI agents should be an important selection criterion when choosing a package or SaaS from a company.

Of the three points we raised above, the most important and most challenging is “(1) Improving inference quality.” In contrast to simple information referencing or command execution, inferences involve integrating multiple contexts and assumptions, and high-level processing to formulate non-contradictory plans. This makes ensuring quality difficult and the risk of errors occurring high. Formalizing implicit knowledge using generative AI is an area that many companies do not have experience in, so designing and evaluating such processes requires a great deal of care. Companies need to advance such initiatives with and understanding of this point.

Summary: Towards an Era Where AI Agents Are Incorporated Into Management Platforms

The evolution of AI has made incorporating AI into enterprise architecture a realistic future. To date, corporate activities have been realized through the combined work of people and programs. Going forward, a new component in the form of AI agents will be incorporated into that. At that point in time, what AI agents replace will not be the areas previously carried out by programs, but the parts carried out by people. To build a management platform for the AI era that captures those changes, the key will be for companies to advance: “(1) improvements to process inference through the formalization of organizational know-how;” “(2) improvements to the quality of inputs through data management;” and “(3) securing executive capacity through the selection of AI-native systems.”

However, taken to extremes, this could even suggest that people will become unnecessary for business activities, because, in the extreme case, owners will be able to set their corporate philosophies as goals and leave it up to AI to find the optimal solution for achieving that. In practice, the potential of AI is unknowable, with some even having spoken of a future in which AI will allow a single person to build a billion dollar company. However, the decision making involved in setting goals and asking questions remains an important area for people to do. Companies need to think about and study the right way to relate to AI in order for people to actively operate companies, rather than be used by AI agents. In the process of such study, companies will train people to work with cooperation with AI agents in mind. In such efforts, people will come to focus on more creative and strategic roles, while AI will come to function as a partner supporting the execution side of those roles.

ABeam Consulting has so far worked side by side with a variety of companies and public bodies to support them in corporate and operational transformations that have their starting points in generative AI-enabled problem solving and technology, covering everything from strategy formulation for such projects to architecture design, and the establishment of data management platforms. Building up “’organizational know-how’ to improve the quality of process inference” is a key prescription of this Insight. At ABeam, we have already carried out multiple proof-of-concept initiatives using AI agents, and can offer efficient services based on the methodologies we have developed through those efforts. Leveraging this knowledge and experience, and working with AI, ABeam contributes not only to the use of AI agents as a means for companies to deal with the decline in the working age population, but also to improving the competitiveness of companies, while continuing to ask essential questions about how people should work and live rich lives.

Insights TOP