Driving Full-Scale AI Transformation Through AI CoEs: Key Points in Launching an AI CoE – Barriers and Openings as Revealed by the Survey of AI Success Factors in the Use of Generative AI | Insights

AI transformation (AIX) is getting fully underway, with the range of application of generative AI growing to encompass the automation and semi-automation of operational processes. In response to these developments, more and more companies are starting up AI Centers of Excellence (CoEs) with the aims of accelerating, optimizing investment in and overseeing company-wide AIX.
In this series (spanning four parts), we go over our approach to the entire process from starting up AI CoEs to executing on projects. In this first part, we investigate the functional design needed to effectively launch an AI CoE, based on case studies and survey results.

Jun Miyamoto

Director
Nobuyuki Shimizu

Senior Manager

AI CoEs That Accelerate AI Transformation

To date, many companies have tried to make use of generative AI, but, because of the ease of leading with trials on the level of individual departments, we have often seen such efforts begin and end with improvements to the productivity of individual employees. The emergence of AI agents, however, has expanded the scope of application of generative AI to the automation and semi-automation of operational processes. This means that AIX is getting fully underway.
With that said, as the scope of application of AI and its degree of autonomy (the degree to which AI makes business decisions and executes operations without a human in the loop) grow, the risks of impacts on operations and customers increase. This is to say that companies are in a position where they need to think not only of implementing technology, but, at the same time, of oversight, the division of roles between AI and humans, and vision for business and system operations.
Against this backdrop, more and more companies are opting to launch AI CoEs with the aims of accelerating, optimizing investment in and overseeing company-wide AIX. However, we often hear from leaders charged with the task that setting up is an uphill struggle as they face challenges such as knowing where to start, knowing the extent to which CoEs should take charge of work and how to divide roles between the CoE and existing organizations. In this series, we organize the current state of and challenges involved in utilizing AI in Japanese companies, and go over specific approaches to starting up AI CoEs and putting them into operation.

The Current State of AI Transformation and the Challenges Companies Face

The “Survey of AI Success Factors in the Use of Generative AI” conducted by ABeam Consulting in November 2025, targeting AI promotion officers at 53 major Japanese companies, revealed that AI was already positioned as a management agenda item at most of them.
Specifically, 93% of companies were officially promoting the use of generative AI as of the end of December 2025 (see Figure 1), while 47% had formulated AI visions, and 43% had formulated AI roadmaps (see Figure 2).

In terms of companies’ aims in generative AI utilization, streamlining operations stood out at 96% of respondents. This was followed by reducing costs, enhancing product and service features and improving customer experience (see Figure 3). These results indicate that companies are simultaneously expecting both short-term increases to productivity and long-term value creation.

We also observe that among companies responding that they have achieved greater-than-anticipated impacts, there is a trend for these companies to have company-wide specialist organizations such as DX departments or AI CoEs at the heart of AI promotion efforts. The most commonly given key functions for these specialist organizations are training and awareness raising, policy formulation and use case creation (see Figure 4).

As to the challenges of generative AI utilization, however, we found that companies most commonly cited managing organizations, people and risks, cost-benefit calculations, preparing data and governance (see Figure 5). Particularly in the domain of organizations and people, 53% of companies lacked personnel. Among these, personnel familiar with generative AI and capable of applying to businesses and operations or of driving projects are in particularly short supply. With labor shortages deepening across all industries in recent years, half of companies facing such labor shortages are supplementing their labor forces through internal development. The fact that it is difficult to secure AI personnel also suggests that a key role of specialist organizations is organizational learning in the form of developing personnel and building up the know-how and shared assets that serve as prerequisites for such development.

Put another way, the technical effectiveness of AI is being better understood by companies, but strategy, implementation, oversight and the people who carry out those things can be thought of as the bottlenecks towards company-wide operation of the technology (see Figure 6). In the next section, we systematize the functions needed to address these bottlenecks and maximize the impact of generative AI, and present archetypal organizational patterns for achieving this.

The Functions of AI CoEs and Organizational Patterns

The functions that should be considered for AI CoEs can be organized into the following nine categories (see Figures 7 and 8). Their order of priority in AI CoE function design varies chiefly based on whether said function will answer strategy (5, 6), implementation (1, 2 , 3 ,4) or oversight (7) challenges.

In any event, the important thing is to avoid the idea that “the AI CoE will take on everything.” Companies need to clarify the roles of corporate departments, business units and even partner organizations, including responsibility for existing IT, IT security, HR, legal affairs, finance and accounting and audit, while aligning with higher-level strategies such as corporate strategy and DX strategy. Clarifying roles allows AI CoEs to concentrate on functions that are essential to driving AIX. It is also important for companies to choose models that fit with their corporate and organizational cultures. This is because driving transformation in a way that does not fit with a company’s culture will create internal resistance that reduces the speed at which the company can produce results. Here we present three examples of models often used in practice.

(1) The Leadership Model (Focusing on (5) and (6))

This is a model in which the AI CoE defines company-wide key areas and investment priorities, connecting leadership and on-the-ground decision making. The AI CoE takes the lead in updating the AI vision and roadmap, managing the use case portfolio and creating standards for testing value. Implementation is delegated to departments, while the AI CoE is responsible for the reproducibility of outcomes and for optimizing investment (preventing duplication, standardization and making decisions on withdrawing from investments). It is a model that suits cultures where changes come from the top down.

(2) The Support Model (Focusing on (3), (4), and (6))

This is a model that maximizes the speed of implementation of AI on the ground. The AI CoE is responsible for preparing design reference materials, assessment procedures, prompt and component re-usage, providing a point of contact for consultations and expanding the number of users through training and awareness raising. The AI CoE goes beyond mere proof of concept (PoC) projects, working side by side with people to produce outcomes. This is a model suited to companies with cultures that are good at bottom-up promotion of change by people on the ground.

(3) The Governance Model (Focusing on (6) and (7))

This is a model focused on putting in place guardrails that allow employees to use AI with peace of mind. The AI CoE standardizes data categorization, usage rules, audit logs, decision-making on provision to external parties and incident response, and incorporates risk into specific measures. This represents a method for clarifying the permitted scope of experimentation and realizing conditions for “safe, quick learning,” while avoiding excessive prohibitions. When deciding the division of functions, an effective approach is to ask questions from the perspectives of “decision making,” “responsibility for execution” and “quality assurance” (see Figure 9). This is a model that is well suited to companies that prioritize stability.

Having considered these functions, companies need to also consider the form the organization should take. Different models exist. AI CoEs can be independent organizations that follow a centrally concentrated model, or they can follow an outpost model in which specialist personnel are deployed in business units. The centrally concentrated model excels in terms of standardization and oversight, but may undermine implementation capacity. The outpost model is stronger in implementation capacity, but can tend to create duplicate investment. Consequently, a hybrid model in which key areas are promoted using outposts while shared standards and rules are overseen centrally tends to be a practical solution (see Figure 10).
Another option open to companies is a reverse outpost model which aims to develop personnel and deploy assets and knowledge, in which personnel from across departments are gathered together at AI CoEs before returning after gaining experience over a set period of time. This approach is effective during phases when initiatives are being scaled with a mature AI CoE at the heart of the company.

Additionally, no matter which of the above patterns a company adopts, what decides the success or failure of its horizontal deployment is the organizational “learning framework.” In the next section, we touch on the learning functions necessary to consolidating the use of AI.

Organizational Learning Functions of AI Utilization Needed for Maximizing Impact

As described above, the major barriers to the utilization of generative AI are, in large part, strategy, implementation and oversight for company-wide promotion of AI and shortages of the personnel needed to implement those things. To supplement the human resources they lack, companies need to make progress on securing personnel through internal development and recruitment. On the other hand, frameworks for allowing newly acquired personnel to learn things in a short span of time and become fully effective in leveraging the company’s assets are key from a long-term perspective. In addition, the question of how companies can reuse shared assets and know-how formalized as knowledge is also important. Properly incorporating such organizational learning functions helps achieve optimization of investment company-wide. The true value of an AI CoE lies not only in increasing the success rates of individual projects, but in increasing the number of repetitions the learning process goes through, turning learning into an asset and extending it to the whole company. Summing up, to build a learning function that facilitates AI utilization, it is important for companies to practice three “reuses” (see Figure 11).

Specifically, these are: (1) arranging standards at entry points, (2) deriving success patterns, and (3) designing cycles reflected in templates, components and training (see Figure 12). Going through these cycles helps companies create projects that incorporate AI and visualize and control the costs of driving those projects, and to increase the cost-benefits of individual initiatives. Additionally, building up assets makes it easier to deal with shortages of personnel and skills.
By building up shared items, knowledge and personnel while continuously revising these reuse functions, companies can establish continuous learning in their use of generative AI.

Towards Full-Scale Promotion of Generative AI Utilization

In this article, we stated based on survey results that many major Japanese companies have put forward AI as management agenda items, and that a certain number of companies have even formulated AI visions and roadmaps. At the same time, we also found that there are many companies for which organizations and people, risk management, ROI calculation, data preparation and governance are challenges in utilizing generative AI. To drive the corporate value improvement and operational streamlining that companies are targeting, the key to success will be not just putting in place the right functions, but also storing project successes and failures as knowledge assets and establishing AI CoEs that can be responsible for organizational learning functions that enable the reuse of such knowledge. Operating such learning functions and taking the trials of individual departments and making them reproducible company-wide helps companies horizontally deploy AI while limiting duplicate investments.
ABeam Consulting boasts experience in working with major Japanese companies to launch AI CoEs and lead AIX. Based on the insights derived from this experience, ABeam excels at addressing the bottlenecks companies face in strategy, implementation and oversight. We also have the capacity to effectively incorporate organizational learning functions. ABeam achieves this by going beyond standalone PoCs to work side by side with clients, helping them promote such PoCs to company-wide initiatives that maximize management impact through AI.

In this series, we investigate other important functions beyond organizational learning functions based on our insights and survey results.
In the second entry in the series, we go over the key factors in formulating and implementing generative AI utilization strategies aligned with company-wide strategy. Specifically, we cover the identification of key agenda items to address, the incorporation of AI into short and medium-term strategies for addressing key agenda items, the formulation of roadmaps and how to set KPIs that contribute to organizational activity, rather than just being measuring indices.
In the third entry, we go over the alliance functions that companies should possess through their AI CoE organizations. In a present defined by new generative AI technologies and new generative AI-enabled businesses emerging day by day, it is important for companies to quickly identify and implement the functions that they lack and should acquire. We go over the key points in designing partnerships when making alliances, dividing work between alliance partners and internal work, knowledge transfer and oversight in this context.
In the fourth entry, we go over internal and external communications that accelerate the generative AI utilization. In parallel with external messaging aimed at market expectations and corporate value improvement, such communications, including internal attitude transformations such as promotion of autonomous generative AI utilization across departments, accelerate AI utilization. Here, we explain how companies should communicate about specific deployment models, as well as their generative AI utilization policies, successful case studies and outcomes.
We hope you will join us in the coming installments.