In recent years, design thinking (DT) has become a pervasive innovation approach in the managerial community as a structured way of solving challenging problems through a human-centered perspective. The approach fosters lateral thinking and has a huge impact on the organizational culture and the core elements of the innovation process. At the same time, the number of organizations trying to solve problems through the application of artificial intelligence (AI) is constantly growing, although more than 80% of AI projects never reach deployment due to the wrong strategic approach, poor data quality, or a lack of AI awareness among employees—and those that do remain below profitability expectations. The aim of this paper is therefore to connect the AI and DT communities by proposing a first framework supporting the development of AI solutions in a human-centered way. The contribution of this paper is thus twofold: First, we investigate ways in which design thinking adds real value along the design process of intelligent solutions, thereby enriching the present body of design management literature. Second, we propose a first framework by linking core elements of both worlds to support strategic designers and data engineers in designing such applications. Finally, we propose a research agenda that can serve as a basis for future research directions in order to advance contributions at the intersection of the two fields.
Trang 1New Frontier for Designing Human-Centered AI Solutions
by Martin B €ockle PhD, Iana Kouris PhD
Martin B€ockle
Lead Strategic
Designer, Boston
Consulting Group,
BCGX, London,
United Kingdom
Iana Kouris
Managing Director,
Boston Consulting
Group, BCGX, London,
United Kingdom
Abstract
In recent years, design thinking (DT) has become a pervasive innovation approach in the managerial community as a structured way of solving challenging problems through a human-centered perspective The approach fosters lateral thinking and has a huge impact on the organizational culture and the core elements of the innovation process At the same time, the number of organizations trying to solve problems through the application of artificial intelligence (AI) is constantly growing, although more than 80% of
AI projects never reach deployment due to the wrong strategic approach, poor data quality, or a lack of AI awareness among employees—and those that do remain below profitability expectations The aim of this paper is therefore to connect the AI and DT communities by proposing a first framework supporting the development of AI solutions in a human-centered way The contribution of this paper is thus twofold: First, we investigate ways in which design thinking adds real value along the design process of intelligent solutions, thereby enriching the present body of design
management literature Second, we propose a first framework by linking core elements of both worlds to support strategic designers and data engineers in designing such applications Finally, we propose a research agenda that can serve as a basis for future research directions in order to advance
contributions at the intersection of the two fields
Key words: artificial intelligence, design thinking, human-centered design
Introduction
I n recent years, design thinking (DT) has attracted significant interestand increasing attention from both industry and academia as a novel problem-solving methodology The community is constantly growing,
Trang 2recently by management scholars
(Liedtka, 2014; Verganti, 2020b)
who are focusing on design methods
and how they are applied to
innovation challenges, since the ways
and forms in which design can
impact the first phase of newly
emerging technologies is an
unexplored field (Verganti, 2020b)
Although the term design thinking
has been widely discussed in
prominent product management
journals (Di Benedetto, 2012; Seidel
and Fixson,2013) such as the
Academy of Management(Dunne and
Martin, 2012) as well as in business
publications like The Economist,
Harvard Business Review
(Liedtka 2018), The Wall Street
Journal(e.g., Wladawsky-Berger,
2015), or The New York
Times(e.g., Parker-Pope, 2016), a
generally accepted definition is still
lacking
This is not surprising, since
even the research stream around
product design, which consists of a
body of literature with a higher
level of maturity, still suffers from a
large number of differing definitions
introduced over time (Liedtka,
2014) Nevertheless, one of the
most widely known definitions,
introduced by Tim Brown, the
CEO of the innovation consulting
firm IDEO, which focuses on
product development and, more
recently, also on service and strategy
design, defines design thinking as a
“human-centered approach to
innovation that draws from the
designer’s toolkit to integrate the
needs of people, the possibilities
of technology, and the requirements for business success” (Brown, 2014)
Generally, this definition highlights the need to connect designers, principles, approaches, methods, and tools in a problem-solving context, while describing the three spaces of innovation:
First, inspiration, which refers to the actual problem and opportunity that aims to motivate the search for solutions; second, ideation, the process through which ideas are developed and tested; and third, implementation, the phase in which the level of maturity is reached and the developed artifact turns its focus towards the market (Brown and Katz, 2011; Liedtka, 2014)
Brown and Katz (2011) also mention that for any organization, these skills need to be dispersed and moved up to the executive level, to support but also inform the strategic decision-making process Over time, different models of design thinking have been proposed, for instance by IDEO or the Stanford Design School (e.g., empathize, define, ideate, prototype, and test), which represent iterative cycles of exploration, idea generation, prototyping, and experimentation
Moreover, Kimbell (2011) introduces different ways of describing and clustering design thinking, for instance through cognitive styles, theory of design,
or as an organizational resource
These characteristics of design thinking also relate to other
emergent approaches like agile product development, in which iteration and experimentation represent key components, or the lean startup approach, which highlights rapid iteration and the development of a minimum viable product (MVP) to validate and test the product with end users at
an early point (Micheli et al.,
2018)
Similarly situated, the process of designing artificial intelligence (AI) solutions highlights the need for a more human-centered thought process, since more than 80% of AI projects never reach deployment, due the wrong strategic approach, poor data quality, or a lack of AI awareness among employees—and those that do remain below profitability expectations (Chawla, 2020) Although the current body of literature does not provide a widely accepted process for designing human-centered artificial intelligence (HCAI) solutions, we believe that the intersection of design thinking and AI provides a valuable contribution to both communities Consequently, we aim
to answer the following research questions:
RQ1: How can design thinking be enabled towards a human-centered
AI approach?
What are the main challenges for the development of intelligent solutions within organizations, and how can they be tackled through HCAI design practices?
Trang 3Thus, the aim of this paper is to
connect the AI and DT communities
by proposing a first framework that
informs the design process of
intelligent solutions in a more
human-centered way The main
focus is placed on challenges like data
collection, the intersection of user
needs and AI capabilities, and the
explanation (xAI) of results to key
stakeholder groups The contribution
of this paper is thus twofold: First,
we investigate ways in which design
thinking practices can be enabled and
shifted towards a more
human-centered AI approach along the
design process of intelligent solutions
and thereby enrich the present body
of design management literature
Design methods and skills are
mentioned by Borja de Mozota and
Wolff (2019) as criteria of design
management, which leans more
towards co-creation with other
experts in project teams (e.g., data
scientists) when designing intelligent
solutions Furthermore, there is a
discussion around the role and
potential of applying design thinking
in user-centered AI scenarios, a
crucial element of design
management tools (Borja de Mozota
and Wolff,2019) Second, we
propose a first framework linking
core elements of the two worlds to
support strategic designers and data
engineers in designing such
applications and overcoming
challenges within organizations
Finally, we propose a research agenda
that can serve as a basis for future
research directions in order to
advance contributions at the
intersection of the two fields within the field of design management
Research background and related work
Human-Centered Artificial Intelligence
Many different research streams are related to the domain of human-centered AI First, ethically responsible AI (Xu, 2009), which highlights the objective of avoiding discrimination and achieving fairness and transparency of the intended solution Second, the well-known research stream of explainable AI (xAI) aims to design explainable, usable, and useful AI solutions (Xu, 2009; Riedl, 2019), since these aspects have not been considered in the past due the strong technology-driven focus The application of xAI practices is becoming more relevant
as AI is increasingly applied in different end user applications, while most end users, specifically those with a limited technical background, perceive intelligent systems as a black box and view them with a low level
of trust Consequently, the design of
AI solutions needs to consider how
to present the output to different user groups in order to increase the level of trust Questions like “Why did you succeed or fail? Why did you do that? Why is this the result?
When can I trust you?” (Xu, 2009; Riedl,2019; Preece,2018) reveal the challenge of a one-size-fits-all approach when interpreting the output of such systems Recent work
shows the potential of classifying the explanations of AI solutions for different types of users—such as developers, AI researchers, and domain experts—by considering their needs, objectives, and goals in different application contexts (Ribera, 2019) Furthermore, the framework developed by
Wang (2019) includes relevant theory on human decision-making, for instance how end users should reason in order to inform the xAI techniques (Wang, 2019) by highlighting a taxonomy of questions:“What is explained?” (e.g., data or model), “How is it
explained?” (e.g., direct/post-hoc, static/interactive), and “At what level?” (e.g., local versus global) (Arya,2019) Such works show that
AI policy and governance play an important role in the future of intelligent solutions
Design Thinking and HCAI
The current body of literature building a bridge between design thinking and AI is limited, but the few studies that have been identified reveal the potential of a meaningful intersection of the two communities, although the question of how AI impacts design thinking practices in the form of tools has not been considered While design thinking is discussed within the design
management literature as a way to address new problems in
organizations (Cooper et al.,2009), the link to AI has been discussed very little, although it has been considered
Trang 4as a future involvement of designers
(Borja de Mozota and Wolff,2019)
Relevant work on how to look at
design in the age of AI has recently
been proposed by Verganti
et al (2020a,2020b), highlighting
that design work for AI solutions
should not focus on the ideation of
products to be commercialized at
scale, but rather on the design of
problem-solving loops, described as
human-capital-free design systems
that replace people with computing
power, which then will develop the
specific solution and deliver it to
the end user
The authors also mention that
AI does not undermine the basic
principles of design thinking but
rather reinforces them by overcoming
past limitations (Verganti
et al.,2020b) Intelligent solutions
are much more user-centered due the
high level of granularity and
the provision of unique user
experiences (UX) for every single
user of the application A recent
study by Iansiti and Lakhani (2020)
reveals that scale, scope, and learning
are highlighted as key differentiators
in designing such solutions (Verganti
et al 2020a), described through the
term AI factory, a software that, for
instance, runs millions of daily ad
auctions at Google or Baidu, or
decides about the availability of rides
on Didi, Grab, Lyft, and Uber
(Iansiti and Lakhani,2020) These
algorithms also set the prices of
products on platforms like Amazon
However, at their heart, they use
internal and external data for
predictions, insights, and choices:
• Scale and standard design processes produce solutions that target several users of predefined customer segments or average archetypes, which have been considered through “personas”
along the design thinking process
AI capabilities remove this scale limitation in design, since intelligent solutions embed design rules that are inherently user-centered, for instance in the case of Netflix, leveraging a rich stream of data on each individual user (Verganti et al.2020a) to develop a specific user experience
The development of these experiences becomes very powerful as the number of users and the complexity of insights grows
• Scope and abduction refers to the fact that, in traditional design practices, products and services are designed for a specific industry, with little flexibility to apply the developed solution to a different application context The design of AI solutions removes these limitations by reframing design artefacts in different domains Verganti et al (2020b) refer to the example of reusing developed experiences in Netflix for AirBnB
• Learning and iterations represent
a limitation in standard design processes to a certain degree, once the developed product is launched Input for future developments and revised versions are informed by product usage, while in intelligent
solutions the algorithm directs the learning strategy towards a better experience, with significant implications on innovation (Iansiti and Lakhani, 2020) End users always experience the best solution, which evolves over time
Consequently, in AI factories the
“design-build-test “learning loop
is fully automated, which means that a new version of the product
is released once the end user accesses the service (Verganti
et al., 2020b)
In addition to the academic body
of knowledge, resources like Google PAIR (2021) inform towards a practice-driven approach by providing design guidelines for developing such intelligent solutions
Similarly, the contribution of Microsoft (Amershi et al.,2019) provides such guidelines for
human-AI interaction in the categories
“initially, during interaction, when wrong, over time.” We strongly believe that these practices provide a strong contribution towards our proposed framework
Research design
To answer the defined research questions and build on a solid baseline, we conducted a structured literature review (SLR) with the keywords “design thinking AI” and
“design thinking artificial intelligence”
and analyzed the existing literature at the intersection of the two research communities The literature review addresses general and specialized
Trang 5scholars within the domain of AI and
design thinking who are interested in
novel AI and design thinking
research from various domains In
the conceptualization phase of the
review, an initial explorative search
using Google Scholar was conducted
to conceptually identify current
approaches and trends Furthermore,
to identify core challenges in the
development of AI solutions within
organizations, we carried outfive
semi-structured interviews with
selected data scientists who
experienced the end-to-end process,
starting from data collection to the
actual business application
Demographic information reveals
thatfive out of five participants were
male data scientists, two of them with
a senior profile and three within a
leadership role in the age group 26–
35 All participants had a master’s
degree and were based in France The
semi-structured interviews were carried out by two strategic designers for a duration of one hour
Based on the developed insights,
we created afirst framework that was designed in an iterative process by adding veins of knowledge from the design thinking community to guide data scientists as well as business professionals along the process of designing AI solutions The defined research challenges, which inform the proposed research agenda, were derived from the structure of the proposed framework
Results
A structured design process becomes even more relevant in the
development of AI applications due the rapid evolution of AI
possibilities and the fact that every
AI solution is highly unique in its
character An important aspect to highlight is that value is added by combining AI with changes in business processes and not through
AI itself Consequently, organizations should aim to design a deployment process in which technology is improved continuously
by providing the right quality of training data to the developed algorithms, while products and business processes are adapted at the same time
Results were collected from two workstreams First, from the structured literature review, and second, from the semi-structured interview sessions The framework proposed in Figure1 is organized in three layers, where the top one follows the design thinking model proposed by the d.School (HPI)
This layer aims to connect towards a human-centered design approach by
FIGURE 1 Design Thinking/AI Process Framework [Color figure can be viewed at wileyonlinelibrary.com]
Trang 6proposing the second layer, a strategic
HCAI design layer that highlights an
AI-informed, end-to-end design
process The bottom layer highlights
challenges identified within
organizations that inform the layers
described above
Empathize
In this phase of the strategic design
layer, the framework starts with the
pillar of the AI baseline, which
focuses onfinding the intersection of
user needs and AI strengths These
opportunities can be identified
through a process of mapping the
existing workflow (Google PAIR)
Consequently, the list of
improvements needs a decision as to
where and how AI might add unique
value (Google PAIR) Generally,
there are tasks where AI can add real
value, for instance by recommending
content to different user groups or
predicting future events, but tasks
that might require complete
transparency are still a huge
limitation of the technology and
should be considered in the design
process
From a design thinking and
product development perspective, the
aim is to understand the target group
by analyzing their needs, goals, and
objectives through close interaction
with them Depending on the
context, Liedtka (2014) proposes
ethnographic research, a qualitative
approach to develop an
understanding by observing and
interacting with them in their daily
environment Methods include
participant observation, interviews, and customer journey mapping in which identified problems and pain points are highlighted By mapping existing workflows, AI opportunities can be identified, especially those that have not been articulated
Quantitative approaches are helpful if existing data is provided to
understand historic behaviors of the end user group The AI canvas (Maillet,2019) provides a toolbox to better understand the intersection of user needs and AI strengths, but also
to place the outcomes of the abovementioned methods Generally, the aim of this tool is to find a consensus between data scientists, designers, and key stakeholders of the intended AI efforts The proposed canvas includes topics like the collaboration between human and machine activities, benefits for humans, and considerations and implications, but it also covers change management activities The best practice, provided by Google PAIR, also emphasizes whether AI is being used to automate a task or to improve a person’s ability to carry out the task themself (Google PAIR)
The research activities described above should conclude on this topic within the “empathize” phase
Organizational challenges at the beginning of an AI initiative are not limited to understanding the technological landscape or the availability and quality of data, although it is important that key decision makers comprehend the technological shift that might be needed as part of the future solution
Very often, challenges are linked to
an absence of the AI mindset needed
to develop such solutions For instance, AI needs to be embedded in the company’s strategy, vision, and purpose in order to develop a clear perspective of where AI will drive business outcomes Furthermore, AI development needs to be treated as a business transformation, where redesign is collaboratively organized with business owners, including iteration and improvement, based on the human-AI learning process
The strategic design layer proposes the identification and clear definition
of the intelligent behavior and what
AI is expected to do This exercise is
in line with the second design thinking phase, where the aim is to define a clear problem statement
Guidance is provided by Google PAIR, which suggest the definition
of the reward function used to determine“right” versus “wrong”
predictions The design of these functions aims to be a collaborative process across disciplines, where UX designers, product designers, and engineers share their perspectives A simple template for defining true positives and false negatives is helpful for the final definition of the intelligent behavior (e.g., our AI model will be optimized for {precision/recall} because {user benefit} (Google PAIR)
The ambition for value creation through AI needs to be clearly articulated from an organizational
Trang 7perspective This also includes setting
the focus and identifying the desired
AI outcome at the department level
to support the corporate strategy
(Ransbotham et al.,2019), for
instance in marketing (e.g., cross- and
upselling, churn and retention, and
next best product and action) or the
supply chain (advanced forecasting,
simulations, and route optimization)
Challenging choices within the
“define” phase include the definition
of performance metrics, which
requires an understanding of how the
performance frontier may be evolving
within the domain in which the
intended AI solution might be
developed This is very challenging
due the fast-moving capabilities
of AI
Other well-known
organizational challenges reveal the
availability of data, which includes
the cleanliness and understanding of
the source data and quality
Organizations may lack, for instance,
the setup of an internal data
economy, such as when data
scientists hoard data with no
organizational support for ease of
publishing Data access is often
limited, orflexibility of access
management may not be guaranteed,
including strong data compliance
guidelines that make the
development of AI solutions
challenging Finally, individuals
within organizations may not be
aware of what data is available to
them or where tofind it In most of
the cases, the data quality does not
meet certain thresholds such as those
regarding completeness and accuracy
Consequently, before starting to develop models in the process, it is necessary to assess the maturity of data capability and understand the organizational limitations Design thinking activities that support organizations within this step include the application of the data landscape canvas in order to understand and categorize data assets in terms of their source and ownership (Wirth and Szugat, 2020)
Ideate
The business process comes into play
in this phase Based on the design principles proposed by Google PAIR, the decision to use AI to automate a task or augment a person’s ability to carry out the task is crucial at this stage, since the different ideas might consider certain ways in which AI can solve the problem and support the end users in accomplishing their goals (Google PAIR) Newly developed ideas might focus on increasing efficiency or reducing tedious tasks, while others might consider a higher level of involvement
by the end users, where AI augments their existing abilities A successful augmentation usually increases the end user’s enjoyment of a task or increases their responsibility and control
Thus, to tackle this challenge, previous insights from the
“empathize” phase might be helpful
in understanding the end users’
motivation of their daily tasks, combined with analyzing current business processes in order to assess
where AI can create the greatest value This activity is usually done through ideation sessions led by designers, where several solutions are created and discussed with data scientists and engineers The identified opportunities should create clarity on how AI will be used to compete long-term in the economy
In this case, the delta model proposed by Hax and Wilde (1999) provides three different strategic options to define the scope of the intelligent solution Thefirst is to be the best product on the market, either through low cost or through differentiation The second is a customer solution that might satisfy
a wider group of people, with a focus
on the customer rather than on the product economics (Hax and Wilde1999) The third is having a system lock-in that covers all the players that contribute to the creation of economic value while providing the widest scope of all three options through bonding
Furthermore, in the process of creating an idea andfinding the focus
of the developed solution, design thinking provides a meaningful framework for choosing and ranking developed ideas The well-known Desirability, Viability, Feasibility (DVF) approach is a valuable tool in the innovation process to move to a more structured level of assessment and sophistication with the planned business model
With a clear vision through ideation, organizations often struggle
to integrate a clear data strategy that embodies the baseline for designing
Trang 8and developing the intended AI
solution In the ideation phase, the
overall data strategy needs to be
strong and have a high level of
maturity before starting the software
development process This means
that it has to enable more clarity and
understanding of data, as well as data
product discoverability, for instance
through a data marketplace Research
shows that every AI strategy needs to
be complemented by a data strategy
Recent research shows that three out
of four organizations declare AI as a
core component of their
transformation plans, although only
11% of them have started
implementing a solid data strategy,
i.e., designing effective data platforms
and processes that enable effective
machine learning approaches (Kiron
and Schrage, 2019) Further
elements of a proper data strategy
include evaluating the data and the
collection method in terms of
whether they are appropriate for the
intended project, including
the documentation of contents and
decisions made during the data
collection process (Google PAIR)
Prototype
This section emphasizes the
software development approach
around the intended solution The
AI technology on the strategic layer
highlights clarity regarding the
selected data strategy to follow One
of the most critical issues is how
data is labeled (e.g., data, time,
content description) since data that
is not properly tagged comes with
certain limitations attached Best practice shows that the more metadata is available, the more options there are for developing a successful model, which also impacts the quality of the user experience (UX) (Google PAIR) in the subsequent steps For instance, for supervised learning, one of the initial steps is creating a labeled data set,
to be divided into training and validation (Verganti et al., 2019)
The company Netflix is using this approach for recommendations based on labeled data sets (e.g., watched and liked movies), where a large group of user choices can lead
to effective recommendations In this case, design thinking could be used to identify opportunities for incentivizing end users or trained subject matter experts to participate
in labeling activities The design of the incentive approach, the labeler experience, diversity, as well as existing tools need to be taken into consideration Generally, approaches like gamification have the potential
to motivate the end user to carry out certain tasks (B€ockle, 2017; B€ockle, 2018); such approaches are connected to either intrinsic or extrinsic motivational theories
The decision on the software development approach has huge implications and a large impact on the business outcomes There are several reasons to pivot in the development of AI solutions, for instance when a new source of data increases accuracy or lowers the computational costs, when a new channel offers the same service, or
when an updated version of software development tools (e.g., PyTorch) requires a code reassessment The development process thus requires a high level of flexibility, combined with an adaptive design process
Solutions developed by any organization always deal with certain limitations when it comes to AI One
of them is the lack of generalization (e.g., face recognition) or the bias of data to be used to train the
algorithm Explainability is an ongoing issue that is being researched heavily under the term xAI
Furthermore, unintended behaviors (e.g., output of a GPT3-based chatbot or self-driving cars) need to
be taken into consideration as well
At this stage, design thinking has huge potential to intervene with more creative approaches such as designing communication patterns that enable the right type of information to be presented to the right end user in the expected format, based on the user’s needs
Test
In the testing phase of the developed model, the strategic design layer refers to the user experience (UX) of
AI From early on, it is necessary to receive qualitative feedback through a diverse group of end users
Dashboards and customer data visualizations, which are also part of human-centered design practices, enable the UX quality of the developed system to be monitored
Testing and tuning is an ongoing process for adjusting the ML model
Trang 9(Google PAIR) and needs to be
considered carefully
Trust plays a central role in
testing the developed AI solutions
with the end user group If the
designed user experience does not
convey trust instantly and
consistently over time, end users
might quickly stop using the product
or service Thus, to increase the level
of trust between the end user and the
developed intelligent solutions,
certain design best practices have
been proposed These include how to
meaningfully introduce the end user
to the AI system and how to set
expectations for adaptation and
create effective mental models, which
need to be considered in the testing
scenarios
Furthermore, guidelines show
that feedback is crucial to developing
trust in AI-enabled user interfaces, as
is explaining how to review, collect,
and connect implicit and explicit
feedback to inform and enhance the
user’s product experience (Google
PAIR) The Google design team
proposed further guidelines on how
to define errors and failure by
providing a path forward from
failure, since AI capabilities can
change over time As explainability is
considered one of the major drivers
for increasing trust in these systems,
there are best practices that explain
how AI systems work by connecting
explanations to the end users’
interactions with the AI system The
proposed guidelines follow a
human-centered design approach and should
be considered in the developed
testing scenarios:
• Mental model: This set of guidelines concerns the end users’
understanding of how AI systems work and how their interactions affect the interface Generally, mental models aim to set expectations about functionalities and communication limitations
• Explainability and trust: These guidelines address how the end user receives an appropriate level
of explanation regarding how the system works and the degree of confidence in its output After developing a clear mental model and awareness of the system’s overall capabilities, these guidelines help end users learn how and when to trust the underlying system
• Feedback and control: This set of guidelines concerns the design
of feedback and control mechanisms that provide a meaningful end user experience (UX) when suggesting
personalized content These mechanisms can also be used to improve the underlying AI model output
• Errors and graceful failures:
These guidelines help identify and diagnose AI context errors and communicate the way forward Context errors include false starts, misunderstandings, and edge cases that cannot be foreseen within the development process Google suggests that these errors should be seen as opportunities to correct the end user’s mental model, encourage the end user to provide feedback,
and enhance the overall learning process through experimentation and error resolution processes
These best practices provide a solid baseline for designing and testing a meaningful user experience while at the same time fostering a high level of trust Organizations need to have a clear plan for their testing strategy; for instance, what does the plan for early testing of the model look like? Are the users diverse enough? Which metrics might be useful to measure whether the tuning process is successful? Since design thinking provides a large repository
of methods for testing, the application of structured problem-solving approaches enables the quality of the developed model to be improved
Conclusion
The result of the present work aims
to support researchers and designers
in developing human-centered AI applications by applying a structured design thinking AI process
framework to increase the level of maturity and success of the intended
AI solutions Generally, the developed framework can be used as
a tool to guide and control the design process of such solutions, since there
is currently no structured process that applies patterns of human-centered design principles To answer the first research question, we strongly believe that the development process of AI solutions is lacking creative approaches that actually have
Trang 10the potential to increase the
likelihood of success We therefore
suggest that these approaches have a
strong impact at the beginning of the
design process, where baseline
elements like the vision or mission of
the intended solution are still unclear
or very loosely defined Furthermore,
design thinking evolves its potential
in the right selection of the intelligent
behavior through the definition of a
clear statement, while considering the
users’ needs at the same time Since
the development of such solutions is
closely linked to the user needs and
the overall organizational design,
there is a need to closely connect AI
with design thinking due the nature
of the structured problem-solving
approach Consequently, by
intertwining design thinking practices
with phases of the AI development
process, we propose thefirst
structured approach of an AI-enabled
design process by shifting the
standard design thinking approach
towards a more human-centered AI
approach Therefore, for RQ1, we
believe that design thinking adds real
value in thefirst two phases
(empathize, define), but also in the
testing phase where the user
experience (UX) of the proposed AI
solution needs to match the end user
needs Regarding RQ2, the identified challenges were not limited to data quality but more importantly included a lack of collaboration between departments including frictions in sharing data on the operational level or the overall missing AI mindset on the organizational level We believe that design thinking is very well suited for organizational change in the long run but might be limited to having a large impact on the operational level
Therefore, we suggest the application
of behavioral design approaches such
as gamification, which are very powerful in enabling behavior change
A crucial element is the definition of the data strategy, which is often one
of the reasons why AI projects fail in the first run We identified several meaningful points of contact between the design thinking approach and the
AI development process and believe that this first framework serves as a baseline for further revision and improvements
Limitations and future research directions
The present paper is also subject to several limitations First, the current body of literature investigating the
connection between design thinking and AI is limited This is also one of the results of our structured
literature review Consequently, the developed framework did not receive strong input from the theoretical perspective but rather presents afirst approach where existing practices have been merged with a non-structured AI development process
Second, most of the best practice examples were selected from Google PAIR, which provides a rich repository of examples for developing such solutions Third, since we only carried out five interviews with data scientists and AI experts, the presented results are limited to the organization types (in terms of size, structure, existing IT
architecture, etc.) where the participants provided their insights
Fourth, the developed framework has not been tested and validated in a real-world scenario and thus provides
afirst approach to showcase the connection between the existing literature in design thinking and AI, mixed with insights from practice, highlighted through organizational challenges While there are many unexplored issues in this domain, we strongly believe that this paper makes
a valuable contribution to the design
TABLE 1 Proposed research challenges
Research
challenge Description
RC1 How can the proposed framework be extended and shaped towards different types of organizations?
RC2 Which design thinking practices need to be revisited in order to meet the fast-evolving AI technology?
RC3 Which additional research streams should be merged with those of design thinking and AI in order to improve current practices
(e.g., behavioral design, gami fication)?