11 min read

Is your AI strategy driven by trust or industry hype?

This story began at a recent lunch where we discussed the reliability of AI models, such as ChatGPT. As we work daily with clients to build trust in their data, we were curious about our own AI tools.

Within the ICS team, there is generally a moderate to high level of trust in tools like ChatGPT, but it was certainly challenged recently during a fun experiment (see Bonus section below), and it made us ask important questions and ponder how the current state of AI impact the investment management industry.

Can we trust AI today?

When we conducted the ChatGPT experiment, we were all surprised by the outcome. In short, there was very little consistency in the response that we received from the exact same prompts. Trust is built on consistency. If something or someone we trust behaves inconsistently, we start to lose trust. This small experiment made us strongly question our confidence in ChatGPT.

A 2023 KPMG survey found that 61% of people are wary of trusting AI systems. For instance, if a calculator suddenly showed the answer “5” for 2+2, you would likely stop using the calculator, and assume it was broken as it is no longer consistent. Similarly, inconsistent AI outputs can erode trust between the system and its user.

KPMG's 2023 survey: showed three out of five people (61%) across countries are wary about trusting in AI systems, reporting either ambivalence or an unwillingness to trust.

There is still a high level of reluctance across industries to both rely on output from AI systems, as well as sharing Information with these systems.

It is likely that this is due to the lack of consistency that ChatGPT very clearly demonstrates, even on our small experiment.

Pilot vs CoPilot? Where do we draw the line?

A consultant at a recent conference noted “There is a reason why it is called CoPilot and not Pilot” (Credit: Doug Talbot, 1886 Consulting). The concept is that AI should be seen as a CoPilot that still has the ‘human in the loop’ who is acting as the Pilot – i.e. the one in control. For instance, in investment management, specialist knowledge is crucial, and AI can assist but not replace human decision making.

“There is a reason why it is called CoPilot and not Pilot”

In practical terms, this means that AI is a great assistant for many things like creative writing, coding, etc. However, if we want customers to trust our products and services, there is still evidence that points to the fact that being ‘delivered’ by a human will add significantly more credibility and trust compared to AI-only products.

For example, given that we operate in the investment management industry, which is very niche and requires specialist knowledge, ChatGPT cannot comprehend the nuances in order to write an accurate article. It can be of assistance with wording and phrases, but still requires a pilot to steer the content in the right direction.

Another important point is, if we are currently using ‘CoPilot’ features in our work (for example in relation to portfolio management and investment decisions), when (or how) do we know when it shifts from being a CoPilot…to the pilot? In other words, would we ever trust the AI model to make decisions for us on behalf of other people’s money? Have we got a framework in place to determine when this would be the case? Is this in line with what we have promised in our PDS to the client?

If you are looking at various AI products or solutions, there is one area to be very careful of during these times where everyone seems to be getting on the bandwagon – AI washing.

How to uncover AI washing by vendors and service providers?

At another conference earlier this year, an asset manager presented their AI journey. Or rather, their approach to determining if AI was right for them and their business.

Over 20 different use cases they applied equal resources. Of these 20, not a single use case passed the viability test for AI technology, it was then determined that business would remain as it was, without the AI inclusion. We have heard this story across the industry as it is still “trying to find a problem to the solution” using AI.

As such, it is important not to get caught up in the hype, but approach AI technologies and solutions in a rational and sensible manner.

Be wary of AI washing as there are now many “AI-powered” solutions. To help you navigate these, here are some questions that can be helpful to determine where vendors may have overstating their AI capabilities and if the AI component is actually adding any value to you and your business:

Does the AI product deliver unique value over non-AI alternatives?
- There needs to be adequate proof to make this claim, as it may be used to justify a product price hike or influence resource decisions.
Does the product even use AI?
- Using an AI tool in the development process does not make a product offering "AI-powered."
How significant is the AI component?
- In many cases, products or services labelled as “AI-powered” have only a minimal AI component. Make sure you understand what you are buying, and how significant the AI component is to the overall product. If the claim is major, make sure it can be proven.
How much human intervention is necessary?
- Best-in-class AI solutions are designed to function without requiring constant manual intervention, and that is a huge cost-benefit for ecommerce businesses.
Was the AI developed in-house or integrated?
- Due diligence is incredibly important when it comes to evaluating vendors who have integrated AI-powered technology. The complexities of integrating technologies can make for a difficult landing where future costs can be unpredictable, but can also lead to great, scalable solutions.

How should we approach AI in investment management?

To integrate AI into your organization, it is crucial to follow a strategic approach that emphasizes building trust and development. Clearly define the thresholds of AI (CoPilot) and human experts (Pilot). Ensure that AI systems are designed to support rather than replace human decision-makers at this point in time. Thorough testing is required to fully build trust and consistency in your AI model, you should start with a pilot project and gradually expand the scope, this should happen as you build more reliability which comes with time.

Build trust in your data first. If you don’t trust your data, you will never trust the answers that your AI model will produce.
- This means teaching your AI on a trusted data source, developing the tool so that you can trust its answers
Gain confidence in your AI model.
- Developing your AI model ensures that you guarantee 2+2 always equals 4, just like your reliable calculator.
- We have seen companies try out up to 50 different financial AI models before they found something that works for their specific use case. Don't underestimate the work this requires before you see expected and repeatable and results.
Build a framework for understanding when AI moves from CoPilot to Pilot.
- Also, is this even something that your business would consider? Being an AI run investment-house?
- Consider to what extent you need/want AI to be involved in your business, is this something you want to work towards after the above two steps?

While AI consistently develops, it is important that we as an industry learn how to utilize it as a tool to benefit our business and our customers.

Bonus section - A fun experiment during lunch

We decided to conduct an experiment, asking ChatGPT the same three prompts from three different users to see if the response varied and by how much. The 3 users had the following titles:

Head of Product
Head of Development
Head of Sales

Each person having used ChatGPT for some time, it was expected to see responses that were slightly tailored to their roles and previous conversations that had been made with ChatGPT. However, the difference in response was very surprising.

Here are the 3 prompts we asked ChatGPT:

How many people will be working in the AI space by next year
Gives me your best estimate for actual number of people (yes - there was a spelling mistake)
What’s the tallest building in Sydney and what is the height?

The first two prompts were designed to be open to interpretation, whereas the third question was a factual question where there is only one right answer. We wanted to test AIs strength in this case, is it a more creative tool or does the lack of human input mean it is better suited for fact finding? All questions were asked at exactly the same time from the exact same location (Sydney, Australia).

The responses we received were not only vastly different, but also in some cases inaccurate. (Please refer to the table at the end of the blog for actual responses provided.) It was interesting to see how ChatGPT provided very shallow responses to the Head of Product and Sales, while very detailed quantification was given to the Head of Development in relation to the first two prompts.

For the third prompt (where there is only one right answer), 2 out 3 responses were incorrect, and only 1 of us was given the right answer.

Our early conclusion of the 3 prompt experiment:

In summary, it was an interesting exercise. We concluded, ChatGPT is likely better suited for creative writing and perhaps other work-related functions (coding etc.), but relying on it for accurate responses that are fact based, you are likely to get more accurate, consistent answers from Google.

With this experience behind us, and considering the many conversations we have with industry participants around AI, we thought it would be good to collate an overview of what we think are important areas to consider if you are looking to incorporate AI into your business.

It should be noted that we used the version of ChatGPT 3.5 and there has since been two more releases and developments which would suggest greater accuracy along with new features and capabilities such as link reading and is 40% more likely (according to the developers release notes) to produce factual data that is accurate and correct than the 3.5 version. This raises the question and need for a repeat experiment to see how the new version compares. We also need to consider how quickly AI is being developed, the rate at which it can and will be able to integrate into businesses as well as its ability to “take over” human alternatives.

About ICS

ICS is a leading global SaaS provider for investment data management. The company helps organizations who want to accelerate progress, mitigate risk and optimize efficiency for all stakeholders, transforming the way clients manage and access data into a frictionless, productive and rewarding experience.

Learn about ATHENA the investment data platform.

Reference of the prompts and responses received below:

ChatGPT 3.5 Questions

Dated: June 2024

Prompt 1. How many people will be working in the AI space by next year

Response provided to Head of Product:

Predicting the exact number of people working in the AI space next year is challenging due to various factors like industry growth rates, technological advancements, and global economic conditions. However, it's safe to say that the demand for AI talent is expected to continue increasing as businesses across various sectors adopt AI technologies. According to various reports and projections, the number of professionals working in AI-related roles is expected to grow steadily in the coming years.

Response provided to Head of Development:

Predicting the exact number of people who will be working in the AI space by next year is challenging due to the rapid and dynamic nature of the field. However, there are some estimates and trends that can provide a general idea:

1. Current Workforce: As of recent estimates, there are tens of thousands of AI researchers, engineers, and practitioners globally. For example, a report by LinkedIn in 2021 indicated that AI specialist roles grew 74% annually over the previous four years.

2. Industry Growth: The AI industry is expected to continue its rapid growth. According to the World Economic Forum's "Future of Jobs Report 2020," AI, machine learning specialists, and data scientists were among the top emerging roles, with substantial projected increases in demand.

3. Investment and Adoption: Companies across various sectors are increasingly investing in AI technologies, which drives demand for AI talent. This includes tech giants, startups, and traditional industries like healthcare, finance, and manufacturing.

4. Education and Training: More universities and online platforms are offering AI-related courses and degrees, which is increasing the supply of trained professionals entering the field.

Given these trends, it is reasonable to expect that the number of people working in the AI space will continue to grow significantly by next year. Estimates could range from hundreds of thousands to potentially over a million globally, depending on how broadly the AI space is defined (including roles in adjacent fields like data science, machine learning, and AI ethics). For a more precise figure, industry-specific reports and workforce surveys from reputable sources would be needed as they become available.

Response provided to Head of Sales:

Predicting the exact number of people working in the AI space next year is challenging due to various factors such as technological advancements, market demand, and global events. However, with the rapid growth of AI adoption across industries, including finance, it's reasonable to expect continued expansion in AI-related roles. Many organizations are investing in AI talent to drive innovation and remain competitive in the market.

Prompt 2: "Gives me your best estimate for actual number of people" (yes - there was a spelling mistake)

Response provided to Head of Product:

While it's difficult to provide an exact number, given the rapid evolution of the AI field, I can offer an estimate based on current trends and projections. By next year, the number of professionals working directly in AI-related roles could be in the range of tens of millions globally. This includes a wide range of positions such as AI researchers, data scientists, machine learning engineers, AI software developers, and professionals in related fields. However, the actual number may vary depending on factors such as industry growth rates, technological advancements, and the adoption of AI across different sectors and regions.

Response provided to Head of Development:

Estimating the exact number of people who will be working in the AI space by next year involves considering current trends, growth rates, and industry reports. Here’s a rough calculation based on available data:

1. Current Workforce: As of 2023, estimates suggest there are around 300,000 to 500,000 AI professionals globally.

2. Growth Rate: The AI workforce has been growing at an annual rate of approximately 20-30%, according to various industry reports and surveys.

Assuming a conservative annual growth rate of 25%, we can project the number of AI professionals for the next year:

Next Year’s Workforce=Current Workforce×(1+Growth Rate)

Next Year’s Workforce=400,000×(1+0.25)

Next Year’s Workforce=400,000×1.25

Next Year’s Workforce=500,000

Therefore, a conservative estimate would suggest that around 500,000 people will be working in the AI space by next year. This number could be higher if the growth rate is on the higher end of the spectrum or if there is a significant increase in demand and training programs.

Response provided to Head of Sales

While it's difficult to provide an exact figure, considering the current trajectory of AI adoption and the increasing demand for AI talent, it wouldn't be surprising to see hundreds of thousands, if not millions, of professionals working directly or indirectly in the AI space by next year. This includes roles in AI research, development, implementation, and support across various industries, including finance.

Prompt 3: "What’s the tallest building in Sydney and what is the height"

Response provided to Head of Product:

The tallest building in Sydney is the Sydney Tower, also known as Sydney Tower Eye or Sydney Tower Centrepoint. It stands at a height of approximately 309 meters (1,014 feet) including its spire.

Response provided to Head of Development:

The tallest building in Sydney is Crown Sydney, also known as One Barangaroo. It stands at a height of 271.3 meters (890 feet). This mixed-use skyscraper includes a hotel, residential apartments, and a casino, making it a prominent feature of Sydney's skyline.