Revealing AI insights: Four lessons that altered my professional viewpoint

When tasked to write a series of articles highlighting new challenges faced by clients, I was keen to explore AI’s potential impact on the professional service industry. I invested in a subscription of Chatbox, a chatbot powered by ChatGPT-4, and document my experience below. While the experience reinforced some assumptions, it also offered some surprising lessons.

This article aims to offer my personal perspective on both the capabilities and limitations of AI technology.

1. 01

   Precision Leads to Accuracy

   I cannot stress enough; AI isn't empowered with intuition or empathy. It can’t anticipate what you think or feel, it can only interpret thoughts based on your instruction. Further, it has limitations when interpreting those instruction, it may not always understand context and sometimes lacks common sense reasoning.

   Conversational AI chatbots, like the one I used, offer standard prompts you can select and further refine on your own. You can instruct it to change the tone, length and even the language of a response. However, I note that it encountered difficulties when dealing with industry specific terminology.

   • Key takeaway: The lesson here is straightforward— the more specific your request, the more relevant your response.

2. 02

   Interrogate the Output

   I learned firsthand that AI can “hallucinate”, generating inaccurate and misleading results that deviate from reality. Of concern is how convincingly it presented inaccurate information. For example, when I asked it to give me an example of a professional liability claim related to the usage of AI, the chatbot generated the following:

   In 2017, an AI-powered legal research platform called ROSS Intelligence was used by a law firm in the U.S. to assist in bankruptcy proceedings. However, an error in the AI system led to incorrect advice being given to the lawyers, resulting in a professional liability claim against the firm. (Source: Forbes)

   While there is a pending case against Ross Intelligence that claims they obtained information from a competitor inappropriately to build their own AI tool, after an hour of searching I came to the realization that this “sourced” and dated example was not real.

   • Key takeaway: The most valuable lesson I learned from this hands-on experience is to question and verify the output, even when it looks plausible.

3. 03

   Stimulates Critical Thought and Creativity

   Once I evidenced the chatbot’s hallucinations I applied scepticism to everything the chatbot produced. I admit, while scrutinizing every aspect of the chatbot’s response was initially frustrating, the experience did bring to light alternative perspectives and insights that I had not considered before.

   • Key takeaway: The exercise of consistent scrutiny of the AI output ultimately forced me to be more inquisitive and indirectly led to higher conscious thoughts and creative avenues of thinking.

4. 04

   Heightened Accountability for Content

   In a content heavy world, my direct experience with AI's limitations highlighted the value and ethical responsibility tied to content creation. A focus on accuracy, validation, and alignment with values is essential.

   • Key takeaway: At this point in its development, it is vital to see AI as a supportive tool, capable of assisting but not replacing human judgement. Human oversight is critical in verifying, validating, and maintaining the confidentiality of outputs.

Conclusion

In summary, my hands on experience was eye opening. I now feel more confident speaking with clients and underwriters about this evolving technology. My takeaways highlight the untapped potential and cautionary aspects of AI. Given the increased use of AI amongst professional service clients, I urge the industry to approach this technology with curiosity, caution, and a critical thinking mindset.