The Virgin Money chatbot: Quite Unfortunate

I had to do a double-take earlier.

It was one of those "You what?" moments.

One of those, "Sorry, say that again?" situations.

The FT reports that a customer was using the Virgin Money chatbot which, apparently, chided them for using the phrase 'virgin'.

You absolutely, positively, cannot make this stuff up.

The customer in question was international FinTech legend David Birch. Here's his original post, complete with 300+ reactions and hundreds of comments. (I'm a little frustrated I totally missed this one as I am a followed of the great man himself).

If you click in on the post you'll see David's perfectly innocuous comment in his screenshot, followed by the chatbot getting all uppity ("please don't use words like that").

Oh dear.

Did.No.One.Test.It?

The FT has this explanation:

One person familiar with the situation at Virgin Money said the mishap had occurred on one of its oldest chatbots, powered by basic natural language processing and which was not running on its most recently upgraded models.

A lot of those older generation chatbots are far less likely to dream up stuff, simply because they are pre-Gen-AI and procedure based.

You can see from the screenshot of David's chat that there's a typo in the last message which would suggest it's brought that out of a database that a human typed in.

For all the hallucinating joy that you sometimes get with Generative AI systems, they don't generally make typos.

I reckon it's got a little system checking for profanity and it's simply triggered that response. It looks like no one thought to look for 'virgin' in the profanity filter and take it out.

This can't have been the only time it's happened though - but was David the only one to have written about it on LinkedIn (and have 21,148 followers)?

I wasn't able to find the chatbot on the Virgin Money homepage. I presume the next generation will be tested to the nth degree now though.