NatWest develops 'LLM Shield' in-house guardrail service for their GenAI activities
Hot on the heels of my post about NatWest's Ask Archie internal Virtual Assistant, I caught this post from the bank's Head of Applied AI, Euan Wielewski, outlining the steps the team has taken in the context of Generative AI guardrails.
He gives a bit more detail about Ask Archie:
Archie+ is a RAG-based service that uses a vector database-powered retrieval system and LLMs to generate accurate answers to colleague's HR questions.
He then goes on to detail the bank's LLM Shield system:
To allow us to safely deploy this AI capability, we also built 🛡 LLM Shield - an in-house AI guardrails service that protects NatWest AI services from prompt injection and jailbreak attacks, while also moderating our LLMs to ensure there are no toxic or biased outputs sent to users.
Very impressive.
Euan continues:
We've had to develop a range of new capabilities to get these lighthouse generative AI use cases live in production:
- Secure, private access to commercial LLM services
- Production-ready vector databases
- Robust information retrieval systems
- Scalable self-hosting of open source LLMs
- Human and automated LLM output evaluation frameworks
- Safe and ethical LLM red teaming processes
- Responsible AI governance and compliance controls
It's interesting, but not surprising, that NatWest would choose to leverage commercial LLM services.
Quite a few other institutions have taken a slightly different approach and are aiming to use the likes of Meta's Llama - but running on their own 'internal' (cloud, typically) infrastructure. I would imagine from Euan's bullet list above that NatWest is also doing something like this too.
Equally, I know quite a few banks that have decided they will not become experts in the nuts and bolts of AI: Instead, they are buying services from third-party vendors. I think this is a very sensible approach too. There ultimately isn't necessarily a wrong answer, especially so early on in the development of this marketplace.
It generally comes down to philosophy. I've worked in massive banks. I've also worked in much smaller banks. The key question for the senior executives is this: What do you want to be good at? What is critical for you to have in-house and what are you going buy-in from the best providers?
There is little hope for smaller banks being able to rival the kind of spend that the likes of NatWest will be investing here. The good news is there are a lot of vendors working hard to develop and deliver a fantastic array of AI services for their corporate clients.
Here's just one example: SalesTalk AI from OpenFi. Sam and his team have created a Conversational AI that does one thing really, really well – mining your list of prospects and engaging them patiently until they're ready to buy. The key target industry for them right now is the mortgage broking process. I've posted a written interview with Sam a few weeks ago and I've just published a podcast interview with him if you'd like to hear him talking about the product.
And here's a quick note for anyone who hasn't built their own LLM Shield: I'll be interviewing an executive from Applause – one of their latest offerings is exactly this: Generative AI testing.
Euan's LinkedIn profile is here:
