Case Studies | FinTech

Retrieval-Augmented Generation for safer code refactoring

RAG Feature Flag Removal on Amazon Bedrock

About

A FinTech business working with Cloud Combinator on AWS. The client is anonymised at their request.

Challenge

The challenge had four focus areas, each of which had to be met for the solution to be trustworthy in a production codebase.

Accurate feature flag removal

Designated flags had to be removed without impacting functionality, including the harder cases of nested flags and downstream dependencies where a naive edit would break the build.

Security

The code is sensitive intellectual property, so the whole pipeline had to run in a controlled environment with no exposure over the public internet.

Knowledge base integration

For the model to reason about complex flag scenarios it needed context across the entire codebase, not just the file in front of it, which called for a retrieval layer feeding the model.

Preservation of code integrity

The original structure, comments and any non-targeted sections of the code had to remain untouched after refactoring, so that reviewers could trust the output.

Solution

The codebase is stored securely in Amazon S3, which acts as the knowledge source for the architecture. That code is indexed into an Amazon OpenSearch cluster running inside the VPC, giving the system a searchable vector index of relevant code snippets.

$0.0075

Amazon Bedrock inference cost per feature flag removed

VPC-

Whole pipeline private, no code exposed to the internet

Claude 3.5

Model delivered via Amazon Bedrock, replacing the prior external API

By the numbers:

  • $0.0075 - Amazon Bedrock inference cost per feature flag removed
  • VPC- - Whole pipeline private, no code exposed to the internet
  • Claude 3.5 - Model delivered via Amazon Bedrock, replacing the prior external API
Changes

The solution was delivered against the agreed success criteria and demonstrated to the client, validating Amazon Bedrock as a suitable platform for continued AI-assisted development. The RAG tool removes feature flags with full codebase context while keeping the entire process inside a private, encrypted environment.

  • Accurate removal — The tool identifies and removes flags across multiple files, handling nested flags and downstream impacts rather than simple single-file cases.
  • Security by design — All services run inside a secure VPC using VPC endpoints and AWS PrivateLink, with least-privilege IAM roles and encryption at rest.
  • Whole-codebase context — The OpenSearch vector index gives Claude relevant snippets so it understands feature flag scenarios in context.
  • Code integrity maintained — Original structure, functionality and comments are preserved, with no changes to non-targeted sections.
  • Handover — The tool was left running in the client's own AWS account, with knowledge transfer sessions and self-guided AWS workshops for their team.

With Bedrock proven on a real engineering problem, the client has a foundation to extend AI-assisted tooling further across their platform. We recommend a Well-Architected review as a next step to harden the solution ahead of wider rollout.

AWS Stack

Amazon Bedrock

(Claude 3.5) for the large language model that performs context-aware feature flag removal.

Amazon S3

For secure, versioned codebase storage that serves as the RAG knowledge source.

Amazon OpenSearch Service

For vector indexing and retrieval of relevant code snippets.

AWS Lambda

For orchestration between the Node.js tool, OpenSearch and Bedrock.

AWS Key Management Service

For encryption at rest of the stored codebase.

AWS Identity

And Access Management for least-privilege access across all components.

YOU MIGHT LIKE

Related success stories

View all case studies

Case Studies | Insights

Utilising Language Recognition, Speed, and Enhanced Security to Make Social Media a Force for Good

  • Here, we take a detailed look at how the Cloud Combinator team collaborated with another cutting-edge AI service provider that provides intelligent systems to “make social media more social” for brands and users alike.
  • Arwen AI is a UK-based startup specialising in AI solutions to manage and enhance brands’ social media interactions. Founded in 2020 by Matt McGrory, Dr. David Cole, and Joel Bailey, Arwen. AI focuses on using AI to automatically detect and remove spam, toxic comments, and other unwanted content from social media platforms.
  • The team at Arwen have three core products. ‘Moderate’ is focused on identifying and removing toxic content from social media channels. ‘Engage’ helps brands identify and engage with meaningful conversations on social media, and ‘Customize’ allows brands to apply bespoke algorithms to their channels - creating an even more effective moderation and engagement.
Read more
CONTACT US

Ready to turn AI into impact?

We'll help you spot the highest-value opportunities, reduce risk around your first AI initiative, and define a clear path to results from day one.

Why talk to us:

Outcome-driven recommendations

AWS-recognised delivery expertise

Risk-aware AI adoption

Clear next step, not a sales pitch

Start with a focused 20-minute conversation about your goals — no pressure, no commitment.

We use cookies

This website uses cookies to enhance user experience and to analyze performance and traffic on our website.

See our Privacy Policy for details.