← Writing

How I Built an AI Agent for Personal Finance

When building an AI agent to answer financial questions for users, I had to make some architectural decisions to avoid building a chatbot that happens to talk about money.

The biggest architectural decision I made was to separate the AI from the financial system of record.

The model is responsible for understanding a user's question, deciding what information it needs, and turning the results into a useful response. It is not responsible for being the source of truth for financial data.

The Architecture


                         User
                           │
                           ▼
                    ┌─────────────┐
                    │  Web App    │
                    └──────┬──────┘
                           │
                           ▼
                    ┌─────────────┐
                    │ API Gateway │
                    └──────┬──────┘
                           │
                           ▼
              ┌─────────────────────────┐
              │      AI Agent Service   │
              │                         │
              │  Context + Memory       │
              │          │              │
              │          ▼              │
              │     LLM / Agent         │
              │          │              │
              │          ▼              │
              │      Tool Calls         │
              └──────────┬──────────────┘
                         │
                         │ Authenticated request
                         ▼
              ┌─────────────────────────┐
              │     Finance API         │
              │                         │
              │  Financial Tools        │
              │  Account Data           │
              │  Transactions           │
              │  Financial Calculations │
              └──────────┬──────────────┘
                         │
                         ▼
              ┌─────────────────────────┐
              │ Financial Data Layer    │
              │                         │
              │ Banks / Transactions    │
              │ Accounts / User Data    │
              └─────────────────────────┘

1. The AI Agent Doesn't Directly Access the Database

For security reasons, the first thing I wanted to avoid was giving the LLM unrestricted access to financial data.

Instead, I created a set of controlled financial tools.

These tools expose specific capabilities to the agent without exposing the underlying implementation.

At a high level, the agent can request information such as:

The important part is that the model doesn't need to know how those operations are implemented. 


LLM
 │
 │ "I need this financial information"
 ▼
Finance Tool
 │
 │ authenticated request
 ▼
Financial Backend
 │
 │ deterministic operation
 ▼
Structured Result
 │
 ▼
LLM

The model can reason about the result, but it doesn't become the source of truth for the underlying numbers.

2. The Agent Decides Which Tools to Use

Rather than creating a large collection of hardcoded intent handlers, I gave the agent access to a defined set of tools and let it determine which ones were relevant to the user's question.

This means the same agent can handle very different financial questions.

A simple question might require one tool call. A more complex question can require multiple calls and allow the agent to reason over the combined results. 


User question
      │
      ▼
    LLM
      │
      ├──────► Tool A
      │          │
      │          ▼
      │       Result
      │
      ├──────► Tool B
      │          │
      │          ▼
      │       Result
      │
      ▼
    LLM
      │
      ▼
 Final response

This tool loop is what makes the system an agent rather than simply sending a prompt to an LLM and returning the response.

3. Financial Calculations Stay Outside the LLM

This was one of the most important design decisions.

LLMs are good at reasoning over information. They are not where I want financial calculations to live.

The finance backend handles the underlying queries and calculations, then returns structured data to the agent.

The LLM can then explain those results in natural language.

Instead of asking the model to calculate a user's financial data directly, the financial service performs the operation and returns structured data: 

{
  "period": "...",
  "total": "...",
  "transaction_count": "...",
  "currency": "..."
}

The agent then turns that structured result into an answer.

Keeping those responsibilities separate makes the system easier to reason about, test, and change.

4. Financial Truth and Conversational Memory Are Separate

The agent also has access to conversation history and relevant user context.

But I deliberately don't treat conversational memory as a financial database.

Memory can help the agent understand preferences and previous context. Financial numbers still come from the financial tools. 


Conversation / Memory
        │
        │ context
        ▼
      Agent
        ▲
        │ financial facts
        │
Finance Tools

Relevant context is selected based on the current question rather than blindly putting an entire history into every prompt.

This keeps the context bounded while allowing the agent to become more useful over time.

5. The Banking Layer Is Isolated From the AI Service

Another important architectural boundary is separating the AI service from the underlying financial integrations.

The financial API owns the connection to the financial data. The AI service communicates with it through authenticated requests.

This means the AI layer doesn't need to directly own or understand every detail of the underlying financial infrastructure.

It also gives us a useful separation between AI concerns and financial data security.

6. The Response Goes Through Validation

The system isn't simply: 

LLM response → User

There are additional checks around the response and the tool execution flow.

The goal is to make sure the response is grounded in information the agent actually retrieved and conforms to the expected structure. 


User
  ↓
Agent
  ↓
Financial tools
  ↓
Structured financial results
  ↓
Agent response
  ↓
Validation / Guardrails
  ↓
User

7. The Architecture Is Model-Agnostic

I also wanted the application to avoid being tightly coupled to one particular model provider.

The agent interacts through an abstraction layer, while the underlying model can be configured independently.

This makes it possible to experiment with different models without rewriting the financial system around each one.

The Main Lesson

The biggest thing I learned building this wasn't how to write a better prompt.

It was deciding what the model should be allowed to do.

The model is responsible for:

And the system outside of the model is responsible for:

The result is less of a chatbot sitting on top of a database and more of a reasoning layer sitting on top of a controlled financial API.

That separation is what allowed me to build an AI agent around personal finance without making the LLM itself responsible for the financial system.