Let’s build an AI Agent - Setup and deployment

Let’s build on Part 1, where we introduced the project and defined its architecture.

Local setup

Dev stacks are a fairly subjective topic, I’ll go through what I’m using for this project, and why. Other tools may bring the same features and benefits.

I’m running Windows for work, and I have to say that it’s in a much better place since the introduction of WSL 2 (Windows subsystem for Linux) a few years ago. WSL allow to seamlessly run a virtualized Linux environment directly from Windows. It makes a lot of sense for me since most of the target systems are running on some flavours of Unix, and so are operating systems.

Building some proficiency with basic Unix commands served me well and most of it is transferable to pretty much any Unix based OS, being a Linux distribution or even macOS which is an ever-popular development platform. And of course, Windows too thanks to WSL.

This helps with consistency – Developing on Linux to deploy on Linux can already prevent some instances of the infamous But it works on my machine syndrom. One more helpful step is the fact that these environments can be created and deleted at will and are isolated from each other. It’s quite common for me to work on multiple projects at once, and I usually have one VM for each as they will most likely have different targets and dependencies.

But can we go even further in replicating the target environment locally? We sure can, thanks to containers. Containers are a way to package code and dependencies together while having full control on the environment.

Containers come with a lot of benefits, I’ll highlight a few:

• They ensure that the local environment mirrors the production environment. We all probably had this one small, niggling discrepancy that causes failures in production despite working fine on my machine.
• They package their own dependencies and are isolated from the host system. It is very easy to have dependencies conflicts over time, I often work with PySpark and keeping Java, Python and Spark versions aligned to mirror let’s say a Databricks cluster can be challenging. Containerisation greatly helps with this matter.
• They make developer onboarding very straight forward and easily reproducible. Rather than having to install dependencies manually, we can just spin a container and have a ready to use, identical environment. This is exactly what we are about to do.

Requirements

Docker Destop

Git

Git or any Git client to clone the repository:

Azure

You will need an Azure subscription to deploy the resources. If you don’t already have an account, you can get started with $200 worth of free credits.

Recommended: Visual Studio Code

This one is a personal choice and any editor will do. For the rest of this series, I will assume that you are using Visual Studio Code as its integration with development containers and unit testing is great.
You can download it for your platform directly from their website.

Make sure to install the Dev Containers extension provided by Microsoft – It will allow you to open the project in a Docker container.

Repository

The GitHub repository contains all the code for this series. Clone it to get started, for example from the command line:

git clone https://github.com/vfc2/oaitoolsample.git

The repository is separated in 2 main folders, infra and app.

• infra contains the project’s Infrastructure as Code, made of Bicep and bash scripts to manage the deployments. The deployment is managed using the Azure CLI – made available in our dev container.
• app contains the application code and tests in Python.

As mentioned earlier, the development environment will run in a container. The .devcontainer folder contains a Dockerfile, a script that defines how to build a Docker container image.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

FROM python:3.12-bullseye AS base_image

# System update and install dependencies
RUN apt-get update && \
    apt-get -y install unzip && \
    apt-get -y install curl && \
    apt-get -y install gnupg && \
    apt-get -y install wget

# Azure SQL ODBC Driver
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
    curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
    apt-get update && \
    ACCEPT_EULA=Y apt-get install -y msodbcsql18 && \
    ACCEPT_EULA=Y apt-get install -y mssql-tools18 && \
    echo 'export PATH="$PATH:/opt/mssql-tools18/bin"' >> ~/.bashrc && \
    apt-get install -y unixodbc-dev && \
    apt-get install -y libgssapi-krb5-2

ENV PATH="$PATH:/opt/mssql-tools18/bin"

Line 1 defines the base image that the container will use, here Debian Bullseye with Python 3.12 pre-installed.
Lines 4 to 8 are performing a simple system update.
From line 11, it installs the ODBC driver that we will need to connect to Azure SQL from Python.

Then we have devcontainer.json, that contains our development container configurations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

{
	"name": "Python 3",
	"build": {
		"dockerfile": "Dockerfile"
	},
	"features": {
		"ghcr.io/devcontainers/features/azure-cli:1": {}
	},
	"customizations": {
		"vscode": {
			"extensions": [
				"ms-python.python",
				"ms-python.black-formatter",
				"ms-python.pylint",
				"ms-azuretools.vscode-bicep",
				"ms-mssql.mssql",
				"njpwerner.autodocstring"
			]
		}
	},
	"postCreateCommand": "cd app && python3 -m venv .venv && . .venv/bin/activate && python -m pip install --upgrade pip && python -m pip install -r requirements.txt"
}

Line 7 make the Azure CLI available in the container. This features system is quite a cool feature and more details can be found on their GitHub.

Line 12 to 17 are a list of Visual Studio Code extensions that will be automatically installed when running in the development containers – It’s great for consistency as it allows the development environment to be perfectly reproducible.

Finally, line 21 allows to run arbitrary commands once the container is created. In our case, a Python virtual environment is created and activated, and the dependencies contained in requirements.txt are installed.

Starting the project in VS Code

Before this, make sure that Docker Desktop is installed and is running, and that the Visual Studio Code Dev Containers extension is installed.

For the first time, open Visual Studio then press Ctrl + Shift + P (Cmd + Shift + P on macOS) to open the Command Palette and type:

Dev Containers: Open Folder in Container… 

Visual Studio Code will automatically use the container definition file that we described above and will build and load the environment. Because it’s the first time, the operation may take a few minutes. You can follow the progress in the built-in terminal.
Once the build completed, you can verify that the workspace in indeed open in the Dev Container on the bottom-left of Visual Studio Code:

In the future, you can use the following command to re-use a Dev Container and have a significantly faster boot-up time:

Dev Containers: Reopen in Container… 

Azure deployment

Before we can run the integration tests, we need to deploy the resources to Azure. The Azure CLI is pre-installed in the Dev Container, make sure to login and select the right subscription to use for the deployment (in my case, 2).

Open a bash terminal (Ctrl + ’) and type:

az login

We will deploy the minimum amount of resources to get the tests running, a resource group containing an Azure OpenAI instance and a SQL Database. We will also deploy an Entra Group called AI Developers. We will assign roles directly to this group as part of the deployment.

We will make sure to add any user (including yourself) to this group so they can access and use the resources.

Files organisation

The resources deployment is organised under the infra folder. The modules folder contains a template for each type of resources (in our case databases, Open AI and one module dedicated to role assignments).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#/bin/bash
if [ -z "$2" ]
  then
    echo "Invalid parameters - Usage: deploy.sh <project-name> <region>"
    exit 1
fi
project_name=$1
region=$2

echo "Creating the 'AI Developers' group..."
group_id=$(az ad group create --display-name "AI Developers" --mail-nickname "AIDevelopers" --output tsv --query id)

echo "Deploying project $project_name to Azure..."
outputs=$(az deployment sub create \
  --name "${project_name}_deployment" \
  --location $region \
  --template-file infra/main.bicep \
  --parameters projectName=$project_name aiDevelopersGroupPrincipalId=$group_id \
  --output tsv \
  --query "[properties.outputs.openAiEndpoint.value, properties.outputs.dbConnectionString.value]"
)
echo "Project $project_name deployed successfully."

echo "Creating an .env file in app/.env ..."
openai_endpoint=$(echo $outputs | cut -d' ' -f1)
db_connection_string=$(echo $outputs | cut -d' ' -f2)

echo -e "OPENAI_ENDPOINT=$openai_endpoint\nAZURE_SQL_CONNECTIONSTRING=Driver={ODBC Driver 18 for SQL Server};$db_connection_string" > 'app/.env'

The script deploy.sh orchestrates the deployment by:

1. Creating the AI Developers Entra Group if it doesn’t already exist, and get its Object ID.
2. Deploying the resources, using main.bicep as the entry point.
3. Creating an .env file in app/.env and output the endpoints of the resources that were created. Since we are using managed identities, no credentials are stored.

Modules are grouped by concerns and are called from main.bicep. I kept them as simple as possible.

Resources naming follow the pattern {type}-{project}-{number}, {project} being a parameter passed to deploy.sh. I strongly recommend using the resources abbreviations from the Cloud Adoption Framework.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

targetScope='subscription'

@description('The project name to use when naming resources.')
param projectName string

@description('The Principal ID of the AI Developers Entra group')
param aiDevelopersGroupPrincipalId string

resource resourceGroup 'Microsoft.Resources/resourceGroups@2024-11-01' = {
    name: 'rg-${projectName}-001'
    location: deployment().location
}

module openAi './modules/open-ai.bicep' = {
    name: 'openAiDeployment'
    scope: resourceGroup
    params: {
        projectName: projectName
    }
}

module database './modules/database.bicep' = {
    name: 'databaseDeployment'
    scope: resourceGroup
    params: {
        projectName: projectName
        aiDevelopersGroupPrincipalId: aiDevelopersGroupPrincipalId
    }
}

module roleAssignments './modules/role-assignments.bicep' = {
    name: 'roleAssignmentsDeployment'
    scope: resourceGroup
    params: {
        aiDevelopersGroupPrincipalId: aiDevelopersGroupPrincipalId
        openAiServiceName: openAi.outputs.openAiServiceName
    }
}

output openAiEndpoint string = openAi.outputs.openAiEndpoint
output dbConnectionString string = 'SERVER=${database.outputs.sqlServerName}${environment().suffixes.sqlServerHostname};DATABASE=${database.outputs.sqlDatabaseName}'

One thing to note is the deployment of the gpt-4o-mini model. The capacity attribute is the Rate Limit in tokens per minute (in this case 500 means 500,000 tokens per minute). You may have to adjust this depending on your subscription type or region (the deployment will throw a self-explanatory error if it’s the case).

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

resource modelGpt4oMini 'Microsoft.CognitiveServices/accounts/deployments@2024-10-01' = {
    name: 'gpt-4o-mini'
    parent: openAiService
    sku: {
        name: 'GlobalStandard'
        capacity: 500
    }
    properties: {
        model: {
            format: 'OpenAI'
            name: 'gpt-4o-mini'
            version: '2024-07-18'
        }
    }
}

Deployment

That being said, let’s start the deployment by running this in the terminal from the project root folder:

bash infra/deploy.sh {your_project_name} swedencentral

The first positional parameter is the name of the project – It’s totally arbitrary and will be used to name the resource group and the subsequent resources (lowercase alphanumeric characters only). I’ve used oaitoolsample in my case.

The second parameter is the target region. I am using the Sweden Central region (as they tend to be on the bleeding edge of feature previews) but feel free to change it.

After a tense few minutes, we can see the result:

Adding users to Entra Group

We have created an Entra Group called AI Developers that has now access to resources. We want to add users to this group to allow them to use the resources, starting with ourselves (even if you are the owner of the subscription).

We could script this operation, but mostly by lack of time, I did it manually in the portal, from the Microsoft Entra ID section, then the Groups section. From there, either in the All groups section or by typing AI Developers, you can find and access the group details. Then navigate to Members and you can now add your user to the group.

Running the tests

With the environment being setup and the resources deployed, we are now ready to run our tests. There are exactly 50 at the time of writing and they are a mix of unit and integration tests.

I have setup the project to run them directly from the Visual Studio Code Testing pane, settings.json allow us to configure how to run them.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

{
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.cwd": "${workspaceFolder}/app",
    "python.testing.pytestArgs": [
        "tests",
        "-rP",
    ],
    "python.defaultInterpreterPath": "${workspaceFolder}/app/.venv/bin/python",
    "python.analysis.extraPaths": [
        "${workspaceFolder}/app"
    ],
    "pylint.args": [
        "--extension-pkg-whitelist=pyodbc"
    ]
}

We can now use the rich Visual Studio Code testing integration. Start by opening the Testing pane (red arrow – If it doesn’t appear yet, open any Python file from the project) then click the Run Tests icon (orange arrow).

What’s next

Thank you for reading. In the next part, we will run interact with our Agent and explore how it works in details.