What adding AI to an existing ASP.NET Core application actually involves - integration patterns, Microsoft Agent Framework, Azure OpenAI, and what to expect.
What adding AI to an existing ASP.NET Core application actually involves - integration patterns, Microsoft Agent Framework, Azure OpenAI, and what to expect.
If you have an existing ASP.NET Core application and you are evaluating whether to add AI features, the first thing worth getting clear on is what "adding AI" actually means in a .NET context in 2026 - because the vendor descriptions are vague enough to mean almost anything, and the gap between "we integrated ChatGPT" and "we built a production AI feature that works reliably with your business data" is significant.
The short version: adding AI to an existing ASP.NET Core application is a software integration project, not an AI research project. You are not training models, running experiments, or building anything from scratch that resembles an AI lab. You are adding a new capability to an application that already works - using Microsoft's current production AI stack to make the application smarter about specific, defined tasks. The AI part is the model (accessed via Azure OpenAI API). The engineering part - which is most of the work - is connecting that model to your existing data, your existing authentication, and your existing workflows in a way that's reliable, secure, and maintainable.
This guide explains what that engineering work actually involves, what patterns it follows, and what decisions you'll need to make before a development team can give you an accurate scope.
When a development team says they are adding AI to your ASP.NET Core application, they are doing one or more of these three things. Understanding which one applies to your use case clarifies what the work involves.
Option 1: Adding an AI-powered feature to your existing application
A new feature - a document search, a chatbot, a summarization button, a classification step - that uses an LLM (large language model) to do something your application couldn't do before. The AI feature sits alongside your existing features. Your application structure doesn't change. A new service class, a new set of API endpoints, a new UI component. The integration work connects the AI service to your existing database, authentication, and business logic.
This is the most common scenario for established ASP.NET Core applications and the one this guide focuses on.
Option 2: Adding AI to an existing workflow
An existing process in your application - invoice approval, support ticket routing, new user onboarding - gains an AI step that handles part of the process automatically. The workflow still exists; it just has an intelligent step inserted. A human reviews invoices: the AI now pre-classifies them and extracts the key fields before the human sees them. A support ticket comes in: the AI categorizes it and suggests an answer before a human responds.
This is AI augmenting a process rather than replacing a feature, and it's the dominant pattern in operations-heavy businesses adding AI in 2026.
Option 3: Replacing an existing feature with a better AI-powered version
Keyword search replaced with semantic search. A rule-based chatbot replaced with an LLM-based one. A manual data extraction step replaced with document intelligence. The feature already exists; the AI version does it better. The integration challenge here is matching the existing feature's behavior closely enough that existing users aren't disrupted while the AI version handles edge cases the rule-based version couldn't.
Understanding the tools clarifies why AI integration is an engineering project rather than an API connection. The Microsoft AI stack for ASP.NET Core developers in 2026 has three layers:
Layer 1 - Microsoft.Extensions.AI
The lowest-level abstraction. Provider-agnostic interfaces for calling LLMs through ASP.NET Core's standard dependency injection system. You register an AI client in Program.cs the same way you register a database context. Code that calls IChatClient works whether the underlying model is Azure OpenAI GPT-4o, a local Ollama model, or anything else. This is the entry point for simple AI features - single-turn completions, text classification, basic summarization.
// Registering an AI client in Program.cs
builder.Services.AddAzureOpenAIChatClient(
deploymentName: "gpt-4o",
endpoint: new Uri(builder.Configuration["AzureOpenAI:Endpoint"]!),
credential: new DefaultAzureCredential());
// Using it in a service - exactly like any other injected dependency
public class DocumentSummaryService(IChatClient chatClient)
{
public async Task<string> SummarizeAsync(string documentText)
{
var response = await chatClient.CompleteAsync(
$"Summarize this document concisely:\n\n{documentText}");
return response.Message.Text;
}
}Layer 2 - Microsoft Agent Framework 1.0
The orchestration layer. Released April 2026 as the production successor to Semantic Kernel and AutoGen. Handles everything that goes beyond a single AI call: managing conversation history, defining and calling tools/plugins (functions that give the AI access to your business data and APIs), orchestrating multi-step reasoning, and coordinating multiple specialized agents for complex workflows.
This is what your integration looks like when the AI feature needs to do more than generate text - when it needs to look up a customer record, call an internal API, check a database value, and then respond based on what it found.
// Registering Microsoft Agent Framework in Program.cs
builder.Services.AddAgentFramework(options =>
{
options.AddAzureOpenAI(config =>
{
config.DeploymentName = "gpt-4o";
config.Endpoint = builder.Configuration["AzureOpenAI:Endpoint"]!;
});
});
// A plugin that gives the AI access to your existing data
public class CustomerPlugin(ICustomerRepository customerRepo)
{
[KernelFunction]
[Description("Gets customer account details by customer ID")]
public async Task<CustomerDto?> GetCustomerAsync(int customerId)
=> await customerRepo.GetByIdAsync(customerId);
[KernelFunction]
[Description("Gets a customer's recent orders")]
public async Task<IEnumerable<OrderDto>> GetRecentOrdersAsync(
int customerId, int count = 5)
=> await customerRepo.GetRecentOrdersAsync(customerId, count);
}Layer 3 - Azure AI Search (for RAG features)
The vector database layer. When your AI feature needs to answer questions from your documents or provide semantically relevant results, the documents are converted to embeddings and stored in Azure AI Search. At query time, the user's question is also converted to an embedding, and the most relevant document chunks are retrieved and provided to the LLM as context. This is Retrieval-Augmented Generation (RAG) - the pattern behind document Q&A, knowledge base search, and any AI feature that needs to reason about content stored in your system.
Regardless of which feature type you are adding, the integration architecture in an ASP.NET Core application follows a consistent pattern. Here's what gets built and where it fits in your existing application.
The AI service layer
A new service class (or set of service classes) that encapsulates all AI logic. This service is registered in ASP.NET Core's DI container and injected wherever it's needed - into controllers, background jobs, or existing services. The AI logic doesn't leak into the rest of the application any more than your database access logic does.
public interface IDocumentQAService
{
Task<QAResponse> AskQuestionAsync(string question, string userId);
Task IndexDocumentAsync(Stream documentStream, string documentName);
}
// The implementation wires together Azure AI Search + Azure OpenAI
// Your controllers just call IDocumentQAService.AskQuestionAsync()The plugin layer (for agent-based features)
Each data source or action the AI agent can access becomes a plugin - a C# class with methods annotated to tell the AI what they do and when to call them. Your existing repository classes and service classes often become plugins with minimal changes - you are adding metadata (descriptions that tell the AI what the method does) rather than rewriting logic.
The API endpoints
New ASP.NET Core endpoints (Minimal API or controller-based) that your frontend calls to interact with the AI feature. These look identical to any other endpoint in your application - they go through your existing authentication middleware, your existing rate limiting, your existing error handling. The AI is just another service being called in the endpoint handler.
The data pipeline (for RAG features)
A background process that takes your documents, converts them to embeddings via the Azure OpenAI embeddings API, and stores them in Azure AI Search. This runs on initial document upload and whenever documents are updated. For most ASP.NET Core applications this is a Hangfire background job or an Azure Function triggered by blob storage events.
Before a development team can accurately scope an AI integration, several things about your existing ASP.NET Core application determine how straightforward or complex the integration will be.
Authentication and authorization you can enforce on AI features. The AI feature should respect the same access rules as the rest of your application. If a user can only see their own records, the AI feature should only answer questions about their records. If your authentication is through ASP.NET Core Identity, Azure AD, or a JWT-based system, this is typically straightforward - the AI service receives the authenticated user's identity and applies the same data access rules as every other service. If your authentication is unusual (session-based, custom token schemes), this needs assessment.
APIs or repositories for your data that the AI can call. For chatbot and agent features, the AI needs to access your data through defined methods - not by running arbitrary SQL against your database. If your ASP.NET Core application has a service layer with typed repository interfaces, the AI integration has clean extension points. If business logic is embedded in stored procedures with no application-layer abstraction, building the AI integration also involves building that abstraction layer first.
Documents in a processable format (for Q&A features). Azure Document Intelligence can extract text from PDFs, Word documents, Excel files, and HTML. Scanned PDFs (images of text rather than digital text) require OCR and produce lower-quality extractions. If your documents are scanned, handwritten, or in unusual formats, this adds both cost and complexity to the document processing pipeline.
A reasonable data quality baseline. AI features that summarize, classify, or answer questions about your data can only be as good as the data they work with. If your database has significant data quality issues - missing fields, inconsistent formats, duplicate records - the AI feature will surface those inconsistencies in its responses. Data preparation isn't always required, but it needs to be assessed.
Before a development team can scope an AI integration accurately, three decisions need to be made. These aren't technical decisions - they are product decisions that determine what gets built.
Decision 1: What is the specific feature, and how does success get measured?
"Add AI to our application" is not a feature. "Add a document Q&A feature that lets users ask questions about their uploaded contracts and get answers with source citations, with a target of answering 80% of questions accurately based on a test set of 50 representative questions" is a feature. The difference matters because the second version can be scoped, costed, and tested. The first version can't. If you can't define what "good" looks like for the AI feature before development starts, the scope will expand during development in ways that are difficult to manage.
Decision 2: Where does the data come from, and is it ready?
If the AI feature needs to know about your customers, orders, products, or documents - where is that data, what format is it in, and is there an existing API or repository method to access it? A chatbot that needs to answer questions about a customer's account needs to be able to look up that account data. If the lookup mechanism doesn't exist yet, it needs to be built before the AI can use it.
Decision 3: What happens when the AI is wrong?
AI features produce incorrect outputs. GPT-4o is highly capable but not infallible. A document Q&A system will occasionally give an incorrect answer. A chatbot will occasionally misunderstand a question. A classification step will occasionally misclassify an item. Before building the feature, decide: what happens in those cases? Does the user see a confidence score? Is there a "I'm not sure - here's how to find a human" fallback? Is there a review step before AI output becomes a system record? These decisions affect the architecture and need to be made before the design phase, not after.
A typical AI integration engagement follows this sequence regardless of which feature type is being built:
Discovery (Weeks 1–2): We review your existing ASP.NET Core codebase, your data architecture, and your existing authentication. We identify the integration points, assess data readiness, and validate that the planned feature is technically feasible with your current application structure. Discovery produces a scoped proposal with a fixed price.
Architecture design (Week 2–3): The integration architecture is designed in detail - the service layer structure, the plugin definitions, the API endpoints, the data pipeline if applicable. This is reviewed with your team before development starts. Changes at this stage cost hours; changes during development cost days.
Development in sprints (Weeks 3–10+): Development runs in two-week sprints. The first sprint typically produces a working proof of concept with real data - the AI feature working end-to-end against your actual application, not a demo. This is the earliest possible point where the integration can be validated against real-world inputs.
Testing and tuning: AI features require a specific type of testing that isn't covered by standard unit tests - evaluation of AI output quality across a representative set of inputs. For document Q&A, this means testing against a set of representative questions with known correct answers. For classification features, this means testing against a labeled sample of items. Prompt engineering adjustments happen during this phase.
Deployment: The AI feature is deployed into your existing Azure infrastructure. It uses your existing App Service or Container Apps deployment, your existing Key Vault for the Azure OpenAI API key, your existing Application Insights for monitoring. The deployment adds new components (an Azure AI Search instance if required, new configuration in App Configuration) but doesn't change your existing deployment process.
Do we need to rebuild our ASP.NET Core application to add AI?
No. AI integration in ASP.NET Core is additive - new services, new API endpoints, new background jobs if required. Your existing controllers, your existing database, your existing authentication, and your existing frontend all remain unchanged. The AI feature integrates with what exists rather than replacing it. The only exception is if your existing application lacks a service/repository layer - if business logic is tightly coupled to the UI or embedded in stored procedures - in which case building that abstraction layer is a prerequisite for clean AI integration.
How long does it take to see a working AI feature?
In a well-run engagement, a working proof of concept - the AI feature operating end-to-end against your actual data, not a demo - is typically deliverable in the first sprint (2 weeks of development), after a discovery phase. The POC will not yet have production-level error handling, UI polish, or full test coverage, but it will answer the question "does this AI integration actually work for our use case" before significant investment has been made.
Can we add AI without using Azure? We use AWS.
Yes. Microsoft.Extensions.AI is provider-agnostic - Azure OpenAI can be swapped for Amazon Bedrock (which hosts Claude Opus 4.8 models, among others) with a configuration change rather than a code change. Azure AI Search has AWS equivalents (Amazon OpenSearch Service with vector search, or Amazon Bedrock Knowledge Bases). The architectural patterns are the same; the specific Azure services are substitutable. That said, for ASP.NET Core applications already on Azure, the native integration between Azure OpenAI, Azure AI Search, Azure Key Vault, and Azure Managed Identity removes significant configuration overhead compared to cross-cloud setups.
What's the difference between using Microsoft Agent Framework and calling Azure OpenAI directly?
Calling Azure OpenAI directly works for single-turn completions - send a prompt, get a response. Microsoft Agent Framework is the layer above that handles everything a production AI feature needs beyond a single API call: conversation history management, plugin/tool calling (letting the AI access your data and APIs), multi-step reasoning where the AI decides what to call and in what order, and the orchestration for multi-agent workflows. For any AI feature more complex than a single-turn text generation, Agent Framework reduces the amount of infrastructure code you maintain and provides a tested, production-supported foundation.
How do we ensure the AI feature doesn't expose one user's data to another?
By enforcing your existing authorization rules in the AI service layer. The same IAuthorizationService and user identity claims used in your existing controllers are used in the AI service. When the AI agent calls a plugin to retrieve data, the plugin method receives the authenticated user's context and applies the same data access rules as every other part of the application. This is a design requirement, not an automatic feature - it needs to be explicitly addressed in the architecture phase. For regulated industries, this is one of the first architectural decisions we document.
What ongoing maintenance does an AI feature require?
Less than you might expect, but some. The main maintenance tasks are: monitoring AI output quality over time (user inputs evolve and prompts sometimes need adjustment), managing Azure OpenAI model version updates (Microsoft periodically retires older model versions with advance notice), monitoring Azure costs as usage scales, and updating the RAG index when your document corpus changes. The Agent Framework SDK releases updates on a regular cadence - staying reasonably current is recommended but not immediately critical. We recommend a light ongoing retainer or periodic review for production AI features rather than assuming they are fully hands-off post-launch.
Adding AI to an existing ASP.NET Core application is well-understood engineering work in 2026 - not research, not experimentation, not a major platform change. The patterns are established, the tools are production-grade, and the integration points in ASP.NET Core's architecture are clean. The questions that need answering before you can get an accurate scope are product questions as much as technical ones: what exactly is the feature, where does the data come from, and what happens when the AI is wrong.
If you are at the stage of evaluating whether and how to add AI to your existing application, our AI integration team can walk through your specific situation in a discovery conversation - no commitment required to understand what the integration would actually involve for your application.
