This path is for developers building an application that calls a model, not for choosing a coding CLI. Keep the model endpoint, credentials, prompt, output contract, tool policy and retrieval data separate so each can be tested.
Work through the path
Open a card when you need it. Marking a lesson complete is optional and stays only in this browser.
1Put the model behind a small API boundary
Keep provider-specific request code, credentials, timeouts and application behavior behind a small interface.
An LLM feature should not scatter provider SDK calls across controllers, jobs and UI code. Define an application boundary such as GenerateAnswer or ClassifyTicket, then keep model selection, authentication, retries and response normalization behind it.
Treat the model response as untrusted external input. Log request metadata and latency without logging user secrets or full private prompts by default. Set timeouts, cancellation and a cost or token budget before production.
Try it
Write a fake ILanguageModel for tests that returns a deterministic response and a failure variant for timeout or provider errors.
public sealed record GenerateRequest(string Input);
public sealed record GenerateResult(string Text, string Model, int? Tokens);
public interface ILanguageModel
{
Task<GenerateResult> GenerateAsync(
GenerateRequest request,
CancellationToken cancellationToken);
}
// Controller/service depends on ILanguageModel,
// not on one provider SDK.2Stream for responsiveness, not for correctness
Send partial model output to the UI while keeping cancellation, buffering, errors and final usage explicit.
Streaming reduces perceived latency because the user sees progress before the complete answer is ready. It does not make the model more accurate, and it makes error handling more visible: a connection can stop after a partial sentence or incomplete JSON.
Choose a transport your client can consume, such as server-sent events or a provider-supported stream. Send typed events for text, status, tool activity, errors and completion rather than making the browser guess from raw text.
Try it
Add a cancel button and verify that disconnecting the browser cancels the upstream request instead of leaving an expensive generation running.
event: token
data: {"text":"A"}
event: token
data: {"text":" short answer"}
event: done
data: {"requestId":"req_123"}
// On disconnect: cancel the upstream request.
// On error: send a safe error event and record details server-side.3Validate structured output at the application boundary
Use a schema and validation step when model output feeds code, a database, a workflow or a user-facing decision.
A request for JSON in a prompt is not the same as a validated application contract. Prefer a provider-supported schema or response format when available, then parse and validate the result with the same discipline you would apply to an HTTP request.
Keep the schema small. On validation failure, record a redacted diagnostic, retry only when the failure is recoverable and provide a safe fallback. Do not silently coerce an unsafe or incomplete value into a production action.
Try it
Add a test for missing fields, an unknown category, confidence outside 0–1 and a response containing extra prose around the JSON.
public sealed record TicketLabel(
string Category,
double Confidence,
string[] Reasons);
var result = await model.GenerateStructuredAsync<TicketLabel>(
prompt, schema, cancellationToken);
if (result.Confidence is < 0 or > 1)
throw new InvalidDataException("Invalid confidence");4Keep tool and function calling under application control
Treat a model tool call as a proposal that your application validates, authorizes, executes and reports back.
Tool calling is a protocol for asking your application to run a named function with arguments. The model does not become authorized merely because it selected a tool. Your code must validate arguments, check the user and resource permissions, apply rate limits and decide whether confirmation is required.
Keep tools narrow and typed. Return the minimum result needed for the next model step, redact secrets and cap the number of iterations. For writes, prefer a preview or approval step before the irreversible operation.
Try it
Design a read-only GetOrderStatus tool, then list the checks you would add before allowing a CancelOrder tool.
var call = await model.NextToolCallAsync(messages, tools, ct);
var args = toolSchema.Validate(call.Arguments);
if (!authorization.CanRun(user, call.Name, args))
return ToolResult.Denied("Approval required");
var output = await toolRegistry.RunAsync(
call.Name, args, timeout, ct);
return await model.ContinueAsync(call, output, ct);5Use embeddings for similarity, not magic search
Turn text into vectors for approximate semantic retrieval, then combine similarity with metadata and normal application filters.
An embedding maps text into a numeric vector where related meanings may be near each other. The vector is not a database of facts and it does not know whether a document is current, authorized or relevant to this tenant.
Chunk documents with enough surrounding meaning, store source metadata and choose a distance metric and index appropriate to the store. Evaluate retrieval with real questions before tuning a similarity threshold.
Try it
Create five realistic questions and label which source chunks are acceptable answers. Measure recall before changing chunk size or top-k.
var chunks = Chunk(document.Text, maxTokens: 400);
foreach (var chunk in chunks)
{
var vector = await embeddings.CreateAsync(chunk.Text, ct);
await index.UpsertAsync(new SearchChunk(
chunk.Id, document.TenantId, chunk.Text,
vector, document.SourceUrl, document.UpdatedAt), ct);
}
// Query: filter tenant and permissions first,
// then rank candidate vectors and inspect the sources.6Build basic RAG as retrieve, cite and verify
Ground a model answer in retrieved passages while preserving source identity, access control and a fallback for “not enough evidence”.
A basic retrieval-augmented generation flow is: authorize the query, retrieve candidate chunks, filter and rerank them, build a bounded context, ask the model to answer only from that context and return citations or an explicit unknown.
Keep retrieval and generation observable as separate stages. When an answer is wrong, you need to know whether the source was missing, the ranking was poor, the context was truncated or the model ignored a good citation.
Try it
Remove the best source from the index and check that the application says it lacks evidence instead of confidently inventing an answer.
var candidates = await index.SearchAsync(
query, tenantId, userPermissions, topK: 8, ct);
var context = SelectWithSources(candidates, maxTokens: 3000);
var answer = await model.GenerateAsync($"""
Answer only from SOURCES. If evidence is missing, say so.
Return source ids with each claim.
SOURCES:
{context}
QUESTION:
{query}
""", ct);
return new Answer(answer.Text, answer.SourceIds);No lesson matches that phrase.
Related tools
Turn the concept into a working example with the browser-based tools.