Foundry is designed to make model selection more structured and less guesswork. Instead of starting from the model list, you start from your use case and work backward. Here’s how it helps you choose: 1. **Start with your use case** Foundry encourages you to define what you’re trying to achieve first: - Is it real-time conversation, predictive analytics, content generation, or computer vision? - Is it high-stakes (like healthcare diagnostics or fraud detection) or lower-risk experimentation? - Do you need fast responses, very high accuracy, or strict cost control? Your answers shape decisions about model type, size, and deployment. 2. **Discover and filter models in one catalog** Foundry offers access to **more than 11,000 AI models**, including foundation, frontier, open-source, and bring-your-own models. In the Model Catalog you can: - Filter by **task**, **provider**, and **capability** - Compare **model and service costs** - See options from providers like OpenAI, Mistral, Cohere, Meta, xAI, NVIDIA, and others 3. **Map tasks to models** You frame your needs in terms of capabilities: - Inputs: natural language, audio, images, multimodal, etc. - Outputs: summaries, decisions, predictions, transcriptions, code, and more. - Complexity: simple classification vs. complex reasoning and multi-step workflows. This helps you quickly eliminate models that don’t fit functionally, before you worry about cost or infrastructure. 4. **Benchmark and prototype in a controlled environment** Foundry lets you benchmark and test models side by side using: - **Standard benchmarks** (intelligence, reasoning, code generation) - **Domain-specific benchmarks** (healthcare, finance, ecommerce, security, etc.) - Your **own data** for realistic evaluation You can use evaluators for: - Ground truth and correctness (exact match, F1, precision, recall) - Safety and risk (bias, hallucinations, fairness) - Semantic quality (text similarity for summarization, translation, and generation) 5. **Iterate and optimize, not just pick once** Model selection is treated as an **iterative process**: - Prototype with a flagship or general-purpose model (for example, GPT-4o series in Azure OpenAI) to test feasibility. - Fine-tune or use retrieval-augmented generation (RAG) with your data to improve accuracy. - Use A/B testing and load testing to see how models behave under real workloads. - Switch models without rewriting your entire application. 6. **Use the Foundry model router over time** Foundry’s model router can help you: - Route queries to different models based on complexity, cost, or performance. - Maintain the best-performing mix of models as new options appear. In practice, this means you can start with a strong, stable model from a leading provider, validate it against your own metrics, and then refine or swap models as your requirements evolve—without starting from scratch each time.

Foundry encourages you to treat performance, cost, and resource usage as a set of trade-offs that you manage deliberately rather than in isolation. Here are the main dimensions to consider: 1. **Performance and accuracy** Define what “good enough” looks like before you choose a model: - Latency and throughput: how fast responses need to be and how many requests you must support. - Accuracy, precision, recall, and exact match (EM) scores: how correct the model must be. - Safety and reliability: tolerance for hallucinations, bias, or inconsistent answers. In Foundry you can: - Use **general-purpose, RAG, and agent evaluators** to assess quality and safety. - Run **load tests** to see how performance holds up under production-like traffic. - Use **A/B testing** to compare different models or versions on your own scenarios. 2. **Resource management and deployment environment** Your infrastructure constraints strongly influence model choice: - **Cloud deployments** can support large foundation or frontier models. - **Edge, mobile, or offline scenarios** often need smaller models, such as small language models (SLMs) or ultra-lightweight nanomodels that can run on microcontrollers. - **On-device models** help keep data local for privacy and latency reasons. Foundry best practices include: - Checking **compute requirements, memory footprint, and loading times** before committing. - Using Foundry to **monitor and optimize** compute usage over time. - Starting with **GitHub models** for low-cost early testing, then moving to Foundry for secure, scalable production. - Using **Foundry Local** models (for example, the Microsoft Phi family) for on-device inference, and scaling to hybrid deployments with Azure Arc when you need centralized management. 3. **Cost and pricing models** Cost is more than just per-token pricing. You need to consider: - Inference costs per request and **cost per inference at scale**. - Training and fine-tuning costs when using your own data. - Infrastructure and supporting services (storage, networking, orchestration, plug-ins, multi-agent setups). Foundry supports multiple pricing options, including: - **Token-based Pay-As-You-Go** - **Provisioned Throughput Units (PTUs)** for capacity reservations - **Standard, Global Standard, and Regional** provisioned throughput tiers - **1-month and 1-year reservations**, where some customers have seen **up to 70% savings** with annual PTU reservations You can also use **Microsoft Cost Management** to monitor spending trends and identify overspending. 4. **Practical cost strategy** A common approach is: - Start with **open-source models** to prototype a minimum viable product with lower risk. - Move to **pretrained foundation models** from leading providers when you need reliability and scale. - Use **serverless or autoscaling infrastructure** to match capacity to demand. - Reserve capacity for predictable workloads to reduce unit costs. 5. **Flexibility and adaptability to optimize over time** To keep improving cost-performance balance, look for models that support: - Fine-tuning, RAG, LoRA, adapters, and prompt tuning. - Quantization, pruning, or distillation to reduce resource usage with minimal accuracy loss. - Simple deployment as API endpoints and compatibility with your existing stack. By combining these practices in Foundry, you can right-size your models: powerful enough to meet your performance and accuracy needs, but efficient enough to stay within budget and infrastructure limits.

Foundry is built to help you design AI solutions that are not only capable, but also safe, compliant, and reliable over time—especially important in regulated or high-stakes environments. Here’s how it supports you across those dimensions: 1. **Safety and responsible AI** Foundry is aligned with Microsoft’s Responsible AI principles, covering security, safety, privacy, fairness, transparency, and accountability. In practice, this means you can: - Use **risk and safety evaluators** to detect bias, hallucinations, and fairness issues in model outputs. - Configure how models handle sensitive or personal data using **Responsible AI tools**. - Continuously monitor AI behavior across the lifecycle using Foundry’s unified toolchain. 2. **Compliance and data protection** Foundry includes built-in security features and supports compliance with major regulations such as: - **GDPR** (General Data Protection Regulation) - **HIPAA** (Health Insurance Portability and Accountability Act) You can: - Choose models and deployment modes that align with **regional and regulatory requirements**. - Use **local inferencing** with open or licensed models when you need data to stay on-premises or on-device. - Extend secure deployment using **Azure Arc** or **Azure Kubernetes Service (AKS)** for private infrastructure. 3. **End-to-end governance and observability** Foundry supports governance across the full AI lifecycle: - Controls and checkpoints from experimentation through deployment. - Monitoring of performance, safety, and cost in one environment. - Integration with CI/CD pipelines so you can update models in a controlled, auditable way. 4. **Consistency vs. innovation** Many organizations need to balance stability with the desire to adopt newer, more capable models. Foundry helps you manage that balance by: - Letting you choose models backed by strong research and ongoing improvements, so your apps can benefit as models evolve. - Encouraging you to check how often a model is updated, what versioning and rollback policies exist, and how changes are communicated. - Highlighting that models sold directly by Azure typically offer **enhanced integration, optimized performance, and direct Microsoft support**, which can contribute to more predictable behavior. For use cases that demand long-term consistency—such as customer support scripts, compliance workflows, or clinical decision support—you can: - Select models designed for stability and reliability. - Lock in specific versions and manage updates through your CI/CD and governance processes. 5. **Real-world examples of safe, reliable use** The e-book highlights customers using Foundry and Azure AI in sensitive or business-critical contexts, such as: - **Mars Science Diagnostics**, which uses Mistral models via the Azure AI catalog to automate veterinary diagnostics, with radiologists validating outputs before production use. - **DraftWise**, which uses the Cohere model family through Foundry to build a legal platform tailored to contract work, reporting a **60% improvement in developer efficiency** over traditional methods. By combining responsible AI tooling, compliance-ready infrastructure, and strong governance practices, Foundry gives you a framework to build AI solutions that can scale, stay compliant, and remain dependable as your organization and regulations evolve.