AI-300: Implementing Machine Learning and Generative AI Solutions

• Prepare data to use AutoML for classification.
• Set up and run an AutoML experiment.
• Evaluate and compare AutoML models.
• Configure MLflow for tracking models in notebooks.
• Use MLflow for tracking models in notebooks.
• Evaluate a trained model using the Responsible AI dashboard

Perform hyperparameter tuning with Azure Machine Learning

• Define a hyperparameter search space.
• Configure hyperparameter sampling.
• Select an early shutdown strategy.
• Run a scan job.

Run pipelines in Azure Machine Learning

• Create components.
• Build an Azure Machine Learning pipeline.
• Run an Azure Machine Learning pipeline.

Trigger Azure Machine Learning jobs with GitHub Actions

• Create a service principal and assign it the necessary permissions to run an Azure Machine Learning job
• Store Azure credentials securely by using secrets in GitHub Secrets.
• Create a GitHub action using a YAML file using stored Azure credentials to run an Azure Machine Learning job

Trigger GitHub Actions with feature-based development

• Use feature-based development.
• Protect the primary branch.
• Trigger a GitHub Actions workflow by merging a pull request

Use environments in GitHub Actions

• Configure environments in GitHub.
• Use environments in GitHub Actions.
• Add approval steps to define the necessary reviewers before routing the model to the next environment.

Deploy a template with GitHub Actions

• Deploy a template to a managed endpoint.
• Trigger template deployment with GitHub Actions.
• Test the deployed model.

Plan and prepare a GenAIOps solution

• Identify use cases for generative AI applications.
• Select a model for your Generative AI application.
• Describe what GenAIOps is and how it defines the application lifecycle.

Manage agent prompts in Microsoft Foundry with GitHub

• Apply versioning principles to manage prompts as code resources.
• Learn how prompts integrate with Microsoft Foundry agents and versioning policies.
• Design a GitHub repository structure to request version control and collaboration.
• Develop a workflow to test and deploy commands securely.

Evaluate and optimize AI agents through structured experiences

• Design evaluation experiences with clear metrics for quality, cost, and performance
• Apply Git workflows to systematically organize and compare agent variants
• Create assessment rubrics that ensure consistent scoring across human graders
• Compare the results of the experiment to make evidence-based optimization decisions

Automate AI assessments with Microsoft Foundry and GitHub Actions

• Explain why automated assessments complement human assessments in AI quality assurance.
• Select reviewers who align with the human evaluation criteria for validation.
• Create assessment datasets with appropriate composition for full tests.
• Implement batch evaluations using Python scripts with Microsoft Foundry.
• Integrate automated assessment workflows into GitHub Actions for continuous testing.

Monitor your generative AI application

• Understand why monitoring is critical when moving Gen AI applications to production readiness.
• Identify and interpret key performance metrics: latency, throughput, token utilization, and error rate.
• Use Azure Monitor with Microsoft Foundry to observe and analyze application behavior.
• Apply insights to optimize performance, cost, and user experience in Gen AI solutions.

Analyze and debug your generative AI application with tracing

• Set up the tracking infrastructure with Microsoft Foundry and Application Insights.
• Implement custom ranges for AI model calls and business logic operations.
• Analyze trace data to identify bottlenecks and performance failure patterns.

Published on 04/16/2026