AWS Certified Machine Learning Engineer - Associate

Practice Exam Questions & Study Guide

Section 1 of this course is free Notes for every focus area and practice questions. Create a free account, no card needed.
No matching topics. Try a different term.
1

Data Preparation for ML and AIFree

Covers collecting and storing data, performing data transformation and feature engineering, and validating data quality and managing bias for ML and AI workloads. (28% of scored content)

1.1: Collect and store data

3 Question Sets Available

Covers extracting, ingesting, storing, and merging data from various sources and formats for ML and AI applications.

Focus Areas:
  • Extract data from data sources (for example, Amazon S3, Amazon EBS, Amazon EFS, Amazon RDS, Amazon DynamoDB, Amazon OpenSearch Service)
  • Make storage decisions and configure storage services based on cost, performance, data structure, and data compliance
  • Troubleshoot and debug data ingestion and storage issues that involve capacity and scalability
  • Use AWS streaming data sources to ingest data (for example, by using Amazon Kinesis, Apache Flink, Apache Kafka)
  • Ingest from and write by using appropriate data formats (for example, Apache Parquet, JSON, CSV, ORC) based on data access patterns
  • Merge data from multiple sources (for example, by using programming techniques, AWS Glue, Apache Spark)
  • Configure scalable vector databases for AI applications (for example, OpenSearch Service, Amazon RDS with pgvector, Amazon S3) based on specifications
  • Ingest and store diverse data types (for example, text, images, audio) for AI and ML applications
  • Ingest data into SageMaker Feature Store

1.2: Perform data transformation, feature engineering, and pre-processing

1 Question Sets Available

Covers transforming data, engineering features, and preparing data for ML models and AI applications including RAG and FM fine-tuning.

Focus Areas:
  • Transform data by using AWS tools (for example, AWS Glue, AWS Glue DataBrew, Spark on Amazon EMR, SageMaker Data Wrangler)
  • Create and manage features by using AWS tools (for example, SageMaker Feature Store)
  • Transform streaming data (for example, by using AWS Lambda, Spark)
  • Perform feature engineering (for example, scaling, standardization, feature splitting, binning, log transformation, normalization)
  • Configure and use embedding models to transform text and image data into numerical representations
  • Apply advanced text pre-processing techniques (for example, tokenization, domain-specific augmentation)
  • Prepare documents for Retrieval Augmented Generation (RAG) applications (for example, chunking strategies, metadata extraction)
  • Mask, redact, and anonymize data
  • Prepare data for FM fine-tuning, continuous pre-training, and model distillation

1.3: Validate data quality and manage bias

1 Question Sets Available

Covers validating data quality, labeling data, identifying and mitigating bias, and cleaning datasets for ML and AI training.

Focus Areas:
  • Validate data quality (for example, by using DataBrew, AWS Glue Data Quality)
  • Label and annotate data
  • Identify and mitigate sources of bias in data by using AWS tools and techniques (for example, dataset splitting, shuffling, augmentation)
  • Optimize multimodal data distributions by applying bias metrics across numeric, text, and image assets
  • Resolve class imbalance in numeric, text, and image datasets
  • Validate AI training data integrity (for example, prompt-response pair validation, content safety screening)
  • Clean data (for example, by detecting outliers, imputing missing data, deduplication)
2

ML Model and Foundation Model (FM) Development

Covers choosing modeling approaches, training and fine-tuning models, and analyzing and evaluating the performance of ML and AI systems. (24% of scored content)

2.1: Choose appropriate modeling approaches for ML and AI solutions

1 Question Sets Available

Covers evaluating and selecting appropriate ML models, FMs, and AI architectures based on business requirements and performance tradeoffs.

Focus Areas:
  • Evaluate and select appropriate FMs from Amazon Bedrock based on task requirements and performance criteria
  • Identify fine-tuning strategies for pre-trained FMs to meet business needs
  • Compare and select appropriate ML models, generative AI (GenAI) models, algorithms, and solution templates to meet business needs (for example, interpretability, domain-specific performance, latency)
  • Evaluate tradeoffs between custom solutions, managed services, pre-trained models, and FMs to meet business needs
  • Select Retrieval Augmented Generation (RAG) architecture patterns based on use case requirements
  • Assess tradeoffs between ML model performance, training time, and cost
  • Assess tradeoffs between AI model performance, latency, and cost
  • Apply AWS AI services to solve specific business problems (for example, Amazon Textract, Amazon Rekognition, Amazon Comprehend, Amazon Transcribe)

2.2: Train, fine-tune, and customize models for ML and AI solutions

1 Question Sets Available

Covers applying SageMaker AI algorithms, configuring training, optimizing hyperparameters, and customizing AI solutions.

Focus Areas:
  • Apply Amazon SageMaker AI built-in algorithms and common ML libraries
  • Configure SageMaker AI script mode with supported frameworks for simplicity and performance
  • Implement hyperparameter optimization (for example, SageMaker AI automatic model tuning [AMT])
  • Implement training time reduction techniques (for example, early stopping, distributed training)
  • Prevent model overfitting, underfitting, and catastrophic forgetting
  • Combine multiple ML models to improve performance or reduce cost
  • Adjust fundamental hyperparameters (for example, epoch, steps, batch size)
  • Apply customization techniques for AI solutions (for example, task-specific prompt engineering, fine-tuning)
  • Optimize retrieval components and embedding models

2.3: Analyze and evaluate the performance of ML and AI systems

1 Question Sets Available

Covers running reproducible experiments, evaluating model performance, detecting drift, and applying evaluation metrics for traditional ML and GenAI models.

Focus Areas:
  • Perform reproducible experiments (for example, by using MLflow on SageMaker AI, Amazon Bedrock evaluations, Amazon Bedrock Prompt Management)
  • Create model performance baselines and implement drift detection
  • Compare the performance of shadow variants to production variants
  • Explain model outputs
  • Debug model convergence issues
  • Apply comprehensive model evaluation techniques for traditional ML and GenAI models
  • Implement integrated human evaluation frameworks (for example, human-in-the-loop workflows, text generation quality assessment)
  • Apply natural language processing (NLP) evaluation metrics (for example, bilingual evaluation understudy [BLEU], Recall-Oriented Understudy for Gisting Evaluation [ROUGE], BERTScore, semantic similarity)
  • Perform AI evaluation (for example, model output assessment, content quality validation, bias detection, LLM-as-a-judge frameworks)
  • Configure RAG system monitoring, including retrieval accuracy assessment
3

Deployment and Orchestration of ML and AI Workflows

Covers managing deployment infrastructure, provisioning and configuring resources, and implementing automated CI/CD pipelines for MLOps and AI workloads. (24% of scored content)

3.1: Manage deployment infrastructure for ML and AI model types

1 Question Sets Available

Covers selecting compute environments, deployment strategies, inference strategies, and deploying and configuring models and agents in AWS environments.

Focus Areas:
  • Select appropriate compute environments and deployment targets
  • Select deployment orchestrators and multi-model or multi-container deployment strategies
  • Select model inference strategies (for example, real-time and batch processing)
  • Evaluate and select appropriate foundation model (FM) deployment options
  • Deploy models that were built outside of AWS into AWS environments (for example, Amazon SageMaker AI, Amazon Bedrock Custom Model Import)
  • Deploy and configure agents for specific tasks, integration with other services and tools, and agent communication protocols
  • Configure FM deployment, model hosting, and resource allocation
  • Apply Retrieval Augmented Generation (RAG) system configurations (for example, retrieval strategies, reranking)

3.2: Provision and configure resources for ML and AI workloads based on existing architecture and requirements

1 Question Sets Available

Covers optimizing resource provisioning, building containers, configuring endpoints, implementing auto scaling, and deploying agentic workflow infrastructure.

Focus Areas:
  • Optimize resource provisioning between on-demand and provisioned resources for performance and cost efficiency
  • Automate compute resource provisioning with integrated communication between stacks and orchestration services
  • Build and maintain containers for ML and AI workloads
  • Configure SageMaker AI endpoints within VPC network environments
  • Deploy and host models programmatically (for example, by using the SageMaker AI SDK for Python, AWS CLI, Boto3)
  • Select specific metrics for auto scaling implementations
  • Create and manage Amazon Bedrock knowledge bases with vector database configurations, document indexing, and retrieval optimization
  • Implement retrieval pipelines to meet business needs
  • Implement agent state management systems
  • Implement AI-specific resource scaling for GPU workloads
  • Deploy agentic workflow infrastructure

3.3: Implement automated orchestration and continuous integration and continuous delivery (CI-CD) pipelines for MLOps and AI workloads

1 Question Sets Available

Covers implementing automated deployment strategies, configuring CI/CD tools, managing model versions, and building agent deployment pipelines for ML and AI workloads.

Focus Areas:
  • Implement automated deployment strategies and rollback actions
  • Configure and troubleshoot AWS CodeBuild; AWS CodeCommit; AWS CodeDeploy; AWS CodePipeline, and AWS CodeConnections
  • Configure training and inference jobs
  • Configure automated testing strategies within CI/CD pipelines for traditional ML and AI workloads
  • Build and integrate mechanisms to re-train models
  • Manage model versions for repeatability and audits (for example, SageMaker Model Registry, MLflow on SageMaker AI)
  • Manage prompts (for example, Amazon Bedrock Prompt Management)
  • Implement automated agent deployment pipelines and agent version management
  • Implement AI model testing frameworks, including prompt testing
  • Configure FM deployment automation with fine-tuned model versioning
  • Configure AI-specific pipeline orchestration for RAG system updates and knowledge base refresh cycles
4

Operating, Monitoring, and Securing ML and AI Solutions

Covers monitoring model inference and performance, optimizing infrastructure costs, and securing ML and AI workloads and endpoints. (24% of scored content)

4.1: Monitor ML and AI model inference and performance

1 Question Sets Available

Covers monitoring model performance in production, detecting data drift and anomalies, and managing agent performance for ML and AI systems.

Focus Areas:
  • Monitor model performance in production by using Amazon CloudWatch generative AI observability; Amazon Bedrock Model Evaluation, and drift detection pipelines
  • Monitor workflows to detect anomalies or errors in data processing or model inference
  • Detect changes in data distribution that can affect model performance
  • Monitor model performance in production by using A/B testing
  • Monitor and automate the management of agent performance and coordination (for example, coordination failure detection, truncated streaming, tool failures)
  • Configure AI-specific performance monitoring for foundation models (FMs), such as Amazon Bedrock evaluations

4.2: Optimize and manage ML and AI infrastructure costs and performance

1 Question Sets Available

Covers selecting instance families, using monitoring tools, optimizing capacity and costs, and managing FM inference costs for ML and AI infrastructure.

Focus Areas:
  • Select inference instance families to optimize performance and cost
  • Configure and use tools to troubleshoot and analyze resources (for example, Amazon CloudWatch, Amazon Bedrock AgentCore Observability, AWS X-Ray)
  • Set up dashboards to monitor performance metrics
  • Optimize capacity for cost, performance, and reliability
  • Optimize costs and set cost quotas by using appropriate cost management tools
  • Optimize infrastructure costs by selecting purchasing options
  • Evaluate cost implications of using FMs for inference in production
  • Monitor agent resource consumption patterns
  • Manage FM inference costs with usage optimization
  • Monitor AI-specific cost patterns (for example, token usage optimization, embedding computation costs, vector database storage optimization)

4.3: Secure ML and AI workloads and model endpoints

1 Question Sets Available

Covers securing CI/CD pipelines, configuring access controls, implementing monitoring and compliance, and implementing safeguards for ML and AI systems.

Focus Areas:
  • Secure continuous integration and continuous delivery (CI/CD) pipelines by checking for code and image vulnerabilities (for example, by using Amazon CodeGuru, Amazon Inspector)
  • Configure least privilege access to ML and AI artifacts
  • Configure IAM policies and roles for users and applications in ML and AI systems
  • Configure comprehensive monitoring, auditing, compliance, and logging for ML and AI systems (for example, AWS CloudTrail, AWS Config)
  • Troubleshoot and debug security issues in ML and AI systems
  • Create VPCs, subnets, and security groups to securely isolate ML and AI systems
  • Identify and mitigate security risks and vulnerabilities in ML and AI systems
  • Select the appropriate credential type to access FMs (for example, Amazon Bedrock API keys, IAM credentials)
  • Implement safeguards and sensitive data protection to meet application requirements and responsible AI policies (for example, by using Amazon Bedrock Guardrails)