AWS Certified Machine Learning Engineer - Associate
Practice Exam Questions & Study Guide
Data Preparation for ML and AIFree
Covers collecting and storing data, performing data transformation and feature engineering, and validating data quality and managing bias for ML and AI workloads. (28% of scored content)
1.1: Collect and store data
Covers extracting, ingesting, storing, and merging data from various sources and formats for ML and AI applications.
- Extract data from data sources (for example, Amazon S3, Amazon EBS, Amazon EFS, Amazon RDS, Amazon DynamoDB, Amazon OpenSearch Service)
- Make storage decisions and configure storage services based on cost, performance, data structure, and data compliance
- Troubleshoot and debug data ingestion and storage issues that involve capacity and scalability
- Use AWS streaming data sources to ingest data (for example, by using Amazon Kinesis, Apache Flink, Apache Kafka)
- Ingest from and write by using appropriate data formats (for example, Apache Parquet, JSON, CSV, ORC) based on data access patterns
- Merge data from multiple sources (for example, by using programming techniques, AWS Glue, Apache Spark)
- Configure scalable vector databases for AI applications (for example, OpenSearch Service, Amazon RDS with pgvector, Amazon S3) based on specifications
- Ingest and store diverse data types (for example, text, images, audio) for AI and ML applications
- Ingest data into SageMaker Feature Store
1.2: Perform data transformation, feature engineering, and pre-processing
Covers transforming data, engineering features, and preparing data for ML models and AI applications including RAG and FM fine-tuning.
- Transform data by using AWS tools (for example, AWS Glue, AWS Glue DataBrew, Spark on Amazon EMR, SageMaker Data Wrangler)
- Create and manage features by using AWS tools (for example, SageMaker Feature Store)
- Transform streaming data (for example, by using AWS Lambda, Spark)
- Perform feature engineering (for example, scaling, standardization, feature splitting, binning, log transformation, normalization)
- Configure and use embedding models to transform text and image data into numerical representations
- Apply advanced text pre-processing techniques (for example, tokenization, domain-specific augmentation)
- Prepare documents for Retrieval Augmented Generation (RAG) applications (for example, chunking strategies, metadata extraction)
- Mask, redact, and anonymize data
- Prepare data for FM fine-tuning, continuous pre-training, and model distillation
1.3: Validate data quality and manage bias
Covers validating data quality, labeling data, identifying and mitigating bias, and cleaning datasets for ML and AI training.
- Validate data quality (for example, by using DataBrew, AWS Glue Data Quality)
- Label and annotate data
- Identify and mitigate sources of bias in data by using AWS tools and techniques (for example, dataset splitting, shuffling, augmentation)
- Optimize multimodal data distributions by applying bias metrics across numeric, text, and image assets
- Resolve class imbalance in numeric, text, and image datasets
- Validate AI training data integrity (for example, prompt-response pair validation, content safety screening)
- Clean data (for example, by detecting outliers, imputing missing data, deduplication)
ML Model and Foundation Model (FM) Development
Covers choosing modeling approaches, training and fine-tuning models, and analyzing and evaluating the performance of ML and AI systems. (24% of scored content)
2.1: Choose appropriate modeling approaches for ML and AI solutions
Covers evaluating and selecting appropriate ML models, FMs, and AI architectures based on business requirements and performance tradeoffs.
- Evaluate and select appropriate FMs from Amazon Bedrock based on task requirements and performance criteria
- Identify fine-tuning strategies for pre-trained FMs to meet business needs
- Compare and select appropriate ML models, generative AI (GenAI) models, algorithms, and solution templates to meet business needs (for example, interpretability, domain-specific performance, latency)
- Evaluate tradeoffs between custom solutions, managed services, pre-trained models, and FMs to meet business needs
- Select Retrieval Augmented Generation (RAG) architecture patterns based on use case requirements
- Assess tradeoffs between ML model performance, training time, and cost
- Assess tradeoffs between AI model performance, latency, and cost
- Apply AWS AI services to solve specific business problems (for example, Amazon Textract, Amazon Rekognition, Amazon Comprehend, Amazon Transcribe)
2.2: Train, fine-tune, and customize models for ML and AI solutions
Covers applying SageMaker AI algorithms, configuring training, optimizing hyperparameters, and customizing AI solutions.
- Apply Amazon SageMaker AI built-in algorithms and common ML libraries
- Configure SageMaker AI script mode with supported frameworks for simplicity and performance
- Implement hyperparameter optimization (for example, SageMaker AI automatic model tuning [AMT])
- Implement training time reduction techniques (for example, early stopping, distributed training)
- Prevent model overfitting, underfitting, and catastrophic forgetting
- Combine multiple ML models to improve performance or reduce cost
- Adjust fundamental hyperparameters (for example, epoch, steps, batch size)
- Apply customization techniques for AI solutions (for example, task-specific prompt engineering, fine-tuning)
- Optimize retrieval components and embedding models
2.3: Analyze and evaluate the performance of ML and AI systems
Covers running reproducible experiments, evaluating model performance, detecting drift, and applying evaluation metrics for traditional ML and GenAI models.
- Perform reproducible experiments (for example, by using MLflow on SageMaker AI, Amazon Bedrock evaluations, Amazon Bedrock Prompt Management)
- Create model performance baselines and implement drift detection
- Compare the performance of shadow variants to production variants
- Explain model outputs
- Debug model convergence issues
- Apply comprehensive model evaluation techniques for traditional ML and GenAI models
- Implement integrated human evaluation frameworks (for example, human-in-the-loop workflows, text generation quality assessment)
- Apply natural language processing (NLP) evaluation metrics (for example, bilingual evaluation understudy [BLEU], Recall-Oriented Understudy for Gisting Evaluation [ROUGE], BERTScore, semantic similarity)
- Perform AI evaluation (for example, model output assessment, content quality validation, bias detection, LLM-as-a-judge frameworks)
- Configure RAG system monitoring, including retrieval accuracy assessment
Deployment and Orchestration of ML and AI Workflows
Covers managing deployment infrastructure, provisioning and configuring resources, and implementing automated CI/CD pipelines for MLOps and AI workloads. (24% of scored content)
3.1: Manage deployment infrastructure for ML and AI model types
Covers selecting compute environments, deployment strategies, inference strategies, and deploying and configuring models and agents in AWS environments.
- Select appropriate compute environments and deployment targets
- Select deployment orchestrators and multi-model or multi-container deployment strategies
- Select model inference strategies (for example, real-time and batch processing)
- Evaluate and select appropriate foundation model (FM) deployment options
- Deploy models that were built outside of AWS into AWS environments (for example, Amazon SageMaker AI, Amazon Bedrock Custom Model Import)
- Deploy and configure agents for specific tasks, integration with other services and tools, and agent communication protocols
- Configure FM deployment, model hosting, and resource allocation
- Apply Retrieval Augmented Generation (RAG) system configurations (for example, retrieval strategies, reranking)
3.2: Provision and configure resources for ML and AI workloads based on existing architecture and requirements
Covers optimizing resource provisioning, building containers, configuring endpoints, implementing auto scaling, and deploying agentic workflow infrastructure.
- Optimize resource provisioning between on-demand and provisioned resources for performance and cost efficiency
- Automate compute resource provisioning with integrated communication between stacks and orchestration services
- Build and maintain containers for ML and AI workloads
- Configure SageMaker AI endpoints within VPC network environments
- Deploy and host models programmatically (for example, by using the SageMaker AI SDK for Python, AWS CLI, Boto3)
- Select specific metrics for auto scaling implementations
- Create and manage Amazon Bedrock knowledge bases with vector database configurations, document indexing, and retrieval optimization
- Implement retrieval pipelines to meet business needs
- Implement agent state management systems
- Implement AI-specific resource scaling for GPU workloads
- Deploy agentic workflow infrastructure
3.3: Implement automated orchestration and continuous integration and continuous delivery (CI-CD) pipelines for MLOps and AI workloads
Covers implementing automated deployment strategies, configuring CI/CD tools, managing model versions, and building agent deployment pipelines for ML and AI workloads.
- Implement automated deployment strategies and rollback actions
- Configure and troubleshoot AWS CodeBuild; AWS CodeCommit; AWS CodeDeploy; AWS CodePipeline, and AWS CodeConnections
- Configure training and inference jobs
- Configure automated testing strategies within CI/CD pipelines for traditional ML and AI workloads
- Build and integrate mechanisms to re-train models
- Manage model versions for repeatability and audits (for example, SageMaker Model Registry, MLflow on SageMaker AI)
- Manage prompts (for example, Amazon Bedrock Prompt Management)
- Implement automated agent deployment pipelines and agent version management
- Implement AI model testing frameworks, including prompt testing
- Configure FM deployment automation with fine-tuned model versioning
- Configure AI-specific pipeline orchestration for RAG system updates and knowledge base refresh cycles
Operating, Monitoring, and Securing ML and AI Solutions
Covers monitoring model inference and performance, optimizing infrastructure costs, and securing ML and AI workloads and endpoints. (24% of scored content)
4.1: Monitor ML and AI model inference and performance
Covers monitoring model performance in production, detecting data drift and anomalies, and managing agent performance for ML and AI systems.
- Monitor model performance in production by using Amazon CloudWatch generative AI observability; Amazon Bedrock Model Evaluation, and drift detection pipelines
- Monitor workflows to detect anomalies or errors in data processing or model inference
- Detect changes in data distribution that can affect model performance
- Monitor model performance in production by using A/B testing
- Monitor and automate the management of agent performance and coordination (for example, coordination failure detection, truncated streaming, tool failures)
- Configure AI-specific performance monitoring for foundation models (FMs), such as Amazon Bedrock evaluations
4.2: Optimize and manage ML and AI infrastructure costs and performance
Covers selecting instance families, using monitoring tools, optimizing capacity and costs, and managing FM inference costs for ML and AI infrastructure.
- Select inference instance families to optimize performance and cost
- Configure and use tools to troubleshoot and analyze resources (for example, Amazon CloudWatch, Amazon Bedrock AgentCore Observability, AWS X-Ray)
- Set up dashboards to monitor performance metrics
- Optimize capacity for cost, performance, and reliability
- Optimize costs and set cost quotas by using appropriate cost management tools
- Optimize infrastructure costs by selecting purchasing options
- Evaluate cost implications of using FMs for inference in production
- Monitor agent resource consumption patterns
- Manage FM inference costs with usage optimization
- Monitor AI-specific cost patterns (for example, token usage optimization, embedding computation costs, vector database storage optimization)
4.3: Secure ML and AI workloads and model endpoints
Covers securing CI/CD pipelines, configuring access controls, implementing monitoring and compliance, and implementing safeguards for ML and AI systems.
- Secure continuous integration and continuous delivery (CI/CD) pipelines by checking for code and image vulnerabilities (for example, by using Amazon CodeGuru, Amazon Inspector)
- Configure least privilege access to ML and AI artifacts
- Configure IAM policies and roles for users and applications in ML and AI systems
- Configure comprehensive monitoring, auditing, compliance, and logging for ML and AI systems (for example, AWS CloudTrail, AWS Config)
- Troubleshoot and debug security issues in ML and AI systems
- Create VPCs, subnets, and security groups to securely isolate ML and AI systems
- Identify and mitigate security risks and vulnerabilities in ML and AI systems
- Select the appropriate credential type to access FMs (for example, Amazon Bedrock API keys, IAM credentials)
- Implement safeguards and sensitive data protection to meet application requirements and responsible AI policies (for example, by using Amazon Bedrock Guardrails)