Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Continuous Variables Spread of Data KDE plots for Continuous Variable Overview of Distribution for Continuous variables Normal Distribution Skewed Distribution Skeweness and Kurtosis Distribution for Continuous Variable
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

What is AWS Config? Benefits, How to Set Up, Application and Tips

T
Trupti Dekate Last Updated : 21 Aug, 2023
8 min read

Introduction

In the ever-evolving landscape of cloud computing, ensuring your AWS resources’ security, compliance, and overall health is paramount. This is where AWS Config steps in as a crucial ally. AWS Config is a powerful service provided by Amazon Web Services that enables you to maintain a comprehensive and up-to-date record of your AWS resource configurations and changes over time. By capturing a detailed inventory of your resource settings and relationships, AWS Config provides valuable insights into your environment’s compliance, security posture, and operational health. This article delves into the functionalities and benefits of AWS Config, shedding light on how it empowers organizations to enhance their governance, mitigate risks, and ensure the best practices in their cloud infrastructure.

This article was published as a part of the Data Science Blogathon.

Table of contents

Introduction to AWS Config

It can be easy to forget alerts in your AWS account’s health dashboard, even though you might want to check one particular alert. In particular, if you use AWS Config, the upcoming September 15, 2021, changes may stop supporting your cloud architecture components. Let’s take a look at the AWS statement below:

As part of our ongoing effort to optimize the costs associated with recording changes related to certain transient workloads, AWS Config is scheduled to release an update to relationships modeled within configuration items (CI) for 7 types of EC2 resources on September 15, 2021. Examples of transient workloads include changes to Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances, Amazon Elastic MapReduce jobs, and Amazon EC2 Autoscaling. This update optimizes CI models for EC2 instance resource types, Security Group, Network Interface, Subnet, VPC, VPN Gateway, and Customer Gateway to record direct relationships and discard indirect relationships.

While you’ll find a breakdown of the announcements here, we’ll cover the basics of AWS Config, dive into upcoming changes, and offer examples of solutions your organization can implement throughout this post.

Benefits of Using AWS Config

AWS Config offers these advantages, enhancing resource management, compliance, security, and overall governance within your AWS environment:

Continuous VisibilityMonitor resource state and changes in real-time.
Resource InventoryMaintain an automated inventory of AWS resources.
Change TrackingTrack configuration changes and their history.
Compliance MonitoringEvaluate resources against predefined rules.
Automated RemediationAutomatically correct non-compliant resources.
Security AnalysisIdentify security vulnerabilities and misconfigurations.
Customizable RulesCreate rules tailored to your organization’s needs.
Historical AnalysisAnalyze past events for troubleshooting and auditing.
Multi-Account GovernanceEnforce policies across multiple AWS accounts.
Resource RelationshipsUnderstand dependencies and associations.
Notifications and AlertsReceive notifications on configuration deviations.
Evidentiary SupportComprehensive record for audits and investigations.

AWS Config Rules

AWS Config Rules are a powerful feature of AWS Config that allow you to define and enforce specific desired configurations and compliance policies for your AWS resources. These rules help you maintain the desired state of your resources and ensure that they adhere to specific guidelines. 

  1. Rule Definition: AWS Config Rules are defined using AWS Lambda functions or predefined rules provided by AWS. They evaluate the compliance of your resources against desired configurations.
  2. Real-Time Evaluation: Config Rules continuously monitor your resources in real-time, assessing their compliance against the specified configurations. An AWS Config non-compliance event is triggered if a resource violates a rule.
  3. Customizable: You can create custom rules using AWS Lambda functions tailored to your organization’s requirements. This allows you to define complex compliance checks beyond the scope of managed rules.
  4. Managed Rules: AWS also provides a set of managed rules covering common compliance requirements. These rules are preconfigured and can be easily applied to your resources.
  5. Automated Remediation: Some AWS Config Rules can automatically trigger remediation actions to bring non-compliant resources back into compliance. This helps maintain the desired state of resources without manual intervention.
  6. Notifications: AWS Config can send notifications when a rule is triggered or a resource becomes non-compliant. These notifications can be sent via Amazon SNS or Amazon CloudWatch Events.
  7. Integration with AWS Organizations: Config Rules can be used to enforce compliance policies across multiple AWS accounts in an AWS Organization.
  8. Scalability: AWS Config Rules can scale to handle large numbers of resources across your AWS environment.

Configuration Concept

As you may know, AWS Config allows you to track current and historical resource configurations in your AWS account. AWS Config takes a snapshot of your deployed resources’ settings and stores them in a configuration item, which AWS refers to as a record of the settings of your specific resources at a specific time. These can be generated immediately when your resources are updated or on a schedule, such as every six hours.

A configuration item represents a point-in-time view of various attributes of a supported AWS resource in your account. Configuration item components include metadata, attributes, relationships, current configuration, and related events. AWS Config creates a configuration item whenever it detects a change in the type of resource it records. For example, if AWS Config logs Amazon S3 buckets, AWS Config creates a configuration entry whenever the bucket is created, updated, or deleted.

Example of Configuration of AWS Config

Configuration entries play a major role in AWS Config by serving as the source of truth for historical changes. AWS Config bases many things on these items; for example, when AWS Config gives you a timeline of the states of your resources, it uses configuration items in the background. This is how a configuration history of an I AM user looks. Each configuration item is represented by a gray rectangle highlighted in blue. Selecting one of these items can drill down into the relationships and changes associated with the AWS resource for the item’s timestamp.

configuration concept

Using the above example, we can dive into the changes of the IAM user on November 12, 2020, by selecting the Changes dropdown menu in the pink box above.

CLI AP

AWS Config creates configuration items with the same API calls as the AWS CLI. Specifically, they use CLI list and describe commands such as AWS ec2 describe_instances –filters Name=tag-name, Values=example-tag. For each configuration item created, AWS Config can make multiple API calls.

Types of relationships:

  • Direct
  • Indirect Relationships.

A direct relation is the one returned when the Describe API is called. For example, the security groups attached to the instance are returned when AWS Config [makes] the Describe Instances call.

An indirect relationship is not returned when you call the Describe API. This requires multiple calls to Describe. For example, when AWS Config calls [s] Describe Security Groups, the API call does *not* return* a list of instances to which the security group is attached. Config must make another API call to find this information. In short, the direct relationships of a resource configuration item can be created and documented using the description CLI command of the same resource service. On the other hand, indirect relationships for a resource are relationships that AWS Config creates by calling another service’s description CLI API.

It can be confusing as to what is considered indirect and direct, so let’s dive into a specific example. Looking at the sample AWS configuration item timeline below, we’ve highlighted information for indirect relationships in pink and direct relationships in green; however, a useful rule of thumb is that if the information you want does not come from a CLI command that is bound to the resource you are referring to, then the information comes from an indirect relationship. Please note that when you call describe-security groups, we get information about the ingress/egress rules and the VPC the security lives in.

What is changing?

To increase the scalability of AWS Config, AWS reduces the number of API calls that AWS Config makes on the backend. Because of these changes, AWS will end support for calling indirect relationships for seven different resource types. Here is a list of the indirect relationships of the affected resources:

  • Customer Gateway configuration items will no longer include VPN connections.
  • EC2 instance configuration entries will no longer include EIPs or routing tables associated with an EC2 instance.
  • Network interface configuration entries will no longer include the EIP or routing tables associated with the network interface.
  • Security group configuration entries will not include attached EC2 instances or network interfaces.
  • Subnet configuration entries will no longer include EC2 instances within the subnet, NACLs, network interfaces, and routing tables associated with the subnet.
  • VPC configuration items will no longer include EC2 instances inside VPC, Internet gateways, NACLs, network interfaces, routing tables, subnets, security groups, and VPN gateways.
  • Finally, VPN gateway configuration entries will no longer include associated routing tables or VPN connections.

To be clear, AWS has confirmed that there will be no impact on any current AWS configuration rules, only configuration items associated with resources. If you use AWS Config notifications for configuration item changes, this will affect you. Specifically, if you are using notifications for any of the above indirect relationship changes for your alerts or runbooks, your current setup will end on September 15, 2021.

Solution

If we need information for a rule currently configured, we can use AWS Config’s partially new advanced query feature to query this information using SQL. Using the example with security group sg-a1a1a1, we could see which EC2 instances are connected by either going to the EC2 console and manually checking each instance, or you can also use these queries.

solution | AWS Config

We’ve also expanded AWS’s documentation’s SQL queries to include additional queries to return indirect relationship information for the seven primary resources affected by upcoming AWS configuration changes.

AWs Config

Unfortunately, suppose you’re using SNS notifications for configuration item changes. In that case, this is a breaking change with no easy solution, so you’ll need to create your solution, such as spinning up a Lambda and generating the SNS notifications yourself.

AWS Config vs CloudTrail

FeatureAWS ConfigCloudTrail
PurposeTracks and manages resource configurationsMonitors and logs API activity
CoverageTracks configuration changes and complianceMonitors API calls and events
GranularityGranular details of resource configurationsHigh-level overview of API activity
Resource TypesSupports a wide range of AWS resource typesFocuses on API actions and events
Change TrackingTracks historical changes to resource configsLogs API calls, including who and when
ComplianceEnsures resources adhere to desired settingsAssists with compliance auditing and analysis
AutomationCan trigger AWS Lambda functions for changesFocuses on logging API events, not automation
CostCosts vary based on the number of resourcesCosts based on data events logged

Conclusion

AWS Config is scheduled to release an update to relationships modeled within configuration items (CI) for 7 types of EC2 resources on September 15, 2021. Examples of transient workloads include changes to Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances, Amazon Elastic MapReduce jobs, and Amazon EC2 Autoscaling.

  • AWS has confirmed that there will be no impact on any current AWS configuration rules, only configuration items associated with resources. This will affect you if you use AWS Config notifications for configuration item changes
  • AWS Config takes a snapshot of your deployed resources’ settings and stores them in a configuration item, which is what AWS refers to as a record of the settings of your specific resources at a specific time.
  •  AWS reduces the number of API calls that AWS Config makes on the backend. Because of these changes, AWS will end support for calling indirect relationships for seven different resource types.

Frequently Asked Questions

Q1. What is AWS configuration?

A. AWS Config is a service that assesses, audits, and records configurations of AWS resources. It helps maintain compliance and security by continuously monitoring resource configurations for changes and assessing compliance against predefined rules.

Q2. What is the difference between AWS config and CloudTrail?

A. While AWS Config and CloudTrail provide visibility into AWS resource activities, AWS Config focuses on resource configurations and changes over time. CloudTrail records API activity to track user actions and resource changes.

Q3. What is AWS config rules?

A. AWS Config Rules are customizable guidelines used to evaluate AWS resource configurations. They ensure that resources adhere to best practices and organizational policies, helping maintain compliance and security.

Q4. How do I run AWS configuration?

A. To run AWS Config, enable the service in your AWS Management Console, configure desired rules, and set up notifications. AWS Config will then continuously monitor and evaluate your resource configurations based on your defined rules.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion. 

AWSAWS Configblogathon
T
Trupti Dekate

I am an Accountant at Global private Analytics Services working with the Data Analysis Team for handling the budget of various Growing Companies. We provide service of analytics and made the work of new tech companies easy by helping them manage their total investment and giving suggestions.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Master MS Excel for data analysis with key formulas, functions, and LookUp tools in this comprehensive course.

Recommended Articles

Responses From Readers

Clear

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag
Congratulations, You Did It!
Well Done on Completing Your Learning Journey. Stay curious and keep exploring!
Top 100 Data Science Interview Questions & Answers 2024
Top 100 Data Science Interview Questions & Answers 2024
How do AWS S3 Buckets and Security Work?
How do AWS S3 Buckets and Security Work?

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details