AWS Solutions Architect - Associate (SAA-C03) Prep Notes

Last updated on
59 min read

Table of Contents

Notes collected when following the - Udemy course Ultimate AWS Certified Solutions Architect Associate by Stephane Maarek

AWS Technical Essentials

  • Regions and Availability Zones
  • Shared Responsibility Model
  • AWS root user - first user when creating account and has access to everything in that account. Also enable MFA to that user
  • AWS supports MFA from virtual device (auntheticator etc), Hardware TOTP token, FIDO security keys (USB device)
  • AWS IAM - Global service (not specific to any region) which manages users authentication and authorization. It’s a free service.
  • Admin can force users to rotate their passwords
  • Managing permissions of users is hard when the number of users grow. So you can use IAM groups to manage that.
  • You create different groups like security, admin etc provide relevant permissions to those groups and then place all the users to respective groups.
  • Users can belong to multiple groups
  • Groups cannot be part of another group
  • IAM policies manage access and permissions to AWS services and resources.
  • Most policies are stored as JSON documents. And they contain version, affect, actions, resource
  • IAM roles

IAM

  • Users, Groups. User can be part of multiple groups. Group cannot be part of another group
  • IAM Policy - JSON doc which mentions what kind of permissions are allowed or denied. Individual user can have inline policy
  • Policy structure - Version, Id(optional), statement. Statement contains sid(optional), effect (alow or deny), principal(account/user/role this is applied to), action (list of actions), Resource (list of resources it is applied to), condition
  • IAM Password Policy - You can define password policy, expiration time, password reuse policy
  • IAM MFA
  • Use access keys for cli based aws usage
  • AWS CLI, AWS SDK
  • AWS Cloudshell - not available in all regions It also allows to download and upload files
  • IAM Roles - perform actions on users behalf. Similar to user but they are intended to be used by services. It’s more like service account
  • IAM Security Tools - Credentials report (admins), Access Advisor (user level. Shows when a service was used). Access Analyzer b
  • Identity policy
    • Policy attached to user, group, resources which defines what permissions they have
    • They basically contains principal, action, resource
    • Three types - AWS managed, customer managed, inline
    • You can add conditions
  • Policy types
    • Identity policy -attached to users, group,
    • Resource policy - attached to aws resource
    • Permissions boundary - A permissions boundary sets the maximum permissions that an identity-based policy can grant to an IAM entity.
    • SCPs (service control policy) - AWS Organizations is a service for grouping and centrally managing AWS accounts. If you enable all features in an organization, then you can apply SCPs to any or all of your accounts.
    • Sessions policy - helps in limiting the permissions provided by identity and resource policies
    • ACL
  • Attribute-based access control (ABAC) is an authorization strategy that defines permissions based on attributes. In AWS, these attributes are called tags. Tags can be attached to IAM principals (users or roles) and to AWS resources.
  • Condition Keys - CalledVia, calledViaFirst, CalledViaLast, PassedToService, AssociatedResourceArn, ViaAWSService, TagKeys, RequestKeys
  • STS (Security Token Service)
    • grant temporary access and limit access to users
    • Lasts for few minutes to hours
    • Assume role request
    • Cross account access so that you can temporarily access other aws accounts
  • Session Tags -
  • Policy Simulator - you can test and troubleshoot identity-based policies, IAM permissions boundaries, AWS Organizations service control policies, and resource-based policie
  • Access Analyzer - you can proactively address any resource policies that violate their security and governance best practices around resource sharing and protect their resources from unintended access
  • Access Advisor
  • Cloudtrail

EC2

  • AWS Billing, you can create budgets which will send emails to you based on certain conditions that are met.
  • Bootstrap EC2 with custom script. Will run it as root user.
  • Security Group is the firewall which allows or blocks traffic from and to the instance. By default there is a rule to allow all outbound traffic. And there are no rules for inbound traffic, so all inbound traffic is blocked by default. It has only allow option, no deny option
  • Security Group is the firewall which allows or blocks traffic from and to the instance. By default there is a rule to allow all outbound traffic. And there are no rules for inbound traffic, so all inbound traffic is blocked by default
  • AWS changes the public if you stop an instance. It will assign a new one when you change.
  • Very important - https://aws.amazon.com/ec2/instance-types/
  • Security group contains inbound and outbound rules
  • You can attach security groups to multiple instances and multiple security groups to a single instance.
  • Security groups are locked to a specific region
  • If application is not reachable then it’s a security group issue. If you get an error saying connection refuse d then it means that the traffic went through the security group but not an error occurred on application
  • By default all inbound traffic is blocked and all outbound traffic in allowed
  • EC2 Instance Connect allows you to ssh into ec2 right from the aws console from browser istelf
  • EC2 Instance Role - Attach a IAM role to EC2 instance to perform aws cli actions in the ssh of ec2 instance without setting up access key of a IAM user
  • EC2 Purchasing Options - Very Important
    • On demand
    • Reserved Instance.
      • Standard reserved Instance - attributes of ec2 hardware can’t be modified. Can be sold in reserved instance marketplace
      • Convertible Reserved Instance - Can be modified. Can’t be sold
    • Savings Plan
    • Spot Instance
    • Dedicated Host
    • Dedicated Instance
    • Capacity Reservations
  • Spot Instance - Spot Block
  • Spot fleet - Spot instance + On demand
  • Private IP - within Virtual private cloud. Public IP - publicly can be connected to, but it is changed when the ec2 is terminated. Elastic IP - Public IP which you save for yourself and it does not change. You can have max of 5 elastic ip, but can be increased by raising a request
  • Elastic ip is charged when the attached instance is terminated
  • EC2 Placement groups - Strategies for placement of ec2 instances.
    • Cluster - clusters instances in a single AZ. Low latency usage
    • Spread - Span across AZs. Instances are on different physical hardware. Limited to 7 instaces per AZ per placement group - Critical application
    • Partition - spreads instances across many different partitions (which may be on diferent racks) within AZ. Scales to 100s of ec2 per group (apache, kafka), Big data
    • You attach a placement group to an EC2 to define placement strategy
  • Elastic Network Interface (ENI)
    • Bound to an AZ
    • You can move ENI from one ec2 to another
    • One EC2 can have two ENI’s
    • An ENI can have - private ip, elastic ip, mac address, security group, delete on termination, source/destination check flagEB
    • You can detach secondary network interfaces when the instance is running or stopped. However, you can’t detach the primary network interface.
    • You can move a network interface from one instance to another, if the instances are in the same Availability Zone and VPC but in different subnets.
  • EC2 Hibernate

Elastic Block Store (EBS)

  • Network drive that is connected to EC2
  • Can be connected to one instance at a time
  • They are bound to specific AZ
  • It can be called as network USB stick
  • Can be resized as and when required
  • EBS snapshots - not necessary to detach during snapshotting, but recommended. Can copy snaptshots across AZ. And this how EBS volumes are moved across AZ’s
    • EBS snaptshot archive - takes 24 to 72 hours to restore
    • Recycle bin for ebs snaptshots and provide retention policy to prevent accidental deletion
    • fast snapshot restore
    • EBS snapshots are stored in S3
  • EC2 Instance store - high performance hardware disk use instance store. They lose their storage if ec2 is stopped. Risk of data loss if hardware fails. Only specific Amazon EC2 instance types support instance stores. Data is lost if instance terminates or hibernates. In case of reboot, data persists
  • EBS volume types
    • gp2/gp3 - SSD, gp3 volumes start with a baseline of 3,000 IOPS and 125 MB/s throughput. With General Purpose SSD gp2 volumes you can burst to 3,000 IOPS using burst credits, however you must scale the storage volume size to increase your performance
    • io/io2 - Provisioned IOPS, 64000 IOPS, 1000MB/s, io2 has tiered pricing structure
    • st1/sc1 - Optimized HDD, Cold SDD
  • EBS Multi Attach - attach same ebs volumes in to multiple ec2 in same AZ. Only available for io/io2 types. Can attach 16 instances at a time. Must use file system that is cluster aware
  • Compute Optimizer - uses Amazon CloudWatch metrics to analyze your EBS volumes and provide recommendations to assist you in optimizing your Amazon Elastic Block Store (Amazon EBS) costs
  • EBS Encryption - Uses KMS (AES - 256). Everything is encrypted like in transit, at rest, snapshots, volumes created from snapshots. To create a encrypted EBS from unencrypted ebs, make a snapshot, then create a new snapshot from that with encryption, then create a volume from that encrypted snapshot
  • AMI are locked to a region. Even the custom ones that you build

EFS - Elastic File System

  • Network file system
  • Works with ec2 in multi az
  • NFS 4.0 and 4.1 support
  • Pay per use
  • It expands as you store
  • Compatible only with linux based systems (not windows)
  • You can share EFS across ec2 instances
  • Storage Classes
    • Standard Storage - Standard and Standard Infrequent Access which, MultiAZ
    • One Zone Storage - One Zone and One Zone Infrequesnt Access, Single AZ
  • Throughput mode
    • Standard
    • Bursting
  • Performance Mode
    • Elastic
    • Provisioned

Elastic Load Balancer - ELB

  • For high scalability and availability
  • Balances load to multiple servers
  • It’s a managed load balancer
  • Access logs is an optional feature of Elastic Load Balancing that is disabled by default. After you enable access logs for your load balancer, Elastic Load Balancing captures the logs and stores them in the Amazon S3 bucket that you specify as compressed files. You can disable access logs at any time.
  • Host based routing is supported. This helps in sending traffic from mobile.google.com to some and api.google.com to some other target group
  • They can reside in more than 1 AZs for high availability
  • Health checks and done if not healthy then traffic is not sent to it
  • Load Balancer Security group
  • Sticky session (Sesion affinity) - implement stickiness so that same client is always redirected to the same load balancer. A cookie is used for stickiness with custom expiration date. Supported by ALB, NLB, GWLB. NLB works without cookies. This stickiness is assigned to target groups
    • Applicaiton based cookie - custom cookie and aws managed cookie.
    • Duration based cookie - generated by LB
  • Cross zone load balancing - distribute traffic evenly across az’s across instances equally. For example if 2 instance in one AZ, and 8 instance in another AZ, then all instances euqally get 10% each
  • SSL/TLS - Uses X.509 certificate. You can load certificates using ACM (AWS certificate manager). You can upload your own certificates. SNI (Server name indication) solves the problem of loading multiple ssl certificates to one web server
  • Connection Draining (Deregistration delay) - time to complete inflight requests while instance is deregistering or unhealthy. It stops sending new requests to ec2 that is deregistering. Between 1 to 3600 seconds. Can be disabled by setting it to 0. Set to low value if requests are short. Basically it is about waiting for existing connections to complete.
  • Types
    • Classic LB (CLB) - Old generation
    • Application LB (ALB)
      • HTTP, HTTPS, Websocket (Layer 7)
      • Create target group when creating ALB to mention which instances this ALB needs to work with
      • Routing based on path of url, hostname, based on querystring, headers
      • X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Port headers are included by ALB before sending it to the servers
      • YOu can create rules in ALB to define and redirect to different target groups. Like for /users then one target group.
    • Network LB (NLB)
      • TCP, TLS, UDP (Layer 4)
      • Less latency compared to NLB
      • Routing to target groups based on private ips
      • Health checks support http, https, TCP
    • Gateway LB (GWLB)
      • Layer 3 - IP protcol
      • For Firewal, DPI, Payload manipulation etc
      • Used to deplay, scale and manage a fleet of 3rd party network virtual applicances in aws
      • Basically it sits between user and application and intercepts everything
      • Uses GENEVE protocol 6081

ASG - Auto scaling group

  • Automatically scale out/in based on load
  • Automatically register the instances to ELB
  • They are free
  • Set min, desired and max capacity
  • Use launch template (versioning support) instead of launch config (old way and immutable)
  • ASG attributes - launch template containing ebs volumes, security group ssh key pair, iam role, network + subnet info, load balancer info
  • It is possible to scale based on cloudwatch alarms
  • Scaling Group Policies - Dynamic
    • Target Tracking Scaling -you specify an Amazon CloudWatch metric and a target value that represents the ideal average utilization or throughput level for your application. Like if cpu reached 40% then do this. ASGAverageCPUUtilization, ASGAverageNetworkIn/Out, ALBRequestCountPerTarget
    • Simple / Step Scaling - When cloudwatch alarm is triggered then do this
    • predictive scaling - to increase the number of EC2 instances in your Auto Scaling group in advance of daily and weekly patterns in traffic flows.
  • Scheduled Actions - based on time like at 5 pm bring 10 up
  • Scaling Group Policy - Predictive
    • Predictive Scaling - continuously forecast load and schedule scaling ahead
  • Scaling cooldown - cooldown period default 300, asg will not launch or terminate inorder for the metrics to stabilize

Amazon RDS

  • Managed DB service
  • Postgresql, mysql, maria, mssql, oracle, aurora
  • Storage Auto Scaling - auto increase storage when increases. You can set max storage threshold
  • RDS Read Replicas - Upto 15 read replicas. Can same AZ, cross AZ, cross region. Eventually consistent. ASYNC replication. No network cost for data for replication within AZ. In case of cross region replica, there is a cost
  • Multi AZ - Disaster Recovery. SYNC replication. Increased availability. Read replicas can be setup as multi az for disaster recovery.
  • How does single az converter to multi az -it takes a snapshot, then creates new rds instance in another az, then syncs data between them eventually.
  • RDS Custom - custom access to underlying ec2using ssh. you can install patches and manage os. Only available for mssql and oracle.
  • RDS Proxy

Aurora - propreittory tech from aws

  • Managed service
  • Drivers of mysql and postgre will work with this
  • AWS claims 5x performance compared to mysql and 3x compared to postgre
  • It storage automatically grows in increments of 10GB upto 125TB
  • Upto 15 read replicas, and faster than mysql
  • It slightly expensive
  • It maintaines 6 copies of data across az’s. 4 out of 6 needs to be written for writes
  • 3 out of 6 needed for reads
  • Self healing with p2p replication
  • Storage is striped across volumes
  • One instance takes writes (master).
  • Supports cross region replication
  • It has read and write separate endpoints to manage reads and writes
  • Backtrack :restore data any point of time without using backups
  • You can define custom reader endpoints to specific aurora instances
  • Aurora serverless - auto db instantiation, scaling, used for unexpected workloads. No capacity planning needed. Pay per second
  • Aurora Multi Master - continuous write availability for writer nodes. Every node does R/W instead of promoting a read replica as master
  • Global Aurora - Aurora cross region read replicas, aurora global database
  • Aurora Machine Learning - sagemaker and comprehend are supported
  • Automated Backups, Manual DB snapshots
  • Aurora DB Cloning
  • Aurora auto scaling -
  • cluster endpoint - primary active instance writer endpoint
  • reader endpoint - load balances itself across read replicas
  • custom endpoint - you select the instances you want, it will load balance and send traffic
  • instance endpoint - points to specific db instnace

ElastiCache - managed redis or memcached

  • Low latency
  • VPC is required fot setting up the cluster
  • Redis - Multi AZ with auto failover, read replicas, backup and restore features, supports sets and sorted sets
  • Memcached - Sharding (multi node support), no replication, non persistent, no backup and restore, multi threaded architecture
  • It supports IAM auth for redis. IAM policies are only used for api level security
  • Memcached supports SASLauth

Route 53

  • It’s a domain registrar
  • Authoritative DNS
  • Supports health checks
  • Provides 100% SLA
  • Record contains - domain/subdomain name, record type, value, routing policy, ttl
  • Record types - A, AAAA, CNAME (maps to another hostname), NS (name server for hosted zone)
  • Hosted Zone - a container for records that define how to route traffic to domains and sub domains. These can be public or private hosted zone (within vpc). 0.5 dollars per month per hosted zone
  • CNAME - points hostname to another hostname. Works only for non root domain, like app.example.com
  • Alias - Points hostname to aws resource. Works for root and non root domain. Free and health checks supported. An extension of dns functionality. Automatically recognizes changes in ip address. Alias record can only be A/AAAA. No TTL can be set.
    • Routing policies - handles how dns queries are responded
      • Simple - route to single resource. Can have multiple values in same record, but a random one is chosen by client. Can’t be associated with health checks
      • Weighted - Control percentage of requests that go to the resource. Can have health checks
      • Failover - Active-Passive. Master, slave technique
      • Latency based - Redirect to resource with least latency. Health checks supported
      • Geolocation - based on user location, like asia, europe etc. Should create default location which maps to anything that doesn’t match.
      • Multi-Value answer - routing traffic to multiple resources. Return multiple values/resoures. Health checks supported. Similar to simple routing policy, but health checks are supported, so dns query will return multiple values which are only healthy.
      • Geoproximity - route based on geo locaiton of users and resources. Ability to shift traffic to resource based on bias. 1 to 99 more traffic to resource. -1 to -99 less traffic to resource. You must use route 53 traffic flow to use this feature
      • IP-based - Routing based on client’s ip address. You provide CIDR list. User Ip to endpoint mapping.
    • Health checks monitors an endpoint, if some metrics don’t meet the standard then traffic is redirected
    • Calculated health checks
    • Route 53 can’t access private endpoints, so you create cloudwatch metrics and then create health checks that monitors the alarm itself
    • You can use your own domain registrar but you can use route 53 to manage your dns records by upating the nameservers values in 3rd party service
  • Use golden AMI for instantiating instances quickly

Elastic Bean Stalk (EBS)

  • Managed service
  • Handles provisioning, load balancing, scaling, health monitoring,
  • Tier types - Web server(load balancer), Worker tier(uses sqs and send events to ec2 to do the job)
  • Deployment modes - Dev, prod
  • Cloud formation
    • Create aws services using a template, IaaC
    • YAML/JSON support
    • You create a stack which contains resources. You can update these resources by updating template
    • You can create change sets that require approval before making changes
    • StackSet – A named set of stacks that use the same template, but applied across different accounts and Regions.

AWS Lightsail

  • Lightsail provides developers with compute, storage, and networking capacity and capabilities to deploy and manage websites and web applications in the cloud. Lightsail includes everything you need to launch your project quickly for a low, predictable monthly price. This includes VMs, containers, databases, content delivery network (CDN), load balancers, Domain Name System (DNS) management, and so on.

S3 - Simple Storage Service

  • Infinite scaling
  • Stores objects (files) in buckets (directories)
  • Buckets should have globally unique name (across all regions and all accounts)
  • Buckets are defined at regional level
  • Objects have a key
  • Application can achieve at least 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per partitioned Amazon S3 prefix.
  • Content of object in value. Max 5TB size
  • If more that 5GB is being uploaded then use multi part data
  • Custom metadata fields can be added
  • Also contain tags ,version id
  • It’s a global service, but during creation of bucket you have to specify a region name
  • You can create folders within those, but all of the objects are referred by key
  • Object level permissions are managed through bucket policies and ACLs at the bucket level, not directly on individual objects in a bucket. These can be based on prefix, user or groups.
  • Bucket policy don’t apply to objects owned by other accounts. ACLs grant read/write permissions to other accounts.
  • S3 url contains - bucket-name.region.amazonaws.com/object (Virtual hosted style url)
  • Largest file to s3 that can be uploaded through UI is 160GB
  • Upto 5Gb single PUT operation works, but for larger than tht use multi part
  • If a bucket is versioning-enabled, you can either permanently delete an object or have Amazon S3 create a delete marker for the object, which allows the object to be recoverable. You can permanently delete individual versions of an object by invoking a DELETE request with the object’s key and version ID. To completely remove the object from your bucket, you must delete each individual version.
  • Online Data Transfer Services
    • AWS DataSync - move large amounts of on premise data. Uses datasync agent on onprem to transfer data between them
    • AWS Transfer - Fully managed support for file transfers for in and out of s3 by integrating with existing auth mechanisms. SFTp, FTPS, S3, FTP
    • Amazon S3 Transfer Acceleration - transfers of files over long distances. It takes advantage of Amazon CloudFront globally distributed edge locations, routing data to Amazon S3 over an optimized network path. Transfer Acceleration is best suited for scenarios in which you want to transfer data to a central location from all over the world or transfer significant amounts of data across continents regularly. It can also help you use your available bandwidth when uploading to Amazon S3. File transfer over long distance
    • Kinesis Data Firehose and Data Streams
    • Amazon Partner Network - Use of third party connectors
  • Hybrid Cloud Storage Services
    • AWS Direct Connect - dedicated network connection from on prem data center to aws
    • AWS Storage Gateway
  • S3 Service Integration
    • AWS Glue - ETL
    • Amazon Athena - query service that makes it easy for you to analyze data directly in Amazon S3, using standard SQL
    • Redshift Spectrum
  • Security
    • User-based - IAM policies
    • Resource based - bucket wide rules, also allows cross account
    • Object Access Control List (OACL) - finer grain
    • Bucket Access Control List (BACL)
    • S3 bucket policies - Contains version and statement. Statement contains sid, effect, principal, action, resource
    • You can set global account level setting to completely block buckets from getting created with public access
    • s3 supports static website hosting. You just have to enable it in bucket and setup public policy for bucket
    • Versioning is setup at bucket level
    • Versioning can be enabled after creating bucket also. Version ID will be shown as null for items that were uploaded before versioning was enabled
    • Delete marker
    • Replication
      • Must enable versioning in source and destination buckets
      • CRR (cross region replication) and SRR(same region replication)
      • Buckets can be in different aws accounts
      • Copying is asynchronous
      • Chaining of replication doesn’t happen i.e replication from bucket 1 to 2 will not trigger replication from 2 to 3
      • For existing objects to be replicated use S3 batch replication
    • Storage Classes - Defined based on durability and availability
      • Standard - General Purpose
      • Infrequent classes
        • Standard infrequent access
        • One zone infrequent access
      • Glacier Storage
        • Glacier Instant Retrieval - instant retrieve
        • Glacier Flexible Retrieval -minutes to hours retrieval
        • Glacier Deep Archive - retrieval time upto 12 hours
      • Intelligent Tiering - Moves objects intelligently between theree (frequent, infrequent, archive instant)tiers based on usage and two more triers archival and deep archival
    • You can create lifecycle rules
    • S3 Storage analytics and insights
      • S3 Storage Lens - Storage Lens provides visibility into storage usage and activity trends at the organization or account level, with drill-downs by Region, storage class, bucket, and prefix. AWS ORganization level data
      • S3 Storage Class Analysis - analyzes storage access patterns to help you determine when to transition less frequently accessed storage to a lower-cost storage class. Object level view can be found
      • Amazon Quicksight - uses Amazon S3 analytics data to gain insight into your data usage and growth patterns. It uses data from s3 storage class analysis
    • Basically for storage optimization use storage class analysis to get the data, then import it to quicksight to view and draw conclusions, then use s3 lifecycle policies to define how the data moves
    • You can transition objects from the S3 Standard storage class to any other storage class but you cannot create a lifecycle rule that moves objects from any storage class back into to the S3 Standard storage class.
    • You can move objects between different storage classes by modifying properties. You can do this automaically using lifecycle rules. These can be transition actions (move between storage classes) or expiration actions.Rules can be based on vertain prefixes or tags
    • S3 analytics can be used to suggest techniques for moving objects.
    • S3 requestor pays - Normally storage cost and the cost download is paid by the ownser. But with this, owner pays for storage, but requestor pays for the download (networking cost).
    • Event Notifications - object created, deleted, updated. Use case- create thumbnails every time photo is uploaded. These events can be for sqs, sns, lambda, event bridge
    • S3 byte range fetches - Using the Range HTTP header in a GET Object request, you can fetch a byte-range from an object, transferring only the specified portion.
    • S3 Select & Glacier Select - allows to you to do server side filtering. Can filter by rows and columns
    • S3 batche operations - Perform bulk operations iwth single request, like modifying metadata, tags, encryption etc. using s3 select, s3 inventory
    • S3 transfer acceleration
    • S3 Object encryption
      • Server Side Encryption (SSE) - 3 types - S3 Managed Keys (AES-256), KMS, Customer provided keys
      • Client side encryption
      • Encryption in transit - ssl/tls. HTTP, HTTPS
      • DSSE - KMS - Double Server Side encrpytion
      • Bucket policies help you enforce certain encryption standards
      • CORS feature is supported and you can specify who can access the objects
      • MFA delete feature
      • S3 access logs - maintains logs around all acess. For audit trail
      • Pre-signed urls - generate pre signed urls with an expiration date. Use case, temporarily allow a user to dowload a file
      • S3 Glacier Vault Lock - Create vault lock policy, lock policy for future edits. Used for compliance, data retention. You can push data only using api or cli
      • S3 Access Points - Write a policy for a prefix, this way only certain path of the object.

Cloudfront

  • CDN
  • Content is cached at edge
  • 216 edge locations
  • Provides DDos protection, integration with shield, aws waf
  • CF origins can be s3 buckets, custom http origin (ec2, load balancer, c3 website) ,any http backend you want
  • It’s a global service
  • CF Geo Restriction can be set up
  • CF Price classes
    • Price class all
    • Price class 200
    • Price class 100 - only least expensive regions
  • Cache Invalidations - after ttl done. You invalidate you using a request to cf

AWS Global Accelerator

  • You want your application to be global, with lowest latency, but your application ALB is deployed in a particular region. You use the concept of Anycast IP (multiple servers having same ip), so that requests can be send to nearest server.
  • For IPv4, Global Accelerator provides two static IPv4 addresses. For dual-stack, Global Accelerator provides a total of four addresses: two static IPv4 addresses and two static IPv6 addresses.
  • Uses AWS internal network
  • gaming, voip
  • types - standard, custom (gaming scenario where multiple users are from diferent region gaming online, so you want to optimizde the routing and region foe best usage for multiple set of users, same goes for Voip)

AWS Snow Family

  • highly secure, portable devices to collect and process data at the edge, and migrate data into and out of aws
  • Offline devivces to perform data migrations
  • Data migration - File transfer is slow.
  • For data migration - three types - snowcone, snowball edge, snowmobile
  • Snowball edge - contains types - storage optimized, compute optimized
  • Snowcone, snowcone SSD
  • Snowmobile - a truck to move data.
  • For edge computing - snowcone, snowcone ssd, snowball edge (ML) - for edge computing on edge
  • OpsHub - a software for your computer to manage snow family device
  • Snowball to glacier - use snowball to s3, then from s3 to glacier using s3 lifecycle policy

FSx - Launch 3rd party high performace file systems on aws

  • Fully managed service
  • FSx for lustre - Lustre file system, high performace loads, ml workloads, POSIX protocol
  • FSx for NetApp ONTAP - netapp ONTAP operating system, designed to provide both NetApp file and block storage
  • FSx for windows file server - Uses SMB used for microsoft and windows workloads
  • FSx for Open ZFS - OpenZFS protocol. You can migrate from onpremises zfs easily to cloud

Storage Gateway

  • Bridge between on premises data and cloud data
  • Use cases - backup, disaster recovery
  • Types -
    • S3 file gateway - for s3 standard (not glacier)
    • FSx file gateway
    • Volume gateway - Block storage, iCSI
    • Tape gateway - tape volumes
  • Storage gateway hardware appliance - custom hardware for managing data

Edge & Hybrid Cloud Storage Services

  • Edge - Local Compute and Storage
    • Snowball edge - storage (with compute enabled or disabled) and compute optimised, Import, export to s3, cluster can be formed. Storage optim 80TB, Compute optim 42 TB
    • Snowcone - Computing and data transfer device, For Iot Purposes, smallest device, Import, export to s3, 8TB HDD, 14TB SSD limit
    • Snowmobile
  • Hybrid - On premises cloud storage
    • AWS outposts - Run AWS services on premises. They are aws servers that are physically installed on your data center. Fully managed service, provides aws services to virtually any data center. You can access the full range of AWS services available in the Region to build, manage, and scale your on-premises applications using familiar AWS services and tools. S3, EBS supported for local storage
  • Hybrid - On Premise Gateways - Provides integration between on prem users and apps using a software appliance with cloud based storage. You need to deploy a hardware or vm instance on your on prem to act as a storage gateway
    • S3 File Gateway - Connect to store data file and backup images as durable objects. SMB or NFS supported
    • FSx File Gateway - Amazon FSx for windows file server
    • Volume Gateway - presents cloud-backed iSCSI block storage volumes to your on-premises applications. cached or stored volume type. cached - stores all data on s3, but freuqently used data is cached on on prem environment. Stored - stores all data locally, while also being backed up to s3
    • Tape Gateway is used to replace physical tapes on premises with virtual tapes in AWS without changing existing backup workflows.

Data Transfer and Migration Services

  • File Transfer Services
    • AWS transfer family
  • Data Synchronisation and online migration servies
    • AWS datasync - async transfer between source and target. Between on prem and cloud
  • Offline data transfer and migration services
    • Snow family services
  • Migration Services
    • AWS Application MIgration Serivce (MGN) - lift and shift migration,
    • CloudEndure Migration -
  • Data Protection Services
    • Backup and Archive - AWS Backup - EC2, EBS, EFS, S3, FSx, Volume gateway, RDS, Aurora, DynamoDb, Neptune, DocumentDB. Centralized backup mgmt, polucy-based bakup, scheduling, retention management, lifecycle policies, incremental backups, cross-region backup, security, compliance
    • Snapshots - EBS snapshots, FSx for lustre snapshots
    • Replication
    • Disaster Recovery Services - CloudEndure Disaster Recovery continuously replicates your machines into a low-cost staging area in your target AWS account and preferred Region. Uses EBS for storage
  • AWS Transfer family
    • Managed service to move data in and out of s3 or efs using ftp protocol
    • Protocols - ftp, ftps, sftp
  • Datasync - Move data to and from on premises.
    • Supports - S3, EFS, FSx
    • File permissions and metadata are preserved
    • Replication can be schedules based on hourly
    • Can be used to sync between different aws services like efs, s3

SQS

  • Queuing can be synchronous (app to app), asynchronous (app to queue to app)
  • SQS - queue model, SNS - pub/sub model, kinesis - real time streaming model
  • Producers send messages to queue. Consumer polls for messages from queue.
  • Fully managed service
  • Unlimited throughput, unlimited number of messages in queue
  • Default retention of 4 days, max 14 days
  • Low latency
  • Limitation of 256kb per message
  • Consumer can receive 10 messages at a time
  • Producer makes sendMessage api. Consumer sends deleteMessage api to queue
  • SQS can be used with Auto scaling group by setting up a cloud watch alarms which will trigger the auto scaling group
  • In flight and server side encryption can be done
  • Access policies can be setup
  • Message Visiblity timeout - time for which a message that is polled by a consumer will not be visible to other consumers. If message is not deleted by then, the other consumers can consume it again
  • ChangeMessageVisibility api can be used by the consumer to tell the queue that i need more time to process the message and increase the message visibility timeout
  • Consumer can use long polling so that it can wait until a message arrives in the queue
  • Types of queues - Standard, FIFO
  • FIFO queue - ordering of messages in queue, exactly once processing. They have group id and deduplication id
  • The drawback is that FIFO queues can only support 300 send, receive, or delete operations per second. When using message batching at 10 messages, this effectively is increased to 3,000 operations per second. If you require higher throughput, you can enable high throughput mode for FIFO on the Amazon SQS console
  • there is a 120,000 quota for the number of inflight messages for a standard queue and 20,000 for a FIFO queue.

Simple Notification Service (SNS)

  • Pub Sub model
  • Event producer sends message to sns topic
  • Event receivers (subscribers) subscribes to topics. All subscribers receive the messages
  • Upto 12.5 million subscriptions per topic
  • 100,000 topics limit
  • Emails, SMS, HTTP endoints
  • Serverside, Client, KMS encryption supported
  • SNS access policies can be setup
  • SNS + SQS fan out - SNS subscribers might be down at the time of sending, so you create sns to receive messages from producers, and then you have multiple sqs as consumers which will receive the messages and then the actuall consumers will consime the message. You get the benefit of not losing any data in this process
  • SNS FIFO is also supported
  • Message filtering is supported

Amazon Kinesis

  • Collect, process and analyze data in real time
  • data can ve - app logs, metrics, website clicks, telemetry data
  • Types
    • Kinesis Data streams - capture, process and store datastreams
      • It contains shards.
      • It receives data from producers
      • Data contains partition key, blob
      • And consumers consume it
      • retention of 1 to 365 days
      • Once data is inserted it can’t be deleted
      • Capacity modes - Provisioned mode (you chose the number of shards. 1MB/s in and 2MB/s out), On demand mode
    • Kinesis data firehose - load data into aws datastreams. Move data to s3, http endpoint, splunk etc. Before moving , you can transform data using lambda etc
    • Kinesis data analytics - analyze data streams with sql or apache flink
    • Kinesis Video Streams - capture, process and store video streams

Amazon MQ

  • Managed message broker service
  • rabbit mq, active mq

Elastic Container Service (ECS)

  • EC2 launch type - you maintain the ec2 instances underlying
  • Fargate luanch type - managed by aws. No need to maintain ec2 instances
  • EC2 task role - allows each task to have a specific role
  • ECS along with load balancer integrations
  • ECS - Data Volumes can be used with EFS
  • VPC is required for this
  • ECS Auto Scaling (Application Auto Scaling) - task level (not EC2 level)
  • Scaling can be types - Target Tracking, Step tracking, Scheduled Tracking
  • ECS Service - CPU scaling
  • Task definition - defines cpu, docker image, volume type, size, all parameter to run a container
  • Task - an instance of task definition

Elastic Container Registry (ECR)

  • Manage docker images on aws
  • Supports vulnerability scanning etc

Elastic Kubernetes Service (EKS)

  • Managed Kubernetes Service
  • Node types - managed node groups , self-managed node groups, aws fargate
  • Data volumes - EBS, EFS(fargate), fsx lustre and netapp

App Runner

  • Fully managed to deploy web applications
  • Code or docker image

Lambda

  • Pricing - pay per request and compute time
  • Upto 10GB of ephemeral storage
  • Max 15 min timeout
  • Memory can be confired but not CPU
  • Supports node.js… and custom random api also
  • Supports cron job
  • Limits - 128MG to 10 GB, 900 seconds max execution time, 4kb environment variables, disk capacity (512 MB to 10GB)
  • 50 MB max deployment size
  • Functions can run at edge also, reuns at edge and minimizes latency
  • Cloudfront two types - cloudfront functions, lambda@edge. Can be used for A/B testing, seo etc
  • By default lambda runs outside of vpc. But can be run in your own VPC. Example is you want to use lambda with rds, then you use rds proxy to pool all conections from multiple lambda instances, and also rds proxy is never publicly visible.
  • Lambda functions can be triggered from database. RDS for postgresql and Aurora for MySQL are supported
  • Versioning is supported
  • Analyze a cold start using X-ray
  • IAM resource policy - who can invoke the function
  • IAM Execution role - whant function can do
  • Invocation types
    • Synchronous - Service waits until lambda responds. ELB, API gateway
    • Asynchronous - Service doesn’t wait. SNS, S3
    • Polling - SQS, DynamoDB streams, Kinesis. lambda doesn’t support by default but you can build a custom one using lambda
  • Execution phases
    • INIT
    • INVOKE
    • SHUTDOWN
  • Handler contains
    • Event object - data sent to lambda
    • Context object - requrest id, runtime, logging
  • Concurrency Limits
    • Limit concurrency
    • Reserve concurrency
  • AWS SAM - The AWS Serverless Application Model (SAM) is an open-source framework for building serverless applications. It provides shorthand syntax to express functions, APIs, databases, and event source mappings. With just a few lines per resource, you can define the application you want and model it using YAML. During deployment, SAM transforms and expands the SAM syntax into AWS CloudFormation syntax, enabling you to build serverless applications faster.
  • AWS SAM CLI helps you to test and deploy lambda functions

DynamoDB

  • Fully managed, distributed database, transaction support is there
  • primary key - partition and sort key
  • Partition key and sort key – Referred to as a composite primary key
  • A strongly consistent read request of an item up to 4 KB requires one read request unit.
  • An eventually consistent read request of an item up to 4 KB requires one-half read request unit.
  • A transactional read request of an item up to 4 KB requires two read request units.
  • One write request unit represents one write for an item up to 1 KB in size.
  • DynamoDB supports two kinds of indexes:
    • Global secondary index – An index with a partition key and sort key that can be different from those on the table.
    • Local secondary index – An index that has the same partition key as the table, but a different sort key.
  • Each table in DynamoDB has a quota of 20 global secondary indexes (default quota) and 5 local secondary indexes.
  • Local indexes can vary only the sort-key , maintaining the same partition-key as the base table, thus being useful only for different sorting patterns. Global indexes support different attributes for both partition-key and sort-key .
  • You can have table without index, you can have more than one global or local secondary index
  • Read/write capacity modes - Provisioned mode, on-demand mode
  • DAX - DynamoDB Accelerator, in memory cache for dynamodb. Solves read congestion by caching. No app logic needs to be modified. Runs in VPC via a host
  • DynamoDB Streams
  • DynamoDB global tables - In the rare event of a single Region failure, there is no need for a database failover as global tables’ multi-active architecture allows customers to read and write to any replica table. Global tables also eliminate the difficult work of replicating data between Regions and resolving update conflicts for multi-active workloads. In addition, global tables enable your applications to stay highly available even in the rare event of isolation or degradation of an entire Region.
  • DynamoDB - TTL - automatically delete items after the TTL. Use cases - data retention, user session handling
  • Backups for distater recovery with point in timerecovery
  • Export to S3 is supported
  • 400KB is the max size of item in dynamodb
  • Dynamodb streams get you changelog, that allows you to globally replicate data, they capture the addition, updation and deletion of items in dynamodb, they have 24 hours after which they are deleted

API Gateway

  • Supports websocket
  • Versioning, environemnts supported, caching
  • Endpoint types - edge optimized (api gateway still lives in a region but the requests are routed through edge locations), regional, private
  • REST API is better than HTTP API since it offers a whole lot of features like caching, etc

Step Functions

  • Build serverless visual workflow to orchestrate your lambda functions
  • Use case - data worflow, etc
  • Standard (can run upto 1 year) and Express Workflow (runs max for 5 mins) types

AWS Cognito

  • User pools - sign in functionality for app users
  • Identity pools (federated identity) - provide aws creds to users so they can access aws resources directly. Sign in google, saml, facebook

Databases

  • RDS - read plicas, multi az, provisioned instance size, ebs volume type and size, RDS custom, RDS proxy, OLTP, security groups are used to control access
  • Aurora - Poostgre, mysql, 6 replicas, self healing, auto scaling, auto scaling of read replicas, custom writer and reader endpoints, aurora serverless, aurora multi-master, aurora global (upto 16 read replicas in each region), Aurora Machine Learning, Aurora database cloning
  • Elasticache - Redix, memcached, point in time restore, code changes required for migration from on prem
  • DynamoDB - auto scaling, provisioned mode, multiaz by default, DAX cluster, DynamoDB streams to stream data, active-active replication, PITR
  • S3 - key-value store, 5TB max, multiple tiers, lifecycle policy, MFA, CORS, bucket policy
  • DocumentDB - MongoDB copy, similar deployment concept as aurora. Runs within VPC, encryption at rest can be disabled but is enabled by default
  • Neptune - graph database, 3AZ, upto 15 read replicas, billions of relations can be stored. Required VPC, You never pay for loading data into your database, graph query languages: Apache TinkerPop and SPARQL.
  • Keyspaces (for apache cassandra) - no sql distributed database, managed, CQL, ondemand and provisioned
  • QLDB - ledger, record finance transactions, replicaiton across 3az,
  • Amazon Managed Blockchain - decentralization support
  • Timestream - time series data, fully managed, trillions of events per day,
  • Athena
    • Serverless query service to analyze data stored on s3
    • uses sql to query files
    • supports csv, json and a bunch of others
    • Commonly used with Amazon Quicksight to build dashboards
    • Use cases - analytics, business intelligence
    • Use columnar data for performance improvement
    • compress data for small intervals
    • Partition datasets
    • Athena federated query - uses data source connectors that run on lambda to run federated queries on relational, non relational, custom data sources
  • Redshift
    • It’s based on postgresql, but it’s olap, not oltp
    • Columnar storage of data (instead of typical row data)
    • Tableau, quicksight integrate with it
    • Faster queires, joins, aggs due to usage of indexes
    • Redshift cluster has leader node (for query planning and result aggs), and compute node (for performing query). Compute node creates slices that are provided with memory and disk space to do their job
    • You provision node in advance. Use reserved instances for cost savings
    • They can store agggregate values from transactional databases
    • Multi Az for some clusters
    • Snapshots are incremental
    • Redshift Spectrum - query s3 data without loading data into redshift
    • Runs within VPC
  • Opensearch (elasticsearch)
    • partial match support
    • modes - managed , serverless
  • EMR (Elastic Map Reduce)
    • Creating hadoop clusters
    • Can have 100s of ec2 instances
    • Nodes types - Master, core, task
  • Quicksight
    • Business intelligence service to create dashbooards
    • Use cases - business analytics, visualizations
    • Integrates with - athena, rds, aurora, redshift, s3, opensearch, timestream, kira, saleforce, custom data sources
  • Glue
    • Serverless
    • ETL service,
    • Useful to prepare and load data for analytics
    • Convert data to parquet format
    • Glue data catalog, glue data crawler
    • Glue job bookmarks, glue elastic views, glue databrew, glue studio, glue streaming etl
  • Lake Formation
    • Central place to manage all data for analytics
    • Discover, cleanse, transform data
    • Fully managed service
    • One of the biggest use is centralized permission control
  • Kinesis Data Analytics
    • for sql apps
    • for apache flink
  • Amazon Managed Streaming for Amazon Kafka (Amazon MSK)
    • Alternative to Amazon Kinesis
    • Fully managed apache kafka on aws
    • Allows you to create broker nodes, zookeper nodes
    • MSK Serverless is also there
    • Kinesis Data Streams vs Amazon MSK difference important
  • Big data Ingestion Pipeline
    • Data from iot devices is collected by amazon kinesis data streams. Then kinesis data firehose collects it and sends it to s3 every minute. In between this, lambda can be used to transform the data.
    • IoT core can be used to harvest data from iot devices

Rekognition

  • Find objects, texts, scenes in images and videos using ml
  • Facial analysis, ficial search, people counting
  • Content moderation can be done. Set min threshold level and it will flag items, then do an manual review of that

Transcribe

  • Speech to text
  • Uses deep learning called automatic speech recognition
  • Auto removes Personally Identifiable info
  • Auto language identification for multi lingual audio

Polly

  • Text to speech
  • Lexicon (customize pronunciation of words, Acronym conversion) and SSML Speech Synthesis markup language (emphasize certain words, phonetic pronunciation, breathing sounds etc)
  • Translate
    • Language transalction

Amazon Lex

  • powers Alexa
  • NLP to understand the intent of text, callers
  • helps build chat bots, call center bots

Amazon Connect

  • Receive calls, create contact flows, cloud based virtual contact center
  • Intergrates with CRMs

Amazon Comprehend

  • For NLP
  • Uses ML to find insights and relationships in text
  • Comprehend emails from customer to understand the experience
  • Amazon Comprehend Medical
  • Sagemaker - to build ml models for developers. Fully managed service.

Forecast

  • Predict future sales, product demand planing, fully managed service

Kendra

  • Document search service. s3, rds, google drive. natural language searching capabilities

Personalize

  • personlized recommendations service. Same what is used by amazon.com. Read data from s3.

Textract

  • extract text from scanned docs.

Cloudwatch

  • Provides metrics for every service on aws
  • dimension is an attribute of metric (instance id, enviroment etc). Upto 30 dimensions per metric
  • You can create cloudwatch dashboards of metrics. Create custom metrics
  • Cloudwatch metric streams can be used to push this data in real time to splunk, kinesis, new relic etc
  • Cloudwatch uses namespaces to categorize logs from different services
  • Cloudwatch agent and SSM (Systems Manager) agent are different

Cloudwatch logs

  • log groups - random name representing an app
  • log stream - instances with app/ contsainer/ log files
  • log expiration can be setup
  • logs can be sent to s3, kinesis etc
  • Encrypted by default, but custom encrytption can be added
  • Sources - sdk, cloudwatch logs agent, cloudwatch unified agent
  • cloudwatch logs insights - search, query, analyse, visualize and export log data. Can query from multiple aws accounts
  • Cloudwatch logs subscriptions - get real time logs for processing and analysis, sent to kinesis, lambda. You can specify filters.
  • By default no logs are sent from ec2 to cloudwathc logs. You need cloudwatch agent on the machine to send the log data
  • Types - Cloudwatch agent (old version, can only send to cloudwatch logs), cloudwatch unified agent(can also collect system level metrics, centralized control )

Cloudwatch alarms

  • Used to trigger based on metric data
  • Contains alarm state (OK, insuf_data), period,
  • Cloudwatch alarm target - What action needs to be done like stop, terminate, trigger auto scaling action, send notifications to sns
  • Alarms are on single metric, if you want to do it on multiple, Create a bunch on alarms, and then create a composite alarm using and/or operations on these single alarms
  • Instance recovery can be done
  • Use CLI to test alarm for development

Amazon EventBridge (Cloudwatch events)

  • Schedule cron jobs, Event pattern (event rules react to something a service is doing)
  • Various services send events, these can be filtered and sent to event bridge, and then you can do activities based on that. More like jenkins where you push code and it will trigger build
  • Types - default event bus (aws services sending events), partnet event bus (3rd party tools sending event), custom event bus
  • You can archive and replay events
  • Schema registry allows you to generate code for your appthat will know in advance how data is structured in event bus
  • Resource based policy for cross account access

Cloudwatch container insights

  • Collect metrics and logs from containers
  • ECS, EKS, Fargate

Cloudwatch lambda insights

  • for lambda

Cloudwatch contributor insights

  • analyze log data and create time series display for contributor data
  • Like top talkers that affect system performance based on vpc flow logs

Cloudwatch application insights

  • provides dashboards that display app perforamce (go, .net etc) running on ec2

Cloudtrail

  • Governance, compliance and audit for aws account
  • Enabled by default
  • get history of api calls/events
  • If a resource is deleted, then investigat ecloudtrail first
  • Types
    • Management events - operations performed on resources, logged by default. Configuring security, create subnet etc
    • Data events - read, write etc on s3. Disabled by default since they are heavy operations
    • Cloudtrail Insights events - to detect unusual activity in account
  • 90 days retention

AWS Config

  • Helps with recording and auditing compliance of aws resources
  • It helps with finding - is there unrestricted ssh acess to my security groups. Are my buckets having public access
  • It’s a per region service.
  • The results can be aggregated over regions and accounts
  • Store this data in s3
  • AWS has confiugured rules and you can create your custom ones
  • It doesn’t block resources, it just alerts about these things
  • Auto remediation action can be triggered to remediate thiese issue

AWS Organizations

  • Global Service
  • Allows to manage multiple aws acconts
  • Main account in management account. Others are member accounts and they can be part of only one organization
  • Consolidated billing, single payment method. Pricing benefits, discounts etc
  • Savings from reserved instances,
  • It has Root Organizational Unit (ROU) which is management account. Then you have multiple Oganization Units within that.
  • Like Dev/Prod OUs, Sales, HR, Dev OUs etc are the use cases
  • Advantages - security control poicies. Management account has complete control over policies
  • IAM Conditions
    • In resource policy we have ca apply condition in the json. Some examples include - deny based on source ip, deny access to specific regions, restrict actions based on resource tags, force MFA for users to delete resources, PrincipalOrgID,
  • Roles vs resource based policy
    • Consider s3, you can use iam role, so that user assumes that iam role and accesses s3 bucket. Or the other way is user is provided with specific s3 bucket policy.
    • Lambda, SNS, SQS, Cloudwatch logs, api gateway use resource based policy. Kinesis streams, systems manager run command, ECS task - IAM role
  • IAM Permission boundary - can be set to users and roles to limit the permissions
  • IAM Identity center
    • Single sign on for aws accounts in aws orgs
    • Also for business cloud apps (salesforce etc)
    • SAML 2.0 enabled apps
    • EC2 windows instances
    • Identity providers - built in in IAM identity center or can be Okta, AD, OneLogin etc
    • You define permissions sets for it - a collection of one or more iam policies assigned to users and groups to define aws access
    • Attribute based access control

AWS Directory services

  • Uses domain controller. Users can login to any computer which is connected to domain controller
  • Types
    • AWS managed Microsoft AD - Create your own AD in aws. Establish trust connections with on prem AD
    • AD Connector - Directory gateway to redirect to on premise ad. Users are managed on prem AD
    • Simple AD - AD-compatible managed directory on AWS, cannot be joined with on prem AD
  • Active Directory Setup
    • Connect to AWS managed AD - Connect IAM identity center with AWS managed AD
    • Connect to self managed AD - two types
      • Create a two way trust relationship using AWS managed AD and trust it with on prem AD
      • Create AD connector

AWS control tower

  • Easy way to set up and govern a secure and compliant multi account aws environment based on best practises
  • It uses AWS organizations to create accounts
  • Benefits
    • Automate setup of environment in few clicks
    • Automate policy management using guardrails
    • Detect policy violations and remediate them
    • Monitor compliance through dashboard
  • Control Tower with Guardrails
    • Preventive guardrail - using SCP. Restricts regions access etc
    • detective Guardrail - using AWS config. It will notify concerned users

Key Management Service (KMS)

  • Manages encryption key for you
  • You can audit key usage using cloudtrail
  • Seamlessly integrate with other aws services
  • You can use it using cli/api
  • Key types
    • Symmetrics - AES 256. You never get access to the unecrypted key.Some aws services are integrated with it
    • Asymmetric - RSA & ECC pairs. Public key is downloadable, but can’t access private key unecrypted
  • Types of keys
    • AWS owned keys - free, SSE-S3, SSE-SQS,
    • AWS managed key - free, (name starting with aws/service-name), key rotation automatic every year
    • Customer managed keys created in kms - 1$/month
    • customer managed keys imported - manual rotation using alias
  • It’s a region specific service
  • KMS key policy - Default key policy, custom key policy
  • KMS multi region keys - They can be created in one region and decrypt in another region. They are not global. They are primary + replicas
  • Used on global aurora, global dynamo db
  • Sharing AMI with KMS

SSM Parameter Store

  • Secure Storage for confiuguration and secrets
  • Optional seamless integration with kms
  • Version tracking in supported
  • Integration with cloudformation
  • Parameter store is stores in similar to file heirarchy
  • Paramter tiers - standard (free, 10000 max), advanced (paid, 100,000, parameter policy supported)
  • Paramter policy - assgin expiration time, no change notification etc
  • Can be used to store some sensistive environment variables data

AWS Secrets Manager

  • Meant for storing secrets
  • Capability to force rotation of keys exvery few months.
  • Similar capability as ssm parameter store, but much better
  • Well integrated with other services
  • Multi region secrets - secrets replicated with primary + read replica concept
  • Secrets can be stored with KMS

ACM (AWS Certificate Manager)

  • Provision, manage and deploy tls certificates
  • Free of charge for public certificates
  • Supports public and private certificates
  • Automatic TLS certificate renewal
  • ACM sends 45 days before expiration

Web Application Firewall (WAF)

  • Protects layer 7 exploits
  • Deploy on ALB, API gateway, Cloudfront, Cognito pool, AppSync GraphQL api
  • Web ACL (access control list) can be defined - filter based on ip, xss, http headers, body, geo match etc, rate based rules (DDos Protection)
  • Supported on ALB not NLB
  • Shield (DDoS protection)
    • Types
      • AWS field standard - free service, lay 3/4 attacks, syn/udp floods
      • AWS shield advanced - $3000 per month, for more sophisticated attacks, 24/7 DDoS response team support, automatically deploys rules on AWS WAF

Firewall Manger

  • Manage firewall rules across accounts in aws organization
  • Security policy can be set - waf rules, aws sheild advanced, security groups for ec2, network firewall, dns firewall,
  • Automatically applies rules to new resources created in the organization

Amazon Guard Duty

  • Intelligent threat discovery to protect aws account
  • Uses ML, anomaly detection, 3rd party data
  • One click to enable
  • Input inlcudes - cloudtrail logs, vpc flow logs, dns logs,
  • can protect against crypto attacks
  • You can setup eventbridge to send notifications
  • Use case: Guard duty creates cloud watch events which are then sent to lambda which modifies the security group of an instance to block certain ip
  • Amazon Inspector
    • Runs security assesments on EC2 instances that are running
    • Leverage AWS System Manger Agent on ec2 and inspector asesses it.
    • Analyses unintended network accessiblity
    • Analyze OS for known vulnerabilities
    • For container images moved to ECR
    • For lambda functions
    • reports it to AWS security hub

Amazon Macie

  • Fully managed data security and data privacy servicethat uses ML, pattern macthing
  • Matches and alerts for Personally identifiable info on S3

Security Hub

  • Visualize security reports
  • Collect data from guardduty, macie, inspector. Also does compliance checks
  • Can be integrated with other tools
  • Anonymous access via the analysis of AWS CloudTrail logs and events

VPC (Virtual Private cloud)

  • Private IP range - 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16. Rest all ar epublic IPs
  • Default VPC has internet connectivity
  • By default a VPC is created for your aws account
  • You can have 5 vpc in a aws region (soft limit)
  • Default route tables are created for each Amazon VPC and subnet you create.
  • When the CIDR blocks for route table entries overlap, the more specific (smaller range) CIDR block takes priority.
  • Max CIDR size in VPC in /16 and CIDR should not overlap
  • They are tied to a region
  • Subnet ranges cannot be changes after VPC creation
  • VPC is region specific, you can’t use one vpc in another region, subnet can live in only one AZ
  • The only architectural difference between a public and private subnet is that a public subnet has a route to an internet gateway.
  • AWS reserves 5 IPs in each subnet
    • Network address - first ip in the cidr range, e.g 10.0.0.0
    • 10.0.0.1 - reserved by aws for vpc router
    • 10.0.0.2 - reserved by aws for mapping to amazon provided dns
    • 10.0.0.3 - reserved for future use
    • 10.0.0.255 - network broadcast address
  • After you create VPC, you can create public or private subnet, and mention cidr for each subnets
  • Internet Gateway
    • Allows resouces in VPC to connect to internet
    • One VPC can be connected to only one IGW
    • An internet gateway needs to be attached to VPC
  • Route Table
    • Define how you want the data to flow in a network
  • Bastion host - used to ssh into private ec2 instances. Bastion is in the public subnet, which is then connected to all private subnets. Bastion security group must allow inbound from port 22 from restrcited cidr, for example public cidr of the corporate. Security group ec2 instances should allow the ip of bastion host
  • NAT Instance - Allows ec2 in private subnets to connect to internet, must be launched in public subnet. Must disable setting source/destination check. Must have elastic ip attached to it. Route tables should be configured to route traffic from private subnets to nat instance
  • NAT instance source/destination check flag should b edisabled because the outer server from which the response comes will think the request came from NAT instance IP, but it’s actually coming from the private instance hiding behind NAT instance. specific NAT AMI is required to be used for this
  • NAT Gateways
    • AWS managed ,per hour of usage
    • Specific to AZ, uses elastic IP,
    • Highly available, but not really fualt tolerant, but you can use multi az
    • Can’t be used by EC2 instance in the same subnet (only from other subnets)
    • Requires IGW - it’s like Private Subnet > NATGW > IGW
    • No security groups to be managed
    • NAT gateway is resilient in single az. For high availability, create multiple nat’s in different AZ’s
    • NAT gateway should be in public subnet
    • 5Gbps, upto 100Gbps
    • NAT gateway can make calls from inside, not receive calls from outside
  • Network Access Control List (NACL)
    • Security for subnet level
    • The default network ACL allows all traffic in and out by default.
    • It’s stateless, comapred to security groups which are stateful. Security groups are stateful because a request that goes inside the ec2/other then automatically it is allowed to go out also and vice versa also. But in case of stateless, NACL, it will blindly follow what is metioned in inbound and outbound rules. Security groups has allow rules only, NACL has allow and deny rules
    • You setup Inbound and outbound rules for NACL
    • NACL is like a firewall which control traffic from subnets
    • First rule that matches will drive the decision
    • By default NACL allows everything with the subnets it’s associated with
    • Ephemeral Ports -
  • VPC Peering
    • Privately connect two VPC’s using aws network
    • Make them behave as if they were on same network
    • Peering can be done across different regions
    • They must not have overlapping CIDR
    • They are not transitive. That is if peerinf is on for VPC A to VPC B, VPC B to VPC C, then inherently there is no peer connection between A and C, you need to peer it manually for A to C also.
    • You can do this cross accounts and cross regions also
  • VPC Endpoint
    • IGW is used to connect to the internet, but if you want to connect to other AWS services like dynamodb, then you can create a VPC endpoint and all services can use this using the private AWS network
    • Types
      • Interface Endpoint - Provisions ENI (private IP) as an entry point, Paid service, supports most of the services. Uses private link
      • Gateway endpoint - provisions gateway and must be setup in security group, free, support s3, dynamo db, doesn’t use aws private link, add route table entry for it to work.
  • VPC Reachability Analyzer - Analyze if there is a path between the provided source and destination. It aids in troubleshooting network issues by verifying connectivity using automated reasoning. All resource configurations (security groups, routes, firewalls, and so on) that can affect the connectivity of your network are inspected to determine if the network flow is possible. VPN gateways, Network interfaces, internet gateways, vpc endoints, vpc peers, transit gateways. The output will be if reachable or not
  • VPC Flow logs
    • Capture info about traffic going into your interfaces
    • VPC flow logs, subnet flow logs, ENI flow logs, these different levels of logs are supported
    • Monitor and troubleshoot connectivity issues
    • VPC flog log contains - version, interface id, src/dest ip/port, packets, start time, action
    • You can use these logs to do analytics for usage patterns
    • Query VPC flow logs using Athena and S3 or cloudwatch logs insights
    • Look at action field in the log to determine and access related issues
  • Site to Site VPN
    • Used to connect corporate vpn to aws networks
    • IPsec technology
    • One AWS Site-to-Site VPN connection consists of two tunnels. Each tunnel terminates in a different Availability Zone on the AWS side, but it must terminate on the same customer gateway on the customer side.
    • Virtual Private Gateway (VGW) - VPN concentrator on the AWS side of the VPN connection. VGW is created and connected to the VPC from which you want to create a S2S VPN connection
    • Customer Gateway - Software device or hardware on the customer side of vpn connection
    • VGW is on the AWS side and customer gateway is on the customer side
    • IP used for this is the IP addres of the customer gateway device. It it is behind a NAT, then IP of NAT device. You need to enable Route Propogation for virtual private gateway in the route table that is associated with your subnets
    • AWS VPN Cloud Hub - Provides secure communication between multiple sites if you have multiple vpn connection. Low cost hub and spoke model for primary or secondary network connectivity between different locations
  • Direct Connect (DX)
    • Provides a dedicated privat econnection from a remote network to your VPC
    • Dedicated connection must be setup between your DC and AWS Direct connect locations
    • YOu need a VGW on your VPC
    • BEnefit is you can private EC2 and public S3 data on the same network
    • Direct connect gateway to connect one or more VPC in many different regions
    • Takes around a month to get this connection
    • Direct Connect Connection types -
      • Dedicated - 1Gbps, 10, 100. Physical ethernet port provided to customer, request made to aws first then completed by aws direct connect partners
      • Hosted - 50Mbpx, 500, 10Gbps, capactiy can be added on demand,
    • Data in transit is not encrypted but is private, but you can setup vpn on top of it to get that safety
    • High resiliency can be achieved by adding multiple aws direct connect locations in different corporate locations. Max resiliency can be achieved by adding seperate connection for same direct connect location
    • Site to Site VPN can be setup along with this to setup as backup for this setup
    • AWS PrivateLink provides a private network connection between VPCs and AWS services, while AWS Direct Connect is a dedicated, private connection between on-premises infrastructure and an AWS Location
  • Transit Gateway
    • It gets really complicated with so many VPCs, peer to peer vpc connection etc. To solve this, use Transit gateway which allows you to connect 1000s of VPCs to transit gateway. And then you connect it to aws direct connect gateway. It uses hub and spoke connection
    • For example you can habe multiple vpcs in a region connected to transit gateway. And multiple transit gateway in each region are connected by transit gateway peering
    • Site to site VPN ECMP(equal cost multi path). Routing strategy to allow to forward a pacjet over multiple best path. Use case - create multiple site2site vpn to increase the bandwidth of your connection to aws
    • Share direct connect with multiple accounts
    • CIDR should not overlap
    • Can connect - VPC, Direct connect, VPN connection with customer gateway
    • The AWS Transit Gateway Network Manager lets you to centrally manage your networks that are built around transit gateways. You can visualize and monitor your global network across Regions and on-premises locations.
  • Traffic Mirroring
    • Allows you to capture and inspect netowkr traffic in vpc
    • Route the traffic to security appiances that you manage
    • Capture traffic from an ENI and send to an ENI or network load balancer
  • IPv6 in VPC
    • IPv4 cannot be disabled, but ipv6 can be enabled and operated in dual stack mode
  • Egress on Internet Gateway
    • Used only for IPv6 (simlar to NAT gateway used for ipv4)
    • You need to update route tables
  • Networking costs
    • Traffic coming into aws is free
    • Traffic in AZ between instances using private IP is free
    • Between AZ’s if you are using public/elastic IP then 0.02 dollars but if you use private IP then it’s much lesser
    • Inter-region traffic is 0.02 dollars
    • Use VPC endpoint to communicate with S3 rather than Internet gateway, since in case of internet gateway, the traffic goes out of aws network and hence there is higher cost involved for that
  • AWS Network Firewall
    • Used to protect entire VPC
    • Layer 3 to 7 protection
    • Fine grained control based on domain, ip, port, regex, allow/drop action, and logs can be sent to other services for analysis
  • Network Foundation
    • Amazon VPC
    • AWS Transit Gateway - Manage connection between VPCs and on prem data center, instead of doing vpc mirroring on multiple vpc’s
    • AWS Private Link - Helps you build a private connection between VPCs, other aws services and on prem network. No support for ipv6.
  • Hybrid Connectivity
    • AWS Direct Connect - direct connect between your data center and aws
    • AWS Cloud WAN
    • AWS Client VPN
    • AWS Site to Site VPN
  • Edge Networking
    • CloudFront
    • Route53
    • Amazon Global Accelerator - optimizes users traffic from user to application using aws global network along with edge locations
  • Application Networking
    • Amazon API Gateway
    • AWS App Mesh
    • AWS Cloud Map
  • Network Security
    • AWS Firewall Manager - Across aws account
    • AWS Shield - DDos proection
    • AWS Network Firewall - security across vpc
    • AWS WAF
  • Network Design Patterns
    • Hub and Spoke - Transit Gateway
    • High Availability Networks - Using multiple Direct connect partner routers and customer gateways

Disaster Recovery

  • Recovery Point Objective (RPO) - Data loss, from the time when last backup was taken all the way upto disaster
  • Recovery Time Objective(RTO) - Downtime, from the disaster time until it is recovered
  • RPO and RTO numbers affect the cost of architecture and complexity of building aws services
  • Disaster Revocery Strategies
    • Backup & Restore - High RPO, less cost
    • Pilot Light - A small version on app is always running, Useful for critical core, very similar to backup and restore, haster than backu & restore since some systems are already running
    • Warm Standby - Full systems are up and running, but ar minimum size, but upon disaster you scale to production load,
    • Hot Site/Multi Site Approach - very expensive, very low RTO, sull scale is running

Database Migration Service (DMS)

  • Migrate databases to AWS
  • Supports - homogenous migrations (oracle to oracle), heterogenous migraton (mssql to aurora)
  • Source DB remains available during the process
  • You need to create EC2 to perform replication task
  • DMS Sources can be - ec2 instance databases, azure sql, rds, s3, document db
  • DMS Targets - On premises & ec2 instances, rds, redshift, dynamo db, s3, opensearch, document db, kafka, nepture
  • If source and destination are same, the n you use AWS Schema Conversion Tool (SCT)
  • DMS can be used for multi az deployment with continuos replication setup
  • RDS & Aurora Migrations
    • RDS Mysql to Aurora Mysql
      • Option 1: DB snapshot and restore
      • Create read aurora read replica from rds mysql, and once replication lag is 0, promote aurora mysql as own db cluster (time consumeingf and costly)
    • External MySQL to Aurora
    • Use DMS
    • Similar thing can be done for postgresql also
  • On Premise strategy with aws
    • ability to download amazon ami as vm
    • VM Import/Export - allows you to migreate exisitng apps to ec2
    • AWS application discovery service
    • AWS DMS
    • AWS Server migratrion service
  • AWS Backup
    • Managed serice
    • manage and automate backup across aws services
    • create custom scripts and manual processes
    • Supporter - EC2, EBS, S3, RDS, and many more
    • Supports corss region backups
    • Supports cross account backups
    • PITR
    • On demand and scheduled backups
    • You create backup plan with retetion policy and bunch onther parameters
    • AWS Backup Vault Lock - enfore a write once, read many strategy for all backups in aws backup vault. Additional layer of defence to manage threats, delete operations etc. even root user cannot delete a backup.
    • Supported services - EBS, EFS, S3, FSx for windows file server, fsx for lustre, rds, aurora, neptune, dynamodb, documentdb, vmware, aws storage gateway volume gateway

Application Discovery Service

  • Plan migration by gathering info about on prem data centers
  • Server utilization data and dependency mapping is important for migration
  • Types
    • Agentless Discovery
    • Agent based discovery
  • Resulting data can be seen in AWS migration hub

Application Migration Service

  • Lift and shift solution which simplifies migration of applications
  • Converts physical, virtrual, cloud servers to run natively on aws

VMware Cloud on AWS

  • Some customers use vmware to manage on premises data center
  • They want to extend data center capacity to cloud also
  • This helps them to vSphere, vSAN,

EC2 Enhanced Networking (SR-IOV)

  • High bandwidth, higher PPS, low latency
  • Option 1: Elastic Network Adapter (ENA), upto 100Gbps
  • Option 2: Intel 82599 upto 10Gbps, LEGACY
  • Elastic Network Adapters (ENAs) provide traditional IP networking features that are required to support VPC networking. EFAs provide all of the same traditional IP networking features as ENAs, and they also support OS-bypass capabilities. OS-bypass enables HPC and machine learning applications to bypass the operating system kernel and to communicate directly with the EFA device
  • Elastic Fabric Adapter EFA
    • Improved ENA for HPC, only on linux

AWS Batch

  • Specify parameters of compute required
  • Runs on serverless tech on fargate
  • Long running jobs
  • Fully managed service
  • Amazon batch on EKS - dynamic compute provisioning and scaling
  • Batch with Fargate, can also using fargate spot instances

AWS Parellel Cluster

Cloud Formation - declarative way of outlining aws infra. Infrastructure as a code

  • Cloudformation stack designer

Amazon SES - Simple email service

  • Fully managed email service
  • inbound/outbound emails
  • DKIM/SPF

Amazon Pinpoint

  • Scalable 2 way (inbound/outbound) marketing communications service
  • Supports SMS, email, push, voice, and in-app messaging
  • Ability to segment and personlize messages with right content to right customers
  • For running marketing campaingns

SSM (Systems Manager)

  • Once you install SSM agent on EC2 is becomes managed instance
  • Use SSM to connect to your hosts, not exposing port 22.
  • can be installed on ec2, on prem servers etc to monitor and manager the system
  • Run commands, patching, os updates
  • SSM Session Manager
    • Allows you to start a SSH on your EC2 or on prem servers
    • No SSH Access, not ports, no bastion hosts
    • You basically have a SSM agent on the EC2 and then session manger which communicates with agent
    • Supports linux, windows, macos
    • Fleet manager is used to see all ssm managed ec2
  • SSM Other services
    • Run command - run a script, send it ot s3/others, run it across multiple ec2 instances
    • Patch Manager - schedule to perform actions
    • Automation - repeat deployment tasks, automation runbook

AWS Cost Explorer

  • visualize, manage cost over time, you can forecast, choose savings plan,

Elastic Transcoder

  • media file conversion

AWS Batch

  • for batch processing, fully managed, batch will dynamically luanch ec2 instances or spot instances, they should be defined as Docker images and run on ec2
  • Basically it’s similar to lambda, but it has not time limit

Amazon AppFlow

  • Fully managed integration service to transfer data between AssS apps and AWS, example sap, salesforce, zendesk,slack, service now. Destination can be aws services or saleforce

AWS Amplify

  • Set of tools to build mobile and web apps

AWS Trusted advisor

  • a tool to assess all aws services, around security groups, updates, snapshots etc. It offers basic, developer, and advanced paid plans

AWS Networking Gateways

  • Internet gateway is an Amazon VPC component that allows communication between your computer and the internet. Applications include, Elastic Load Balancers, Amazon EC2 instances, Amazon S3, AWS Lambda and so on.
  • Customer Gateway is a physical or software appliance that you own or manage in your on premises network. Applications include manages routing to and from your environment.
  • VPN Gateway is the gateway on the AWS side of site-to-site VPN connection. Applications include Amazon EC2 instances, Amazon S3, Amazon RDS< Amazon Lambda, and so on.
  • Direct Connect Gateway establishes connectivity that spans Amazon VPCs spread across multiple AWS Regions. Applications include Amazon EC2 instances, Amazon RDS, AWS Lambda, and so on.
  • NAT Gateway is a network address translation service that enables instances in a private subnet to connect to services outside your VPC. Applications include Amazon EC2 instances, Amazon RDS, AWS Lambda, and so on.
  • AWS Transit Gateway connects Amazon VPCs, AWS accounts, and on premises networks to a single gateway. Applications include Amazon VPC connections, AWS VPN connection, AWS Direct Connect.
  • Virtual gateway allows resources that are outside of your mesh network to communicate to resources that are inside. Applications include Amazon EC2, Amazon ECS, and Amazon EKS.

STS (Security Token service) - Temporary security credentials in IAM

  • allows you to request temporary security credentials for your AWS resources, for IAM authenticated users and users that are authenticated in AWS such as federated users via OpenID or SAML2.0.
  • Cross-Account Access using AWS STS
  • Identity Federation Use-Case
  • EC2 Instance STS Credentials

AWS Compute Optimizer

  • AWS Compute Optimizer is more specific to compute resources, such as EC2 instances, Auto Scaling groups, Elastic Block Store (EBS) volumes and Lambda functions. If the top contributors to cost are not related to any of these services, then AWS Compute Optimizer becomes less relevant.

S3 Origin Access Identity (OAI)

  • OAI prevents users from viewing your S3 files by simply using the direct URL for the file, Your users can only use the URL of your CloudFront distribution
  • OAC better than OAI
  • cannot set OAI if your S3 bucket is configured as a website endpoint

VPC Sharing

  • VPC sharing allows multiple AWS accounts to use a common vpc
  • VPC owner shares a subnet with a other accounts (participants) that belong to the same organization from AWS Organizations. After sharing participants can create,view modify on that subnet.

Lambda

  • Lambda Layers let you set common base elements for your Lambda functions, reducing code duplication and overall code deployment complexity.
  • Lambda provides your account with a total concurrency limit of 1,000 across all functions in a region.
  • Use reserved concurrency to reserve a portion of your account’s concurrency for a function. This is useful if you don’t want other functions taking up all the available unreserved concurrency.
  • Use provisioned concurrency to pre-initialize a number of environment instances for a function. This is useful for reducing cold start latencies.

Kinesis

SNI - Server name indication

  • With SNI support we’re making it easy to use more than one certificate with the same ALB. The most common reason you might want to use multiple certificates is to handle different domains with the same load balancer.

AWS Resource Access Manager (RAM)

  • With AWS Resource Access Manager (AWS RAM), you can share resources that are created and managed by other AWS services. You can share resources with individual AWS accounts. You can also share resources with the accounts in an organization or organizational units (OUs) in AWS Organizations. Some supported resource types also let you share resources with individual AWS Identity and Access Management (IAM) roles and users.
  • Elastic IP can be assigned to NLB not to ALB
  • SCP if you want to restrict or implement a policy across several accounts in the organization.
  • need to add IPv4 subnet first before you can create an IPv6 subnet.