AWS Storage & Databases

AWS Elasticl Block Storage

Amazon Elastic Block Store (EBS) is a scalable, high-performance block storage service designed for use with Amazon EC2 instances. It provides persistent, low-latency storage volumes that can be attached to and detached from EC2 instances as needed.

• Persistance for EC2 (consistant after reboot)

• snapshots for EBS volumes for backups

• replicates the working like the physical HDDs & SSDs

• Supports upto 16TB of single block storage

• Can survive after reboot

---

AWS S3 Bucket

Amazon S3 is an object storage service that provides scalable, durable, and secure storage for data of any size or format. It is designed for high availability and offers industry-leading performance, making it a foundational service for storing and retrieving data in the cloud.

• Store & retrieve unlimited amount of data

• stores data into the buckets & follows the bucket lifecycle policy

• versioning support

• create ACLs

• Supports various storage tiers

• supports 5TB single object size

• Highly durable (99.99999999999%)

S3 Storage Classes

Amazon S3 offers various storage classes designed to optimize costs based on data access frequency, durability, and latency requirements.

---

1. S3 Standard (General Purpose)

• Use Case: Frequently accessed data.

• Features:

  • 99.999999999% (11 9s) durability across multiple Availability Zones.

  • Low latency and high throughput.

  • Ideal for active datasets like website hosting, mobile apps, and gaming.

---

2. S3 Standard-IA (Infrequent Access)

• Use Case: Data accessed less frequently but requiring rapid access when needed.

• Features:

  • Lower storage cost compared to S3 Standard.

  • Retrieval fees apply for data access.

  • Suitable for backups or disaster recovery.

---

3. S3 One Zone-IA

• Use Case: Infrequently accessed data that does not require high availability.

• Features:

  • Stores data in a single Availability Zone, reducing costs further.

  • Ideal for non-critical, reproducible datasets.

---

4. S3 Intelligent-Tiering

• Use Case: Data with unpredictable or changing access patterns.

• Features:

  • Automatically moves data between access tiers based on usage patterns.

  • No retrieval fees; cost savings without performance trade-offs.

---

5. S3 Glacier Instant Retrieval

• Use Case: Rarely accessed data requiring millisecond retrieval.

• Features:

  • Ideal for long-term storage with occasional need for immediate access.

  • Low-cost storage with instant retrieval.

---

6. S3 Glacier Flexible Retrieval (formerly Glacier)

• Use Case: Archival data with occasional access needs.

• Features:

  • Retrieval options: expedited (minutes), standard (3–5 hours), and bulk (5–12 hours).

  • Cost-effective for archival workloads.

---

7. S3 Glacier Deep Archive

• Use Case: Long-term data archiving with infrequent access.

• Features:

  • Lowest-cost storage class in S3.

  • Retrieval time: standard (12 hours) or bulk (48 hours).

  • Suitable for regulatory compliance and long-term backups.

---

8. S3 on Outposts

• Use Case: Local data storage for low-latency applications or compliance requirements.

• Features:

  • Extends S3 capabilities to AWS Outposts (on-premises infrastructure).

  • Fully managed service providing consistent S3 APIs and functionality.

---

Each storage class is tailored to balance cost and performance for specific use cases, helping optimize your storage strategy in AWS.

---

AWS Elastic File System

Amazon EFS is a scalable and fully managed Network File System (NFS) designed for AWS cloud services. It provides shared file storage that can be accessed concurrently by multiple EC2 instances, containers, and on-premises servers, with seamless scalability and low-latency access.It is based on the linux file system.

---

AWS RDS (Relational Database Service)

Amazon RDS is a fully managed service for relational databases, making it easy to set up, operate, and scale databases in the cloud. It automates database administration tasks like provisioning, patching, backups, and scaling, freeing developers to focus on application development.

---

Supported Database Engines

1. MySQL: Popular open-source database known for scalability and reliability.

2. PostgreSQL: Advanced open-source database with robust features like JSON support and extensibility.

3. Oracle: Enterprise-grade database for high-performance applications with advanced security.

4. MariaDB: Community-developed fork of MySQL with enhanced features.

5. Microsoft SQL Server: Designed for enterprise-level database solutions with tight integration into the Microsoft ecosystem.

6. Amazon Aurora (MySQL and PostgreSQL-compatible): AWS-optimized relational database for high performance and availability.

---

Key Features of Amazon RDS

1. Migration (Lift & Shift):

   • Simplifies migrating on-premises databases to AWS, maintaining data integrity and minimal downtime.

2. Backup and Redundancy:

   • Offers automated backups, manual snapshots, and point-in-time recovery for data protection.

3. Failover and Disaster Recovery (DR):

   • Provides high availability through Multi-AZ deployments, with automatic failover to a standby replica in case of primary instance failure.

4. Managed Service:

   • Handles routine tasks like software patching, database monitoring, and maintenance, reducing administrative overhead.

5. Performance Optimization:

   • Supports scaling compute and storage independently based on workload requirements.

6. Security:

   • Includes encryption at rest and in transit, network isolation via VPC, and AWS IAM integration for fine-grained access control.

7. Monitoring:

   • Integrated with Amazon CloudWatch for performance metrics, alarms, and logging.

---

AWS Aurora

Amazon Aurora

Amazon Aurora is a fully managed relational database service designed for the cloud, offering high performance and availability. It is compatible with MySQL and PostgreSQL, combining the simplicity of open-source databases with the enterprise-grade reliability of commercial databases.It supports upto 15 read replicas & 6 backup copies at a time.

---

Key Features of Amazon Aurora

1. MySQL and PostgreSQL Compatibility:

   • Supports applications written for MySQL or PostgreSQL with minimal changes.

   • Leverages familiar tools, drivers, and ecosystem.

2. Managed Service:

   • Automates tasks like provisioning, patching, backup, recovery, and scaling.

   • Frees developers to focus on application innovation instead of database maintenance.

3. High Performance:

   • Up to 5x faster than standard MySQL databases and 3x faster than standard PostgreSQL databases.

   • Designed for low-latency, high-throughput workloads.

4. Scalability:

   • Automatically scales storage up to 128 TB per database instance.

   • Supports read replicas for handling increased query loads.

5. High Availability:

   • Automatically replicates data across multiple Availability Zones (AZs).

   • Provides continuous backups and point-in-time recovery.

6. Security:

   • Data encryption at rest and in transit using AWS Key Management Service (KMS).

   • Integrated with Amazon VPC for network isolation.

7. Global Database:

   • Allows replication across multiple regions for low-latency global applications.

---

DynamoDB

AWS DynamoDB

Amazon DynamoDB is a fully managed, serverless NoSQL database service designed for high availability, scalability, and low-latency performance. It is ideal for applications requiring fast and flexible data storage.

---

Features

1. Serverless architecture

2. Multi-AZ redundancy

3. Low-latency performance

4. Supports key-value and document data models

5. Auto-scaling for variable workloads

6. Integrated with AWS services (e.g., Lambda, IAM)

7. On-demand and provisioned capacity modes

8. Advanced security with encryption

Amazon DynamoDB is perfect for real-time applications like gaming, IoT, and mobile backends.

---

AWS Redshift

AWS Redshift

Amazon Redshift is a fully managed, petabyte-scale data warehousing service designed for analyzing large volumes of structured and semi-structured data. It supports advanced analytics and big data solutions for operational excellence.

---

Features

1. Handles large volumes of data

2. Optimized for data warehousing

3. Enables complex analytics and business intelligence

4. Scales to petabyte levels

5. Integrates with big data tools and AWS services

---

Other Databases on AWS

1. AWS DocumentDB

A fully managed document database service optimized for JSON-like workloads and MongoDB compatibility.

2. AWS Neptune

A fully managed graph database service designed for applications that work with highly connected datasets.

3. AWS Quantum Ledger Database (QLDB)

A fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log.

4. AWS ElastiCache

A fully managed in-memory caching service for accelerating application performance using Redis or Memcached.

5. AWS DynamoDB Accelerator (DAX)

An in-memory caching service for DynamoDB that significantly improves read performance for applications.

---

AWS FSx

Amazon FSx is a managed service from AWS that provides fully managed file storage solutions optimized for specific workloads. It enables you to set up and run file systems in the cloud, tailored to your needs, with native support for industry-standard protocols and features. FSx is ideal for workloads that require high-performance, low-latency file storage, such as big data analytics, machine learning, media processing, backup and restore, and enterprise applications.

Key Variants of Amazon FSx

AWS offers multiple variants of FSx to support different workloads and use cases:

1. Amazon FSx for Windows File Server

   • Provides a native Microsoft Windows file system.

   • Supports the SMB protocol.

   • Integrates seamlessly with Windows-based applications.

   • Features include Active Directory integration, DFS namespaces, and file system backups.

2. Amazon FSx for Lustre

   • A high-performance file system designed for compute-intensive workloads.

   • Fully compatible with Lustre, a popular open-source file system for high-performance computing (HPC).

   • Integrated with Amazon S3 for seamless data transfer between Lustre and S3 buckets.

3. Amazon FSx for NetApp ONTAP

   • Provides fully managed NetApp ONTAP file systems.

   • Supports NFS, SMB, and iSCSI protocols.

   • Offers enterprise features like data deduplication, compression, and SnapMirror replication.

4. Amazon FSx for OpenZFS

   • Delivers a managed version of the OpenZFS file system.

   • Provides ZFS-based capabilities like snapshotting, cloning, and data integrity checks.

   • Optimized for workloads requiring high throughput and low latency.

---

AWS Database Migration Service (DMS)

AWS DMS is a fully managed service that simplifies migrating databases to AWS with minimal downtime. It supports migrations between homogeneous (e.g., Oracle to Oracle) and heterogeneous (e.g., SQL Server to MySQL) database platforms while ensuring data integrity and replication.

• Secure & easy migration

• No downtime required

• Source DB type & target DB type can be different

• Homogeneous DB migration support

• Conversion from AWS schema conversion tool for heterogeneous migration

• Continuous database replication & data consolidation

• migrates the DB on AWS EC2 Instance

---

AWS Storage Gateways

AWS Storage Gateway is a hybrid cloud storage service that enables on-premises applications to seamlessly use AWS cloud storage. It connects your on-premises infrastructure with AWS, facilitating cloud storage use cases such as backup, archiving, disaster recovery, and data migration while maintaining local access to data.

Types of AWS Storage Gateways

AWS offers three types of gateways, each designed for specific workloads and integration needs:

1. File Gateway

   • Provides a file interface for storing files as objects in Amazon S3.

   • Supports industry-standard file protocols (NFS, SMB).

   • Enables local caching for low-latency access to frequently accessed files.

   • Ideal for use cases like backup, file sharing, and archive.

2. Tape Gateway

   • Enables you to use AWS cloud storage for scalable and cost-effective backup and archiving while maintaining compatibility with existing tape-based backup workflows.

   • Appears to backup applications as a virtual tape library (VTL).

   • Virtual tapes are stored in S3, and archived tapes are moved to Amazon S3 Glacier or S3 Glacier Deep Archive.

3. Volume Gateway

   • Provides cloud-backed storage volumes for block storage applications.

   • Operates in two modes:

     • Cached Mode: Stores frequently accessed data locally, while the full dataset is stored in Amazon S3.

     • Stored Mode: Stores the entire dataset locally and asynchronously backs up data to S3.

   • Supports iSCSI connectivity for use with on-premises servers.