Saturday, June 21, 2025

Dynamo db

Dynamo db

Class 51th AWS Dynamo db 21th

What is nosql database?

  • Nosql databases are a category of database management system that differ from traditional relational  database(SQL) in several key ways.
  • They are designed to handle large volumes of unstructured, semi-structured, or structured data with high performance, scalability, and flexibility.
Key Characteristics of No sql database

Schema Flexibility

Unlike relational databases, which require a predefined schema, Nosql databases allow for dynamic and flexible schema

Horizontal scalability

Nosql databases are designed to scale out distributing data across multiple servers or node,this contrasts with vertical scaling in relational databases,which often involves adding more resources to a single server.

High performance

They are optimized for high read and write performance, often at the expenses of strong consistency. This makes hem suitable for applications requiring low-latency data access.

Types of NoSQL database

Key-value stores

Eample :Redis,dynamoDB

Description:Data is toed as key-value pairs ,These databases are highly performance and are used for caching session management, and real-time data processing

Document store

Example:MongoDB,CouchDB

Description:data is stored in documents(usually Json or Bson),Document stores are flexible and can handle complex, hierarchical data structure, making them suitable for  content management and web applications

Column-family stores

Example:Apache Cassandra,Hbase

Description :Data is stored in columns rather than rows, and column are grouped into families.These database are optimized for read and write performance and are user in applications requiring high throughput and large scale data processing.

Graph database

Example:Neo4j,Amazon Neptune

Desciption:Data is represented as nodes,edges,and properties,graph databases are ideal for applications involving complex relationships and connections,such as social networks,recommendation engines and fraud detection

What is dynamoDB?

Amazon dynamoDB  is  a fully managed Nosql database service provided by  Amazon webservice(AWS)that delivered high performance, scalability and reliability

It is designed to handle large amounts of data with eash and allows developers to offload the administrative burden of operating and scaling a distributed database.

It is a serverless and public database service(database-as-a-service –Dbaas)key/value & document model.

Data stores in tables rows format.we call the rows as items.there is no affixed structure(schema) to the table

Partition key column must be unique. We can also add a sort key(optional) to organize the data more effectively.

AWS takes care of all the basic need like maintenance.Highly resilient across Azs,optinally global availability.

Backups,PITR,encryption at rest.Event-driven integration(on  database record changes)

DynamoDB table

DynamoDB table is a collection of items(in RDS we call rows)

No fiex structure required to maintain (expect partition and sort key columns)

Table can contain only simple key (partition) or composite key(partition+Sort)-primary key

Each item max size is 400 KB

No fixed structure required to maintain (expect partition and sort key columns)

Capacity unite(CUs) are the measurement for the DB performance.

To write the data write capacity units(WCU),1 WCU – 1 Kb per sec.

To read the data –read capacity units(RCU),1 RCU -4 kb per sec


Practical

Step1: Create one table emp,empno is partition key is the unique column 

Choose Provisioned 

Don't do any change click create table 
Tables created

Step2: to insert the records Select the the table action explore item,

Click create item, empno is partition key remaining attributes are non unique columns, click create item

Step3:Records are inserted successfully

Step4: See allow the records, with proper order  

here only we use the query safe performance issue, Scan take more performance issue if big table 

Step5: Query ,filter the record empno 1 

Step6: Scan filter


Step7: Creating one more table dept with sort key this composite key 

Secondary index 

Don't do any changes click create table 
Step8: create item enter the value click create item


Step9: i have deptno.json 20 records in local directory ,plan to upload CLI 
C:\Users\Administrator\Desktop\SofwareEngineering_E0>aws configure
AWS Access Key ID [****************IEW6]:
AWS Secret Access Key [****************xeEr]:
Default region name [eu-west-1]:
Default output format [json]:
File : deptno.json 
{
  "dept": [
    {
      "PutRequest": {
        "Item": {
          "deptno": {"S": "10"},
          "dname": {"S": "Sales"}
        }
      }
    },
    {
      "PutRequest": {
        "Item": {
          "deptno": {"S": "20"},
          "dname": {"S": "Marketing"}
        }
      }
    },
    {
      "PutRequest": {
        "Item": {
          "deptno": {"S": "40"},
          "dname": {"S": "Constuction"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "50"},
          "dname": {"S": "Tax"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "60"},
          "dname": {"S": "travel"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "70"},
          "dname": {"S": "service"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "80"},
          "dname": {"S": "Bus"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "90"},
          "dname": {"S": "tram"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "100"},
          "dname": {"S": "train"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "110"},
          "dname": {"S": "taxy"}
        }
      }
    },
{
      "PutRequest": {
        "Item": {
          "deptno": {"S": "120"},
          "dname": {"S": "boat"}
        }
      }
    }
  ]
}

C:\Users\Administrator\Desktop\SofwareEngineering_E0>aws dynamodb batch-write-item --request-items file://dept.json --region us-east-1
{
    "UnprocessedItems": {}
}
records uploaded successfully.


Scan filter the records

Step10: You can able replicate the table as well, with different region click Globally choose your region, whenever the records are inserted in the your region it was reflected automatically 
Step11: you can choose the region click create.

Step12:replicating creating

Step13: table is creating other region 
Table created 
Step14:Delete one record the main region us-Virginia click delete 120 deptno

Step15:Repicate region deleted the record successfully 
Step16:reverse delete the dept 50 record in Europ(paris)

Step17:deleted record main region also us-virginia


Both ways possible delete, replicate tables
Step18: Delete the table source side us-virginia

Getting error  due toe replicate table exist ,so to delete first target side 

Step19:Target side table delete successfully.
Step20:Table deleted successfully, 
source side us-virginia

--Thanks 






Thursday, June 19, 2025

RDS part3

RDS part3

Class 50th AWS RDS June 20th
Amazon aurora (AWS Deployed Aurora (MySQL Compatible),Aurora (PostgreSQL Compatible))
  • Amazon aurora is a relational database service offered by Amazon web service(AWS) that is designed for high performance, availability ,and compatibility with MySql and Postgresql
  • Amazon Aurora gives us to 128 TB of auto scaling SSD storage(its is 64 TB for RDS Mysql or other DB engines)
  • Up to five times the throughput of mysql community edition and times than Postgressql community edition.
  • Amazon aurora doesn’t available under free limitation, minimum 10 mins bill will be generated on creation (1 sec traffic follow)
  • Aurora Db available with cluster endpoint and read endpoint restore,clone and backtrack features available in the aurora.
backtrack means if any structure changes done wrongly, we can backtrack restore the change previous state. 

Step1:RDS Aurora Serverless 
Amazon Aurora serverless is a deployment option within amazon aurora(both Mysql and postgreSql-compatible editions) that automatically starts up,shutdown, and scales capacity based on your applications needs
Key Feature
Auto-scaling
Automatic capacity adjustment: Aurora server scales the database’s compute capacity seamlessly,based on the current workload.
Minimum and maximum capacity: you can define the minimum and maximum aurora capacity units(ACUs) to ensure that the database scales within the desired range.
Cost-Efficiency:
Pas-as-you-go: You only pay for the database capacity you actually use,billed per second with a minimum of five minutes of usage each time the database is active.
(Serverless means your are not maintained the server, based you demand of Usage, ACUs will increase automatically, we have issue early yar cold start whenever the use the database db container create through container will connect to the database though container, if are not use the database db container will destroy, if are trying to access again database it will take time to create container available to access data these are will complete in seconds we called these period call cold start. to over come the issue serverless always available container. these issue still exists in lambda function. )
Select Don't create aurora replicate,choose VPC and availability zone
select public access if required,

and then click create the database 
Blue-Green Deployment
Developer do the changes the final changes deploy into the production, instead of direct deployed into production here blue-green two instance are available zero downtime, apply the change first green environment, after successfully apply the change switch over to Blue ,so here zero downtime apply the changes.

Practical: Create one Free tier database mysql,with backup enable 
Step1:

Instance configuration :Burstable classes (includes t classes)
Public access :yes
VPC :select you VPC 
db subent grp:choose you subnet grp

Initial schema name give any (ccitdb)

Finally click create database
Step2:
Connect mysql endpoint, successfully.


Step3:Click blue/green deployment 
Step4:Given deployment name ccitdb-green ,no change 


if you want modify any configuration ,you can do else no change click next.
Below is the overview before create and then click create

Creating the database 


Step5:Created successfully Green database, here mysqldb-greenblue (Blue) is primary database
mysqldb-green is the secondary database.

Connected the secondary database mysql-green it is standby ready only replicate of primary

See here i have insert the record in the blue database
Records came automatically green database, here green is the read only database.


unable delete green database ready only, as see emp blue i have created, automatically came green database.

if required easily switchover standby to primary, once click switchover it will automatically changed standby to primary primary to standby.

Step6:Delete the stand by and then primary and then finaly db indentifier 


Step7: This Finally step



--Thanks 



RDS part2

RDS part2

Class 49th AWS RDS June 19th

RDS(Relational database system)

We have Different types of database Deployment available in AWS 

Single-AZ DB instance deployment (Single database instance,without stand by)

Multi-AZ DB Single instance deployment(2 instance, 1 for primary instance and 1for standby instances)

Multi-AZ DB cluster deployment (3 instance, 1for primary instance and 2 for readable standby instances)

Multi-AZ -Single instance 
Key feature of RDS multi-AZ deployments

High Availability.
Automatic failover(60-120 seconds)
Synchronous replication
Maintenance operations
Backups and snapshots on stand by instance
Note: It is not free service and it doesn’t support multi region access.
Below is the structure of the, 1 database instance provide primary avail zone, if any case primary was down, automatically stand by change to primary database without changing endpoint ,both the databases always synchronize replicate stand by with in milli second if changes are done in primary

Practical:
Msql installation:
Step1: https://dev.mysql.com/downloads/installer/ , Click Full 

click next >click execute >click next 
Authentication method  Choose below option
Use Strong password of authentication
Give mysql root user password and repeat password 
Click next 
Click next 
Server file file permission 
Click next  
Apply configuration
Click execute
Click Finish
Click Next 
Click Finish 
Product configuration 
Click Next 
Connect server given root password check 
Connect succeeded 
Click next 
Click execute 
Click finish
Click next 
Click finish
My sql installed successfully.

Synchronize :Once any record or change done database level immediately reflecting change Primary AZ 1-->Stand by after apply change send  acknowledgement to primary, after received acknowledgement then only further change will sent to stand by instance that is called synchronization
Asynchronized :sending the changes only not expecting any acknowledge for the stand by server

Multi-AZ -Single instance Only is available single instance to connect 

Step2: Multi Az DB instance deployment >select Mysql >Templates dev/test

DB instance identifier ccit-multi-az-single, give admin user master password  and instance configuration need to choose only Burstable classes it is less cost compared to standard classes

Standard classes give very large configuration  
Burstable classes give very less configuration 
db t3.micro ,20 GB minimum 

uncheck the enable storage auto scaling, select the existing VPC ,You need create subnet group

Step3:Create subnet group
Aurora and RDS >Subnet groups give any name "ccit_subnet_grp" choose you availability zone  and subnet click create


Step4: As see db subnet group came automatically after refresh ,select public access yes
initial schema created.

Additional monitoring settings
 Enhanced Monitoring
  Uncheck Enable Enhanced monitoring
Additional configuration
 Backup
 Uncheck  Enable automated backups
 Uncheck  Enable encryption
 Uncheck  Enable auto minor version upgrade
Click Create database
take the endpoint 


As you see below ccitdb default schema created automatically

CREATE TABLE `ccitdb`.`users` (
  `id` INT NOT NULL AUTO_INCREMENT,
  uname  varchar2(45),
  PRIMARY KEY (`id`));





Step5: if you want change the time database side , you need change Parameter groups

Current time 
select now()

2025-06-21 03:15:48
Aurora and RDS >Parameter groups




--Thanks

Tuesday, June 17, 2025

RDS part1

 RDS part1

Class 48th AWS RDS June 18th

Create Database RDS Database Cloud environment

Step1:
Aurora and RDS >Create database 
Choose a database creation method : Standard create 
Engine options :Mysql
Templates:Free tier
DB instance identifier:ccit-single-instance (give any name)
Choose Self managed  User Admin give password any

Step2:
Instance configuration: Burstable classes
Select the VPC and subnet Public access yes












Additional configuration: Backup  
uncheck
Enable automated backups
Enable encryption
Maintenance
uncheck 
Enable auto minor version upgrade

Step3: Click Create  database

Step4: After created ,Take the end point connect details connect the mysql in your local

Step5: Give the connect details click connect 


CREATE TABLE `ccitdb`.`users` (
  `id` INT NOT NULL AUTO_INCREMENT,
  `uname` VARCHAR(45) NULL,
  PRIMARY KEY (`id`));



--Thanks