You can create an Amazon Aurora DB cluster as Read Replica in a different AWS Region that the source DB cluster.
Taking this approach can improve your disaster recovery capabilities, let you scale read operations into a region that is closer to your users, and make it easier to migrate from one region to another.
Limits
- The cluster can be created where Aurora is available, so this is tied up to the minimum of three Available Zone in the region. Like you cannot use eu-central-1 (Frankfurt )
- In a cross-region scenario, there is more lag time between the source DB cluster and the Read Replica
- Data transferred for cross-region replication incurs Amazon RDS data transfer charges.
- For each source DB cluster, you can only have one cross-region Read Replica DB cluster
- Because KMS encryption keys are specific to the region that they are created in, you cannot replicate encrypted DB clusters across
Pre-requisites
- Because KMS encryption keys are specific to the region that they are created in, you cannot replicate encrypted DB clusters across regions.
- A best practice is to create even the cross Aurora cluster in a VPC.
- You need to create three subnets each in a different AZ.
- First, to create an Amazon Aurora cross-region Read Replica, you must be running the most recent version of the Aurora database engine
- To upgrade your Aurora database to the most recent version, apply any pending maintenance actions for all of the instances in your DB cluster.
Setting at the beginning
Procedure step by step
- Create subnet group
- Create security group
- Create db parameter group
- Create db cluster parameter group
- Create db cluster
- Create db instance
Steup session variables to call the command in the right environment
export REGION='us-east-1'export VPC='vpc-99999999'
export TAG_COMMON='Key=Role,Value=members Key=Project,Value=migration'
Create subnet group
aws rds create-db-subnet-group --region $REGION --profile p-test \
--db-subnet-group-name gaming-service-database-7-test-subnet-group-name \
--db-subnet-group-description "Subnet group for test in $REGION" \
--subnet-ids $LIST_SUBNETS --tags $TAG_COMMON
Create security group
--group-name gaming-service-database-7-$REGION-security-group-name \
--description "Security group for gaming service in 7" --vpc-id $VPC --output text)
Add tags to the SG
Create db parameter group
--db-parameter-group-family 'aurora5.6' \
--db-parameter-group-name "gaming-service-database-7-$REGION" \
--tags $TAG_COMMON --description 'db parameter for gaming_service_database in 7'
Modify some default value for the parameter group
Create the db_params_group_modified.json parameter file
{
"DBParameterGroupName": "gaming-service-database-test-7-dbparametergroup",
"Parameters": [
{
"ParameterName": "wait_timeout",
"ParameterValue": "120",
"ApplyMethod": "immediate"
},
{
"ParameterName": "max_connections",
"ParameterValue": "16000",
"ApplyMethod": "immediate"
}
]
}
Call the method for generate the changes
--cli-input-json file://db_params_group_modified.json
Create db cluster parameter group
Call the method for creating the cluster parameter group.
--profile p-test --db-parameter-group-family 'aurora5.6' \
--db-cluster-parameter-group-name "gaming-service-database-t-7-$REGION" \
--tags $TAGS_COMMON --description 'DB Cluster Parameter Group'
Modify some default values for the cluster parameter group
Create the file db_params_cluster_group_modified.json json to pass as value
{
"DBClusterParameterGroupName": "gaming-service-database-t-test-7",
"Parameters": [
{
"ParameterName": "binlog_format",
"ParameterValue": "statement",
"ApplyMethod": "pending-reboot"
}
]
}
Then modify the db cluster parameter group
--profile p-test --cli-input-json file://db_params_cluster_group_modified.json
Create db cluster
Write the create_db_cluster.json file with the all information needed.
{
"BackupRetentionPeriod": 3,
"DBClusterIdentifier": "gaming-service-test-cross-region",
"DBClusterParameterGroupName": "gaming-service-database-t-test-7",
"VpcSecurityGroupIds": [
"sg-2c95f951"
],
"DBSubnetGroupName": "gaming-service-database-test-7-subnet-group-name",
"Engine": "aurora",
"ReplicationSourceIdentifier": "arn:aws:rds:eu-west-1:9999999:cluster:gaming-service-database-t-psj53bnufk19",
"Tags": [
{
"Key": "Name",
"Value": "gaming_service_database-7-test"
},
{
"Key": "Environment",
"Value": "test"
}
]
}
Then call the command to start the creation of the cluster
--cli-input-json file://create_db_cluster.json
Monitoring creation of the cluster
--db-cluster-identifier gaming-service-test-cross-region \
--profile p-test \
--region $REGION
Create db instance
--db-cluster-identifier $DB_CLUSTER_NAME \
--db-instance-class 'db.r3.xlarge' \
--db-instance-identifier "$INSTANCE_NAME-7-test-db-instance-1" \
--db-parameter-group $DB_PARAMETER_GROUP \
--engine 'aurora' \
--no-publicly-accessible \
--tags $TAGS_COMMON \
--query 'DBInstance.DBInstanceIdentifier' --output text
Monitoring creation of the instance
--db-instance-identifier "$INSTANCE_NAME-7-test-db-instance-1"
FAQ with Aws support
[1]Is any way that I can estimate how long will take to cross copy roughly 2 TB of data ?
[AWS] There is no way you can estimate the time it will take to create a cross region read replica. I would still encourage you to follow the below steps in the documentation given below before creating a read replica.
While I was creating the cross region read replica(N.Virginia), the source cluster (Ireland) was sometime in 'modifying' state. Do you know why ?
The reason for source cluster was in the modifying state could be because it was because of the more lag time or because of the unavailability of a availability zone that means no capacity of your db cluster class in the availability zone you requested, our system has to provision additional capacity and this process complete in sometime.
[3]Can the cluster creation process be stopped without any impacts on the source cluster ? Can I put in stand-by the creation process ?
[AWS] Once you trigger the creation process of promoting read replica or creating a cross region read replica, you cannot terminate/stand-by the creation process until it ends. So, I would encourage you to double check before triggering the create button.
[4]Can the source cluster have worse performance during the creation ? In term of performance, what can on the source be impacted ?
In this process, the database of the source cluster will be up and running but there will be a certain lag which will depend on many different factors. Replication is indeed slower over the region, than over the availability zone. Network route adds up to replication lag, however the lag is mostly influenced by the following two factors:
This is the reason why I am not able to provide you particular numbers. This has to be tested in the test environment first, then you should come up with a baseline and further on you have to monitor the latency to see if it's acceptable or not.
Can I figure out how much would be the replication lag ?
Network route adds up to replication lag, however the lag is mostly influenced by the following two factors:
- you might have long running transactions, that update data, stay open and later commit - causing the replication lag increase in numbers, however this is not caused by the performance of servers or replication;
- master instances use multiple threads to access and write the data, but replication is performed with a single thread, which leads to latency as well;
- you might have long running transactions, that update data, stay open and later commit - causing the replication lag increase in numbers, however this is not caused by the performance of servers or replication;
- master instances use multiple threads to access and write the data, but replication is performed with a single thread, which leads to latency as well;
Since the dataset is roughly 2TB, the time to create the cross read replica will be quite long, I'm worry if the existing cluster (Ireland) can suffer of bad performance while the creation is ongioing. Is that true ?
I understand the dataset is 2tb which is huge, but I cannot assure you the estimated time it would take to create a read replica. It will not be minutes but in hours for sure. As I mentioned earlier the performance depends on the depends on many different factors. Replication is indeed slower over the region, than over the availability zone.
Just for my understanding, the creation process will restore a snapshot taken and saved in S3, and then a read replica instance is created from the snapshot. That means the creation process doesn't read directly from the source cluster in Ireland until the cluster in created. Then once it is ready it will kick off the replication process which reads directly from the cluster(Ireland). Do you confirm this ?
When you create a read replica, you specify an existing DB Instance as the source. Amazon RDS takes a snapshot of the source instance and creates a read-only instance from the snapshot. For Aurora, Amazon RDS uses those engines' native asynchronous replication to update the read replica whenever there is a change to the source DB instance. The read replica operates as a DB instance that allows only read-only connections; applications can connect to a read replica just as they would to any DB instance. Amazon RDS replicates all databases in the source DB instance.
Do you assure me that there won't be any impacts on the source cluster in Ireland during the creation cross read replica phase and then when the replication is set up ?
I apologize but I cannot assure that there won't be any impacts on the source cluster. I can provide you the documentations which will be the best practice to follow in order to avoid any impacts after creating a read replica.
Documentations