Upgrade In-Place Aurora MySQL V1 to V2

 Mysql 5.6 is already the end of life back in Feb 2021, however, AWS is still supporting the last minor 5.6 version but they are pushing clients to migrate to MySQL 5.7. 
While the RDS MySQL 5.6 can't be created any longer they still support Aurora cluster V1 MySQL 5.6 compatible. 
AWS can upgrade to the major version on your behalf but It is preferable running some application tests beforehand.

Cloudformation and Aurora upgrade in-place 

An Aurora feature is to upgrade in-place the cluster which is currently the quickest way and AWS does the upgrade for you. The problem raises up when the DB cluster resource is created via Cloudformation. Currently, it is not possible to upgrade in-place via CFN, it fails with the error that the DB cluster and parameter groups are not supported.
Why is that?
The exiting stack defines the DB resources with the parameter groups compatible with MySQL 5.6.
When you deploy the newer template version, that updates the DB resources properties to the MySQL 5.7 engine version, the DB resources need to use the DB parameter groups compatible with MySQL 5.7 version. During the upgrade steps, AWS fails and it raises an error about the not compatible parameter groups with the engine version.

First Workaround to still maintain the DB resources in CFN

Ok, let's see how we can proceed to upgrade the DB cluster and still keeping the stack in sync.
By using the export / import CFN mechanism, we will upgrade the DB cluster in-place

1. Take a manual snapshot 
2. Add the DeletionPolicy: Retain to the DBCluster and DBInstances
3. Add the AllowMajorVersionUpgrade: true to the DBInstances
4. Remove from the template all the CW alarms, these can't be imported
5. Remove from the template all the DNSRecords, these can't be imported
6. Remove from the template the RDSsecretattachment, these can't be imported
7. Upload the CFN template
8. Testing if dropping the RDS Secret attachment hasn't changed  the master password
9. Comment the DBCluster, DBInstances resources in the template.
10. Upload the CFN template
11.  The DB Cluster and DB instances will be retained
1.  Check the stack's update logs, the DB resources should be in "DELETE_SKIPPED"
12. Add to the template the 5.7 cluster and parameter groups
13. Upgrade via console the Aurora cluster to 57 [1]
1. In the modification console, change the DB parameter and cluster groups to use the 5.7 created in the template
14.  Monitor the logs by console
15.  The DB instances need to be rebooted, the parameter group is in the pending-reboot state.
16.  Testing the DB is again accessible with the master user
17.  Update the template
1. Replace the DBcluster property to use the new cluster parameters group
2. Replace the DBcluster property engineVersion with the current new MySQL version
3. Replace the DBcluster property engine to aurora-mysql
4. Replace the dbinstances property to use the new parameters group
5. Replace the dbinstances property engineVersion with the current new MySQL version
6. Replace the dbinstances property engine to aurora-mysql
7. Uncommented the DBcluster, dbinstances definitions
18.  From the CFN console, choose the Import resources operation
19.  Remove from the template the old parameter groups 5.6
20.  Uncommented RDS Secret, CW, DNS Aliases
21. Upload the CFN template

Upgrade output 

[1]

August 06, 2021, 11:29:21 AM UTC DB cluster created

August 06, 2021, 12:06:08 PM UTC Upgrade in progress: Performing online pre-upgrade checks.

August 06, 2021, 12:06:59 PM UTC Upgrade in progress: Performing offline pre-upgrade checks.

August 06, 2021, 12:08:21 PM UTC Upgrade in progress: Creating pre-upgrade snapshot [preupgrade-testing-upgrade-cluster-5-6-mysql-aurora-1-22-2-to-5-7-mysql-aurora-2-10-0-2021-08-06-12-04].

August 06, 2021, 12:09:44 PM UTC Upgrade in progress: Cloning volume.

August 06, 2021, 12:11:57 PM UTC Upgrade in progress: Purging undo records for old row versions. Records remaining: 991

August 06, 2021, 12:13:53 PM UTC Upgrade in progress: Purging undo records for old row versions. Records remaining: 991

August 06, 2021, 12:13:57 PM UTC Upgrade in progress: Upgrading database objects.

August 06, 2021, 12:17:59 PM UTC Database cluster major version has been upgraded

Conclusion

This workaround will be still valid until either Cloudformation team or RDS team will have applied a bugfix to upgrading in-place Aurora Mysql via Cloudformation.

Thank you for reading it.