/*Here i feel is the placement of version dropdown*/

Upgrade CCM and Database in HA Mode

Overview

Icon

Be sure to review Upgrade Overview before starting this procedure!

This section provides details on upgrading your CloudCenter deployment in HA mode. 

Icon

Prerequisites

Icon

Be aware that the CCM and DB servers will be offline during the upgrade process. Schedule some down time for your enterprise before starting this process.

Verify these requirements before you begin the upgrade process:

  • Review the information provided in the Upgrade Overview section and validate the following requirements for the release to which you are upgrading:

    • Is an upgrade path available?
    • Is the core_upgrade.bin file required?
  • See High Availability Best Practices for HA considerations.

  • For each CCM (CCM primary and CCM secondary) and each PostgreSQL (DB master and DB slave) instance that must be upgraded, verify the following prerequisites:

    • Ensure that a version file (/usr/local/osmosix/etc/version) exists in both CCMs to be upgraded.

    • Verify that the version file contains the correct version number (for example, if your current CloudCenter release version is 4.7.2, ensure that the corresponding version value is 4.7.2).

    • See the corresponding release notes for release-specific information on the CloudCenter version to which you are upgrading. For example, the CloudCenter 4.8.0 Release Notes.

Backup Database

Backup your database and application (the following example uses /mnt, you can change this directory as applicable).

Backup from 4.5.x
Icon

Osmosix users do not have permission to use the -R option. CloudCenter uses the GetVendorList routine. To backup this routine along with the rest of the database, you must provide the -R option using your root user credentials.

Backup from 4.6.x

Download the Upgrade Packages

Download package files:

Icon

See Virtual Appliance Overview to understand the required components.

See Phase 4 Overview  to understand the various types of software download files.

  1. SSH into the VM instance designated for this component by using the key pair that you used to launch the VM.

    Icon

    Along with the key pair, you may need to use your login credentials for sudo or root access based on your environment.

  2. Download the following required files for this component from software.cisco.com:

    • ccm-installer.jar
    • ccm-response.xml

    • core_upgrade.bin

Select Your Upgrade Scenario

Your upgrade process differs depending on your instance setup. Ascertain the following considerations before you begin the CloudCenter upgrade.

Scenario
Existing Scenario
Upgrade Scenario
Follow the Process in this Section

Scenario 1


 

  • Instance 1 = CCM primary
  • Instance 2 = CCM secondary
  • Instance 3 = MySQL
  • Instance 4 = MySQL
  • Instance 1 = CCM primary
  • Instance 2 = CCM secondary
  • Instance 3 = PostgreSQL master
  • Instance 4 = PostgreSQL slave
Double CCM and Double PostgreSQL – A

Scenario 2

 

  • Instance 1 = CCM primary
  • Instance 2 = CCM secondary
  • Instance 3 = PostgreSQL
  • Instance 4 = PostgreSQL
  • Instance 1 = CCM primary
  • Instance 2 = CCM secondary
  • Instance 3 = PostgreSQL master
  • Instance 4 = PostgreSQL slave
Double CCM and Double PostgreSQL B
Scenario 3
  • Instance 1 = CCM primary  
  • Instance 2 = CCM secondary   
  • MySql  in either or both CCM instances
  • Instance 1 = CCM primary  
  • Instance 2 = CCM secondary
  • Instance 3 = PostgreSQL master  
  • Instance 4 = PostgreSQL slave
MySql in Either or Both CCM Instances
Scenario 4
  • Instance 1 = CCM  
  • Instance 2 = MySQL  
  • Instance 3 = MySQL
  • Instance 1 = CCM  
  • Instance 2 = PostgreSQL master  
  • Instance 3 = PostgreSQL slave
Single CCM and Double PostgreSQL
Scenario 5
  • Instance 1 = CCM primary
  • Instance 2 = CCM secondary
  • Instance 3 = MySQL
  • Instance 1 = CCM primary
  • Instance 2 = CCM secondary
  • Instance 3 = PostgreSQL
Double CCM and Single PostgreSQL

Scenario 1: Double CCM and Double PostgreSQL – A

 Scenario 1: Each in Separate Instances

Follow this process to upgrade the CCM-DB instances in HA mode.

  1. To upgrade both PostgreSQL database instances, follow this procedure.
    1. Install both database servers on the same cloud or datacenter.
    2. Ensure that the master and slave database servers are located on the same subnet. If the master and slave database servers are on different subnets (or VPCs) ensure that they are able to communicate with each other over private IP.

    3. Launch both instances using the MGMTPOSTGRES_MASTER IP and MGMTPOSTGRES_SLAVE IP.
    4. Run the following commands on each MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7 

      <cloudtype> = amazon, openstack, vmware

    5. Set up SSH connectivity between the master and slave VMs. Exchange the SSH keys between the MGMTPOSTGRES_MASTER IP and MGMTPOSTGRES_SLAVE VMs.

      1. On the MGMTPOSTGRES_MASTER, execute the following to generate a new SSH key. 

      2. Copy the id_rsa files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from the MGMTPOSTGRES_MASTER to the same location on the MGMTPOSTGRES_SLAVE. On the MGMTPOSTGRES_SLAVE, if the .ssh directory does not exist, create it using the following commands before copying the files.

      3. On the MGMTPOSTGRES_SLAVE, execute the following to generate a new SSH key.

      4. Verify mutual SSH access between the MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE by running the following command on each VM.

    6.  Start the database wizard and allow the CCM to access the database.

      MGMTPOSTGRES_MASTER – Configure High Availability Properties

      To configure high availability for MGMTPOSTGRES_MASTER, follow this procedure.

        1. SSH into the DB instance as a centos user.
        2. Run the following command:

      1. Invoke the wizard.

        MGMTPOSTGRES Wizard Path
      2. Configure Postgres HA to ensure the PostgreSQL database HA and enter the information in each field as follows:

        Write this down for future reference!

        Icon
        Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

        See Configure CCM Wizard Properties for other settings.

        Wizard Menu

        Field

        Description

        Configure_Postgres_HADB MasterThe hostname for the master database VM – not configurable.
        DB Master Private IPThe private IP address of the master database VM
        DB Slave HostnameThe hostname for the slave database VM
        DB Slave Private IPThe private IP address of the slave database VM

        VIP or EIP 

        The VIP/EIP IP for the database

        Icon

        Use your mouse to select this option.

         AWS Cloud Nuances for EIP
        To setup PostgreSQL as an RDS service in the SA or HA modes, see Configuring HA for PostgreSQL Database on AWS. 

        Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

        • Configuring database for HA ...

        • Configuring database for replication

      3. Exit the configuration wizard.

      4. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
        # pcs status

        1. Ensure that the PCSD Status for both database servers are Online.
        2. Ensure that the Daemon Status for Corosync, Pacemaker and the pcsd service are active/disabled.

       

      Back to CCM (Required)

      Icon

      If your database upgrade was successful, be aware that you can now stop both instances of MySQL as you no longer need these instances.

  2. Upgrade the CCM primary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, openstack, vmware

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder.

  3. Upgrade the CCM secondary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack,  azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder:

Scenario 2: Double CCM and Double PostgreSQL B

 Scenario 2: Each in Separate Instances

Follow this process to upgrade the CCM-DB instances in HA mode.

  1. Follow this procedure to upgrade both PostgreSQL database instances.
    1. Install both database servers on the same cloud or datacenter.
    2. Ensure that the master and slave database servers are located on the same subnet. If the master and slave database servers are on different subnets (or VPCs) ensure that they are able to communicate with each other over private IP.

    3. Launch both instances using the master IP and slave IP.
    4. Set up SSH connectivity between the master and slave VMs. Exchange the SSH keys between the DB master and DB slave servers.

      1. On the DB master, execute the following to generate a new SSH key. 

      2. Copy the id_rsa files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from DB master to the same location on DB slave. On DB slave, if the .ssh directory does not exist, create it using the following commands before copying the files.

      3. On the DB slave, execute the following to generate a new SSH key.

      4. Verify mutual SSH access between the DB master and DB slave by running the following command on each VM.

    5. Create a security group with the specified ports and launch the instances using this security group.
    6. Run the following commands on each PostgreSQL instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7 

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    7.  Start the database wizard and allow the CCM to access the database.

      MGMTPOSTGRES_MASTER – Configure High Availability Properties

      To configure high availability for MGMTPOSTGRES_MASTER, follow this procedure.

        1. SSH into the DB instance as a centos user.
        2. Run the following command:

      1. Invoke the wizard.

        MGMTPOSTGRES Wizard Path
      2. Configure Postgres HA to ensure the PostgreSQL database HA and enter the information in each field as follows:

        Write this down for future reference!

        Icon
        Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

        See Configure CCM Wizard Properties for other settings.

        Wizard Menu

        Field

        Description

        Configure_Postgres_HADB MasterThe hostname for the master database VM – not configurable.
        DB Master Private IPThe private IP address of the master database VM
        DB Slave HostnameThe hostname for the slave database VM
        DB Slave Private IPThe private IP address of the slave database VM

        VIP or EIP 

        The VIP/EIP IP for the database

        Icon

        Use your mouse to select this option.

         AWS Cloud Nuances for EIP
        To setup PostgreSQL as an RDS service in the SA or HA modes, see Configuring HA for PostgreSQL Database on AWS. 

        Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

        • Configuring database for HA ...

        • Configuring database for replication

      3. Exit the configuration wizard.

      4. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
        # pcs status

        1. Ensure that the PCSD Status for both database servers are Online.
        2. Ensure that the Daemon Status for Corosync, Pacemaker and the pcsd service are active/disabled.

       

      Back to CCM (Required)

  2. Upgrade the CCM primary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder.

  3. Upgrade the CCM secondary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder:

Scenario 3: MySql in Either or Both CCM Instances

 Scenario 3: Each in a Separate Instance and MySQL with CCM

Follow this process to upgrade the CCM-DB instances in HA mode.

  1. Follow this procedure to upgrade both PostgreSQL database instances.
    1. Install both database servers on the same cloud or datacenter.
    2. Ensure that the master and slave database servers are located on the same subnet. If the master and slave database servers are on different subnets (or VPCs) ensure that they are able to communicate with each other over private IP.

    3. Launch both instances using the master IP and slave IP.
    4. Set up SSH connectivity between the master and slave VMs. Exchange the SSH keys between the DB master and DB slave servers.

      1. On the DB master, execute the following to generate a new SSH key. 

      2. Copy the id_rsa files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from DB master to the same location on DB slave. On DB slave, if the .ssh directory does not exist, create it using the following commands before copying the files.

      3. On the DB slave, execute the following to generate a new SSH key.

      4. Verify mutual SSH access between the DB master and DB slave by running the following command on each VM.

    5. Create a security group with the specified ports and launch the instances using this security group.
    6. Run the following commands on each PostgreSQL instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7 

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    7.  Start the database wizard and allow the CCM to access the database.

      MGMTPOSTGRES_MASTER – Configure High Availability Properties

      To configure high availability for MGMTPOSTGRES_MASTER, follow this procedure.

        1. SSH into the DB instance as a centos user.
        2. Run the following command:

      1. Invoke the wizard.

        MGMTPOSTGRES Wizard Path
      2. Configure Postgres HA to ensure the PostgreSQL database HA and enter the information in each field as follows:

        Write this down for future reference!

        Icon
        Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

        See Configure CCM Wizard Properties for other settings.

        Wizard Menu

        Field

        Description

        Configure_Postgres_HADB MasterThe hostname for the master database VM – not configurable.
        DB Master Private IPThe private IP address of the master database VM
        DB Slave HostnameThe hostname for the slave database VM
        DB Slave Private IPThe private IP address of the slave database VM

        VIP or EIP 

        The VIP/EIP IP for the database

        Icon

        Use your mouse to select this option.

         AWS Cloud Nuances for EIP
        To setup PostgreSQL as an RDS service in the SA or HA modes, see Configuring HA for PostgreSQL Database on AWS. 

        Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

        • Configuring database for HA ...

        • Configuring database for replication

      3. Exit the configuration wizard.

      4. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
        # pcs status

        1. Ensure that the PCSD Status for both database servers are Online.
        2. Ensure that the Daemon Status for Corosync, Pacemaker and the pcsd service are active/disabled.

       

      Back to CCM (Required)

  2. Upgrade the CCM primary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder.

  3. Upgrade the CCM secondary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

      Icon

      if you have two MySQL servers (in cased of high availability setup), the mysql_host=<mysql_ip> value displays as mysql_host=<default>. In this case, you do not need to change the default value to mysql_ip.

    3. Run the following commands from your download folder:

Scenario 4: Single CCM and Double PostgreSQL

 Scenario 4: DBs in Separate Instances

Follow this process to upgrade the CCM-DB instances in HA mode.

  1. Follow this procedure to upgrade both PostgreSQL database instances.
    1. Install both database servers on the same cloud or datacenter.
    2. Ensure that the master and slave database servers are located on the same subnet. If the master and slave database servers are on different subnets (or VPCs) ensure that they are able to communicate with each other over private IP.

    3. Launch both instances using the master IP and slave IP.
    4. Set up SSH connectivity between the master and slave VMs. Exchange the SSH keys between the DB master and DB slave servers.

      1. On the DB master, execute the following to generate a new SSH key. 

      2. Copy the id_rsa files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from DB master to the same location on DB slave. On DB slave, if the .ssh directory does not exist, create it using the following commands before copying the files.

      3. On the DB slave, execute the following to generate a new SSH key.

      4. Verify mutual SSH access between the DB master and DB slave by running the following command on each VM.

    5. Create a security group with the specified ports and launch the instances using this security group.
    6. Run the following commands on each PostgreSQL instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7 

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    7.  Start the database wizard and allow the CCM to access the database.

      MGMTPOSTGRES_MASTER – Configure High Availability Properties

      To configure high availability for MGMTPOSTGRES_MASTER, follow this procedure.

        1. SSH into the DB instance as a centos user.
        2. Run the following command:

      1. Invoke the wizard.

        MGMTPOSTGRES Wizard Path
      2. Configure Postgres HA to ensure the PostgreSQL database HA and enter the information in each field as follows:

        Write this down for future reference!

        Icon
        Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

        See Configure CCM Wizard Properties for other settings.

        Wizard Menu

        Field

        Description

        Configure_Postgres_HADB MasterThe hostname for the master database VM – not configurable.
        DB Master Private IPThe private IP address of the master database VM
        DB Slave HostnameThe hostname for the slave database VM
        DB Slave Private IPThe private IP address of the slave database VM

        VIP or EIP 

        The VIP/EIP IP for the database

        Icon

        Use your mouse to select this option.

         AWS Cloud Nuances for EIP
        To setup PostgreSQL as an RDS service in the SA or HA modes, see Configuring HA for PostgreSQL Database on AWS. 

        Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

        • Configuring database for HA ...

        • Configuring database for replication

      3. Exit the configuration wizard.

      4. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
        # pcs status

        1. Ensure that the PCSD Status for both database servers are Online.
        2. Ensure that the Daemon Status for Corosync, Pacemaker and the pcsd service are active/disabled.

       

      Back to CCM (Required)

  2. Upgrade the CCM server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azurepack, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder.

    4. If your database upgrade was successful, be aware that you can now stop both instances of MySQL as you no longer need these instances.

Scenario 5: Double CCM and Single PostgreSQL

 Scenario 5: CCMs in Separate Instances

Follow this process to upgrade the CCM-DB instances in HA mode.

  1. Upgrade the PostgreSQL instance.
    1. Create a security group with the specified ports and launch the instance using this security group.
    2. Run the following commands on the PostgreSQL instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7 

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    3.  Start the database wizard and allow the CCM to access the database.

      MGMTPOSTGRES_MASTER – Configure High Availability Properties

      To configure high availability for MGMTPOSTGRES_MASTER, follow this procedure.

        1. SSH into the DB instance as a centos user.
        2. Run the following command:

      1. Invoke the wizard.

        MGMTPOSTGRES Wizard Path
      2. Configure Postgres HA to ensure the PostgreSQL database HA and enter the information in each field as follows:

        Write this down for future reference!

        Icon
        Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

        See Configure CCM Wizard Properties for other settings.

        Wizard Menu

        Field

        Description

        Configure_Postgres_HADB MasterThe hostname for the master database VM – not configurable.
        DB Master Private IPThe private IP address of the master database VM
        DB Slave HostnameThe hostname for the slave database VM
        DB Slave Private IPThe private IP address of the slave database VM

        VIP or EIP 

        The VIP/EIP IP for the database

        Icon

        Use your mouse to select this option.

         AWS Cloud Nuances for EIP
        To setup PostgreSQL as an RDS service in the SA or HA modes, see Configuring HA for PostgreSQL Database on AWS. 

        Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

        • Configuring database for HA ...

        • Configuring database for replication

      3. Exit the configuration wizard.

      4. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
        # pcs status

        1. Ensure that the PCSD Status for both database servers are Online.
        2. Ensure that the Daemon Status for Corosync, Pacemaker and the pcsd service are active/disabled.

       

      Back to CCM (Required)

    4. If your database upgrade was successful, be aware that you can now stop the MySQL instance as you no longer need it.
  2. Upgrade the CCM primary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder.

  3. Upgrade the CCM secondary server.

    1. Run the following commands on the CCM instance.

       Syntax

      <ostype> = centos6, centos7, rhel6, rhel7, ubuntu1204 (Ubuntu12.04 is not recommended for a new install)

      <cloudtype> = amazon, azureclassic, azurepack, azurerm, google, opsource, openstack, softlayer, vmware, vcd

    2. Edit the ccm-response.xml file to include the following values:

    3. Run the following commands from your download folder:

You have upgraded the CCM and PostgreSQL servers in HA mode.

Post Upgrade Tasks

  1. Verify Your Upgrade – Ensure that the version file (/usr/local/osmosix/etc/version) reflects the new release in each VM.

  2. Reboot the CCM Servers.

  • No labels