CliQr is now part of Cisco Learn More About Cisco

Backup and Recovery in HA Mode

Recommendations

To effectively manage your CloudCenter deployment, backup your deployment on a daily basis – you can setup a cronjob to automatically perform this backup.

Icon

If you are upgrading the CloudCenter deployment, be aware that the process may differ. See the Upgrade in Non-HA Mode or Upgrade in HA Mode for additional context.

Icon

Use this procedure to backup and recover data for the following releases on a per-component basis:

  • CloudCenter 4.6.x
  • CloudCenter 4.7.x

The backup and recovery procedure is performed on a per component basis and the procedure for backup is the same for all components. Instead of repeating this procedure for each role, the procedure calls out the applicable roles for each component in the HA and standalone modes. For the non-HA modes, see Backup and Recover in Non-HA Mode.

Database (PostgreSQL)

Use this procedure for the following roles (see Component Modes and Roles for additional context).

  • MGMTPOSTGRES_MASTER
  • MGMTPOSTGRES_SLAVE

Backup

Backup your database and application (the following example uses /mnt, you can change this directory as applicable).

Backup from 4.6.x

Recover

This procedure assumes that the MGMTPOSTGRES_MASTER is terminated in the HA set up.

  1. On the existing MGMTPOSTGRES_SLAVE run below command.

  2. Launch the MGMTPOSTGRES_(new)SLAVE VM.
    1. Phase 1: Prepare Infrastructure > MGMTPOSTGRES_MASTER/SLAVE
    2. Phase 2: Configure Network Rules > MGMTPOSTGRES_MASTER/SLAVE
    3. Phase 3: Perform Network Compliance Check > MGMTPOSTGRES_MASTER/SLAVE
    4. Phase 4: Install Components > MGMTPOSTGRES_MASTER/SLAVE – Use one of the processes (Installer Process or Appliance Process provided after this section) for explicit instructions.

Installer Process

On the new MGMTPOSTGRES VM, follow this procedure.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Download the following files from software.cisco.com to the /tmp folder. See Installation Overview > Installation Download Details for additional context.

    • core_installer.bin
    • ccm-installer.jar
    • ccm-response.xml
  3. Run Core installer to setup core system components using the following commands.

    For example:

    Syntax:

    <ostype> = centos6, centos7, rhel6, rhel7

    <cloudtype> = amazon, openstack, vmware

  4. Remove the core_installer.bin file.

  5. REQUIRED: At this point, you must continue with the Appliance Process to configure the wizard properties and set up the VM.

Appliance Process

Launch an appliance using the MGMTPOSTGRES appliance file. On the new MGMTPOSTGRES_MASTER, follow this procedure.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Exchange SSH keys between the new VM and existing MGMTPOSTGRES servers.
    1. Copy the files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from the same location on the existing VM to the same location on new VM.

      If  the .ssh directory doesn’t exist on the New VM, first create it (using the following commands) before copying the files. 

    2. On new VM, run the following commands.

    3. Verify mutual SSH access between the existing and new VM by running the following command on each server.

    1. SSH into the DB instance as a centos user.
    2. Run the following command:

  3. Invoke the wizard.

    CCM Wizard Path
  4. Configure Postgres HA to ensure that the PostgreSQL database HA and enter the information in each field as follows:

    Database Properties

    Field

    Description

    Configure_Postgres_HA
    • Master Hostname
    The host name for the master database VM
    • Master Private IP
    The private IP address of the master database VM
    • Slave Hostname
    The host name for the slave database VM
    • Slave Private IP
    The private IP address of the slave database VM
    • VIP
      or
    • EIP

    The VIP/EIP IP for the database

    Icon

    Use your mouse to select this option.

    Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

    • Configuring database for HA ...

    • Configuring database for replication

  5. Exit the configuration wizard.

  6. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
    # pcs status

    1. Ensure that the PCSD Status for both database servers are Online.
    2. Ensure that the Daemon Status for corosync, pacemaker and pcsd are active/disabled.

 

CCM

Use this procedure for the following roles (see Component Modes and Roles for additional context).

  • CCM_SA_PRIMARY
  • CCM_SA_SECONDARY

Backup

This procedure assumes that the CCM_SA_PRIMARY is terminated in a HA setup.

  1. To backup your database and application, issue the following commands.

    Icon

    The following example uses /mnt, you can change this directory as applicable.

    Backup from 4.6.x

Recover

This procedure assumes that the CCM_SA_PRIMARY is terminated in the HA set up.

On the existing CCM_ SA_SECONDARY, perform this procedure.

  1. Navigate to /home/cliqruser.

  2. Remove the .unison folder:

  3. Verify and delete any running cron jobs containing the name unison.
  4. Navigate to osmosix/etc folder and delete the harole file.

  5. To launch the CCM_(new)SA_PRIMARY VM, follow this procedure.

    1. Phase 1: Prepare Infrastructure > CCM_SA_PRIMARY

    2. Phase 2: Configure Network Rules > CCM_SA_PRIMARY

    3. Phase 3: Perform Network Compliance Check > CCM_SA_PRIMARY

    4. Phase 4: Install ComponentsCCM_SA_PRIMARY – Use one of the processes (Installer Process or Appliance Process provided after this section) for explicit instructions.

Installer Process

On the new CCM_SA_PRIMARY VM, perform this procedure.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Download the CCM installer files to the /tmp folder:

    1. core_installer.bin
    2. ccm-installer.jar
    3. ccm-response.xml
  3. Run Core installer to setup core system components using the following commands.

    For example:

    Syntax:

    <ostype> = centos6, centos7, rhel6, rhel7

    <cloudtype> = amazon, openstack, vmware

  4. Remove the core_installer.bin file.

  5. Log off and log back in as the root user to ensure JAVA Home is set
  6. Modify the ccm-response.xml file as follows:

    Response file option

    Value

    Notes

    <entry key="db_host" value="localhost"/> 

    Replace localhost with VIP (MGMTPOSTGRES_VIP_IP)

    Required for the CCMs  to connect to the master Postgres database

  7. Run the appliance installer to setup CCM.

  8. Reboot the CCM VM.
  9. REQUIRED: At this point, you must continue with the Appliance Process to configure the wizard properties and set up the VM.

Appliance Process

Launch a VM using the CCM_SA appliance image. To configure the CCM wizard properties, follow this procedure.

  1. On the CCM_SA_PRIMARY server, if the .ssh directory does not exist for the cliqruser, create it using the following commands before copying the files. 

  2. Copy the files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from the CCM_SA_SECONDARY to the same location on CCM_SA_PRIMARY server. 

  3. On the new CCM_SA_PRIMARY server, execute the following commands.

  4. Verify mutual SSH access between the CCM_SA_PRIMARY and CCM_SA_PRIMARY servers by running below command on each server.

  5. On the CCM_SA_ SECONDARY, invoke the CCM configuration wizard and configure the properties. See CCM_SA_SECONDARY – Run Appliance Install for additional details.

  6. Navigate to the Configure HA screen and enter the information in each field as follows:

    CCM HA Properties

    Field

    Description

    Configure_HA
    • Primary Node Private IP

    The private IP address of the primary server (CCM_SA_ SECONDARY).

    • Secondary Node Private IP

    The private IP address of the secondary server (CCM_SA_ PRIMARY).

    • Public DNS
    The public DNS of the slave database VM
    • Hazelcast IP
    • Comma separated IP addresses of the CCMs (CCM_SA_ SECONDARY, CCM_SA_PRIMARY).

    • Used internally by the CloudCenter platform.
    • External URL

    The VIP/EIP IP for the database

    Icon

    Use your mouse to select this option.

    Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

    1. Configuring database for HA ...

    2. Configuring database for replication

  7. Exit the configuration wizard.

  8. Restart both servers.

  9. Update the CCM_LB with the new IP address of the new CCM_SA_PRIMARY server.
    1. SSH into the VM instance using the key pair that you used to launch the VM.

    2. Modify the HAProxy config file as follows to replace the old IP with the new IP address.


    3. Start the HAProxy service and check the status to ensure that it is active.

AMQP

Use this procedure for the following roles (see Component Modes and Roles for additional context).

  • AMQP_PRIMARY
  • AMQP_SECONDARY

Backup

Backup the webapp folder containing the exploded war files to a backup folder (the following example uses /mnt, you can change this directory as applicable). This backup only applies to the Guacamole server, not the AMQP server.

Recover

This procedure assumes that the AMQP_PRIMARY is terminated in a HA setup.

Launch the CCM_(new)SA_PRIMARY VM.

  1. Phase 1: Prepare InfrastructureAMQP_PRIMARY
  2. Phase 2: Configure Network Rules > AMQP_PRIMARY
  3. Phase 3: Perform Network Compliance Check > AMQP_PRIMARY
  4. Phase 4: Install ComponentsAMQP_PRIMARY – Use one of the processes (Installer Process or Appliance Process provided after this section) for explicit instructions.

Installer Process

On the existing AMQP_SECONDARY, perform this procedure.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Download the AMQP installer files to the /tmp folder:

    1. core_installer.bin
    2. cco-installer.jar
    3. conn_broker-response.xml
  3. Run Core installer to setup core system components using the following commands.

    For example:

    Syntax:

    <ostype> = centos6, centos7, rhel6, rhel7

    <cloudtype> = amazon, azure, azurerm, azurepack, google,  opsource, openstack, softlayer, vmware, vcd

  4. Remove the core_installer.bin file.

  5. Log off and log back in as the root user to ensure JAVA Home is set.

  6. Run the appliance installer to setup AMQP.

  7. Reboot the AMQP VM.
  8. REQUIRED: At this point, you must continue with the Appliance Process to configure the wizard properties and set up the VM.

Appliance Process

Launch a VM using the AMQP appliance image. To configure the GUA wizard properties, follow this procedure.

  1. Copy the cookie from AMQP_SECONDARY to AMQP_PRIMARY node of location /var/lib/rabbitm.  

  2. Detach AMQP servers using the following command.

  3. On the new AMQP_PRIMARY server, execute the following commands.

  4. On the new AMQP_SECONDARY server, execute the following commands.

  5. Check AQMP cluster status on both the servers.

  6. Set the mirroring policy by issuing below command in one of the server.

  7. Verify the policy change using the following command.

  8. Modify the rabbitmq config file on both servers by adding the following line to rabbit array.

    For example:

  9. Restart rabbitmq server on the both the servers.

  10. Update the AMQP_LB with the new IP address of the new AMQP_PRIMARY server.
    1. SSH into the VM instance using the key pair that you used to launch the VM.

    2. Modify the HAProxy config file as follows to replace the old IP with new


    3. Start the HAProxy service and check the status, it should be active

       

    1. Configure the CCO and CCM properties. The wizard includes multiple menu groups with different properties. The table below lists each property and highlights the common properties in bold text.

      Write this down for future reference!

      Icon
      Write down the Field details in a printed version of the Your Notes section for later use.
    2. Configure the properties for the CCM and CCO VMs:

      GroupHostPossible IP Addresses

      CCM_Info

      CCM Host

      CCM_IP or  CCM_SA_IP or CCM_LB_IP
      CCO_InfoCCO HostCCO_IP or  CCO_LB_IP
    3. Verify your changes and Exit the GUA configuration wizard.

  11. Exit the configuration wizard.

  12. Restart the server.

CCO

Use this procedure for the following roles (see Component Modes and Roles for additional context).

  • CCO
  • CCO_PRIMARY
  • CCO_SECONDARY
  • CCO_TERTIARY
  • CCO_LB

Backup

Backup the exploded war files to a backup folder (the following example uses /mnt, you can change this directory as applicable).

Recover

This procedure assumes that the:

  • CCO_TERTIARY is terminated in a HA setup.
  • Node1, Node2, Node 3 are part of CCO HA.

  • Node3 is down.

  • Node4 is the new Node

To launch the CCM_(new)SA_PRIMARY VM, follow this procedure.

  1. Phase 1: Prepare Infrastructure > CCO_PRIMARY/SECONDARY/TERTIARY
  2. Phase 2: Configure Network Rules > CCO_PRIMARY/SECONDARY/TERTIARY
  3. Phase 3: Perform Network Compliance Check > CCO_PRIMARY/SECONDARY/TERTIARY
  4. Phase 4: Install ComponentsCCO_PRIMARY/SECONDARY/TERTIARY – Use one of the processes (Installer Process or Appliance Process provided after this section) for explicit instructions.

Installer Process

On the existing CCO_SECONDARY VM, perform this procedure.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Download the CCO installer files to the /tmp folder:

    1. core_installer.bin
    2. cco-installer.jar
    3. cco-response.xml
  3. Run Core installer to setup core system components using the following commands.

    For example:

    Syntax:

    <ostype> = centos6, centos7, rhel6, rhel7

    <cloudtype> = amazon, openstack, vmware

  4. Remove the core_installer.bin file.

  5. Log off and log back in as the root user to ensure JAVA Home is set

  6. Run the appliance installer to setup CCO.

  7. Reboot the CCO VM.
  8. REQUIRED: At this point, you must continue with the Appliance Process to configure the wizard properties and set up the VM.

Appliance Process

Launch a VM using the CCO appliance image. To launch the CCO wizard, follow this procedure.

    1. SSH into the CCO instance as a centos user.
    2. Edit the /etc/mongod.conf file and add the following line. 

    3. Modify bind_ip as follows.

    4. Restart mongod.

  1. CCO_LB with the new IP address of the new CCO_PRIMARY server.
    1. SSH into the VM instance using the key pair that you used to launch the VM.

    2. Modify the HAProxy config file as follows to replace the old IP with the new IP address.


    3. Start the HAProxy service and check the status, to ensure that it is active

Health Monitor

The back up scenario is not applicable in the non-HA mode as there is no known data that can be persisted.

To restore, simply launch a new VM and reconfigure the VM.

 

  • No labels