Deploy project on GCP virtual machine via Gitlab CI/CD

Introduction

This article is about:

1. Open a GCP instance with gcloud
2. Import ssh key into an instance with gcloud
3. Running your project in the background with Daemon
4. Deploy your project on GCP virtual machine via gitlab pusher

Environment

Create a GCP virtual machine

Here is how I would do, you could have your own way.

• Ray use Mac, so I install Google Cloud SDK locally. As to the installation, you could refer to official document
• Create a VM
  1. Create a VM called example-instance-1
  2. The boot drive storage is 10GB
  3. Pull image we need from ubuntu-os-cloud
  4. We use ubuntu-1804-lts as the image-family, so the latest version of this family will be used automatically.
  5. The type of boot-drive is pd-stand, you could check the types with command gcloud compute disk-types list
  6. The machine type is f1-micro, you could check the types wit command gcloud compute machine-types list
  7. We identify each instance with tags, and we will it when we want to create a firewall-rules.
  8. We specify the zone of the instance. Some resources are limited in certain zone and region

As follows:

gcloud compute instances create example-instance-1 \
--image-project=ubuntu-os-cloud \
--image-family=ubuntu-1804-lts \
--boot-disk-size=10GB \
--boot-disk-type=pd-standard \
--machine-type=f1-micro \
--tags=example-instance-1,http-server,https-server \
--zone=asia-east1-a 

After creating, let’s produce ssh-key first

ssh-keygen -t rsa -b 4096 -C "root@example"

Assume that the key is named example

cat example.pub > instanceSSHConfig && vim instanceSSHList

Put root before the key in instanceSSHList file, whose format is as follows:

[USERNAME]:ssh-rsa [KEY] [USERNAME]

Get the name of the instance

gcloud compute instances list

Add the public key into the instance
(Be careful! This command will replace all of your SSH keys on this instance, that said, any keys without appearing on this file will be gone)

gcloud compute instances add-metadata instanceName --metadata-from-file ssh-keys=instanceSSHList

Installation

Here we mainly install nvm, node with version v12.1.0, and npm. You could get more detail via the official document

apt-get update -y && apt-get install curl -y && curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash && export NVM_DIR="$HOME/.nvm" && [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" && [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" && nvm install v12.1.0 && apt-get install npm -y

Daemon

Now, let’s config Daemon. We will run our service with Daemon so it will be run automatically when it’s disconnected.

sudo vim /etc/init.d/serviceName

#!/bin/sh
### BEGIN INIT INFO
# Provides:          yourServiceName (optional)
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start daemon at boot time
# Description:       Enable service provided by daemon.
### END INIT INFO

dir="yourProjectLocation"
cmd="theCommandItRequiresToStartYourService"
user="root"

name=`basename $0`
pid_file="/var/run/$name.pid"
stdout_log="/var/log/$name.log"
stderr_log="/var/log/$name.log"

get_pid() {
    cat "$pid_file"
}

is_running() {
    [ -f "$pid_file" ] && ps -p `get_pid` > /dev/null 2>&1
}

case "$1" in
    start)
    if is_running; then
        echo "Already started"
    else
        echo "Starting $name"
        cd "$dir"
        export NODE_ENV=test
        if [ -z "$user" ]; then
            sudo $cmd >> "$stdout_log" 2>> "$stderr_log" &
        else
            sudo -u "$user" $cmd >> "$stdout_log" 2>> "$stderr_log" &
        fi
        echo $! > "$pid_file"
        if ! is_running; then
            echo "Unable to start, see $stdout_log and $stderr_log"
            exit 1
        fi
    fi
    ;;
    stop)
    if is_running; then
        echo -n "Stopping $name.."
        kill `get_pid`
        for i in 1 2 3 4 5 6 7 8 9 10
        # for i in `seq 10`
        do
            if ! is_running; then
                break
            fi

            echo -n "."
            sleep 1
        done
        echo

        if is_running; then
            echo "Not stopped; may still be shutting down or shutdown may have failed"
            exit 1
        else
            echo "Stopped"
            if [ -f "$pid_file" ]; then
                rm "$pid_file"
            fi
        fi
    else
        echo "Not running"
    fi
    ;;
    restart)
    $0 stop
    if is_running; then
        echo "Unable to stop, will not attempt to start"
        exit 1
    fi
    $0 start
    ;;
    status)
    if is_running; then
        echo "Running"
    else
        echo "Stopped"
        exit 1
    fi
    ;;
    *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
    ;;
esac

exit 0

• If it shows that the service is not found, we need to reload the daemon

  sudo systemctl daemon-reload

• Note that the authority has to been revised, making Daemon executable

  sudo chmod 755 serviceName

• Set up auto-restart, when VM reboots, the service will auto start

  sudo systemctl enable serviceName

• In this example, the name of Daemon will be equal to the name of project

CI/CD

Gitlab variables setting

• We are going to do CI/CD with Gitlab pusher, so we have to create a pair of ssh key , and set the private key as $SSH_PRIVATE_KEY in variable setting

  ssh-keyscan to-be-conneted-instance-ip 

Gitlab yaml config file

We will set up the yaml file of Gitlab pusher
In our project

vim .gitlab-ci.yml

# This file is a template, and might need editing before it works on your project.
# Official framework image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/node/tags/
image: node:8

# This folder is cached between builds
# http://docs.gitlab.com/ce/ci/yaml/README.html#cache
cache:
  paths:
  - node_modules/

stages:
- build
- deploy

npm-build:
  stage: build
  script:
  - rm -rf node_modules/ && npm i npm@latest -g && npm install
  
depoly-test:
  stage: deploy
  script:
  # cfr. https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  # Install ssh-agent if not already installed, it is required by Docker.
  # (change apt-get to yum if you use a CentOS-based image)
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  
  # Run ssh-agent (inside the build environment)
  - eval $(ssh-agent -s)
  
  # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  
  - mkdir -m 700 -p /root/.ssh
  
  - tar zcf ../$CI_PROJECT_NAME.tar.gz ./
  
  - scp -o StrictHostKeyChecking=no ../$CI_PROJECT_NAME.tar.gz root@35.201.171.244:/locationYouPrefer
  
  - ssh root@yourIP "rm -rf /locationYouPrefer/$CI_PROJECT_NAME && mkdir -p locationYouPrefer/$CI_PROJECT_NAME && tar zxf locationYouPrefer/$CI_PROJECT_NAME.tar.gz -C locationYouPrefer/$CI_PROJECT_NAME && chmod -R 655 locationYouPrefer/$CI_PROJECT_NAME && cd locationYouPrefer/$CI_PROJECT_NAME && npm rebuild && /etc/init.d/$CI_PROJECT_NAME restart"
  
  only:
  - branchYouPrefer

Conclusion

Until now, when we use git push to specified branch, we should be able to trigger gitlab pusher to achieve automatic deployment