【系列】CentOS 7.3 离线安装（无网络环境）CI CD环境之gitlab + gitlab runner(docker in docker)

1.打包镜像

开发机器执行：

docker save -o E:gitlab-ce-latest.tar gitlab/gitlab-ce:latest

docker save -o E:gitlab-runner-latest.tar gitlab/gitlab-runner:latest

docker save -o E:microsoft-dotnet-core-sdk-3-1.tar mcr.microsoft.com/dotnet/core/sdk:3.1

2.服务器上执行docker load 加载镜像

 docker load -i gitlab-ce-latest.tar

docker load -i gitlab-runner-latest.tar

docker load -i microsoft-dotnet-core-sdk-3-1.tar

3.上传gitlab + gitlab runner 的yaml脚本

上传至：/opt/gitlab文件夹下，脚本内容如下

version: '3.7'

services:
  gitlab:
    image: gitlab/gitlab-ce:latest
    container_name: gitlab
    restart: always
    hostname: gitlab
    privileged: true
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        gitlab_rails['gitlab_shell_ssh_port'] = 2222
        registry_external_url 'http://' # ip or hostname
        registry['enable'] = true
        gitlab_rails['smtp_enable'] = true
        gitlab_rails['smtp_address'] = "****"
        gitlab_rails['smtp_port'] = 25
        gitlab_rails['smtp_user_name'] = "****"
        gitlab_rails['smtp_password'] = "123456"
        gitlab_rails['smtp_authentication'] = "login"
        gitlab_rails['smtp_enable_starttls_auto'] = true
    ports:
      - '2222:22'
      - '80:80'
      - '4443:443'
    networks:
      - gitlab-network-overlay
    volumes:
      - /opt/gitlab/config:/etc/gitlab
      - /opt/gitlab/data:/var/opt/gitlab
      - /opt/gitlab/logs:/var/log/gitlab
    

  gitlab-runner1:
    image: gitlab/gitlab-runner:latest
    container_name: gitlab-runner1
    hostname: gitlab-runner1
    restart: always
    privileged: true
    # network_mode: "host"
    volumes:
      - '/opt/gitlab-runner1-config/config:/etc/gitlab-runner:Z'
      - '/var/run/docker.sock:/var/run/docker.sock' #这里一定要映射到宿主机的docker.sock文件上 否则会出现无法访问的情况
    networks:
      - gitlab-network-overlay
networks:
  gitlab-network-overlay:
    #name: gitlab-network
    external: true

4.创建overlay网络，使runner可以连接到外部网络，否则会导致runner 运行时连接不到gitlab harbor nexus等私库

#初始化swarm
docker swarm init

docker network create --driver=overlay --attachable gitlab-network-overlay

docker network ls

5.创建映射文件夹

mkdir {config,data,logs}

mkdir -vp /opt/gitlab-runner1-config/cofig

6.使用docker stack启动gitlab 和gitlab runner

docker stack deploy -c docker-compose.yaml gitlab-swarm

这是可能会因为selinux问题出现错误：

Creating gitlab ...
Creating gitlab-runner1 ... error

Creating gitlab ... error
c/self/attr/keycreate: permission denied"": unknown

ERROR: for gitlab Cannot start service gitlab: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused "write /proc/self/attr/keycreate: permission denied"": unknown

ERROR: for gitlab-runner1 Cannot start service gitlab-runner1: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused "write /proc/self/attr/keycreate: permission denied"": unknown

ERROR: for gitlab Cannot start service gitlab: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused "write /proc/self/attr/keycreate: permission denied"": unknown
ERROR: Encountered errors while bringing up the project.

这时候，修改selinux的模式为permissive:

vi /etc/selinux/config

然后重启服务器，重新执行5中的命令，即可正确启动gitlab 和 gitlab runner

7.配置gitlab

vi config/gitlab.rb

修改：

## GitLab URL
##! URL on which GitLab will be reachable.
##! For more details on configuring external_url see:
##! https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab
##!
##! Note: During installation/upgrades, the value of the environment variable
##! EXTERNAL_URL will be used to populate/replace this value.
##! On AWS EC2 instances, we also attempt to fetch the public hostname/IP
##! address from AWS. For more details, see:
##! https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
external_url 'http://188.6.7.131'

修改后重启服务

docker exec -it gitlab bash

gitlab-ctl reconfigure

exit

这时就可以说使用IP地址访问gitlab了，然后进行一些相关配置，这里不再赘述

8.配置gitlab runner （具体参考官网文档相关配置）

进入runner后执行：

gitlab-runner register

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
http://188.6.7.131/
Please enter the gitlab-ci token for this runner:
QncXTQE42GLjeLfvcfJD
Please enter the gitlab-ci description for this runner:
[gitlab-runner1]: group-runner1
Please enter the gitlab-ci tags for this runner (comma separated):

Registering runner... succeeded                     runner=QncXTQE4
Please enter the executor: virtualbox, docker-ssh+machine, docker-ssh, shell, ssh, docker+machine, kubernetes, custom, docker, parallels:
docker
Please enter the default Docker image (e.g. ruby:2.6):
Docker:19.03.11
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

这里使用docker运行runner 所以executor 选择docker

配置完成后先不急进行流水线测试，由于我们是在无网络环境下，还没有搭建私有docker仓库，所以需要修改runner的相关配置：

修改/gitlab-runner1/config/config.toml:

concurrent = 1
check_interval = 0

[session_server]
session_timeout = 1800

[[runners]]
name = "gis-group-runner1"
url = "http://188.6.7.131/"
token = "qQZ1p2hETvf94F7hNFJt"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.docker]
tls_verify = false
image = "Docker:19.03.11"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
pull_policy = "if-not-present"

重启容器：

docker ps 

docker restart 22516cf45d71

常见问题：重启服务后 gitlab一直处于restart状态通过docker logs gitlab 看到如下错误：

/opt/gitlab/embedded/bin/runsvdir-start: No such file or directory

解决办法：删除容器重启创建，相关数据已经持久化到服务器，并不会影响数据

docker stack deploy -c docker-compose.yaml gitlab-swarm

参考：

https://blog.51cto.com/xiaozhagn/2488779