背景介绍
笔者创建了一个镜像，然后在不同的主机上启动，发现有的能启动，有的却不行，报错信息为找不到文件。
犹记得当初有人介绍，只要docker镜像做好了，拿到任何地方都可以用，此处好像不成呢，好诡异的事。
初步怀疑docker存储驱动导致的问题。
制作镜像步骤还原
1、在某一台主机做dockerfile，该主机环境如下
[root@ ~]# docker info Containers: 1 Running: 0 Paused: 0 Stopped: 1 Images: 46 Server Version: 1.13.1 Storage Driver: btrfs Build Version: Btrfs v4.9.1 Library Version: 102 Logging Driver: journald Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Swarm: inactive Runtimes: docker-runc runc Default Runtime: docker-runc Init Binary: /usr/libexec/docker/docker-init-current containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1) runc version: 9c3c5f853ebf0ffac0d087e94daef462133b69c7 (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f) init version: fec3683b971d9c3ef73f284f176672c44b448662 (expected: 949e6facb77383876aeff8a6944dde66b3089574) Security Options: seccomp WARNING: You're not using the default seccomp profile Profile: /etc/docker/seccomp.json Kernel Version: 3.10.0-514.16.1.el7.x86_64 Operating System: CentOS Linux 7 (Core) OSType: linux Architecture: x86_64 Number of Docker Hooks: 3 CPUs: 40 Total Memory: 125.7 GiB Name: cndh1325-10-12.ptmind.com ID: 2AHA:F2JK:ECEX:U2WY:OAET:BFOS:CY77:MNAK:2Y2P:7DRS:KU72:6VI5 Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ WARNING: bridge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Registries: docker.io (secure)
[root@test ~]# xfs_info / meta-data=/dev/sda5 isize=256 agcount=4, agsize=116621952 blks = sectsz=512 attr=2, projid32bit=1 = crc=0 finobt=0 spinodes=0 data = bsize=4096 blocks=466487808, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=0 log =internal bsize=4096 blocks=227777, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
可以看到，制作镜像的主机存储驱动为 Storage Driver: btrfs ，文件系统xfs不支持d_type。
将镜像上传镜像库，已提供使用。
2、在A主机拉取镜像，镜像不能启动
笔者是创建的elasticsearch:6.2.4版本镜像，从官方拉取，官方镜像默认包含ingest-geoip插件,在dockerfile中已经将插件删除
dockfile如下
FROM docker.elastic.co/elasticsearch/elasticsearch:6.2.4 COPY elasticsearch.yml /usr/share/elasticsearch/config/ COPY jvm.options /usr/share/elasticsearch/config/ RUN /usr/share/elasticsearch/bin/elasticsearch-plugin remove ingest-geoip --purge && /usr/share/elasticsearch/bin/elasticsearch-plugin remove ingest-user-agent && chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/config
报错信息如下：
Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config/ingest-geoip Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config/ingest-geoip at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
A主机环境如下
[root@A ~]# docker info Containers: 20 Running: 20 Paused: 0 Stopped: 0 Images: 21 Server Version: 1.13.1 Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: false Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Swarm: inactive Runtimes: docker-runc runc Default Runtime: docker-runc Init Binary: /usr/libexec/docker/docker-init-current containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1) runc version: 5eda6f6fd0c2884c2c8e78a6e7119e8d0ecedb77 (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f) init version: fec3683b971d9c3ef73f284f176672c44b448662 (expected: 949e6facb77383876aeff8a6944dde66b3089574) Security Options: seccomp WARNING: You're not using the default seccomp profile Profile: /etc/docker/seccomp.json Kernel Version: 3.10.0-862.9.1.el7.x86_64 Operating System: CentOS Linux 7 (Core) OSType: linux Architecture: x86_64 Number of Docker Hooks: 3 CPUs: 16 Total Memory: 62.74 GiB Name: jp33e505-10-14.ptengine.com ID: PDSY:4YPU:AUSW:HA4V:223D:YBSG:E67E:TUJF:3WUJ:EL7J:VNGH:LG3O Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Registries: docker.io (secure)
[root@A ~]# xfs_info / meta-data=/dev/sdc2 isize=256 agcount=4, agsize=6553600 blks = sectsz=512 attr=2, projid32bit=1 = crc=0 finobt=0 spinodes=0 data = bsize=4096 blocks=26214400, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=0 log =internal bsize=4096 blocks=12800, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
检查发现存储驱动为overlay2，文件系统xfs不支持d_type。
3、在B主机拉取镜像，镜像能启动
B主机环境如下
[root@B ~]# docker info Containers: 21 Running: 21 Paused: 0 Stopped: 0 Images: 22 Server Version: 1.13.1 Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Swarm: inactive Runtimes: docker-runc runc Default Runtime: docker-runc Init Binary: /usr/libexec/docker/docker-init-current containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1) runc version: df5c38a9167e87f53a9894d77c0950e178a745e7 (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f) init version: fec3683b971d9c3ef73f284f176672c44b448662 (expected: 949e6facb77383876aeff8a6944dde66b3089574) Security Options: seccomp WARNING: You're not using the default seccomp profile Profile: /etc/docker/seccomp.json Kernel Version: 3.10.0-957.10.1.el7.x86_64 Operating System: CentOS Linux 7 (Core) OSType: linux Architecture: x86_64 Number of Docker Hooks: 3 CPUs: 16 Total Memory: 62.75 GiB Name: jp33e506-10-19.ptengine.com ID: OFVX:ELT3:RNT4:OM6W:L7GC:OC47:R5QJ:XBVL:T5HW:UGIL:PJUJ:66ZO Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Registries: docker.io (secure)
[root@B ~]# xfs_info / meta-data=/dev/md126p3 isize=512 agcount=4, agsize=7169408 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=0 spinodes=0 data = bsize=4096 blocks=28677632, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal bsize=4096 blocks=14002, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
现在已经很明显了，A和B唯一不同的就是文件系统xfs是否支持d_type，而官方已明确说明如果使用xfs文件系统，需要d_type的支持，即 ftype=1，具体参考 https://docs.docker.com/storage/storagedriver/overlayfs-driver/
修改A主机的d_type
mkfs.xfs -f -n ftype=1 /dev/sdg5
重启docker，一切正常。