OpenShift on VM Bare metal

Let’s try to install OpenShift 4 on bare metal, where bare metal is virtual machine managed outside of OpenShift reach. We are trying to follow the OpenShift Install on Bare metal official documentation as much as possible.


In my infrastructure I do have at my disposal 2 machines that both have libvirt (qemu+kvm) enabled (with nested virtualization support even 😝).

  • okinawa: 8 cores, 32GiB memory
  • wakasu: 8 cores, 64GiB memory

Because we are going to install the machine bare metal on virtual machine, some assumptions are made (see Infrastructure on those).

  • VMs are created using a pre-defined Mac address, and the DHCP server on the network assign them a static IP (based on the mac address).
  • This allows to create ahead of time DNS entries on the network so that the bootstrap and the cluster can find its own by domain name.

My home network is using home as domain name tld. So VMs will be vm1.home, … and for the ocp cluster, aliased with ocp.home. We will have 3 master and 5 workers 🙃. The DNS entry looks more or less like

;; Load balancer
vm0.home. IN A
api.ocp.home. IN A
api-int.ocp.home. IN A
*.apps.ocp.home. IN A

;; Masters
vm1.home. IN A
master1.ocp.home. IN A
vm2.home. IN A
master2.ocp.home. IN A
vm3.home. IN A
master3.ocp.home. IN A

;; Workers
vm4.home. IN A
worker1.ocp.home. IN A
vm5.home. IN A
worker2.ocp.home. IN A
vm6.home. IN A
worker3.ocp.home. IN A
vm7.home. IN A
worker4.ocp.home. IN A
vm8.home. IN A
worker5.ocp.home. IN A

;; Bootstrap machine
vm9.home. IN A
bootstrap.ocp.home. IN A

;; etcd records
etcd-0.ocp.home. IN A
etcd-1.ocp.home. IN A
etcd-2.ocp.home. IN A
_etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-0.ocp.home.
_etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-1.ocp.home.
_etcd-server-ssl._tcp.ocp.home. IN SRV 0 10 2380 etcd-2.ocp.haome.

As you can see, aside from the 3 masters and the 5 workers, we will have a load-balancer VM as documented in the official documentation.

For reference, VMs Mac address compared to their IP.

MAC address IP Name
52:54:00:dd:a3:20 VM0
52:54:00:dd:a3:21 VM1
52:54:00:dd:a3:22 VM2
52:54:00:dd:a3:23 VM3
52:54:00:dd:a3:24 VM4
52:54:00:dd:a3:25 VM5
52:54:00:dd:a3:26 VM6
52:54:00:dd:a3:27 VM7
52:54:00:dd:a3:28 VM8
52:54:00:dd:a3:29 VM9

Although I am tempted to use ocp4-helpernode, I am going to try to do as much as I can with my bare hands ✋.

Provisionning Machines

We will need to provision some machines, using virsh, virt-install or whatever works the best to be honest. This is also gonna be heavily based on ocp4-helpernode/


For the bootstrap VM (vm9), we can go with any system but, let’s follow closely the quistart and use Centos EL8. The boostraper machine will be on wakasu so we will need to target the libvirt daemon from there.

export QEMU_URI=qemu+ssh://vincent@wakasu.home/system
virt-install --connect=${QEMU_URI} \
             --name="ocp4-bootstrap" --vcpus=4 --ram=8192 \
             --disk path=/var/lib/libvirt/images/ocp-bootstrap.qcow2,bus=virtio,size=120 \
             --boot menu=on --print-xml > ocp4-bootstrap.xml
virsh --connect=${QEMU_URI} \
      define --file ocp4-bootstrap.xml

Load balancer

Master and worker nodes

Let’s group machines between wakasu and okinawa. There will be 2 masters on wakasu and one on okinawa, 3 workers on wakasu, and 2 on okinawa.

  • wakasu

    export QEMU_URI=qemu+ssh://vincent@wakasu.home/system
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-master1" --vcpus=4 --ram=12288 \
                 --disk path=/var/lib/libvirt/images/ocp4-master1.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:21 \
                 --boot menu=on --print-xml > ocp4-wakasu-master1.xml
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-master2" --vcpus=4 --ram=12288 \
                 --disk path=/var/lib/libvirt/images/ocp4-master2.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:22 \
                 --boot menu=on --print-xml > ocp4-wakasu-master2.xml
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-worker1" --vcpus=4 --ram=8192 \
                 --disk path=/var/lib/libvirt/images/ocp4-worker1.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:24 \
                 --boot menu=on --print-xml > ocp4-wakasu-worker1.xml
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-worker2" --vcpus=4 --ram=8192 \
                 --disk path=/var/lib/libvirt/images/ocp4-worker2.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:25 \
                 --boot menu=on --print-xml > ocp4-wakasu-worker2.xml
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-worker3" --vcpus=4 --ram=8192 \
                 --disk path=/var/lib/libvirt/images/ocp4-worker3.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:26 \
                 --boot menu=on --print-xml > ocp4-wakasu-worker3.xml
    for f in ocp4-wakasu-*.xml; do
        virsh --connect=${QEMU_URI} define \
              --file $f

    Domain ocp4-master1 defined from ocp4-wakasu-master1.xml

    Domain ocp4-master2 defined from ocp4-wakasu-master2.xml

    Domain ocp4-worker1 defined from ocp4-wakasu-worker1.xml

    Domain ocp4-worker2 defined from ocp4-wakasu-worker2.xml

    Domain ocp4-worker3 defined from ocp4-wakasu-worker3.xml

  • okinawa

    export QEMU_URI=qemu+ssh://vincent@okinawa.home/system
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-master3" --vcpus=4 --ram=12288 \
                 --disk path=/var/lib/libvirt/images/ocp4-master3.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:23 \
                 --boot menu=on --print-xml > ocp4-okinawa-master1.xml
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-worker4" --vcpus=4 --ram=8192 \
                 --disk path=/var/lib/libvirt/images/ocp4-worker4.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:27 \
                 --boot menu=on --print-xml > ocp4-okinawa-worker1.xml
    virt-install --connect=${QEMU_URI} \
                 --name="ocp4-worker5" --vcpus=4 --ram=8192 \
                 --disk path=/var/lib/libvirt/images/ocp4-worker5.qcow2,bus=virtio,size=120 \
                 --os-variant rhel8.0 \
                 --network bridge=br1,mac.address=52:54:00:dd:a3:28 \
                 --boot menu=on --print-xml > ocp4-okinawa-worker2.xml
    for f in ocp4-okinawa-*.xml; do
        virsh --connect=${QEMU_URI} define \
              --file $f
    Domain ocp4-master3 defined from ocp4-okinawa-master1.xml
    Domain ocp4-worker4 defined from ocp4-okinawa-worker1.xml
    Domain ocp4-worker5 defined from ocp4-okinawa-worker2.xml