[RHOCP4] RHEL7 worker 노드 추가

RHOCP4.8.14에 RHEL7 worker노드를 추가하는 방법에 대한 내용입니다.

 

RHEL7 버전은 RHOCP4에서 더이상 사용되지 않고 삭제할 예정입니다.

 

RedHat에서는 RHOCP 4.8에서는 RHEL 7.9만 지원하며, RHOCP 4.9에서는 RHEL7.9와 RHEL8.4를 지원하고 있습니다.

 

기 구성한 저의 cluster의 버전이 4.8이기때문에 RHEL 7.9 OS를 설치한 후 기존 Cluster에 worker노드로 추가하는 방법에 대한 내용으로 정리하였습니다.

 

 

RHEL7.9 작업

[root@worker2 yum.repos.d]# subscription-manager register --username=<user_name> --password=<password>


[root@worker2 yum.repos.d]# subscription-manager list --available --matches '*OpenShift*'


[root@worker2 yum.repos.d]# subscription-manager attach --pool=<pool_id>


[root@worker2 yum.repos.d]# subscription-manager repos \
    --enable="rhel-7-server-rpms" \
    --enable="rhel-7-fast-datapath-rpms" \
    --enable="rhel-7-server-extras-rpms" \
    --enable="rhel-7-server-optional-rpms" \
    --enable="rhel-7-server-ose-4.8-rpms"
    
    
    
[root@worker2 yum.repos.d]# systemctl disable firewalld --now

 

 

1. bastion서버에 ansible 2.9 버전 설치

[root@bastion yum.repos.d]#dnf install -y ansible
Failed to set locale, defaulting to C.UTF-8
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.

rhel-8-for-x86_64-baseos-rpms                                                                                            3.1 MB/s | 4.1 kB     00:00
rhel-8-for-x86_64-appstream-rpms                                                                                         4.4 MB/s | 4.5 kB     00:00
ansible-2.9-for-rhel-8-x86_64-rpms                                                                                        78 MB/s | 2.0 MB     00:00
Dependencies resolved.
=========================================================================================================================================================
 Package                               Architecture            Version                         Repository                                           Size
=========================================================================================================================================================
Installing:
 ansible                               noarch                  2.9.26-1.el8ae                  ansible-2.9-for-rhel-8-x86_64-rpms                   17 M
Installing dependencies:
 python3-babel                         noarch                  2.5.1-5.el8                     rhel-8-for-x86_64-appstream-rpms                    4.8 M
 python3-cffi                          x86_64                  1.11.5-5.el8                    rhel-8-for-x86_64-baseos-rpms                       238 k
 python3-cryptography                  x86_64                  3.2.1-4.el8                     rhel-8-for-x86_64-baseos-rpms                       559 k
 python3-jinja2                        noarch                  2.10.1-2.el8_0                  rhel-8-for-x86_64-appstream-rpms                    538 k

 

 

2. github 사이트에서 openshift-ansible 파일 다운로드

[root@bastion ~]#git clone https://github.com/openshift/openshift-ansible.git
Cloning into 'openshift-ansible'...
remote: Enumerating objects: 150225, done.
remote: Counting objects: 100% (1411/1411), done.
remote: Compressing objects: 100% (512/512), done.
remote: Total 150225 (delta 785), reused 1333 (delta 733), pack-reused 148814
Receiving objects: 100% (150225/150225), 41.77 MiB | 20.31 MiB/s, done.
Resolving deltas: 100% (94080/94080), done.

 

 

3. hosts 파일 작성

[root@bastion ~]#cd openshift-ansible/inventory/

[root@bastion inventory]#ls
README.md  dynamic  hosts.example

[root@bastion inventory]#cp -pr hosts.example hosts


[root@bastion inventory]#cat hosts
[all:vars]
# SSH user, this user should allow ssh based auth without requiring a
# password. If using ssh key based auth, then the key should be managed by an
# ssh agent.
ansible_user=root

# If ansible_user is not root, ansible_become must be set to true and the
# user must be configured for passwordless sudo
#ansible_become=True

###############################################################################
# Required configuration variables                                            #
###############################################################################
#openshift_kubeconfig_path="~/.kube/config"
openshift_kubeconfig_path="/root/ocp/config/auth/kubeconfig"

# For running RHEL worker upgrades
#[workers]
#mycluster-worker-[1:3].example.com

# For running RHEL worker scaleup
[new_workers]
worker2.ocp-dc.hk.com ansible_host=55.55.35.217

 

 

4. scaleup.yml 파일 실행

[root@bastion inventory]#ansible-playbook -i hosts ../playbooks/scaleup.yml

PLAY [Pre-scaleup checks] *************************************************************************************************************************************************************************

TASK [openshift_node : Ensure [new_workers] group is populated] ***********************************************************************************************************************************
skipping: [localhost]

TASK [openshift_node : Get cluster nodes] *********************************************************************************************************************************************************
ok: [localhost]

TASK [openshift_node : Check for nodes which are already part of the cluster] *********************************************************************************************************************
skipping: [localhost] => (item=worker2.ocp-dc.hk.com)

< 생 략 >


TASK [openshift_node : Approve node CSRs] ************************************************************************************************************************
changed: [worker2.ocp-dc.hk.com -> localhost]
TASK [openshift_node : Wait for node to report ready] ************************************************************************************************************
FAILED - RETRYING: Wait for node to report ready (30 retries left).
FAILED - RETRYING: Wait for node to report ready (29 retries left).
ok: [worker2.ocp-dc.hk.com -> localhost]

PLAY RECAP *******************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0
worker2.ocp-dc.hk.com      : ok=42   changed=21   unreachable=0    failed=0    skipped=8    rescued=0    ignored=0

 

 

5. 노드 조인 및 정보 확인

openshift-ansible을 사용하여 RHEL7 worker 노드 추가시에 ansible을 통해 자동으로 노드에 추가할 수 있도록 approve까지 진행해준다.

[root@bastion ~]#oc get node -o wide
NAME                    STATUS   ROLES    AGE     VERSION           INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                 CONTAINER-RUNTIME
master1.ocp-dc.hk.com   Ready    master   7h9m    v1.21.1+a620f50   55.55.35.211   <none>        Red Hat Enterprise Linux CoreOS 48.84.202110042259-0 (Ootpa)   4.18.0-305.19.1.el8_4.x86_64   cri-o://1.21.3-4.rhaos4.8.git2933275.el8
master2.ocp-dc.hk.com   Ready    master   7h6m    v1.21.1+a620f50   55.55.35.212   <none>        Red Hat Enterprise Linux CoreOS 48.84.202110042259-0 (Ootpa)   4.18.0-305.19.1.el8_4.x86_64   cri-o://1.21.3-4.rhaos4.8.git2933275.el8
master3.ocp-dc.hk.com   Ready    master   7h2m    v1.21.1+a620f50   55.55.35.213   <none>        Red Hat Enterprise Linux CoreOS 48.84.202110042259-0 (Ootpa)   4.18.0-305.19.1.el8_4.x86_64   cri-o://1.21.3-4.rhaos4.8.git2933275.el8
worker1.ocp-dc.hk.com   Ready    worker   6h11m   v1.21.1+a620f50   55.55.35.216   <none>        Red Hat Enterprise Linux CoreOS 48.84.202110042259-0 (Ootpa)   4.18.0-305.19.1.el8_4.x86_64   cri-o://1.21.3-4.rhaos4.8.git2933275.el8
worker2.ocp-dc.hk.com   Ready    worker   2m56s   v1.21.1+6438632   55.55.35.217   <none>        OpenShift Enterprise                                           3.10.0-1160.45.1.el7.x86_64    cri-o://1.21.3-8.rhaos4.8.git7415a53.el7

 

 

# 참고

4.8.5 기준으로 동일한 방법과 절차로 진행했을 때는 crio에 버그가 있었는데, 4.8.14에서는 버그가 해결 된 것으로 보입니다.

또한, SELINUX도 disable 또는 permissive로 설치 했었는데 이번에는 오히려 끄니 에러가 발생하였습니다.

210910 기준으로 4.8.5 (RHEL7)에 설치할 때 crio 서비스에 버그가 있음

systemctl status crio
...
conmon validation: invalid conmon path: stat /usr/libexec/crio/con...e or directory


버그 해결
https://access.redhat.com/solutions/5009091
conf 수정은 안해도 되고 ln 으로 해결하면 됨


(libexec 이하 디렉토리 /crio/conmon는 만들어줘야함)
ln -s /usr/bin/conmon /usr/libexec/crio/conmon