Skip to main content

Installing on OpenShift

Preparing the project​

The helm installation operates the same on OpenShift as on Kubernetes however there are some differences such as management of CRDs that must be done by an administrator. There are also RBAC rules that must be put into place that will require admin access.

CRDs​

Generate the CRDs out of the helm chart:

helm template --namespace renku renku renku/renku -f renku-values.yaml --set amalthea.deployCrd=true --set amalthea-sessions.deployCrd=true | yq e '. | select(.kind == "CustomResourceDefinition")' > renku-crds.yaml
warning

The above command will give you the CRDs from the latest version of Renku. But you can select the CRDs from a specific Renku version by adding the --version=x.x.x flag to the helm template command. Using the latest version may cause problems or unpredictable behavior if you extract the resource from a different version of Renku than what you have or will install.

RBAC​

The admin here needs to decide if the "Resource Pool" feature will be used or not. If it is to be used, then the RBAC for resource pool management must also be generated and applied by an admin that has access to ClusterRole and ClusterRoleBinding. If resource pools are not to be used, then the feature can be disabled in the values file by setting dataService.rbac.resourcePools to false (and being admin in the namespace where Renku is installed should be sufficient).

Enabling Resource Pools​

To have the "Resource Pool" feature enabled, some ClusterRoles and ClusterRoleBindings are needed, to generate them run:

 helm template --namespace renku renku renku/renku -f renku-values.yaml --set amalthea.deployCrd=true --set amalthea-session.deployCrd=true --set dataService.rbac.create=true| yq e '. | select(.kind == "*Role*" and (.metadata.name == "renku-data-service" or .metadata.name == "renku-k8s-watcher"))' > data-services-rbac.yaml
warning

The above command will give you the RBAC from the latest version of Renku. But you can select the RBAC from a specific Renku version by adding the --version=x.x.x flag to the helm template command. Using the latest version may cause problems or unpredictable behavior if you extract the resource from a different version of Renku than what you have or will install.

Cluster access​

Login

oc login --user <username> https://url.to.openshift.cluster --web

Create the project

oc new-project renku

Optional:​

If no certificate issuer is available, then a self signed issuer can be created.

Note that this require a ClusterIssuer and thus admin access.

Follow the Cert-Manager documentation for SelfSigned

Prerequisite​

CRDs​

Install CRDs generated earlier (as an admin):

oc apply -f renku-crds.yaml

RBAC​

Install the RBAC rules generated earlier (as an admin):

oc apply -f data-services-rbac.yaml

Depending on how many people will manage the Renku deployment(s), it will be simpler to create a Group so that the required roles do not need to be updated for each every person.

kind: Group
apiVersion: user.openshift.io/v1
metadata:
  name: renku-admin
users:
  - user1
  - user2

As an admin:

oc apply -f - renku-admin.yaml

Required cluster roles, bindings and cluster role bindings for people installing Renku:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: amaltheasession-manager
rules:
  - apiGroups:
      - amalthea.dev
    resources:
      - amaltheasessions
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - amalthea.dev
    resources:
      - amaltheasessions/finalizers
    verbs:
      - update
  - apiGroups:
      - amalthea.dev
    resources:
      - amaltheasessions/status
    verbs:
      - get
      - patch
      - update
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: amaltheasession-manager-for-renku-admin
  namespace: renku
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: renku-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: amaltheasession-manager
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: resourcesquotas-manager-for-renku-admin
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: renku-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: resourcesquotas-manager
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: priorityclasses-manager
rules:
  - apiGroups:
      - "scheduling.k8s.io"
    resources:
      - priorityclasses
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: priorityclasses-manager-for-renku-admin
subjects:
  - kind: User
    apiGroup: rbac.authorization.k8s.io
    name: renku-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: priorityclasses-manager

As an admin, setup renku project admin roles:

oc apply -f renku-roles.yaml

Amalthea session service account​

The default Security Context Constraint (SCC) used to start pods will not allow for Amalthea sessions to run as it uses fixed user values. This means that currently the nonroot-v2 SCC must be used. This can be achieved through the use of a service account with adequate configuration.

Create service account for amalthea-sessions to allow the use of a different SCC:

oc --namespace renku create serviceaccount renku-amalthea-sessions-scc-handler

Add the required SCC to the service account (must be done as an admin)

oc adm policy add-scc-to-user nonroot-v2 -z renku-amalthea-sessions-scc-handler -n renku

To make use of this service account in your cluster, set the localClusterSessionServiceAccount settings under dataService in the Helm chart values file.

dataService:
  localClusterSessionServiceAccount: renku-amalthea-sessions-scc-handler

Amalthea ingress​

The configuration for the sessions ingress have their own settings (which are similar to the one from the main ingress). In order to have them use the proper class, their annotations must be updated:

notebooks:
  sessionIngress:
    className: openshift-default
    annotations:
      # remove default nginx specific annotations
      nginx.ingress.kubernetes.io/proxy-body-size: null
      nginx.ingress.kubernetes.io/proxy-request-buffering: null
      nginx.ingress.kubernetes.io/proxy-buffer-size: null

Network Policies:​

There are some subtle differences in the DNS setup between a vanilla Kubernetes cluster and OpenShift. Without diving in the details, the main issue is that the NetworkPolicy for the AmaltheaSession objects created with Renku won't work as is in OpenShift.

There are three (maybe four) things to take into account:

  • The DNS is using port 5353
  • The DNS resolver is in a different namespace
  • The DNS resolver pods have a different label to match
  • Depending on your network setup there might be IP blocks to unlock

Here is an example of a modified NetworkPolicy that adds port 5353 to the list of allowed ports as well as updates the namespace and pod selectors to reach the appropriate dns pods.

It also shows how to unlock part of the internal network to make it accessible to AmaltheaSession. That last point is completely optional but might be useful depending on the network setup of the infrastructure hosting the cluster.

# values.yaml
networkPolicies:
  sessions:
    egress:
      - ports:
          - port: 53
            protocol: UDP
          - port: 53
            protocol: TCP
          - port: 5353
            protocol: UDP
          - port: 5353
            protocol: TCP
        to:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: openshift-dns
            podSelector:
              matchLabels:
                dns.operator.openshift.io/daemonset-dns: default
      - to:
          - ipBlock:
              cidr: 0.0.0.0/0
              except:
                - 10.0.0.0/8
                - 172.16.0.0/12
                - 192.168.0.0/16
      # Optional: unlock access to part of the internal network
      - to:
          - ipBlock:
              cidr: 172.31.0.0/16

Renku deployment​

With everything in place as listed in the previous steps, Renku can now be installed as usual

helm upgrade  --install --namespace renku renku renk/renku -f renku-values.yaml --timeout 1800s --skip-crds
info

Any Renku user can be made a Renku administrator. This is useful for setting up different global environments, resource pools or integrations. In addition, Renku administrators can access the projects and similar resources of any user in the platform. Read more at Managing Renku Admin Users.

Image building​

Based on Shipwright, OpenShift has a project called Build for RedHat OpenShift that provides all the necessary pieces to enable building images for the Renku platform.

Once the operator is installed, follow the Harbor and Shipwright instructions to get your system ready to build user session images.

Chart dependencies​

Renku depends on several sub-charts to be deployed. The components provided by them do not necessarily start as is on OpenShift.

The configuration example below contains the value changes required for them to start properly.

Example:​

Here follows a full configuration based on the minimal deployment values file.

Beside Renku specific elements, customization of child charts are also required. Here we can see the changes to be applied to make KeycloakX, PostgreSQL, Redis and Solr start properly.

Note that these changes are not Renku specific, they need to be applied in any cases when deployed in OpenShift.

# NOTE: Any resource requests and replica counts here are used only for development and testing purposes
# and likely will not be able to support more than 5 concurrent users.
# Therefore the resource requests, limits, replica counts here and are NOT SUITABLE FOR PRODUCTION.
---
enableInternalGitlab: false

# OpenShift
amalthea:
  deployCrd: false

amalthea-sessions:
  deployCrd: false

podSecurityContext:
  runAsUser: null
  runAsGroup: null
  runAsNonRoot: true
  seccompProfile:
    type: "RuntimeDefault"

securityContext:
  runAsUser: null
  runAsGroup: null
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  capabilities:
    drop: ["ALL"]
  seccompProfile:
    type: "RuntimeDefault"

networkPolicies:
  sessions:
    egress:
      - ports:
          - port: 53
            protocol: UDP
          - port: 53
            protocol: TCP
          - port: 5353
            protocol: UDP
          - port: 5353
            protocol: TCP
        to:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: openshift-dns
            podSelector:
              matchLabels:
                dns.operator.openshift.io/daemonset-dns: default
      - to:
          - ipBlock:
              cidr: 0.0.0.0/0
              except:
                - 10.0.0.0/8
                - 172.16.0.0/12
                - 192.168.0.0/16
      # Unlock access to part of the internal network
      - to:
          - ipBlock:
              cidr: 172.31.0.0/16
# end OpenShift

authz:
  resources:
    limits:
      memory: 75Mi
    requests:
      cpu: 50m
      memory: 75Mi
dataService:
  # added
  rbac:
    create: false

  localClusterSessionServiceAccount: renku-amalthea-sessions-scc-handler
  # end added
  dataTasks:
    resources:
      limits:
        memory: 250Mi
      requests:
        cpu: 50m
        memory: 250Mi
  k8sWatcher:
    resources:
      limits:
        memory: 200Mi
      requests:
        cpu: 20m
        memory: 200Mi
  resources:
    limits:
      memory: 750Mi
    requests:
      cpu: 50m
      memory: 750Mi
  replicaCount: 1
enableV1Services: false
gateway:
  replicaCount: 1
gitlab:
  enabled: false
global:
  anonymousSessions:
    enabled: false # Changed from default
  renku:
    domain: renku.apps.my-openshift.ch
  useHTTPS: true
  redis:
    port: 6379
    host: renku-redis-master
    sentinel:
      enabled: false
ingress:
  className: openshift-default
  enabled: true
  hosts:
    - renku.apps.my-open-shift.ch
  tls:
    - hosts:
        - renku.apps.my-openshift.ch
      secretName: renku-renku-ch-tls
keycloakx:
  resources:
    requests:
      memory: 600Mi
    limits:
      memory: 600Mi
  # added
  podSecurityContext:
    fsGroup: null

  securityContext:
    runAsUser: null
    runAsGroup: null
    runAsNonRoot: true
    allowPrivilegeEscalation: false
    capabilities:
      drop: ["ALL"]
    seccompProfile:
      type: "RuntimeDefault"
  # end added
notebooks:
  oidc:
    allowUnverifiedEmail: true
  sessionIngress:
    host: renku.apps.my-openshift.ch
    tlsSecret: renku-renku-ch-tls
    ingressClassName: openshift-default
postgresql:
  primary:
    # added
    volumePermissions:
      enabled: false
      securityContext:
        runAsUser: "auto"
    podSecurityContext:
      enabled: false

    shmVolume:
      chmod:
        enabled: false

    containerSecurityContext:
      enabled: false
    # end added
    resources:
      limits:
        memory: 300Mi
      requests:
        memory: 300Mi
  # Use Bitnami's PostgreSQL image from Renkulab Harbor registry
  image:
    registry: harbor.renkulab.io
    repository: bitnami-mirror/postgresql
redis:
  architecture: standalone
  master:
    persistence:
      enabled: false
  metrics:
    # Use the bitnami metrics image from the Renkulab Harbor registry
    image:
      registry: harbor.renkulab.io
      repository: bitnami-mirror/redis-exporter
  sentinel:
    enabled: false
    # Use Bitnami's Redis Sentinel image from Renkulab Harbor registry
    image:
      registry: harbor.renkulab.io
      repository: bitnami-mirror/redis-sentinel
  # Use Bitnami's Redis image from Renkulab Harbor registry
  image:
    registry: harbor.renkulab.io
    repository: bitnami-mirror/redis
secretsStorage:
  resources:
    limits:
      memory: 500Mi
    requests:
      cpu: 50m
      memory: 500Mi
solr:
  resources:
    limits:
      memory: 800Mi
    requests:
      cpu: 50m
      memory: 400Mi
  # Use Bitnami's Solr image from Renkulab Harbor registry
  image:
    registry: harbor.renkulab.io
    repository: bitnami-mirror/solr
  # added
  podSecurityContext:
    fsGroup: null
  containerSecurityContext:
    runAsUser: null
  # end added
ui:
  client:
    # added
    welcomePage:
      text: |
        ## Welcome to the OpenShift deployment of Renku!
    # end added
    resources:
      limits:
        memory: 150Mi
      requests:
        cpu: 10m
        memory: 150Mi
  server:
    resources:
      limits:
        memory: 75Mi
      requests:
        memory: 75Mi