How to permit outbound access to internet hosts in deployments with government community network connectivity

Overview

This article outlines the necessary steps to add domain names to an allow-list within the Squid proxy. Squid is an enabled service within v3.11 OpenShift clusters deployed with connectivity to government community networks (such as: HSCN, PSN, Janet). Domains on the allow-list can be accessed on the internet via the proxy, enabling you to request external resources/data on nodes that previously only had connectivity to government community networks. Added domains should be scrutinised as the relevant authority of the community network may require you to submit documentation regarding these for you to receive accreditation.

Prerequisites

To complete the steps in this guide, you must have access to a cluster that is running OpenShift Container Platform v3.11 (or newer) with connectivity to a government community network. Squid proxy is not currently enabled within environments that do not have access to a government community network.

You must also have access to either:

the OpenShift Web console
oc, the OpenShift command-line client (CLI). For more information, see OpenShift's Get Started with the CLI

Destination domain allow-list

Squid proxy is installed as a service on the control plane load balancers. Within the Squid configuration, we have specified an ACL (Access Control List) as being a list of destination domains (present in a file on the file system), which we will refer to as the allow-list for the remainder of this article. Outbound traffic from the cluster's internal network will pass through the proxy with the destination domain being compared against the allow-list for each connection attempt; if the domain exists then outbound internet traffic is permitted, otherwise it is denied with a HTTP 403 error.

By default, the following domains are added to the allow-list (and cannot be removed) to facilitate installation, testing and ongoing operation of the cluster:

registry.access.redhat.com
registry.redhat.io
<UKCloud Object Storage Endpoint - cas.cor00005.ukcloud.com OR cas.frn00006.ukcloud.com>
<UKCloud SSO Endpoint - idp.ukcloud.com>

Warning

Adding any subdomains of domains that are already within the proxy allow-list will lead to squid being unable to reconfigure and any domains added after this point will not be accessible. Please keep the above domains in mind when adding to the allow-list as these will not be shown in the ConfigMap. To give an example, you should avoid adding .ukcloud.com or .redhat.com to your proxy allow-list as both registry.access.redhat.com and idp.ukcloud.com are already on the allow-list.

A scheduled job (which runs at 0 minutes past every hour) on the OpenShift cluster Bastion host reads a Config Map named proxy-whitelist within the whitelist project. If there are any modifications to this Config Map, the job overwrites the previous custom entries within the allow-list and triggers a reconfigure task on the Squid proxy to enable the updated domains to be accessed.

Assigning non cluster-admin users rights to edit the allow-list

By default, only users who have been assigned the cluster-admin role will be able to view/edit the proxy-whitelist ConfigMap object within the whitelist project. It is possible to grant non cluster-admin users rights to edit this object but this should be given careful consideration. The ability to access hosts on the internet, from nodes that previously only had access to government community networks, exposes the cluster to additional risk (should a domain be added to the allow-list that hosts malicious content for example). For this reason, only trusted users should be permitted to determine these domains to reduce the risk of a malicious domain being unwittingly added.

To allow non cluster-admin users to edit this object, use the following Role Based Access Control (RBAC) commands:

Tip

To add multiple users to these roles, separate each username within the username variable with a space.

export username=<replace-with-username>
# Create roles
oc create role view-project --verb=get,list,watch --resource=namespaces --verb get --resource=projects -n whitelist
oc create role get-configmap --verb=get,list --resource=configmaps -n whitelist
oc create role edit-proxy-whitelist --verb=patch,update --resource=configmaps --resource-name=proxy-whitelist -n whitelist
# Bind roles to desired user
oc policy add-role-to-user view-project $username -n whitelist --role-namespace='whitelist'
oc policy add-role-to-user get-configmap $username -n whitelist --role-namespace='whitelist'
oc policy add-role-to-user edit-proxy-whitelist $username -n whitelist --role-namespace='whitelist'

Modifying the allow-list

There are two tools you can use to modify the proxy-whitelist Config Map. As previously stated you will need to have the necessary rights to edit this object within the whitelist project.

Web console

To view the proxy-whitelist Config Map within the whitelist project, navigate to the following URL (ensuring you replace <your-deployment-name>):

https://ocp.<your-deployment-name>.cna.ukcloud.com:8443/console/project/whitelist/browse/config-maps/proxy-whitelist

Tip

To add all subdomains to the allow-list, precede the domain with a . for example .domain.com

Click Actions then Edit to modify the values within the proxy-whitelist.txt key. Each domain should be on its own line:
Click Save when finished.

oc client

Note

To authenticate using the oc client you will need an API token. Retrieve this by logging in to the Web Console and clicking your name in the top right then choosing Copy Login Command. This can then be pasted from your buffer into a terminal where you have the oc client installed.

Change to the whitelist project:
```
$ oc project whitelist
```

Edit the proxy-whitelist Config Map:

$ oc edit cm proxy-whitelist

This will display the object definition in yaml format in your default text editor:

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  proxy-whitelist.txt: ""
kind: ConfigMap
metadata:
  creationTimestamp: 2019-01-18T01:18:44Z
  name: proxy-whitelist
  namespace: whitelist
  resourceVersion: "5385199"
  selfLink: /api/v1/namespaces/whitelist/configmaps/proxy-whitelist
  uid: ff7df4a9-1abe-11e9-965a-fa163ecc60e0

Within the object definition data attribute is the proxy-whitelist.txt key. By default this will be an empty single line string but you can modify this to be a multi-line string by changing "" to |- as below and indenting the domains you want to add with two spaces. For example:
```
proxy-whitelist.txt: |-
  firstdomain.com
  .seconddomain.com
```
Save your changes to persist to the Config Map, otherwise cancel your edit to not make any changes.

Feedback

If you find a problem with this article, click Improve this Doc to make the change yourself or raise an issue in GitHub. If you have an idea for how we could improve any of our services, send an email to feedback@ukcloud.com.