Hi everyone! I’m excited to introduce this new blog post series to the world. I hope to share many more posts and invite you all to join me on these adventures.

What is the Data Product Portal?

The Data Product Portal is an open-source tool designed to help organisations build and manage data products at scale. It’s intuitive, flexible, and aimed at making data product management straightforward and effective.

For more information, check out our announcement blogpost. It covers the “why, what and how” of the Data Product Portal and introduces key concepts. Throughout this series, I’ll refer to the Data Product Portal as “the Portal”.

The Portal is open-source! Visit our Github and leave a star if you like what you see. You can also join our Slack community to chat with us directly.

What is This Blog Post Series?

The Portal is designed for easy setup. Simply run docker compose up or helm install on your K8S cluster, and you’re ready to go. This will present you with a bare-bones version of Portal. However, the Portal’s functionality becomes truly exciting when integrated with other services, such as AWS for role management or OIDC for user authentication.

This blog series will guide you through various integrations for the Portal with technical examples. Topics include integrating the Portal with OIDC, linking the Portal to AWS, and connecting to Conveyor.

Following this series will help you fully utilize the Portal by integrating it with essential technologies, creating a comprehensive hub for your data professionals.

This blogpost focuses on integration with OIDC.

Why Do You Need OIDC Integration?

The Portal allows large-scale data access within your organization. A crucial feature is the ability to authenticate and authorize users. By verifying user identities, the Portal can securely and correctly grant access to the appropriate data products.

Without OIDC, all users will appear as “John Doe”, making it impossible to differentiate access rights. Enabling OIDC is the first step to making your Portal installation more useful for your entire organization.

What is OIDC?

OpenID Connect (OIDC) is an identity authentication protocol that is an extension of open authorization (OAuth) 2.0 to standardize the process for authenticating and authorizing users when they sign in to access digital services.

Implementing OIDC with the Portal allows you to link the Portal to your organization’s SSO or identity provider, enabling secure user sign-in and user-specific data access.

How to Integrate OIDC with the Portal

Supported Providers

The Portal has been tested with AWS Cognito but should work with other OIDC providers with minimal code changes. As an open-source project, contributions to support additional OIDC providers are welcome.

OIDC returns identity , refresh and access tokens. Providers may vary in the information they pass in tokens, and the application depends on this information, so integrating a new provider might require some code changes there as well.

The claims our provider is required to pass are at least email, family_name and name.

Cognito setup

While I won’t cover the complete setup of a Cognito user pool (you can find that here), here are the key points for the Portal:

• Add email, family_name and name to the required_attributesin your user pool.

• Create an app integration and note the client ID and secret.

• Find the authority URL using the format:

https://cognito-idp.<COGNITO_REGION>.amazonaws.com/<COGNITO_USER_POOL_ID>

• Set the correct callback URLs

I will dive deeper into these steps below.

Portal makes use of Cognito User Pools. When creating your user pool, make sure to add at least email, family_name and name to the required_attributes. These claims are used in portal.

Your user pool will need exactly one app integration.

When you click on this app integration you can find out the client ID and secret of your app client. These values are important below.

The last value you need to find is the authority url. In case of Cognito you can create it with the following template.

https://cognito-idp.<COGNITO_REGION>.amazonaws.com/<COGNITO_USER_POOL_ID>

For other OIDC providers we require the URL that hosts the .well-known/openid-configuration file of your provider.

Callback URLs

The Portal requires specific callback URLs allowed by Cognito. These are necessary for both local development and production deployment. Ensure the following URLs are in your allowed sign-out callback list:

For local development:

For production (HOST is the CNAME of your application — the DNS record that maps the alias name of your application to it’s true or canonical domain name):

https://<HOST>/https://<HOST>/api/auth/device/callback/https://<HOST>

The following list needs to be added to the allowed sign-out callback list.

http://localhost:3000/logout/ # For local developmenthttps://<HOST>

Integrating OIDC Locally / Docker Integration

Backend

When running the portal locally, most of the integration of external services happens in a .env configuration file.
Running docker compose up uses .env.docker by default. If you want to build your dockers locally for deploying, you need to create a .env file in the root of the project. Please refer to the README in backend/ for further explanation.

Make sure the following configuration values are present in the .env file used in your Docker deployment pipeline. The values can be derived from your Cognito user pool as explained above.

OIDC_CLIENT_ID=<ID> # Cognito Client IDOIDC_CLIENT_SECRET=<SECRET> # Cognito client secretOIDC_AUTHORITY=<AUTH_URL>

Frontend

The frontend docker used when running locally or when executing npm dev uses a config.js file. Please refer to the README in frontend/ for further explanation on how to set this up. The file should contain the following code:

const config = (() => {    return {        API_BASE_URL: "http://localhost:5050",        OIDC_DISABLED: false,        OIDC_CLIENT_ID: "<CLIENT_ID>",        OIDC_CLIENT_SECRET: "<CLIENT_SECRET>",        OIDC_AUTHORITY: "<AUTH_URL>

Integrating OIDC in Your Production Installation

The easiest way to deploy portal in a production environment is through Helm. You can enable OIDC in Helm by passing the following configuration in the values.yaml file provided through Helm.

Make sure your values.yaml contains the following entries.

oidc:  disabled: false  client_id: <client_id>  client_secret: <client_secret>  authority: <auth_url>  redirect_uri: <host>

Conclusions

Setting up OIDC support for your Portal deployment is straightforward. Adjust the .env and config.local.js files for local development, or pass values in values.yaml for Helm installs.

Let me know if this tutorial was clear and feel free to reach out with questions. Join us on Github or in our Slack community. Stay tuned for the next installment of this series!