GitHub Integration Guide

warning

Experimental Feature: The GitHub Integration is currently in an experimental phase and may undergo changes that could break existing implementations in future updates. Please use it with caution and stay updated with the latest releases.

Overview

The GitHub Integration Guide offers instructions to set up and configure GitHub within Querybook. Follow these steps to allow interaction between Querybook and your GitHub repositories.

info

Please check the GitHub User Guide for detailed instructions on using GitHub features.

Implementation

To integrate GitHub with Querybook, follow the steps below. This setup involves configuring GitHub OAuth, setting up necessary environment variables, and enabling the GitHub Integration feature.

1. Setup GitHub OAuth Application

Before integrating GitHub with Querybook, you need to create an OAuth application on GitHub to obtain the necessary credentials.

1. Navigate to GitHub Settings:

   • Go to your GitHub account settings.
   • Click on Developer settings.
   • Select OAuth Apps and then click New OAuth App.

2. Register a New Application:

   • Application Name: Choose a name for your application, e.g., Querybook Integration.
   • Homepage URL: Enter your Querybook instance URL, e.g., https://your-querybook-domain.com.
   • Authorization Callback URL: Set this to https://your-querybook-domain.com/github/oauth2callback.

3. Save the Application:

   • After registering, GitHub will provide a Client ID and Client Secret. Keep these credentials secure as they are required for the integration.

2. Install Dependencies

Ensure that the required Python packages are installed. GitHub Integration relies on OAuth libraries and other dependencies.

Add the following line to your requirements/local.txt:

-r github.txt

Note: The github.txt file includes pygithub==2.4.0 and cryptography==3.4.8, which are essential for interacting with the GitHub API and securing tokens. For more details, refer to infra_installation.mdx.

3. Configure GitHub Integration

Configure Querybook to use the GitHub feature by setting the necessary environment variables and updating configuration files. Secrets such as GITHUB_CLIENT_SECRET and GITHUB_CRYPTO_SECRET should be stored securely in environment variables, while non-sensitive information can be placed in querybook_config.yaml.

1. Set Config Variables:

   GITHUB_CLIENT_ID=github_app_client_id
   GITHUB_CLIENT_SECRET=github_app_client_secret
   GITHUB_CRYPTO_SECRET=crypto_secret
   GITHUB_REPO_NAME=github_username/github_repository
   GITHUB_REPO_BRANCH=main  # Optional, defaults to 'main' branch
   • GITHUB_CLIENT_ID: The Client ID obtained from GitHub OAuth App.
   • GITHUB_CLIENT_SECRET: The Client Secret obtained from GitHub OAuth App.
   • GITHUB_CRYPTO_SECRET: A secret key used for encrypting GitHub tokens in the database.
   • GITHUB_REPO_NAME: The repository name in the format user/repo_name (e.g., github_username/querybook-datadocs).
   • GITHUB_REPO_BRANCH: The branch to which commits are pushed. Defaults to main if not set.

   Note: To obtain GITHUB_REPO_NAME, format your repository name as username/repository. For example:

   1. Navigate to your GitHub profile and click on Repositories.
   2. Select the repository you want to link.
   3. Enter the repository name in the format username/repository_name. You can obtain this from your repository's GitHub URL. For instance, if your repository URL is https://github.com/username123/querybook-datadocs, the repository name would be username123/querybook-datadocs.

4. Enable the Feature in Querybook

To display the GitHub button on the Querybook UI for DataDocs, edit the querybook_public_config.yaml to enable GitHub Integration feature:

github_integration:
    enabled: true

Example Configuration

Below is an example configuration snippet demonstrating how to set up GitHub Integration in querybook_config.yaml and querybook_public_config.yaml:

querybook_config:
    GITHUB_CLIENT_ID: 'your_github_client_id'
    GITHUB_CLIENT_SECRET: '---Redacted---'
    GITHUB_CRYPTO_SECRET: '---Redacted---'
    GITHUB_REPO_NAME: 'github_username/querybook-datadocs'
    GITHUB_REPO_BRANCH: 'main'

public_config:
    github_integration:
        enabled: true

Additional Tips for Developers

• OAuth Permissions: Ensure that users completing the GitHub OAuth process have write permissions to the configured GitHub repository (GITHUB_REPO_NAME). This is necessary for Querybook to perform actions such as committing changes or managing the repository effectively.
• Security: Keep your GitHub OAuth credentials secure. Avoid hardcoding sensitive information in configuration files. Store secrets safely and securely using environment variables.
• Testing: After setting up, perform test commits to verify that the integration works as expected before deploying to production environments.

For more information, refer to the GitHub User Guide and GitHub's OAuth Apps Documentation.