Here are the list of breaking changes that you should be aware of when updating Querybook:
If you use the public docker image on https://hub.docker.com/r/querybook/querybook, no action is required.
To ensure the build time is scalable with the increasing amount of custom integrations, optional python packages such as pyhive (for Hive, Presto support) and oauth are removed from the default installation.
The following integrations will now require custom installation:
- Query Engine:
- BigQuery (google-cloud-bigquery)
- Druid (pydruid)
- Hive (pyhive)
- Presto (pyhive)
- Snowflake (snowflake-sqlalchemy)
- Trino (trino)
- Hive Metastore (w/ Thrift) (hmsclient)
- Glue (boto)
- OAuth (requests-oauthlib)
- LDAP (python-ldap)
- GSpread exporter (gspread)
- Result Store:
- AWS S3 (boto3)
- Google GCS (google-cloud-storage)
- AWS Based (requests-aws4auth)
Even though these packages are removed from the default installation, these dependencies are quite easy to add back! Checkout the Infra Installation Guide to learn how.
Depending on deployment of Querybook, re-initialization of indices in ElasticSearch 7 cluster might be needed.
This may happen for example when your
web component is not started with
querybook/scripts/bundled_docker_run_web script with
In such a cases, one has the following options how to initialize them manually:
In Docker based deployments, attach to
workercomponent and run
Locally, set following keys to proper values in
FLASK_SECRET_KEY: ...DATABASE_CONN: ...REDIS_URL: ...ELASTICSEARCH_HOST: ...
PYTHONPATH=querybook/server python ./querybook/server/scripts/init_es.py
In case some inconsistency occurs in ES indices source data in development deployment using
or deployment using
docker-compose, one can clear cached ES data by stopping and removing
and then removing Docker volume
docker-compose will create fresh volume again.
Kubernetes base deployments that uses ElasticSearch templates as they are should work without any impact, as the volumes are always recreated with deployment of a new (updated) pod.
Now announcements have two extra fields -
active_till. These fields are not required and a user can still create an announcement without these two fields and if an announcement has two one these fields and the date in these fields are not in the range, this announcement will be filtered.
Result store plugin change. Now BaseReader::get_download_url requires a custom_name field to rename the download file. You only need to add the
custom_name=None in the parameters for it to work.