Note: This documentation was written for an earlier version of cBioPortal. The information presented here may not apply to cBioPortal v7 and onwards.

MSK Maintenance (In Progress)

We provide our cBioPortal's maintenance page publicly in the hope that it may be useful to others

Database Migration

In the database migration process, we are going to have two main steps: building importers and updating database scheme.

Building Importers

Codebases:
Building single importer:
- Take the cbioportal hash and add it to genome nexus annotation pipeline POM -> push to fork
- Take the cbioportal hash and new genome nexus annotation pipeline hash (from above) and propagate to pipelines POM; also update db version if db migration is part of change
- Login to pipelines server
- Sets general env variables needed for building
  - Run sh /data/portal-cron/scripts/automation-environment.sh
- Specific to cbioportal build, looks for properties and xml files off of $PORTAL_HOME
  - Run export PORTAL_HOME=/data/portal-cron/git-repos/cbioportal
- Make sure the local cbioportal codebase is on the correct git hash at /data/portal-cron/git-repos/cbioportal
- Specify importer to be build
  - Copy importer properties to /data/portal-cron/git-repos/cbioportal/src/main/resources (e.g. triage /data/portal-cron/git-repos/pipelines-configuration/properties/import-triage/*)
- Navigate to pipelines folder
  - cd /data/portal-cron/git-repos/pipelines
- Build importer
  - mvn clean install -DskipTests
Build multiple importers
- Take the cbioportal hash and add it to genome nexus annotation pipeline POM -> push to fork
- Take the cbioportal hash and new genome nexus annotation pipeline hash (from above) and propagate to pipelines POM; also update db version if db migration is part of change
- Login to pipelines server
- Sets general env variables needed for building
  - Run sh /data/portal-cron/scripts/automation-environment.sh
- Specific to cbioportal build, looks for properties and xml files off of $PORTAL_HOME
  - Run export PORTAL_HOME=/data/portal-cron/git-repos/cbioportal
- Make sure the local cbioportal codebase is on the correct git hash at /data/portal-cron/git-repos/cbioportal
- Go to build importer jars folder
  - cd /data/portal-cron/git-repo/pipelines-configuration/build-importer-jars
- (Optional) Remove existing jars
  - rm *.jar
- Build all importers at once (build all importers except cmo-pipelines)
  - sh buildproductionjars.sh -sd=true -sgp=true -b=importers
  - Available parameters:
    - [--cbioportal-git-hash|-cgh=<cbioportal_commit_hash>]
    - [--skip-deployment|-sd=<true|false>]
    - [--skip-git-pull|-sgp=<true|false>]
    - [--build|-b=<build_specifier>], build_specifier should be one of the following
      - all (build for all artifacts)
      - importers (all importers except cmo-pipelines)
      - cmo-pipelines (cmo-pipelines artifacts only)
      - triage-cmo-importer
      - msk-dmp-importer
      - msk-cmo-importer
      - public-importer
      - genie-aws-importer
      - genie-archive-importer
      - hgnc-importer

Updating Database Scheme

Database needs to be updated one by one, we have four main databases: triage, private, genie, and public. Take triage database as an example.

Migrate one database (e.g. triage)
- SSH into pipeline server
- Checkout to the commit that contains the latest database scheme
- Check if property sets up correctly to the right database (triage)
  - vi /data/portal-cron/git-repos/cbioportal/src/main/resources/application.properties
- Move to directory
  - cd /data/portal-cron/git-repos/cbioportal
- Run database migration using script:
  - python3 core/src/main/scripts/migrate_db.py --properties-file src/main/resources/application.properties --sql db-scripts/src/main/resources/migration.sql
- Monitor the DB migration process and look for possible errors
- Access database and verify the DB scheme is updated