Everything You Need to Know About The Apache DolphinScheduler Version Upgrade

Apache DolphinScheduler has added an automatic version upgrade feature since version 2.0.1. The official documentation provides a one-click upgrade script:

sh ./script/create-dolphinscheduler.sh 

If it's a minor version update, simply executing the script suffices. However, upgrading across multiple major versions can still encounter issues. Here's a summarized guide.

(Applicable for upgrades from 1.x → 2.x or 2.x → 3.x)

I. Pre-Upgrade Preparations

1. Data Backup
   • Database Backup: Backup the DolphinScheduler metadata database (MySQL/PostgreSQL)

mysqldump -u[username] -p[password] dolphinscheduler > dolphinscheduler_backup.sql

• Configuration Backup: Backup all configuration files under the conf/ directory (e.g., application.yaml, common.properties)
• Resource Backup: Backup custom scripts, JAR packages, and other resources in the resources/ directory

2. Version Compatibility Check

   • Verify whether the current version supports direct upgrades to the target version (e.g., upgrading from 2.0.5 to 3.1.0 requires checking the official compatibility matrix)
   • Ensure JDBC driver, ZooKeeper, and other dependency versions meet the target version requirements

   
   

3. Environment Check

   • Confirm server resources (CPU/memory/disk) meet the new version's minimum requirements
   • Stop all running scheduling tasks to avoid task state loss during upgrade

II. Upgrade Steps

Stop All DolphinScheduler Services

Stop all services based on your deployment method. For cluster deployments, use:

sh ./script/stop-all.sh 

[[2, 3]]

Database Upgrade

1. Modify configurations in ./bin/env/dolphinscheduler_env.sh (replace {user} and {password} with your database credentials).

   For MySQL:

   export DATABASE=${DATABASE:-mysql}
   export SPRING_PROFILES_ACTIVE=${DATABASE}
   export SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
   export SPRING_DATASOURCE_USERNAME={user}
   export SPRING_DATASOURCE_PASSWORD={password}

2. Manually download the mysql-connector-java driver and place it in ./tools/libs [[3, 5]]
3. Execute the database upgrade script:

   sh ./tools/bin/upgrade-schema.sh 

Service Upgrade

1. Update configurations in bin/env/install_config.conf based on your deployment:

   • Pseudo-Cluster: Follow Pseudo-Cluster Deployment Guide

   • Cluster: Follow Cluster Deployment Guide

     
     

2. Restart services:

   sh ./bin/start-all.sh 

III. Post-Upgrade Verification

1. Service Health Check

   curl [http://localhost:12345/dolphinscheduler/actuator/health ](http://localhost:12345/dolphinscheduler/actuator/health )  # Check API health
   tail -n 100 logs/api-server.log                               # Check logs for errors

2. Task Execution Test
   • Manually trigger a test workflow to confirm scheduling, execution, and alerting
   • Verify historical task states are fully migrated
3. Functionality Compatibility Check
   • Ensure APIs, custom plugins, tenant configurations, and UI operations (e.g., workflow definition) work normally

IV. Rollback Plan

1. Database Restoration

   mysql -u[username] -p[password] dolphinscheduler < dolphinscheduler_backup.sql

2. Service Rollback
   • Stop the new version and restore the old installation directory
   • Restart services using old configurations

V. Important Notes

1. Incremental Upgrades
   • For upgrades from 1.x to 3.x, follow sequential steps (e.g., 1.3.9 → 2.0.5 → 3.1.0) [[8, 9]]
2. Database Migration
   • To switch database types (e.g., MySQL → PostgreSQL), rebuild the database using scripts under sql/create/
3. Plugin Compatibility
   • Custom alert plugins and task types must adapt to the new version's SPI interfaces
4. Community Support
   • Refer to the Official Upgrade Documentation for troubleshooting