Skip to content

DBDiff/DBDiff

 
 

DBDiff logo

Build Status Total Downloads Monthly Downloads License

DBDiff is an automated database schema and data diff tool. It compares two databases, local or remote, and produces a migration file of the differences automatically.

When used alongside a compatible database migration tool, it can help enable database version control within your team or enterprise.

Features

  • Works on Windows, Linux & Mac command-line/Terminal because it has been developed in PHP
  • Connects to a source and target database to do the comparison diff, locally and remotely
  • Diffs can include changes to the schema and/or data, both in valid SQL to bring the target up-to-date with the source
  • Deterministic Output: SQL is generated in a stable, predictable order (tables, columns, and data), ensuring consistent migrations and test pass rates across different environments.
  • Some tables and/or fields can be ignored in the comparison with a YAML collection in the config file (see File Examples)
  • Diffs are SUPER fast and this tool has been tested with databases of multiple tables of millions of rows
  • Since this diff tool is being used for migrations, it provides up and down SQL in the same file
  • Works with existing migration tools like Flyway and Simple DB Migrate by specifying output template files/formats
  • Is Unicode aware, can work with UTF8 data, which includes foreign characters/symbols
  • Multi-database support: MySQL, PostgreSQL, and SQLite via the --driver flag
  • Supabase-ready: use --supabase as a one-flag shorthand for connecting to a Supabase PostgreSQL instance over SSL

Pre-requisites

  1. You will need to have access to the command-line (Terminal/CMD/PowerShell)
  2. You will need to have git installed
  3. You will need to have PHP installed (version 7.4.x, 8.3.x, 8.4.x, or 8.5.x)
  4. You will need to have Composer installed

Note: Make a note of where composer.phar is installed as we will need it later on during Setup

  • PHP 7.4.x
  • PHP 8.3.x
  • PHP 8.4.x
  • PHP 8.5.x

Supported Databases

Other versions may work but are not actively supported. Feel free to contribute a PR to add official support.

MySQL

  • MySQL 8.0.x
  • MySQL 8.4.x (LTS)
  • MySQL 9.3.x (Innovation)
  • MySQL 9.6.x (Innovation)

PostgreSQL

  • PostgreSQL 14.x
  • PostgreSQL 15.x
  • PostgreSQL 16.x (LTS)
  • PostgreSQL 17.x
  • PostgreSQL 18.x

Use --driver=pgsql (or pass driver: pgsql in your .dbdiff config file).

SQLite

  • SQLite 3.x (any version supported by the installed pdo_sqlite PHP extension)

Use --driver=sqlite. For SQLite the comparison argument uses the file path as the database name:

./dbdiff --driver=sqlite server1./path/to/source.db:server1./path/to/target.db

Supabase

Use the --supabase shorthand flag — it sets driver=pgsql and enables SSL automatically:

./dbdiff --supabase --server1=user:pass@db.xxx.supabase.co:5432 server1.mydb:server1.mydb

Installation

On the command-line, use git to clone the ssh version:

git clone git@github.com:DBDiff/DBDiff.git

Or use git to clone the https version:

git clone https://github.com/DBDiff/DBDiff.git

Or download the .zip archive and unzip it to a folder of your choosing e.g. dbdiff:

https://github.com/DBDiff/DBDiff/archive/master.zip

Or use composer to include DBDiff as a project dependency:

php composer.phar require "dbdiff/dbdiff:@dev"

Or use composer to install DBDiff globally:

composer global require "dbdiff/dbdiff:@dev"

Create a PHAR build

Please first ensure in your php.ini file the phar.readonly setting is set to false , for example:

[Phar]
; http://php.net/phar.readonly
phar.readonly = false

Then in the root of the dbdiff repository to produce a Phar build simply run:

$ ./scripts/build

A dist folder should be created containing the following files:

  • dbdiff.phar
  • dbdiff.phar.gz

Feel free to rename dbdiff.phar to dbdiff and move it to /usr/local/bin or another directory of your choice.

You can also add it to your system's path if you wish to make it globally available on your system as a utility.

Docker

You may now use docker / docker-compose or Podman / podman-compose to create a local environment for DBDiff (for testing or production), including a PHP server with a database and the DBDiff CLI available as a service.

Please ensure you have one of the following installed locally, as well as a download of the git repository, before continuing:

  • Docker + docker-compose (or Docker Desktop)
  • Podman + podman-compose — a daemonless, rootless Docker-compatible alternative (see DOCKER.md for setup details)

Note: Please run these commands from the root of the DBDiff folder. Also the commands may need to be prepended with sudo on some systems.

Docker Standalone DBDiff CLI with PHP 7.3

# Build DBDiff CLI Image
docker build --tag "dbdiff:latest" --file "docker/Dockerfile" .
# Run DBDiff CLI Image as a Container
docker run -i -t --ipc=host --shm-size="1g" "dbdiff:latest" <command>
# Remove DBDiff CLI Image
docker image rm dbdiff:latest

Cross-Version Testing

You can easily test DBDiff against any combination of PHP and MySQL using the provided start.sh script.

# Run tests for a specific combination
./start.sh 8.3 8.0

# Run all 16 version combinations in parallel (4x speedup)
./start.sh all all --parallel

The CI matrix mirrors this locally: 4 PHP versions × 4 MySQL versions = 16 jobs, plus dedicated jobs for SQLite (4 PHP versions) and PostgreSQL (4 PHP × 5 Postgres versions = 20 jobs).

See the DOCKER.md file for extensive documentation on the Dockerized test runner, including flags for fast restarts, recording fixtures, and automated CI.

Removing Docker Compose DBDiff Environment

docker-compose down

Setup

Make sure you are in the root of your application for all the following steps, using 'cd' to navigate on the command line to where you have placed your "dbdiff" folder

We are going to assume that composer.phar is installed inside your "dbdiff" folder. If it is installed elsewhere you will need to use it's exact path

  1. If you didn't install DBDiff with composer, install the dependencies of the project with: php composer.phar install
  2. Make a .dbdiff file by following the File Examples and place it in the root of your "dbdiff" directory
  3. Type ./dbdiff {dbdiff command here e.g. server1.db1:server1.db2} to start the app! See Command-Line API for more details on which commands you can run.

You should see something like...

ℹ Now calculating schema diff for table `foo`
ℹ Now calculating data diff for table `foo`
ℹ Now generating UP migration
ℹ Now generating DOWN migration
ℹ Writing migration file to /path/to/dbdiff/migration.sql
✔ Completed

Congratulations you have installed and ran DBDiff!

Command-Line API

Note: The command-line parameters will always override the settings in the .dbdiff config file

  • --server1=user:password@host1:port - Specify the source db connection details. If there is only one server the --server1 flag can be omitted
  • --server2=user:password@host2:port - Specify the target db connection details (if it’s different to server1)- --driver=mysql|pgsql|sqlite - Database driver to use. Defaults to mysql. Use pgsql for PostgreSQL (including Supabase), sqlite for file-based SQLite databases.
  • --supabase - Convenience shorthand: sets --driver=pgsql and enables SSL (sslmode=require). Designed for use with Supabase.- --template=templates/simple-db-migrate.tmpl - Specifies the output template, if any. By default will be plain SQL
  • --type=schema or data or all - Specifies the type of diff to do either on the schema, data or both. schema is the default
  • --include=up or down or all - Specified whether to include the up, down or both data in the output. up is the default
  • --nocomments=true - By default automated comments starting with the hash (#) character are included in the output file, which can be removed with this parameter
  • --config=config.yaml - By default, DBDiff will look for a .dbdiff file in the current directory which is valid YAML, which may also be overridden with a config file that lists the database host, user, port and password of the source and target DBs in YAML format (instead of using the command line for it), or any of the other settings e.g. the format, template, type, include, nocomments. Please note: a command-line parameter will always override any config file.
  • server1.db1.table1:server2.db2.table3 or server1.db1:server2.db2 - The penultimate parameter is what to compare. This tool can compare just one table or all tables (entire db) from the database
  • --output=./output-dir/today-up-schema.sql - The last parameter is an output file and/or directory to output the diff to, which by default will output to the same directory the command is run in if no directory is specified. If a directory is specified, it should exist, otherwise an error will be thrown. If this path is not specified, the default file name becomes migration.sql in the current directory

Usage Examples

Example 1 -- MySQL (default)

$ ./dbdiff server1.db1:server2.db2

Looks for the .dbdiff config file for connection details and compares the schemas of db1 and db2, outputting migration.sql in the current directory.

Example 2 -- MySQL table data diff

$ ./dbdiff server1.development.table1:server2.production.table1 --nocomments=true --type=data

Compares only the data of table1 between the two databases; no comment headers in output.

Example 3 -- MySQL with template and full output

$ ./dbdiff --config=config.conf --template=templates/simple-db-migrate.tmpl --include=all server1.db1:server2.db2 --output=./sql/simple-schema.sql

Uses config.conf for connection settings, applies the simple-db-migrate template, and writes both up and down SQL to ./sql/simple-schema.sql.

Example 4 -- PostgreSQL

$ ./dbdiff --driver=pgsql --server1=user:pass@localhost:5432 server1.staging:server1.production

Compares two PostgreSQL databases on the same server. Use --driver=pgsql or set driver: pgsql in your .dbdiff file.

Example 5 -- Supabase

$ ./dbdiff --supabase --server1=postgres:pass@db.xxxx.supabase.co:5432 server1.staging:server1.production

--supabase is equivalent to --driver=pgsql with SSL enabled. No extra config needed.

Example 6 -- SQLite

$ ./dbdiff --driver=sqlite server1./var/db/v1.db:server1./var/db/v2.db

Compares two SQLite files. No --server1 flag is needed -- the file paths are embedded directly in the comparison argument. Paths must not contain dots other than in the filename extension.

File Examples

.dbdiff

server1:
	user: user
	password: password
	port: port # MySQL: 3306 | PostgreSQL: 5432
	host: host1 # usually localhost or 127.0.0.1
server2:
	user: user
	password: password
	port: port # MySQL: 3306 | PostgreSQL: 5432
	host: host1 # usually localhost or 127.0.0.1
driver: mysql # mysql | pgsql | sqlite (default: mysql)
template: templates/simple-db-migrate.tmpl
type: all
include: all
nocomments: true
tablesToIgnore:
- table1
- table2
- table3
fieldsToIgnore:
	table1:
		- field1
		- field2
		- field3
	table4:
		- field1
		- field4

simple-db-migrate.tmpl

SQL_UP = u"""
{{ $up }}
"""
SQL_DOWN = u"""
{{ $down }}
"""

How Does the Diff Actually Work?

The following comparisons run in exactly the following order:

  • When comparing multiple tables: all comparisons should be run
  • When comparing just one table with another: only run the schema and data comparisons

Overall Comparison

  • Check both databases exist and are accessible, if not, throw an error
  • The database collation is then compared between the source and the target and any differences noted for the output

Schema Comparison

  • Looks to see if there are any differences in column numbers, name, type, collation or attributes
  • Any new columns in the source, which are not found in the target, are added

Data Comparison

  • And then for each table, the table storage type (e.g. MyISAM, CSV), the collation (e.g. utf8_general_ci), and number of rows are compared, in that order. If there are any differences they are noted before moving onto the next test
  • Next, both changed rows as well as missing rows from each table are recorded

Compatible Migration Tools

Project Language / Package Manager Description
Simple DB Migrate Python / PIP Generic database migration tool inpired on Rails migrations
Flyway Java / Maven Database Migrations Made Easy

Please do let us know if you're using any other migration tools with DBDiff, other than the ones listed here, so we can add it.

Questions & Support 💡

  • Create a new issue if you can't find yours being addressed
  • Watch this space, as we're in the process of creating a discourse forum for all the DBDiff community
  • The documentation so far is what you see on this page, however this will slowly be expanded onto it's own website
  • If you are a company or organisation interested in commercial support packages for DBDiff please get in touch

Contributions 💖

Please make sure to read the Contributing Guide before making a pull request.

Thank you to all the people who already contributed to DBDiff!

Releasing 🚀

DBDiff uses automated workflows for versioning and distribution:

1. Automated Release (Recommended)

You can trigger a formal release directly from the GitHub Actions tab:

  • Select the "Release DBDiff" workflow.
  • Click "Run workflow" and specify the new version (e.g. v2.0.0).
  • This will automatically:
    • Build the production PHAR using scripts/build.
    • Create and push the Git tag.
    • Create a GitHub Release with build assets attached.
    • Notify Packagist to update the stable version.

2. Manual/Local Release

If you need to tag a release locally:

./scripts/release.sh v2.0.0
git push origin v2.0.0

Then manually upload the files from the dist/ folder to the GitHub Release page.


Feedback 💬

If you've made it down here, you're probably a fan 😉

Could you please kindly spare 2 minutes to give us your feedback on DBDiff:

https://forms.gle/gjdJxZxdVsz7BRxg7

We read each and every suggestion that comes through.

License

MIT

Made with 💖 by Akal Logo

About

Compare MySQL, Postgres or SQLite databases & automatically create schema & data change scripts/migrations automatically for database version control. Supports Flyway, Liquibase and has a built-in migration tool.

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages