4.3 KiB
NMIG - the database migration tool.
WHAT IS IT ALL ABOUT?
NMIG is an app, intended to make a process of migration from MySQL to PostgreSQL as easy and smooth as possible.
KEY FEATURES
- Precise migration of the database structure - NMIG converts MySQL data types to corresponding PostgreSQL data types, creates constraints, indexes, primary and foreign keys exactly as they were before migration.
- Ability to rename tables and columns during migration.
- Ability to recover migration process if disaster took place (without restarting from the beginning).
- Ability to migrate big databases - in order to eliminate "process out of memory" issues NMIG will split each table's data into several chunks.
Each group of chunks will be loaded via separate worker process. - Speed of data transfer - in order to migrate data fast NMIG uses PostgreSQL COPY protocol.
- Ease of monitoring - NMIG will provide detailed output about every step, it takes during the execution.
- Ease of configuration - all the parameters required for migration should be put in one single JSON document.
SYSTEM REQUIREMENTS
- Node.js 5 or greater
USAGE
1. Create a new database.
Sample: CREATE DATABASE my_postgresql_database;
2. Download NMIG package and put it on the machine running your PostgreSQL (not mandatory, but preferably).
Sample: /path/to/nmig
3. Edit configuration file located at /path/to/nmig/config.json with correct details.
- config.json contains brief description of each configuration parameter
- Make sure, that username, you use in your PostgreSQL connection details, defined as superuser (usually "postgres")
More info: http://www.postgresql.org/docs/current/static/app-createuser.html
4. Go to nmig directory, install dependencies, and run the app
Sample:
$ cd /path/to/nmig
$ npm install
$ node nmig.js
5. If a disaster took place during migration (for what ever reason) - simply restart the process
$ node nmig.js
NMIG will restart from the point it was stopped at.
6. At the end of migration check log files, if necessary.
Log files will be located under "logs_directory" folder in the root of the package.
Note: "logs_directory" will be created during script execution.
7. In case of any remarks, misunderstandings or errors during migration,
please feel free to email me
anatolyuss@gmail.com
VERSION
Current version is 2.4.0
(major version . improvements . bug fixes)
TEST
Tested using MySQL Community Server (5.7.17) and PostgreSQL (9.5).
The entire process of migration 59.9 MB database (53 tables, 581275 rows),
which includes data types mapping, creation of tables, constraints, indexes,
PKs, FKs, migration of data, garbage-collection (VACUUM) and analyzing the newly created
PostgreSQL database took 1 minute 2 seconds.
Note: it is highly recommended to VACUUM newly created database!
Just keep in mind, that VACUUM is a very time-consuming procedure.
So if you are short in time - disable VACUUM via config.json ("no_vacuum" parameter).
Such step will save you ~25% of migration time.
The migration process described above without VACUUM took 58 seconds only.
REMARKS
Errors/Exceptions are not passed silently.
Any error will be immediately written into the error log file.
KNOWN ISSUES
- Empty strings in char/varchar columns may be interpreted as NULL.
LICENSE
NMIG is available under "GNU GENERAL PUBLIC LICENSE" (v. 3)
http://www.gnu.org/licenses/gpl.txt.