------------------------------------------------------------------
Opportunity
------------------------------------------------------------------
This is a fantastic opportunity for a seasoned software developer to work on a project which will have an impact in the open source community. All the source code from this project will be released for free for other developers to use and adapt. There may also be ongoing paid work on this project to create updates and add features
------------------------------------------------------------------
Developer Requirements
------------------------------------------------------------------
We are looking for someone who:
* Uses Git regularly for version control
* Is familiar with command-line (CLI) tools
* Is familiar with MySQL/SQL syntax
* Knows a little bit about database migrations, or is willing to learn!
* Puts in a proper bid with solid examples of previous software work done. Generic proposals/messages will be rejected
------------------------------------------------------------------
The Problem
------------------------------------------------------------------
I've found a few solid migration tools like Flyway ([login to view URL]) and Simple DB Migrate ([login to view URL]), the latter being my preference for it's simplicity but the former having a lot more commits and contributors. However, these are just migrators and do not help produce the actual diff/migration to be versioned, which I would not like to do manually if it can be automated.
I found a diff tool by MySQL called mysqldiff ([login to view URL]), which outputs SQL for (some) schema and data changes but it shockingly doesn't produce valid SQL!!!
------------------------------------------------------------------
The Solution
------------------------------------------------------------------
I think a solid migration tool mixed with an automated schema and data diff tool would be a great contribution to the open source community and exactly what we need for our project now. Can just work for MySQL for now.
Let’s call this tool the dbdiff CLI.
------------------------------------------------------------------
Specification
------------------------------------------------------------------
In summary, we would like a tool which can:
* Work on the Linux & Mac command-line/Terminal (can be developed in your choice of programming language e.g. Node.js, Java, Python etc. and can include as many other open source plugins as you like, but cannot include any commercial plugins)
* Connect to a source and target database, or two SQL files made by mysqldump, to do the comparison diff
* Diffs can include changes to the schema, data, both in valid SQL to bring the target up-to-date with the source
* Since this diff tool is being used for migrations, it must be able to provide up and down SQL in the same file e.g. The up SQL may include adding a row of data, but the down SQL would involve deleting that same row
* Work with existing migration tools like Flyway and Simple DB Migrate by specifying output template files/formats, for example, Simple DB Migrate may work with [login to view URL] which includes: SQL_UP = u""" {{ up }} """ SQL_DOWN = u""" {{ down }} """
* Is Unicode aware, can work with UTF8 data, which includes foreign characters/symbols
* Works with just MySQL for now, but can easily be expandable to other DBs in the future
* Common tests should be included in a ‘test’ folder as the tool is being developed
* Must be developed in an open environment via a Git repository which we will provide the details to
Hi,
I am professional in this task.I would like to assist you in your projects. Get it done professionally get it done right the 1st time. I am here to make long term relationship. I would like to get all details from you and ready to start ASAP . Lets discuss.
Regards
Hello,
I have done such a project but for PostgreSQL and Interbase databases.
It was written in Java and Groovy and used to migrate 10 GB
database with real production data. It was commercial project
and I can not provide you its code. For MySQL it should be a bit easier to
develop such a tool but still it is challenging.
It is challenging enough just for schema, but for schema and
data it is really challenging. Moreover the larger database the more time
the comparison itself will be consuming - in my case it took 3 days.
(i.e. there is just one table with 20 000 000 rows and 30 columns and no changes
- one needs roughly 20 000 000 * 29 "manual" comparisons - if we leave out primary keys).
Requirement that tool is to work with SQL dump files adds more complexity
because parsing of the file must be implemented additionally.
Regards,
Michał Olek