It seems to me that data migration remains a challenge partly due to edge cases that are difficult to handle without tight integration between programming language and datastore.
For example:
- Some migration operations may involve complex logic and potentially queries against other objects
- Migration implementations for a given object type have to be available at runtime until all objects of that type have been upgraded
- For large datasets, it may be desirable to control the rate at which migrations are applied, and potentially to allow rollback in the case of unanticipated load or failures
Most approaches I've seen attempt to solve the problem via revision-controlled scripts, and rely on careful co-ordination of development and deployment.
Can anyone share examples of programming languages that have attempted to add 'native' support for data migrations?
[1] - https://alembic.sqlalchemy.org/
[2] - https://github.com/SRoddis/Mongo.Migration