PASS Summit 2019, Seattle, WA — DB Best released a multi-purpose Migration Platform, introducing full automation for all steps of a database upgrade or migration project. DB Best cre...
Our migration team has successfully completed many database migration projects. Database migration has been DB Best’s core expertise for many years. And by working on so many migration projects, we have established a proven 12-step methodology that always leads to a successful migration. Further, our unique Migration Platform automates the migration of both OLTP and data warehouse workloads between different database platforms.
Develop, Migrate, and Test Your Data-Driven Applications
DB Best Migration Platform assists in automating and navigating through these 12 steps of our migration methodology. What’s more, we have integrated all our accumulated migration knowledge and experience in this product. As a result, the Migration Platform ensures that you stay on the right course, stay within the required project path, and follow best practices.
Migration Platform integrates the best parts of industry-leading conversion applications, our own products, and internal tools to dramatically reduce the costs and risks associated with database migration projects.
DB Best Migration Platform ensures accurate migration of your databases to new platforms, so your applications work with minimal changes and better reliability after the migration. Be sure to use the DB Best Migration Platform to move with confidence your most complex or outdated apps into new data platforms, either on-premises or in the cloud.
Brands that trust Migration Platform
Typical database migration steps
DB Best’s unique multi-purpose Migration Platform automates almost everything you need to do in a database migration or upgrade project including:
- Creation of assessment reports to estimate the complexity of your database modernization project
- Automation of database schema and code conversion to provide users with explicit information on how they can address conversion action items
- Conversion of SQL statements embedded into your applications or scripts
- Migration data to take advantage of automated data extraction and the parallel loading of terabytes of data
- Validating that all your data migrated from the source database to the target
- Creation of test cases and running these test cases against the source and target databases to compare the results of the queries’ execution
- Automation of CI/CD processes to minimize downtime for your production system
Still not convinced? Learn more about the key database migration steps and see how our Migration Platform helps you follow the proven process.
Automated solution for key migration steps
Schema Conversion and Comparison
What are the database schema conversion best practices?
The migration of any database system is a complicated process and usually starts with the schema conversion. This includes fundamental aspects such as:
Afterward, you need to convert data objects including tables, constraints, indexes, sequences, and user-defined types. Finally, the schema conversion step ends with the conversion of views, triggers, packages, procedures, and functions.
Migration Platform plays an important role in these steps as the application guides you through the process in order. In addition, Migration Platform helps you take advantage of industry-leading conversion automation tools. And the platform also integrates with Microsoft SQL Server Migration Assistant and AWS Schema Conversion Tool to ensure the highest possible quality of database schema conversion.
You can use the Migration Platform to convert schema and then check a list of conversion action items. You will need to address these issues manually to follow the 12-step process.
After completing the conversion step, you need to compare the source and target database schemas. This is also an important step for database upgrade projects as you need to discover the possible database schema drift and address it if needed. You can export the results of the schema comparison to the report.
Types and Objects Mapping
Why is manual mapping so important?
Mapping is an auxiliary module in the Migration Platform application. It is used in core operations like schema comparison, data migration, data comparison, and the test organizer.
Migration Platform automatically maps the data types and objects in your source and target database schemas. However, in some cases you may need to manually map the data types and objects. Frequently, this is the case in heterogeneous database migrations when the source and target database platforms are different.
Migration Platform includes a set of predefined mapping rules. However, you can create your own mapping rules manually if you wish. You can use the following mapping options:
- Object mapping allows you to map objects in the source and target databases. You can execute the major Migration Platform operations only against a pair of objects. So, you would need to specify the mapping rules to map objects with different names. You can even create rules for the objects from different categories (like tables and views).
- Data type mapping allows you to match different data types for various database platforms. For example, you may want to map the Oracle number data type with the numeric data type in SQL Server.
How can Migration Platform deal with big data assets fast and reliably?
The data migration operation moves data between source and target databases. The important thing is that Migration Platform analyzes the size of the tables you need to migrate. Based on this analysis, the Migration Platform determines the right way to move data quickly and correctly using parallel processes.
You can only proceed to data migration after you make sure that all objects were converted correctly. The following screenshot describes the availabe data migration templates.
DB Best Migration Platform supports the following types of data migration:
- The virtual partitioning option migrates data by splitting data in the source table using the Virtual Partitions filter. In general, it generates WHERE clauses for the source tables split the migrated data.
- The agent option uses a Windows web service that communicates with Migration Platform using REST requests.
You can use several agents to significantly reduce the time of data migration. In this case, Migration Platform distributes data load between several machines.
- The staging tables option allows for migrating data to temporary tables. After completing the data transfer, Data Migrator moves data to the user tables and removes the temporary tables.
Also, you can combine Virtual partitioning and Agents to get a significant performance boost during data migration. Virtual partitioning pre-evaluates the table and determines the optimal number of virtual partitions and creates the necessary migration tasks.
The screenshot above shows the results of data migration. You can see that data migrated successfully.
How can I use Migration Platform to validate data after migration or upgrade operations?
Validating data is an important step in any migration or upgrade project. You always want to avoid data loss and ensure that your data migrated correctly. Migration Platform has a built-in module to compare data in the source and target databases. Here you can see tables with equal data as well as tables with comparison errors.
Migration Platform also integrates our proprietary technology developed in Database Compare Suite. So, you can take advantage of proven and industry-leading techniques.
Migration Platform supports the following data comparison options:
- Detailed data comparison allows for discovering the exact data differences in your database tables or views
- Fast data comparison helps quickly verify whether huge tables (i.e. with millions of rows) are equal or not
You can explore data differences in the pop-up window and then address them using Migration Platform.
However, data validation is not the only required step to confirm whether your new database system works the same as your source system. To answer this general question, you need a comprehensive testing approach. That’s what we call the Test Organizer.
How do I verify that your new database system works as before?
The Test Organizer module is a unique and unrivaled feature of Migration Platform. It provides users with an easy way to create tests for stored procedures, functions, views, tables, and ad-hoc queries. Moreover, Test Organizer can generate tests automatically or manually for the above-mentioned database objects. You can also use the Test Organizer module to test breaking changes and new features during a database upgrade.
Test Organizer allows for the running of test cases against both source and target databases. This helps ensure that your new database system works as well as the source system. You need to check the input and the output of your new application and it should get the same data from the new database as the old one and return exactly the same result.
You can even create a group of tests and specify the order of their execution. Finally, you can overview the results of test execution and clearly see the test cases that ended with errors.
Test Organizer includes the following features:
- A fully-qualified object names option allows for generating tests out of the context of any specific database. By default, this option is set to true. In this case, the object names include the names of the top-level database objects (database name or database schema). Use caution as these tests will not work on other database environments and you will need to update the embedded SQL code.
- The execution context option allows for specifying the context of the test execution. If you don’t use the default option (i.e. the name of the top-level database objects), you need to specify the execution context.
- The tests auto-generation option provides users with automatically generated tests for all object categories using predefined patterns. Consider using the advanced tests auto-generation option to expand the test coverage.
- Trace log manager generates tests from the application trace log that you get on the production environment. Consider using this option after completing the unit testing before moving to the integration testing stage.
- The associated and affected objects option lets automatically load all tables, which can be affected by the test operations. Use this option if your tests execute INSERT, UPDATE, DELETE, and other data operations. Also, consider using the export affected data objects to CSV option to include all affected data objects into a report.
- Advanced comparison options allow us to set up data comparison options at the group or test level.
- The run scenario option allows for defining the test execution order, ignoring tests while executing a group of tests, and defining whether tests should run in-parallel or sequentially in the same session or different sessions.
At the end of the day, you may want to see the summary of tests execution. You can review the statistics at our Report Portal.
After validating the migration, you may need to check that the performance of your new database system is greater than or equal to the benchmark performance levels of your source database system. However, this task goes beyond the Migration Platform scope of features as you will need third-party benchmark tools. All that Migration Platform can do is provide you with a golden copy of your source data in the Data Store module.
How can I be sure that my source data is safe?
At the start of the project, you need to create a golden copy of the source data.
The Data Store preserves this golden copy of data for the whole database or for specified tables. You can then use the Data Restore option to restore the whole golden copy or only the selected tables when needed. This need regularly occurs during the testing phase.
Data backup allows for preserving data in the database using auxiliary tables for storing data.
How can Migration Platform automate DevOps routine operations?
Migration Platform has an intuitive graphic user interface. In addition, it includes CLI versions. So, you can utilize a command-line interface to automate the execution of all the Migration Platform operations.
This may come in handy for routine DevOps operations. For example, you can fully automate data verification when moving between dev/test/stage/production environments.
Migration Platform Use Cases
Take a look at the key scenarios and typical problems that this product can help you handle.
|Scenarios||Supported platforms||Challenges||Solutions using DB Best Migration Platform product features|
|Ongoing Database Development||Your test code coverage of database code is very low. The database code has complex logic, which is not covered by unit tests. It handles mission-critical code, but everyone is scared to touch it. It must be refactored but needs robust testing tools.||Use the Test Organizer feature to create unit tests around every procedure, function or view. Use the Data Store feature to create data sets and database snapshots (a golden copy) for further reuse. Run the tests against the “old” and “new” (refactored) database to ensure they are functionally equivalent after refactoring.|
|Database Upgrade||An upgrade path from an old database version (like SQL Server 2008) is breaking the code. You need to change the SQL code and test it. You need to find breaking changes, fix them and test quickly at the database level, without touching applications.||Use the Schema Conversion and Schema Comparison features to see differences. Use Test Organizer to build tests around changed objects. Validate the tests against “old” and “new” databases.|
Migrate very large data volumes within a very narrow downtime.
|The migration of tens of terabytes of data takes multiple days. Any error in the process requires a full restart. You need to design the process, which is robust, correct and resilient to failures to prevent a full migration restart.||Use the Data Migration feature to design your own scenarios that scale vertically or horizontally: move large tables by using virtual partitioning or run multiple agents to extract and migrate data in parallel. Use staging tables, if necessary. Control all steps on the way.|
|Database Migration and Unification||You are about to start a complex migration project that requires all steps of the migration: schema and data migration, schema and data validation, testing and deployment. Are you looking for a single set of tools to work with across all steps of the journey?||Use all features of DBMP, which is designed exactly for that purpose! Read our blog posts that covers a typical migration project and shows how DB Best Migration Platform helps automate this process. Use the Schema Conversion and Schema Comparison modules to move database schema. Use the Data Migration and Data Comparison features to migrate data and validate the migration. Use Test Organizer to design tests which will validate the functional correctness of the migrated code. Use Object Mapping to define underlying mappings between source and target databases for all DBMP features. Design Automation scenarios using Command-Line Interface (CLI) to integrate with CI/CD or DevOps scenarios.|
|Move to the Cloud|
Migrate your datastore to the cloud while staying with the same database engine.
|The database in the cloud has a limited feature set and can’t be migrated as-is. You need to change the application code. It takes too much time to manually go through all the breaking changes. You need faster tools to finish the project.||Use Schema Conversion to move your database schema to the cloud. It will run SQL Server Migration (SSMA) or AWS Schema Conversion Tool (SCT) conversion engines to handle the differences. Use Test Organizer to build tests around changed objects. Validate the tests against “old” and “new” databases.|
Supported database platforms
Migration Platform supports the following database migration projects:
- Oracle Database to Microsoft SQL Server
- Oracle Database to Azure SQL Database
- SAP IQ (Sybase IQ) to Microsoft SQL Server
- Microsoft SQL Server to Azure SQL Database
- Microsoft SQL Server to Amazon RDS for SQL Server
- Oracle Database to PostgreSQL
- Oracle Database to Amazon Redshift
- Oracle Database to Amazon RDS for Oracle
- PostgreSQL to Amazon RDS for PostgreSQL
In addition, you can use Migration Platform for upgrade projects because you should approach these projects in the same way as migrations. Consider using Migration Platform to upgrade the following databases:
- Microsoft SQL Server 2000, 2005, 2008, and 2012
- Oracle Database
- PostgreSQL 9, 10, and 11
- Windows Server 2008; Windows Server 2012; Windows 7 or higher
- .NET Framework 4.7
- ODAC (to interact with Oracle Database)
- SAP SQL ODBC 16.0 (to interact with SybaseIQ and SybaseASE, installs with SybaseIQ Studio or SybaseASE Studio)
- Amazon Redshift x64 (to interact with Amazon Redshift)
We have some great blog posts covering our prolific database migration projects. Learn how we leveraged various automated tools to perform awesome database migrations.
Database upgrade is inevitable if you're running aging versions of your databases. Alongside with cool new features, the database upgrade usually brings some code changes. Regardless o...
‘Low-performing and costly’ — these were major characteristics of the Oracle system of one of our customers, a leading US automotive corporations. In this regard, it was quite na...
One of our customers decided to modernize their Oracle database to Microsoft SQL Server. However, the thousands of SAP Crystal Reports connected to their existing Oracle database prese...