News Updates - DSG Updates
Global leading provider of data management software and unified application development solutions, dedicated to serving enterprise-level large data centers.
From Oracle to OceanBase: DSG helps Hebei Mobile CRM system move towards domestic replacement.
发布时间：2023.12.11

Recently, DSG company used its rich experience and technical strength to successfully assist Hebei Mobile in migrating their database from Oracle to OceanBase using a bidirectional reflux strategy, achieving localization and ensuring data consistency and security.


01


Background and Explanation of Situation for the Project


In recent years, with the continuous development of domestic database technology, more and more enterprises have begun to focus on the application of domestic databases. Hebei Mobile, as a leading domestic communication operator, is actively promoting the domestic replacement of the CRM system.

Hebei Mobile's original business F database was based on the Oracle 19C database, with a data volume of about 15T and a daily incremental data of about 300G. In order to fundamentally ensure the security of enterprise data and achieve independent and controllable core technology, Hebei Mobile decided to migrate the business F database to the domestic OceanBase database. In this project, DSG, as a senior partner of mobile operators, is responsible for implementing the migration of this data.

Explanation of the situation on the ground

  • The source Oracle 19C database uses a Western character set, while the target OceanBase database uses the utf8 character set;
  • The source is synchronized through ADG, and the synchronization/comparison programs are deployed on the intermediate server;
  • There are 20 primary migration links and 7 backup links, which are split based on partition tables, HOT tables, LOB tables, corrupted tables, and tables that require manual modification using pkuk_uidx_flag;
  • Regularly comparing the consistency of full data migration on the primary real-time migration links.


Synchronization requirement

  • Synchronize 16,410 tables, with 8,033 tables containing primary key (PK) and 8,377 tables without primary key. The project requires full synchronization to be completed within 4 hours, with incremental data generated during the process needing to be caught up within 30 minutes.
  • Conduct a full database content audit within 2 hours through ADG shutdown inspection.
  • During normal business hours, latency is between 0-5 seconds. Batch processing in the evening takes approximately 1.5-3 hours, with around 1 billion DML operations per hour and latency not exceeding 10 minutes per hour.


Migration characteristics

  • There are a large number of tables without primary keys in the Oracle database.
  • Support the unique Western European character set (GB18030) of Hebei Mobile business database to transcode to utf8, ensuring data quality.
  • Output the unconvertible garbled characters through logs to assist businesses in transforming garbled data.


02


Project deployment architecture




Project architecture diagram


The project adopts a bidirectional backflow solution and a middle machine deployment mode. DSG's independently developed domestic large-scale database high-performance replication tool DSG SuperSync is responsible for building the link, realizing the full + incremental data migration from Oracle 19c to OceanBase database, and the incremental data migration from OceanBase to Oracle 19c database. While completing the domestic substitution, the backflow link is also ensured to cope with the possibility of business back-switching in case of emergency.

To ensure data consistency, the project also uses the DSG XCMP data comparison platform to achieve rare data comparison of the Oracle 19c Western character set (WE8ISO8859P1), cross-character set comparison, and comparison of multiple data types, ensuring the validity and integrity of the data.



03


Performance and Challenges of the Project



In this project, the amount of data is large, the data types are complex, and there are high requirements for data consistency and availability. The entire DSG team has made tremendous efforts, overcome various challenges, and ultimately successfully completed this domestic substitution project.

Performance indicators

  • On the night of the cutover, over 600 core system tables underwent data comparison, involving over 20 billion records, which took 50 minutes.
  • Concurrent sorting of billions of records in a single table took 2045 seconds for data comparison.
  • Data comparison for over 10,000 tables in the entire database, involving over 1 terabyte of LOB data and over 40 billion records, took 2 hours.

 

Challenges and Innovations in the Project

  • Resolve the issue of OB non-rowid data type, where Oracle's rowid cannot be inserted into urowid.
  • Automatically correct negative values in the source Oracle storage date.
  • When exporting a large number of empty tables from the source, the speed was too slow. After modifying the configuration, exporting nearly ten thousand tables took only three minutes.
  • Ensure data uniqueness and simplify data verification configuration.
  • Improve sorting efficiency when comparing billions of records in a single table.


04


Conclusion



In the project of localization and replacement of CRM system for Hebei Mobile, DSG  fully leveraged its technical advantages in database migration, providing Hebei Mobile with an efficient, stable, and secure migration solution.

The successful implementation of this project not only enhanced the autonomy and controllability of the Hebei Mobile CRM system, but also reduced operational costs, while also providing a successful case of domestic database migration for other domestic enterprises. In the future, Disjie Company will continue to provide customers with efficient and stable database solutions, making a positive contribution to the promotion of domestic database replacement!

热门动态
警告弹窗