News Updates - DSG Updates
Global leading provider of data management software and unified application development solutions, dedicated to serving enterprise-level large data centers.
From TiDB to Oracle: DSG helped the leading enterprise in the insurance industry, Non-Car Policy Center, achieve a full-stack digital transformation.
发布时间：2023.12.04

Recently, a leading domestic insurance company in the industry has undergone digital transformation, introducing the domestically developed distributed database TiDB to replace the original Oracle database. With the assistance of DSG SuperSync from DSI, the company has been empowered to provide a full-stack innovative solution for switching and returning to the non-vehicle policy center.


A leading company in the domestic insurance industry, a Fortune Global 500 company, a World Brand 500 company, with a long history, strong strength, and professional competitive advantages, as well as a world-renowned brand, occupies the position of the leader in the domestic insurance market.

In recent years, with the rapid development of the insurance industry, the core business systems of the insurance group are facing increasing pressure. In order to improve the system's processing capabilities, reduce software and hardware costs, and increase the autonomy and controllability of the information system, the insurance company has started its digital transformation by introducing the domestically developed distributed database, TiDB, to replace the original Oracle database.


The Policy Center, as the core business system of the insurance company, bears a large volume of policy management and premium income each year, covering a wide range of products including car insurance and non-car insurance (including agricultural insurance). It handles insurance, underwriting, payment, endorsement, and inquiries for hundreds of products across nine major product lines.

Non-car policy center accounts for 40% of the overall business volume of the Policy Center. In order to achieve key technological independence, the insurance company decided to fully localize the underlying software and hardware of the non-car policy center. After a period of effort, the non-car policy center successfully completed a full-stack localization on October 29, 2023. This switch not only eliminated the reliance of the insurance company's core system on foreign software and hardware, but also laid an important foundation for the digital transformation of the insurance company.


Effect of construction


  • Improve high load processing capacity: Using distributed databases to solve single machine bottleneck and improve processing performance under high concurrency and large data volume pressure.

  • Achieve resource elasticity expansion and contraction: Using distributed databases, agile expansion capability improves delivery and operation efficiency, achieves online elastic resource scaling, and meets the rapid development needs of the business.

  • Achieve cost reduction of software and hardware: Server and software are fully localized, meeting business performance and stability requirements through adaptation, transformation, and performance tuning, and reducing software and hardware costs.


In order to ensure the successful switch to the full-stack trust creation center for non-car insurance policies, the insurance group has adopted a plan of one-time switch and dual-track operation. TiDB serves as the main database carrying the full traffic, while also synchronizing in near real-time with Oracle. This ensures that in the event of serious performance issues or functionality problems caused by data consistency, the policy issuing and claims business can be rolled back from TiDB to the Oracle production library through emergency fallback.

During the implementation process, in order to ensure the continuity and security of business data, the insurance company used DSG SuperSync, a real-time synchronization tool provided by DSG, to efficiently and with low latency, and smoothly synchronize TiDB data to the Oracle database in real-time, ensuring no performance impact on the downstream Oracle database.

At the same time, the DSG DMP disaster recovery monitoring management platform realizes real-time monitoring of the disaster recovery link, comprehensively monitoring the entire data synchronization process, and promptly sending out abnormal alarms through various means such as SMS, email, and custom interfaces, ensuring the controllability and visibility of the data link.



Use domestic software


- TiDB TiDB is an open-source distributed relational database designed and developed independently by PingCAP. It is a fusion-type distributed database product that supports both online transaction processing and online analytical processing (Hybrid Transactional and Analytical Processing, HTAP). It has important features such as horizontal scaling or scaling down, financial-grade high availability, real-time HTAP, cloud-native distributed database, compatibility with MySQL 5.7 protocol, and MySQL ecosystem.

- TiCDC TiCDC is an incremental data synchronization tool for TiDB, which pulls data change logs from the upstream TiKV and parses the data into ordered, row-level change data output to the downstream.

- DSG SuperSync It is a large-scale database high-performance data replication product that integrates multiple core technologies, has been developed and polished for more than a decade, and has independent intellectual property rights. Based on a completely logical replication method and using log analysis technology, it generates a unique data loading format for DSG, achieving business initial full synchronization and real-time incremental synchronization. It has the characteristics of multi-source heterogeneity, real-time replication, small resource occupancy, flexible deployment, and easy operation. It is suitable for the vast majority of databases in the market, such as mainstream foreign commercial edition databases, domestic databases, open source databases, and various types of files. The target end supports big data components, distributed databases, data warehouses, and data middle platforms. It comprehensively adapts to customers' complex environments, supporting UNIX, Linux, Windows, AS400, virtualization, and cloud, and more.


- DSG DMP The Disaster Recovery Monitoring Management Platform is a high-performance, large-scale database monitoring platform tool developed by DSG with independent intellectual property rights. It achieves comprehensive monitoring of data synchronization, data change time, data change magnitude, software running status, software resource utilization, host hardware status, and network environment. Main application scenarios include disaster recovery monitoring, disaster recovery operations and maintenance, and disaster recovery statistics. It features graphical monitoring, disaster recovery queue configuration, real-time system detection, automatic alarm via SMS and email, disaster recovery parameter control, operations and maintenance collaboration management, data report statistics, log monitoring and review, autonomous allocation of permissions, and configuration of functional modules.


Architectural deployment





Deployment Instructions


- DSG SuperSync+DMP deployed on the middle server, the three middle machines have resources of 128 cores and 512 GB of memory, with 5 1.6T NVMe disks.

- The real-time updated incremental data in TiDB database is exported to json files by Ticdc. DSG SuperSynnc analyzes the json files exported by Ticdc, converts and generates unique data flow format, and writes it into the Oracle database.


- Three DSG rollback channels are deployed according to the requirements, synchronizing a total of 890 tables, including 789 + 42 + 159.

 Collection requirements


  • The data synchronization has the ability to ensure continuity without the risk of data loss;
  • The delay during normal trading hours shall not exceed 1 minute, and during high trading volume periods, data delay shall not exceed 2 hours;
  • Ensure that data synchronization for the current day is completed by midnight the next day;
  • Real-time monitoring of the synchronization program to provide instant feedback in case of errors.

Performance indicators


  • During each working day, the daily incremental data for each channel reaches around 50 million records, while on non-working days the transaction volume is around 20 million records, with the majority of transaction data being concentrated in the morning at 10 o'clock and during the period of new business launches;

  • During normal business hours during the day, the delay is within 20 seconds, with no backlog of data files.


Challenges and Innovations in the Project


  1. Analyze data from json files and real-time import it into the Oracle database.

  2. Support ddl in the TiDB library and real-time import it into the Oracle database.

  3. Support full and incremental data comparison from TiDB to Oracle.

  4. Provide detailed parsing of specified json files and data repair.

  5. Use TiDB to Oracle synchronization alerts to accurately locate data redundancy caused by business logic conflicts, assist in troubleshooting issues, and optimize logic.


In this digital transformation project, as a partner, DSG successfully achieved the goal of synchronizing data from TiDB to Oracle, providing strong support for the insurance company to achieve independent and controllable key technology, and comprehensively promoting the digital transformation of the enterprise.

The success of the project also marks the further development of DSG in the field of data synchronization, especially in meeting the various data replication flow requirements of domestic innovative databases as production ends has made important breakthroughs! In the future, DSG will continue to focus on independent research and development, technological innovation, and digital transformation, continuously empowering and providing excellent technical support and services for clients to achieve full-stack innovative digitalization.

热门动态
警告弹窗