AutoMQ to OceanBase
BladePipe supports data replication from AutoMQ to OceanBase. View supported migration, sync, verification, and connector capabilities.
| Function | Description |
|---|---|
Incremental Data Sync | Allow subscribing to messages from the source Topic and transforming them into DML operations, namely, INSERT, UPDATE, DELETE. |
Subscription Modification | Add, delete, or modify the subscribed topics. For more information, see Modify Subscription. |
Position Resetting | Reset positions by timestamp to consume the data in a past period again. |
Advanced Functions
| Function | Description |
|---|---|
Message Format | The following message formats are supported. See Message Format.
|
Removal of Target Data before Full Data Migration | Remove the existing data in the Target before running the Full Data Migration, applicable for DataJobs reruning and scheduled Full Data migrations. |
Recreating Target Table | Recreate target tables before running the Full Data Migration, applicable for DataJobs reruning and scheduled Full Data migrations. |
Highly Concurrent Row-Level Write | It has the distributed database characteristics. A highly concurrent row-level write strategy (ROW) is adopted by default. |
Incremental Data Write Conflict Resolution Rule | IGNORE: Ignore primary key conflicts (skip writing), REPLACE: Replace the entire row in case of primary key conflicts. |
Limits
| Limit | Description |
|---|---|
Creating Tables in the Target in Advance | Only support automatic Topic creation for messages. |
Raw Message Format | Only support raw message replication from AutoMQ to AutoMQ, and Raw Message Format needs to be selected at both the Source and the Target. |
OceanBase Version | Currently only OceanBase for MySQL is supported. |
OceanBase Data Types | Geospatial data is not supported. |
Prerequisites
| Prerequisite | Description |
|---|---|
Port Preparation | Allow the migration and sync node (Worker) to connect to the AutoMQ nodes. |
Parameters
| Parameter | Description |
|---|---|
schemaFormat | MQ Message format. For more information, see Message Format. |
consumerGroupId | AutoMQ consumer group ID. |
consumeParallel | Degree of consuming AutoMQ topics in parallel. |
sessionTimeoutMs | AutoMQ session timeout in milliseconds. |
maxPollRecords | Maximum number of messages fetched in one poll from AutoMQ. |
Tips: To modify the general parameters, see General Parameters and Functions.
Prerequisites
| Prerequisite | Description |
|---|---|
Permissions for Account | SELECT, INSERT, DELETE, UPDATE, and common DDL permissions. |
Port Preparation | Allow the migration and sync node (Worker) to connect to the OceanBase port (e.g., 3306). |
Parameters
| Parameter | Description |
|---|---|
keyConflictStrategy | Strategy for handling primary key conflicts during write in Incremental DataTask:
|
writeStrategy | Target write strategies include:
|
Tips: To modify the general parameters, see General Parameters and Functions.