Database Administrators Stack Exchange
Database Administrators Stack Exchange

Questions tagged [amazon-dms]

AWS Database Migration Service (AWS DMS)

AWS DMS is a web service that you can use to migrate data from a source database to a target database. To work with AWS DMS, one of your databases must be on an AWS service. You can't migrate from an on-premises database to another on-premises database.

31 questions
7
votes
2 answers

AWS DMS creates tables with no auto_increment

I am migrating my databases to Amazon's RDS using their DMS tool. Pretty much everything looks great except that columns that were marked as auto_increment on the source database did not get marked as auto_increment on the target database. I also…
InfernoTK
  • 103
  • 1
  • 7
6
votes
1 answer

Spike in AWS RDS Oldest Replication Slot lag

Does anyone know about this "Oldest Replication Slot Lag" in AWS RDS? What does it represent? I have a PostgreSQL instance on RDS (with Logical Replication switched on) for transferring data from PostgreSQL DB to S3 bucket (in CSV form) using AWS…
Zinx
  • 319
  • 2
  • 10
4
votes
1 answer

Create event trigger as non-superuser in Postgres

Is it possible to give a role or a set of attributes to a non-superuser in postgres that would allow the user to create an EVENT TRIGGER? Simple example of what (obviously) doesn't work in psql: => CREATE OR REPLACE FUNCTION do_nothing() -> …
matt
  • 141
  • 1
  • 2
3
votes
1 answer

Getting Error 1236 (Could not find first log file name in binary log index file) reading binlog in AWS DMS

I have created two DMS task (full load + CDC) with RDS MYSQL 'Read' replica as source endpoint and REDSHIFT as target endpoint. For both the task source and target endpoint is same. One task with only 3 tables is running fine while another task with…
Sumant Kumar
  • 83
  • 7
3
votes
0 answers

Postgresql v10.1 - How to fix ERROR: duplicate key value violates unique constraint

I'm not sure how this happened to this table but have an issue with ERROR: duplicate key value violates unique constraint trying to use AWS's DMS service to move a postgres v10.1 DB to RDS v10 The table looks like # \d+ public.brokered_carriers …
Mike
  • 185
  • 6
3
votes
1 answer

Do not replicate Delete operations AWS DMS

I am using AWS DMS to replicate ongoing changes with SQL Server as a source and target endpoint. The tasks are running and replicating data with low latency. However, I need the tasks configured not to replicate DELETE operations from the source…
tOpsDvp
  • 41
  • 1
  • 5
3
votes
2 answers

Failure with ongoing replication from SQL Server RDS using AWS DMS

I'm trying to replicate from an SQL Server RDS database using DMS, and running into problems. I am able to load all data from one table into the destination (~1M rows) as a single 'Migrate existing data' Task. When I run a second 'Replicate data…
Greg Sansom
  • 151
  • 1
  • 7
3
votes
1 answer

AWS DMS endpoint test connection error

I am trying to migrate Microsoft SQL Server databases from on-premise to an RDS instance. I have created a SQL Server 2014 Express Edition instance in RDS and I have SQL Server 2014 Express Edition on premise in in my corporate network. I am able to…
mohan sahu
  • 31
  • 1
  • 3
2
votes
1 answer

AWS DMS - How to select different databases for source and target points for a task

Does aws DMS(Data Migration service) allow to select different databases for source and target points for a task in a replication instance? My source engine is MySQL. example: I want to move table A from database D1 to D2. but i am not able to do…
HiteshRathi
  • 49
  • 8
2
votes
1 answer

Replication from Amazon's RDS to Postgres via DMS errors out

I have a transactional replication from SQL Server 2008 to RDS SQL Server 2016 in 2014 compatibility mode. This works okay. I now want to replicate this RDS database via DMS to Postgresql. This errors prematurely, the Cloudwatch entries: Database…
Willem
  • 66
  • 3
1
vote
0 answers

How to troubleshoot logical replication on Postgresql, when it comes to high replication lag?

I have a use case in which we use AWS DMS to stream data to our data lakes (S3), from time to time the replication lag gets out of control, leading to over 500Gb in replication slot lag. It was also observed that when it happens, the wal_sender…
F S
  • 11
  • 2
1
vote
1 answer

Time zone conversion in DMS from DB2 to Aurora Postgres

Challenge We are using AWS DMS to convert data from IBM DB2 into AWS Auroa Postgres IBM DB2 server is set to Eastern time (ET) zone. Timestamp related data inside DB2 is therefore in ET. Findings AWS DMS basically functions by making a connection to…
Padraic Renaghan
  • 73
  • 1
  • 6
1
vote
0 answers

aws documentation for DMS CDC is confusing

I am following this documentation https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.Security to enable CDC using logical replication. I have pglogical extension installed and added under…
Falcon
  • 101
  • 8
1
vote
1 answer

What does MISSING_TARGET means on Amazon DMS (Database Migration Service) validation?

I'm running Amazon DMS from RDS to Aurora, the error when full restarting (never been successful before) from awsdms_validation_failures_v1 tables are: [ { "TASK_NAME": "TPPHLQOQH3WTSU27DVRVWS2BAXTQ7E7DYITWJCQ", "TABLE_OWNER": "public", …
Kokizzu
  • 1,403
  • 6
  • 18
  • 35
1
vote
2 answers

How does AWS DMS on-going replication works internally?

From the documentation it's mentioned that "DMS collects changes to the database logs using the database engine's native API" and replicates to target. But, I didn't see anywhere that, at what rate it replicates from source to target in terms of…
Pand005
  • 151
  • 1
  • 2
  • 6
1
2 3