Database Administrators Stack Exchange
Database Administrators Stack Exchange

Questions tagged [bulkcopy]

44 questions
18
votes
1 answer

Usage of COPY FROM STDIN in postgres

I just started using Postgres and I'm trying to create a sample db to understand its functions, looking around, I found some scripts in pgfoundry.org. I understand the commands, since I previously used both Oracle and MS-SQL, but all the scripts I'm…
Eugenio Laghi
  • 283
  • 1
  • 2
  • 6
11
votes
4 answers

ORA-01502: index or partition of such index is in usable state problem

I have a table in my Oracle database, where select pkcol, count(*) from myTable group by pkcol having count(*) > 1; yields PKCOL COUNT(*) ------- ---------- 1 2 2 2 Trying to remove the duplicate rows delete…
bernd_k
  • 12,369
  • 24
  • 79
  • 111
10
votes
2 answers

Variable substitution in psql when using \copy

I'm using the psql command \copy and I would like to pass a variable to it from the shell (for table name) like I've done when scripting queries. I've read in the documentation for psql that: The syntax of the command is similar to that of the SQL…
beacon_bonanza
  • 103
  • 1
  • 4
9
votes
5 answers

Copying a table (and all of its data) from one server to another?

I have a massive table, let's say 500,000 rows. I want to copy it (schema and data) from one server to another. This is not an upsert or any kind of update; It's a one-off straight copy and paste. What are the idiomatic approaches to this? I've…
J. Mini
  • 1,161
  • 8
  • 32
7
votes
2 answers

BCP stops after the last "1000 rows sent to SQL Server. Total sent: ..."

I ran SQL Server 2005's bulk copy program BCP like this: bcp mydb.dbo.mytbl in myfile.blk -c -S mysvr -U mylogin -P mypass And it ran and produced output like so: Starting copy... 1000 rows sent to SQL Server. Total sent: 1000 ... 1000 rows sent to…
Endy Tjahjono
  • 381
  • 4
  • 17
6
votes
1 answer

Keep empty string instead of null for /copy command

When using the \copy command with psql I have a file with empty values. These values are not to be null but instead always an empty string. My command: \copy agltransact FROM 'dbo_agltransact.csv' with null as '' CSV HEADER; The error I get: ERROR:…
Asken
  • 317
  • 1
  • 2
  • 9
5
votes
1 answer

System.Data.Entity.Infrastructure.CommitFailedException: C# Multithreading & SQL Server 2012

We have a C# multi threaded (100 threads) program which reads the records from the DB and each thread picks up one record (one Entity Framework connection per thread) and update the a single DB table. For first few minutes (5 minutes) the program…
Balu
  • 71
  • 2
  • 4
5
votes
1 answer

Copy few rows from one database to another

I have two PostgreSQL databases on two different machines and the network connection is slow. I would like to copy few selected rows from the first instance to the second. Unfortunately, the full dump of single table is too big. A CLI solution would…
Michas
  • 185
  • 1
  • 7
4
votes
3 answers

SQL Server - Copy data from one database to another - identical tables, new identity values needed

Here's the thing, I have been running identical copies of my website for two different geographical regions. The site now supports multi-currency pricing so I want to merge both sites together. All tables that I need to merge are Identity based and…
Jordan
  • 41
  • 1
  • 1
  • 2
4
votes
1 answer

Postgres copy data with \xYY as plain string instead of interpreting as encoded string

I have a log file full of URLs, generated by Bro IDS. When Bro logs a URL with non-ascii characters in it, it inserts \xYY where YY is the hexadecimal character code. Also, some URLs contain "\x". Is there a setting or flag I can use with the COPY…
potatohead00
  • 43
  • 1
  • 3
4
votes
0 answers

Can PostgreSQL's COPY create CSV with Locale-specific number formatting?

I want to create a CSV with PostgreSQL 9.1 COPY that directly opens correctly with a double-click in crazy German Excel 2010. Using ";"-Delimiter is easy. The encoding is solved, but Excel only recognizes numbers correctly if they use the , as…
alfonx
  • 857
  • 1
  • 12
  • 22
4
votes
4 answers

What options exist for copying data between servers on a regular basis?

My apologies if this has already been answered. I've searched SO and of course here on DBA, and am surprised to find no close matches. Specifically, I'm looking for a solution which survives schema differences. I finally got a development server…
Jon of All Trades
  • 5,987
  • 7
  • 48
  • 63
4
votes
3 answers

Copy 8 Million Records

I have a table in a local MS SQL Server database that has 72 columns and over 8 million records. Most of the columns are floats except for the primary key which is an INT Identity column. The table also has 3 indexes. These records are the saved…
Bobby Ortiz
  • 141
  • 3
3
votes
1 answer

Diagnosing the error in a Postgresql to Redshift migration through psycopg2

I am running a job class that contains the following: A Postgresql connection that can issue SQL statements A Redshift connection that can do the same an S3 connection to function as an intermediary between the two The current process I am taking…
yburyug
  • 245
  • 1
  • 2
  • 6
3
votes
1 answer

BULK INSERT Data from file but ignore the first occurence of field terminator

I have a file that has data arranged in the following style: 1 1 0.5102 0.4898 0.0000 0.0000 0.0000 1 2 0.3114 0.6886 0.0000 0.0000 0.0000 I need to load this into a table on MS-SQL and process it. I am…
sriramn
  • 429
  • 3
  • 5
  • 13
1
2 3