Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 Problem with .dat file created via BCP

Author  Topic 

yosiasz
Master Smack Fu Yak Hacker

1635 Posts

Posted - 2010-07-20 : 17:56:45
Hi there

when doing bcp to a flat file (.dat) any PK fields become problematic when trying to BULK INSERT into a staging table. Does bcp process change the format of that field? As you may know bcp and BULK INSERT can be very sensitive. One of the many errors I am seeing is "Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 1 (DimDateKey)."

Source table is as follows (the bcp runs without error and produces a .dat file)

CREATE TABLE [dbo].[DimensionDate](
[DimDateKey] [int] NOT NULL,
[CalendarDate] [datetime] NOT NULL

CONSTRAINT [PK_Dim_Date] PRIMARY KEY CLUSTERED
(
[DimDateKey] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]


Destination table is

CREATE TABLE [dbo].[stgDimensionDate](
[DimDateKey] [int] NOT NULL) ON PRIMARY


Wondering if you have some insight on this.


Thanks!

If you don't have the passion to help people, you have no passion

webfred
Master Smack Fu Yak Hacker

8781 Posts

Posted - 2010-07-21 : 01:20:06
First you should have a look into the flatfile.


No, you're never too old to Yak'n'Roll if you're too young to die.
Go to Top of Page

yosiasz
Master Smack Fu Yak Hacker

1635 Posts

Posted - 2010-07-21 : 09:07:19
I am looking at the file and of course since it is a dat file it is not easy to make sense of it but it looks consistent. But I am sure there is something more than what meets the eye



CREATE TABLE [dbo].[DimensionDate](
[DimDateKey] [int] NOT NULL,
Sumitup bigint NOT NULL

CONSTRAINT [PK_Dim_Date] PRIMARY KEY CLUSTERED
(
[DimDateKey] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]



CREATE TABLE [dbo].[stgTestTable](
[DimDateKey] [int] NOT NULL,
Sumitup [bigint] NOT NULL
) ON [PRIMARY]

GO
declare @intLoop int = 0
WHILE @intLoop < 101
BEGIN
SET @intLoop = @intLoop + 1
INSERT INTO DimensionDate
VALUES( CONVERT(VARCHAR(8),GETDATE()+@intLoop,112), CAST(150 +@intLoop AS BIGINT))
END
GO

exec xp_cmdshell @Query=N'BCP "SELECT [DimDateKey], SUM(Sumitup) AS Sumitup FROM dbo.DimensionDate WHERE DimDateKey >= 20100701 GROUP BY dimdatekey" queryout "c:\what_is_up.dat" -N -T -S{Your_server_name} -o "c:\what_is_up.out"',@NoOutput=N'No_Output'
BULK INSERT dbo.stgTestTable FROM "c:\what_is_up.dat" WITH (DATAFILETYPE ='widenative',TABLOCK,BATCHSIZE=512000)

DROP TABLE [dbo].[DimensionDate]
DROP TABLE [dbo].[stgTestTable]



If you don't have the passion to help people, you have no passion
Go to Top of Page
   

- Advertisement -