Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2000 Forums
 SQL Server Administration (2000)
 BCP in Fixed width Format

Author  Topic 

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-11 : 12:00:43
My question is if I bcp in a fixed width file, using a format file, will char/varchar get loaded with spaces padded to the end?

I think they are...ok I know they are

How do I do bcp so it doesn't pad

Will SET ANSI_PADDING help?



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



X002548
Not Just a Number

15586 Posts

Posted - 2007-10-11 : 14:47:14
Well that didn't work


USE Tracker_Data
GO
SET ANSI_PADDING OFF
GO
USE Privacy_Tracker_Data
GO

isp_bcp_in_database
@dbName = 'Tracker_Data'
, @fp = 'D:\data\Tracker_Data\Scramble'
, @Formatpath = 'D:\data\Tracker_Data\FORMATIN'
, @RESTART = 'T00000001'


USE Tracker_Data
GO
SET ANSI_PADDING ON
GO
USE Privacy_Tracker_Data
GO



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-11 : 15:01:45
Trigger?

UPDATE U
SET MyCol = U.MyCol
FROM inserted AS I
ON dboMyTable AS U
ON U.MyPK = I.MyPK

I don't know if
MyCol = U.MyCol
will Trim, but if not you could use
MyCol = RTrim(U.MyCol)
but the trigger causes an extra Update in all instances, of course ... so something "cheaper" would probably be better

Kristen
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-11 : 16:22:06
Wrote a script to generate updates. BUT After it runs, it'll leave some columns with 1 space. These were probably null to begin with.

The Journey continues

EDIT: Modified it to add Transactions in between, and truncating the log with TRUNCATE_ONLY


SELECT SQL FROM (
SELECT DISTINCT 'PRINT ' + '''' + 'UPDATING ' + p.TABLE_NAME + ''''+CHAR(13)+CHAR(10)+'BEGIN TRAN' AS SQL
, p.TABLE_NAME, 1 AS SQLGROUP, 0 AS COLNO
FROM INFORMATION_SCHEMA.COLUMNS c
INNER JOIN Privacy_Tracker_Data..PRIVACY_COLUMN p
ON p.TABLE_CATALOG = c.TABLE_CATALOG
AND p.TABLE_SCHEMA = c.TABLE_SCHEMA
AND p.TABLE_NAME = c.TABLE_NAME
AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')
UNION ALL
SELECT DISTINCT 'UPDATE ' + p.TABLE_NAME + ' SET ' AS SQL
, p.TABLE_NAME, 1 AS SQLGROUP, 1 AS COLNO
FROM INFORMATION_SCHEMA.COLUMNS c
INNER JOIN Privacy_Tracker_Data..PRIVACY_COLUMN p
ON p.TABLE_CATALOG = c.TABLE_CATALOG
AND p.TABLE_SCHEMA = c.TABLE_SCHEMA
AND p.TABLE_NAME = c.TABLE_NAME
AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')
UNION ALL
SELECT ' ' + c.COLUMN_NAME + ' = ' + 'RTRIM('+c.COLUMN_NAME+')' AS SQL
, c.TABLE_NAME, 2 AS SQLGROUP, ORDINAL_POSITION AS COLNO
FROM INFORMATION_SCHEMA.COLUMNS c
INNER JOIN Privacy_Tracker_Data..DSINFO p
ON p.TABLESCHEMA = c.TABLE_SCHEMA
AND p.TABLENAME = c.TABLE_NAME
AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')
AND ORDINAL_POSITION = 1
UNION ALL
SELECT ', ' + c.COLUMN_NAME + ' = ' + 'RTRIM('+c.COLUMN_NAME+')' AS SQL
, c.TABLE_NAME, 3 AS SQLGROUP, ORDINAL_POSITION AS COLNO
FROM INFORMATION_SCHEMA.COLUMNS c
INNER JOIN Privacy_Tracker_Data..DSINFO p
ON p.TABLESCHEMA = c.TABLE_SCHEMA
AND p.TABLENAME = c.TABLE_NAME
AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')
AND ORDINAL_POSITION <> 1
UNION ALL
SELECT DISTINCT 'COMMIT TRAN' + CHAR(13) + CHAR(10) + 'GO'
+ CHAR(13) + CHAR(10) + 'BACKUP LOG Tracker_Data WITH TRUNCATE_ONLY'
+ CHAR(13) + CHAR(10) + 'GO'AS SQL
, p.TABLE_NAME, 4 AS SQLGROUP, 1 AS COLNO
FROM INFORMATION_SCHEMA.COLUMNS c
INNER JOIN Privacy_Tracker_Data..PRIVACY_COLUMN p
ON p.TABLE_CATALOG = c.TABLE_CATALOG
AND p.TABLE_SCHEMA = c.TABLE_SCHEMA
AND p.TABLE_NAME = c.TABLE_NAME
AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')
) AS XXX
ORDER BY TABLE_NAME, SQLGROUP, COLNO





Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-11 : 17:00:42
"After it runs, it'll leave some columns with 1 space."

Why's that then, eh?

Add a wrapper for:

UPDATE U
SET MyCol = NullIf(MyCol, '')

??

Well ...

SET MyCol = NullIf(MyCol, ' ')

in your case perhaps!

Kristen
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 08:58:48
Well, it's not null on load

And the rtrim, doesn't trim all the spaces, as it leaves one

So.......now I probably want to find the columns that have a single space and make it null

any ideas?

I'm working on it

I get no help here (at the orifice) so any help here (at sqlteam) would be great

the short answer would be if I could figure out how to bcp in a fixed width file without it loading space

i.e.

If the bcp sees a blank string, it should be left null



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

pootle_flump

1064 Posts

Posted - 2007-10-12 : 09:06:02
I bcp in fixed width all the time. This is hardly ground breaking but I just dump it all into unconstrained staging tables, cleanse, dedupe etc and load into the live tables. I imagine you have discounted something like this already though....
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-12 : 09:34:30
"And the rtrim, doesn't trim all the spaces, as it leaves one"

Have you got one of those funny spaces in there?

CHAR(160) or somesuch ...

Kristen
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 10:27:15
quote:
Originally posted by pootle_flump

I bcp in fixed width all the time. This is hardly ground breaking but I just dump it all into unconstrained staging tables, cleanse, dedupe etc and load into the live tables. I imagine you have discounted something like this already though....



Yes

This is a separate db that is meant for data privacy to be given over dev environments

It's not one db, but many

To do that would be huge

One db has over 1k tables to scramble



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-12 : 10:33:24
"scramble"

Ah, I forgot that was what you were doing.

perhaps the space has been scrambled, such that RTrim() doesn't think its a space.
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 10:44:12
OK Why do I get the following


UPDATING TRK_ASI_C_Detail

(1193 row(s) affected)

Server: Msg 3021, Level 16, State 1, Line 1
Cannot perform a backup or restore operation within a transaction.
Server: Msg 3013, Level 16, State 1, Line 1
BACKUP LOG is terminating abnormally.


FROM the following? Isn't that outside of the transaction?



PRINT 'UPDATING TRK_ASI_C_Detail'
BEGIN TRAN
UPDATE TRK_ASI_C_Detail SET
PropIDNumber = RTRIM(PropIDNumber)
, PropIDCode = RTRIM(PropIDCode)
COMMIT TRAN
GO
BACKUP LOG Tracker_Data WITH TRUNCATE_ONLY
GO


Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 10:53:21
I noticed that if I open another thread (QA Window) the Command executes no problem



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-12 : 11:13:11
Something else creating the transaction?
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 11:24:05
I'm messing something up

I'm the only one on the box

EM Is pointing at another db (i.e., the privacy one)

I ran this last night without a problem (I think)

I'm gonna close everythng down and start over



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 11:50:27
OK, There are no Miracles (at least in SQL Server)

But now it's working



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 12:34:36
OK, 30 tables took 1/2 hour to do the replace

I then shrunk the db and the files

DBCC SHRINKDATABASE(Tracker_Data,10)


USE Tracker_Data
GO

DBCC SHRINKFILE(Tracker_Data_Data)
GO

USE Privacy_Tracker_Data
GO

And it's still at 13GB, from the original restore that was 4GB


Any other ideas?

DBCC puts out

DbId FileId CurrentSize MinimumSize UsedPages EstimatedPages
------ ------ ----------- ----------- ----------- --------------
22 1 1656296 128 1656256 1656256

(1 row(s) affected)

DBCC execution completed. If DBCC printed error messages, contact your system administrator.




Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 12:45:38
I'm, gonna try to do DBCC DBREINDEX
or DEFRAG

What else would cause it to grow?


Row counts match excatly between the scrambled and pre-scrambled db's





Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-12 : 13:11:03
"What else would cause it to grow?"

That's it. The scrambled data will index differently, and has probably left the indexes shot full of holes ...

Or your junior side-kick has set the Model database to 13GB for a larf - ,maybe hes hooked it up to a date algorithm, and its ready for tomorrow already, so your run on Monday is going to be 16GB already

Kristen

Kristen
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-12 : 13:45:36
Yup, That was it

Thanks everyone



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2007-10-12 : 18:05:26
"Yup, That was it"

Wow! The Junior Sidekick really DID hook the size-of-Model database up to the day-of-the-month?

Did he use MVJ's date function?

Maybe you can sue ...
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2007-10-16 : 10:02:00
no

The indexes where a mess

In DB2, I can tell DB2's load cards to load a null's if the whole field is space

Is there anyway to that with the format file? Or a switch in bcp?



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page
    Next Page

- Advertisement -