Author |
Topic |
X002548
Not Just a Number
15586 Posts |
|
X002548
Not Just a Number
15586 Posts |
Posted - 2007-10-11 : 14:47:14
|
Well that didn't workUSE Tracker_DataGOSET ANSI_PADDING OFFGOUSE Privacy_Tracker_DataGOisp_bcp_in_database @dbName = 'Tracker_Data' , @fp = 'D:\data\Tracker_Data\Scramble' , @Formatpath = 'D:\data\Tracker_Data\FORMATIN' , @RESTART = 'T00000001'USE Tracker_DataGOSET ANSI_PADDING ONGOUSE Privacy_Tracker_DataGO Brett8-)Hint: Want your questions answered fast? Follow the direction in this linkhttp://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspxAdd yourself!http://www.frappr.com/sqlteam |
 |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-11 : 15:01:45
|
Trigger?UPDATE USET MyCol = U.MyColFROM inserted AS I ON dboMyTable AS U ON U.MyPK = I.MyPK I don't know if MyCol = U.MyColwill Trim, but if not you could useMyCol = RTrim(U.MyCol)but the trigger causes an extra Update in all instances, of course ... so something "cheaper" would probably be betterKristen |
 |
|
X002548
Not Just a Number
15586 Posts |
Posted - 2007-10-11 : 16:22:06
|
Wrote a script to generate updates. BUT After it runs, it'll leave some columns with 1 space. These were probably null to begin with.The Journey continuesEDIT: Modified it to add Transactions in between, and truncating the log with TRUNCATE_ONLYSELECT SQL FROM ( SELECT DISTINCT 'PRINT ' + '''' + 'UPDATING ' + p.TABLE_NAME + ''''+CHAR(13)+CHAR(10)+'BEGIN TRAN' AS SQL, p.TABLE_NAME, 1 AS SQLGROUP, 0 AS COLNO FROM INFORMATION_SCHEMA.COLUMNS c INNER JOIN Privacy_Tracker_Data..PRIVACY_COLUMN p ON p.TABLE_CATALOG = c.TABLE_CATALOG AND p.TABLE_SCHEMA = c.TABLE_SCHEMA AND p.TABLE_NAME = c.TABLE_NAME AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')UNION ALL SELECT DISTINCT 'UPDATE ' + p.TABLE_NAME + ' SET ' AS SQL, p.TABLE_NAME, 1 AS SQLGROUP, 1 AS COLNO FROM INFORMATION_SCHEMA.COLUMNS c INNER JOIN Privacy_Tracker_Data..PRIVACY_COLUMN p ON p.TABLE_CATALOG = c.TABLE_CATALOG AND p.TABLE_SCHEMA = c.TABLE_SCHEMA AND p.TABLE_NAME = c.TABLE_NAME AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')UNION ALL SELECT ' ' + c.COLUMN_NAME + ' = ' + 'RTRIM('+c.COLUMN_NAME+')' AS SQL, c.TABLE_NAME, 2 AS SQLGROUP, ORDINAL_POSITION AS COLNO FROM INFORMATION_SCHEMA.COLUMNS c INNER JOIN Privacy_Tracker_Data..DSINFO p ON p.TABLESCHEMA = c.TABLE_SCHEMA AND p.TABLENAME = c.TABLE_NAME AND DATA_TYPE IN ('nchar','nvarchar','char','varchar') AND ORDINAL_POSITION = 1UNION ALL SELECT ', ' + c.COLUMN_NAME + ' = ' + 'RTRIM('+c.COLUMN_NAME+')' AS SQL, c.TABLE_NAME, 3 AS SQLGROUP, ORDINAL_POSITION AS COLNO FROM INFORMATION_SCHEMA.COLUMNS c INNER JOIN Privacy_Tracker_Data..DSINFO p ON p.TABLESCHEMA = c.TABLE_SCHEMA AND p.TABLENAME = c.TABLE_NAME AND DATA_TYPE IN ('nchar','nvarchar','char','varchar') AND ORDINAL_POSITION <> 1UNION ALL SELECT DISTINCT 'COMMIT TRAN' + CHAR(13) + CHAR(10) + 'GO' + CHAR(13) + CHAR(10) + 'BACKUP LOG Tracker_Data WITH TRUNCATE_ONLY' + CHAR(13) + CHAR(10) + 'GO'AS SQL, p.TABLE_NAME, 4 AS SQLGROUP, 1 AS COLNO FROM INFORMATION_SCHEMA.COLUMNS c INNER JOIN Privacy_Tracker_Data..PRIVACY_COLUMN p ON p.TABLE_CATALOG = c.TABLE_CATALOG AND p.TABLE_SCHEMA = c.TABLE_SCHEMA AND p.TABLE_NAME = c.TABLE_NAME AND DATA_TYPE IN ('nchar','nvarchar','char','varchar')) AS XXXORDER BY TABLE_NAME, SQLGROUP, COLNO Brett8-)Hint: Want your questions answered fast? Follow the direction in this linkhttp://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspxAdd yourself!http://www.frappr.com/sqlteam |
 |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-11 : 17:00:42
|
"After it runs, it'll leave some columns with 1 space."Why's that then, eh?Add a wrapper for:UPDATE USET MyCol = NullIf(MyCol, '')??Well ... SET MyCol = NullIf(MyCol, ' ') in your case perhaps!Kristen |
 |
|
X002548
Not Just a Number
15586 Posts |
Posted - 2007-10-12 : 08:58:48
|
Well, it's not null on loadAnd the rtrim, doesn't trim all the spaces, as it leaves oneSo.......now I probably want to find the columns that have a single space and make it nullany ideas?I'm working on itI get no help here (at the orifice) so any help here (at sqlteam) would be greatthe short answer would be if I could figure out how to bcp in a fixed width file without it loading spacei.e.If the bcp sees a blank string, it should be left nullBrett8-)Hint: Want your questions answered fast? Follow the direction in this linkhttp://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspxAdd yourself!http://www.frappr.com/sqlteam |
 |
|
pootle_flump
1064 Posts |
Posted - 2007-10-12 : 09:06:02
|
I bcp in fixed width all the time. This is hardly ground breaking but I just dump it all into unconstrained staging tables, cleanse, dedupe etc and load into the live tables. I imagine you have discounted something like this already though.... |
 |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-12 : 09:34:30
|
"And the rtrim, doesn't trim all the spaces, as it leaves one"Have you got one of those funny spaces in there?CHAR(160) or somesuch ...Kristen |
 |
|
X002548
Not Just a Number
15586 Posts |
Posted - 2007-10-12 : 10:27:15
|
quote: Originally posted by pootle_flump I bcp in fixed width all the time. This is hardly ground breaking but I just dump it all into unconstrained staging tables, cleanse, dedupe etc and load into the live tables. I imagine you have discounted something like this already though....
YesThis is a separate db that is meant for data privacy to be given over dev environmentsIt's not one db, but manyTo do that would be hugeOne db has over 1k tables to scrambleBrett8-)Hint: Want your questions answered fast? Follow the direction in this linkhttp://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspxAdd yourself!http://www.frappr.com/sqlteam |
 |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-12 : 10:33:24
|
"scramble"Ah, I forgot that was what you were doing.perhaps the space has been scrambled, such that RTrim() doesn't think its a space. |
 |
|
X002548
Not Just a Number
15586 Posts |
Posted - 2007-10-12 : 10:44:12
|
OK Why do I get the followingUPDATING TRK_ASI_C_Detail(1193 row(s) affected)Server: Msg 3021, Level 16, State 1, Line 1Cannot perform a backup or restore operation within a transaction.Server: Msg 3013, Level 16, State 1, Line 1BACKUP LOG is terminating abnormally. FROM the following? Isn't that outside of the transaction?PRINT 'UPDATING TRK_ASI_C_Detail'BEGIN TRANUPDATE TRK_ASI_C_Detail SET PropIDNumber = RTRIM(PropIDNumber), PropIDCode = RTRIM(PropIDCode)COMMIT TRANGOBACKUP LOG Tracker_Data WITH TRUNCATE_ONLYGO Brett8-)Hint: Want your questions answered fast? Follow the direction in this linkhttp://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspxAdd yourself!http://www.frappr.com/sqlteam |
 |
|
X002548
Not Just a Number
15586 Posts |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-12 : 11:13:11
|
Something else creating the transaction? |
 |
|
X002548
Not Just a Number
15586 Posts |
|
X002548
Not Just a Number
15586 Posts |
|
X002548
Not Just a Number
15586 Posts |
Posted - 2007-10-12 : 12:34:36
|
OK, 30 tables took 1/2 hour to do the replaceI then shrunk the db and the filesDBCC SHRINKDATABASE(Tracker_Data,10)USE Tracker_DataGODBCC SHRINKFILE(Tracker_Data_Data)GOUSE Privacy_Tracker_DataGOAnd it's still at 13GB, from the original restore that was 4GBAny other ideas?DBCC puts outDbId FileId CurrentSize MinimumSize UsedPages EstimatedPages ------ ------ ----------- ----------- ----------- -------------- 22 1 1656296 128 1656256 1656256(1 row(s) affected)DBCC execution completed. If DBCC printed error messages, contact your system administrator.Brett8-)Hint: Want your questions answered fast? Follow the direction in this linkhttp://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspxAdd yourself!http://www.frappr.com/sqlteam |
 |
|
X002548
Not Just a Number
15586 Posts |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-12 : 13:11:03
|
"What else would cause it to grow?"That's it. The scrambled data will index differently, and has probably left the indexes shot full of holes ...Or your junior side-kick has set the Model database to 13GB for a larf - ,maybe hes hooked it up to a date algorithm, and its ready for tomorrow already, so your run on Monday is going to be 16GB already   KristenKristen |
 |
|
X002548
Not Just a Number
15586 Posts |
|
Kristen
Test
22859 Posts |
Posted - 2007-10-12 : 18:05:26
|
"Yup, That was it"Wow! The Junior Sidekick really DID hook the size-of-Model database up to the day-of-the-month?Did he use MVJ's date function?Maybe you can sue ...   |
 |
|
X002548
Not Just a Number
15586 Posts |
|
Next Page
|