| Author |
Topic  |
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/03/2012 : 14:37:38
|
Hi,
I have to solve the following problem. We used DoubleTake for the event of a disaster. The database server is mirrored 1:1.
After a disaster event, the original database is offline. The system will switch to the mirrored backup system. The backup system is identical to the original system. How can I prove that the backup database is identical in content and data to the prod DB? |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/03/2012 : 14:42:36
|
There is a third party utility I've used before from Red Gate called SQL Data Compare (as well as SQL Compare, which is the schema version). The data compare tool will not work very well on very big databases as you might imagine, but if you have a smaller DB it might suit your needs.
I have no connections to Red Gate, just used their products before. |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/03/2012 : 14:52:37
|
the prod DB is in time of a disaster point no longer available! for that reason I can't compare any schema or table. Please keep in mind that the prod db is physically offline in disaster time. I have only the mirrored DB on hand !!
quote: Originally posted by lazerath
There is a third party utility I've used before from Red Gate called SQL Data Compare (as well as SQL Compare, which is the schema version). The data compare tool will not work very well on very big databases as you might imagine, but if you have a smaller DB it might suit your needs.
I have no connections to Red Gate, just used their products before.
|
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/03/2012 : 16:12:17
|
| This this a puzzle or a real world situation? Has it already happened or are you designing a solution to help in the event of a disaster? |
Edited by - lazerath on 10/03/2012 16:12:38 |
 |
|
|
chadmat
The Chadinator
USA
1955 Posts |
Posted - 10/03/2012 : 16:39:04
|
If the original primary is completely offline, there is no way to guarantee that.
-Chad |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/04/2012 : 02:49:42
|
I been thereby to provide a solution, which will bee use in test case, to proves that, that the two DBs are equal. This can happen in following way. I create a deduction from prod system, then shut down the system and backup system is online. Now I make a deduction from the backup and compare the two.
THX
quote: Originally posted by lazerath
This this a puzzle or a real world situation? Has it already happened or are you designing a solution to help in the event of a disaster?
|
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/04/2012 : 02:50:59
|
see postet ne reply
THX
quote: Originally posted by chadmat
If the original primary is completely offline, there is no way to guarantee that.
-Chad
|
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/04/2012 : 10:50:20
|
quote: Originally posted by zero1de
I been thereby to provide a solution, which will bee use in test case, to proves that, that the two DBs are equal. This can happen in following way. I create a deduction from prod system, then shut down the system and backup system is online. Now I make a deduction from the backup and compare the two.
THX
quote: Originally posted by lazerath
This this a puzzle or a real world situation? Has it already happened or are you designing a solution to help in the event of a disaster?
It is not clear to me what you mean. What is a deduction? How can you compare the two systems post-deduction if "the prod DB is in time of a disaster point no longer available". |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/04/2012 : 11:35:50
|
Hi,
I might be pushing at the beginning wrong. The whole thing should just be a test. The disaster date is determined by me. So I can first collect some data from any key tables and then get the prod DB offline and get backup DB online and compare these values ??there. It should only a test scenario.
THX
quote: Originally posted by lazerath
quote: Originally posted by zero1de
I been thereby to provide a solution, which will bee use in test case, to proves that, that the two DBs are equal. This can happen in following way. I create a deduction from prod system, then shut down the system and backup system is online. Now I make a deduction from the backup and compare the two.
THX
quote: Originally posted by lazerath
This this a puzzle or a real world situation? Has it already happened or are you designing a solution to help in the event of a disaster?
It is not clear to me what you mean. What is a deduction? How can you compare the two systems post-deduction if "the prod DB is in time of a disaster point no longer available".
|
 |
|
|
chadmat
The Chadinator
USA
1955 Posts |
Posted - 10/04/2012 : 13:19:52
|
I don't understand the point? In real life, the disaster will not be determined by you, so if you just want to compare DBs, why simulate a disaster at all? If it is for some sort of DR process, then this test is invalid because you won't know the date/time a disaster will occur.
-Chad |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/04/2012 : 14:24:01
|
I do not understand what's so hard about understanding that. It is only a simulated disaster and want to proves that the data is mirrored correctly. It goes to prove us that the Double Taker Software syncroniziert correctly.
quote: Originally posted by chadmat
I don't understand the point? In real life, the disaster will not be determined by you, so if you just want to compare DBs, why simulate a disaster at all? If it is for some sort of DR process, then this test is invalid because you won't know the date/time a disaster will occur.
-Chad
|
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/04/2012 : 16:27:33
|
Ok, so you'll have to do some work and this isn't perfect, but maybe you can do something like this:
DECLARE @SQL NVARCHAR(MAX);
SELECT @SQL=COALESCE(@SQL+CONVERT(NVARCHAR(MAX),' UNION ALL '),CONVERT(NVARCHAR(MAX),'')) + 'select CHECKSUM_AGG(CHECKSUM(*)) FROM [' + CONVERT(NVARCHAR(MAX),Table_schema) + '].[' + CONVERT(NVARCHAR(MAX),Table_name) + ']' FROM INFORMATION_SCHEMA.Tables ORDER BY TABLE_SCHEMA, TABLE_NAME
select @SQL;
EXEC (@SQL);
Keep in mind CHECKSUM won't work on data types: text, ntext, image, XML, cursor, and sql_variant with any one of the preceding types as its base type. If you don't have any of these datatypes, you could do a CHECKSUM_AGG on the result of the UNION ALL. Otherwise, you'll need to find someway to exclude those datatypes.
|
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/05/2012 : 02:27:52
|
Thanks for your help. But I do not understand what this query makes and what it checkes. Can you tell me please whats happining in this query ?
THX
quote: Originally posted by lazerath
Ok, so you'll have to do some work and this isn't perfect, but maybe you can do something like this:
DECLARE @SQL NVARCHAR(MAX);
SELECT @SQL=COALESCE(@SQL+CONVERT(NVARCHAR(MAX),' UNION ALL '),CONVERT(NVARCHAR(MAX),'')) + 'select CHECKSUM_AGG(CHECKSUM(*)) FROM [' + CONVERT(NVARCHAR(MAX),Table_schema) + '].[' + CONVERT(NVARCHAR(MAX),Table_name) + ']' FROM INFORMATION_SCHEMA.Tables ORDER BY TABLE_SCHEMA, TABLE_NAME
select @SQL;
EXEC (@SQL);
Keep in mind CHECKSUM won't work on data types: text, ntext, image, XML, cursor, and sql_variant with any one of the preceding types as its base type. If you don't have any of these datatypes, you could do a CHECKSUM_AGG on the result of the UNION ALL. Otherwise, you'll need to find someway to exclude those datatypes.
|
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/05/2012 : 14:04:26
|
It's producing a CHECKSUM of all the data in each row in a table, then a CHECKSUM aggregate of those to produce a CHECKSUM of the table. It's then doing that for every table in the database and UNIONing them together. This list can then have a further CHECKSUM_AGG applied to it in order to get a single integer value for the DB.
A check sum in principal is just a way to take a lot of data and get a semi-unique value that represents it. If the data changes, in theory a new checksum will be generated. I say theory, because this function uses INT and there are many chances for a "collision", which means different data can have the same CHECKSUM. For your purposes, it should be good enough.
Again, there are many deficiencies to this approach, but the general principal should be relevant. |
Edited by - lazerath on 10/05/2012 14:04:45 |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/05/2012 : 15:59:35
|
o.k but the db which is to check has fields like ntext, images, text etc.. and there for a get this message when execute the query.
Lookup Error - SQL Server Database Error: Argument data type image is invalid for argument 7 of checksum function.
Lookup Error - SQL Server Database Error: Argument data type ntext is invalid for argument 12 of checksum function.
THX
quote: Originally posted by lazerath
It's producing a CHECKSUM of all the data in each row in a table, then a CHECKSUM aggregate of those to produce a CHECKSUM of the table. It's then doing that for every table in the database and UNIONing them together. This list can then have a further CHECKSUM_AGG applied to it in order to get a single integer value for the DB.
A check sum in principal is just a way to take a lot of data and get a semi-unique value that represents it. If the data changes, in theory a new checksum will be generated. I say theory, because this function uses INT and there are many chances for a "collision", which means different data can have the same CHECKSUM. For your purposes, it should be good enough.
Again, there are many deficiencies to this approach, but the general principal should be relevant.
|
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/05/2012 : 16:27:14
|
| Like I said, you will need to do some work and it's not perfect, but this shows you a path. |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/06/2012 : 06:06:42
|
ok you mean that a should find a way to extract this fields Like ntext etc. But how can i Do that ? can you give me a Exempel
Thx
quote: Originally posted by lazerath
Like I said, you will need to do some work and it's not perfect, but this shows you a path.
|
Edited by - zero1de on 10/09/2012 01:20:39 |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/08/2012 : 12:56:06
|
Here is a Script for the Table that contain ntext Field. Now how can i extract this Field from the query ?
DECLARE @SQL NVARCHAR(MAX);
SELECT @SQL=COALESCE(@SQL+CONVERT(NVARCHAR(MAX),' UNION ALL '),CONVERT(NVARCHAR(MAX),'')) + 'select CHECKSUM_AGG(CHECKSUM(*)) FROM [' + CONVERT(NVARCHAR(MAX),Table_schema) + '].[' + CONVERT(NVARCHAR(MAX),Table_name) + ']' FROM INFORMATION_SCHEMA.Tables ORDER BY TABLE_SCHEMA, TABLE_NAME
select @SQL;
EXEC (@SQL);
CREATE TABLE [dbo].[HL_NEW] ( [DWDOCID] int NOT NULL, [DWPAGECOUNT] int NULL, [DWDISKNO] int NULL, [DWFLAGS] int NULL, [DWOFFSET] int NULL, [DWSTOREDATETIME] datetime NULL, [DWSTOREUSER] nvarchar(20) NULL, [DWMODDATETIME] datetime NULL, [DWMODUSER] nvarchar(20) NULL, [DWLASTACCESSDATETIME] datetime NULL, [DWLASTACCESSUSER] nvarchar(20) NULL, [DWSTORAGEACCESS] ntext NULL, [FORDNR] nvarchar(20) NULL, [ENGAGEMENTNR] nvarchar(40) NULL, [NACHNAME] nvarchar(40) NULL, [VORNAME] nvarchar(40) NULL, [GEBURTSDATUM] datetime NULL, [SPARTE] nvarchar(40) NULL, [TITEL] int NULL, [DOKDATUM] datetime NULL, [BEHLTER] nvarchar(20) NULL, [VBAKTZ] nvarchar(20) NULL, [VBDATUM] datetime NULL, [STATUS] nvarchar(20) NULL, [MATCHCODE] nvarchar(20) NULL, [PartnerID] int NULL, [ProdUnit] int NULL, [BARCODENR] nvarchar(38) NULL, [mig_id] int NULL, [mig_status] nvarchar(38) NULL, [mig_status_detail] nvarchar(255) NULL, [KATEGORIE] nvarchar(80) NULL, [BELEGART] nvarchar(40) NULL, [PHIN_STATUS] int NULL, [STAPELNUMMER] nvarchar(40) NULL, [FORDERGNR] int NULL, [GLAEUBIGERNR] int NULL, [HCSUPDATE] int NULL, [ABGLEICH] nvarchar(255) NULL, [BRIEFID] nvarchar(20) NULL, [BRIEFNR] nvarchar(20) NULL, [BETREFF] nvarchar(255) NULL, [EMAIL] nvarchar(255) NULL, [KONTONR] int NULL, [BEMERKUNG] nvarchar(255) NULL, [HAUPTSCHULDNER] int NULL, [DOKBEZEICHNUNG] nvarchar(255) NULL, [ZUSTSACHB] nvarchar(80) NULL, [DWDOCIDOLD] nvarchar(20) NULL, [EXPORTID] nvarchar(20) NULL, [ARCHIVIERT] int NULL, [DWHEADERSIZE] bigint NULL, [DWDOCSIZE] bigint NULL, [DWEXTENSION] nvarchar(20) NULL, [DWVERID] int NULL, [DWSYSVERSION] int NULL, [DWFLAGSEX] bigint NULL, [DELETE] varchar(5) NULL) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] WITH (DATA_COMPRESSION = NONE);
quote: Originally posted by zero1de
ok you mean that a should find a way to extract this fields Luke ntext etc. Bit how can i Do that ? can you give me a Exempel
Thx
quote: Originally posted by lazerath
Like I said, you will need to do some work and it's not perfect, but this shows you a path.
|
Edited by - zero1de on 10/09/2012 01:19:19 |
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/09/2012 : 16:08:13
|
I was hoping you would do more work and then show us where you were stuck, but I figured I'd just put something together for you. Try this:
DECLARE @SQL NVARCHAR(MAX);
DECLARE @datatype
TABLE (
DATA_TYPE NVARCHAR(128),
ALTERNATE_EXPRESSION NVARCHAR(128)
);
INSERT @datatype VALUES('text','CONVERT(VARCHAR(MAX),%n)')
INSERT @datatype VALUES('ntext','CONVERT(NVARCHAR(MAX),%n)')
INSERT @datatype VALUES('image','CONVERT(VARBINARY(MAX),%n)')
INSERT @datatype VALUES('XML','CONVERT(VARCHAR(MAX),%n)')
DECLARE @TABLE
TABLE (
Table_schema sysname,
Table_name sysname,
column_list NVARCHAR(MAX)
PRIMARY KEY(Table_schema, Table_name)
);
DECLARE @COLUMN
TABLE (
ID INT,
Table_schema sysname,
Table_name sysname,
column_name sysname,
ORDINAL_POSITION INT,
data_type NVARCHAR(128),
max_ordinal INT,
column_list NVARCHAR(MAX)
PRIMARY KEY(ID)
);
-- Capture an overridable column_listl, also enforce an order
-- without using ORDER BY in the following SELECT
INSERT @TABLE
SELECT t.Table_schema, t.Table_name, CASE WHEN c.COLUMN_NAME IS NULL THEN '*' END
FROM INFORMATION_SCHEMA.Tables AS t
OUTER APPLY (
SELECT TOP 1 c.COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS AS c
WHERE c.Table_schema = t.Table_schema AND c.Table_name = t.Table_name
AND c.DATA_TYPE IN (SELECT d.DATA_TYPE FROM @datatype d)
) AS c;
-- need to assemble an ordered list for following update statement to build column_list
INSERT @COLUMN
SELECT ROW_NUMBER() OVER (ORDER BY c.Table_schema, c.Table_name, c.ORDINAL_POSITION) AS ID,
c.Table_schema,
c.Table_name,
c.column_name,
c.ORDINAL_POSITION,
c.data_type,
MAX(c.ORDINAL_POSITION) OVER(PARTITION BY c.Table_schema, c.Table_name) AS max_ordinal,
NULL AS column_list
FROM @TABLE AS t
JOIN INFORMATION_SCHEMA.COLUMNS AS c
ON c.Table_schema = t.Table_schema AND c.Table_name = t.Table_name
WHERE t.COLUMN_LIST IS NULL
ORDER BY c.Table_schema, c.Table_name, c.ORDINAL_POSITION;
DECLARE @last_table_schema sysname, @last_table_name sysname, @column_list NVARCHAR(MAX);
-- String concatenation to get COLUMN_LIST built the hard way -- I wish i had a CONCAT_STRING_AGG() function built in
UPDATE c
SET @column_list = COALESCE(CASE WHEN COALESCE(@last_table_schema,'') = c.table_schema
AND COALESCE(@last_table_name,'') = c.table_name
THEN @column_list + ',' END,'') + REPLACE(COALESCE(d.ALTERNATE_EXPRESSION,'%n'),'%n','[' + c.column_name + ']')
, @last_table_schema = c.table_schema
, @last_table_name = c.table_name
, c.column_list = @column_list
FROM @COLUMN AS c
LEFT JOIN @datatype d ON c.DATA_TYPE = d.DATA_TYPE;
-- go back and update our @TABLE variable with the column list
UPDATE t
SET t.COLUMN_LIST = c.COLUMN_LIST
FROM @TABLE AS t
JOIN @COLUMN AS c
ON c.Table_schema = t.Table_schema AND c.Table_name = t.Table_name
WHERE c.max_ordinal = c.ORDINAL_POSITION -- get only the last column as only it will have the full column list
-- Assemble the CHECKSUM for each row and a CHECKSUM_AGG for each TABLE with a UNION ALL to pull them together
SELECT @SQL=COALESCE(@SQL+CONVERT(NVARCHAR(MAX),' UNION ALL
'),CONVERT(NVARCHAR(MAX),''))
+ 'SELECT CHECKSUM_AGG(CHECKSUM('+ t.column_list + ')) AS CS FROM [' + CONVERT(NVARCHAR(MAX),t.Table_schema) + '].[' + CONVERT(NVARCHAR(MAX),t.Table_name) + ']'
FROM @TABLE t;
-- Include a CHECKSUM_AGG on the resultset so we can get a DB level CHECKSUM
SELECT @SQL = 'SELECT CHECKSUM_AGG(z.CS) AS DB_CHECKSUM FROM (
' + @SQL + ') AS z';
-- Run the Dynamic SQL statement to get a DB checksum
EXEC(@SQL); |
 |
|
|
zero1de
Yak Posting Veteran
Germany
66 Posts |
Posted - 10/11/2012 : 02:56:58
|
Lazerath many thx for your work. It works fine :) Thank you.
quote: Originally posted by lazerath
I was hoping you would do more work and then show us where you were stuck, but I figured I'd just put something together for you. Try this:
DECLARE @SQL NVARCHAR(MAX);
DECLARE @datatype
TABLE (
DATA_TYPE NVARCHAR(128),
ALTERNATE_EXPRESSION NVARCHAR(128)
);
INSERT @datatype VALUES('text','CONVERT(VARCHAR(MAX),%n)')
INSERT @datatype VALUES('ntext','CONVERT(NVARCHAR(MAX),%n)')
INSERT @datatype VALUES('image','CONVERT(VARBINARY(MAX),%n)')
INSERT @datatype VALUES('XML','CONVERT(VARCHAR(MAX),%n)')
DECLARE @TABLE
TABLE (
Table_schema sysname,
Table_name sysname,
column_list NVARCHAR(MAX)
PRIMARY KEY(Table_schema, Table_name)
);
DECLARE @COLUMN
TABLE (
ID INT,
Table_schema sysname,
Table_name sysname,
column_name sysname,
ORDINAL_POSITION INT,
data_type NVARCHAR(128),
max_ordinal INT,
column_list NVARCHAR(MAX)
PRIMARY KEY(ID)
);
-- Capture an overridable column_listl, also enforce an order
-- without using ORDER BY in the following SELECT
INSERT @TABLE
SELECT t.Table_schema, t.Table_name, CASE WHEN c.COLUMN_NAME IS NULL THEN '*' END
FROM INFORMATION_SCHEMA.Tables AS t
OUTER APPLY (
SELECT TOP 1 c.COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS AS c
WHERE c.Table_schema = t.Table_schema AND c.Table_name = t.Table_name
AND c.DATA_TYPE IN (SELECT d.DATA_TYPE FROM @datatype d)
) AS c;
-- need to assemble an ordered list for following update statement to build column_list
INSERT @COLUMN
SELECT ROW_NUMBER() OVER (ORDER BY c.Table_schema, c.Table_name, c.ORDINAL_POSITION) AS ID,
c.Table_schema,
c.Table_name,
c.column_name,
c.ORDINAL_POSITION,
c.data_type,
MAX(c.ORDINAL_POSITION) OVER(PARTITION BY c.Table_schema, c.Table_name) AS max_ordinal,
NULL AS column_list
FROM @TABLE AS t
JOIN INFORMATION_SCHEMA.COLUMNS AS c
ON c.Table_schema = t.Table_schema AND c.Table_name = t.Table_name
WHERE t.COLUMN_LIST IS NULL
ORDER BY c.Table_schema, c.Table_name, c.ORDINAL_POSITION;
DECLARE @last_table_schema sysname, @last_table_name sysname, @column_list NVARCHAR(MAX);
-- String concatenation to get COLUMN_LIST built the hard way -- I wish i had a CONCAT_STRING_AGG() function built in
UPDATE c
SET @column_list = COALESCE(CASE WHEN COALESCE(@last_table_schema,'') = c.table_schema
AND COALESCE(@last_table_name,'') = c.table_name
THEN @column_list + ',' END,'') + REPLACE(COALESCE(d.ALTERNATE_EXPRESSION,'%n'),'%n','[' + c.column_name + ']')
, @last_table_schema = c.table_schema
, @last_table_name = c.table_name
, c.column_list = @column_list
FROM @COLUMN AS c
LEFT JOIN @datatype d ON c.DATA_TYPE = d.DATA_TYPE;
-- go back and update our @TABLE variable with the column list
UPDATE t
SET t.COLUMN_LIST = c.COLUMN_LIST
FROM @TABLE AS t
JOIN @COLUMN AS c
ON c.Table_schema = t.Table_schema AND c.Table_name = t.Table_name
WHERE c.max_ordinal = c.ORDINAL_POSITION -- get only the last column as only it will have the full column list
-- Assemble the CHECKSUM for each row and a CHECKSUM_AGG for each TABLE with a UNION ALL to pull them together
SELECT @SQL=COALESCE(@SQL+CONVERT(NVARCHAR(MAX),' UNION ALL
'),CONVERT(NVARCHAR(MAX),''))
+ 'SELECT CHECKSUM_AGG(CHECKSUM('+ t.column_list + ')) AS CS FROM [' + CONVERT(NVARCHAR(MAX),t.Table_schema) + '].[' + CONVERT(NVARCHAR(MAX),t.Table_name) + ']'
FROM @TABLE t;
-- Include a CHECKSUM_AGG on the resultset so we can get a DB level CHECKSUM
SELECT @SQL = 'SELECT CHECKSUM_AGG(z.CS) AS DB_CHECKSUM FROM (
' + @SQL + ') AS z';
-- Run the Dynamic SQL statement to get a DB checksum
EXEC(@SQL);
|
 |
|
|
lazerath
Constraint Violating Yak Guru
USA
278 Posts |
Posted - 10/11/2012 : 11:10:31
|
| Thanks for the feedback. Glad to help. |
 |
|
Topic  |
|
|
|