Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Transact-SQL (2005)
 SQL XML SOAP X.509, .net and performance, oh my!

Author  Topic 

djseng
Starting Member

25 Posts

Posted - 2008-01-16 : 10:12:34
Basically what this procedure does is takes a book of business and loads it into our database. What it does to do that.... a lot of stuff.

Currently the procedure involves executing a piece of, for most part, auto-generated .net code which creates a SOAP request with some criteria and a x509 certificate attached to it, which then returns this fully formed object, auto-generated .net object representation of the book, which is then that object is serialized into an xml stream, then removes the utf-8 encoding declaration and the namespace declaration, then calls a procedure on the to then insert the generated xml into the database.

This seems like a lot of work to me, and I've been diligently trying to increase the performance of this procedure, but for the most part I've been making things worse.

When I saw that the code was removing the namespace and the utf-8 declarations, i immediately believed this to be bad, so i altered the stored procedure and declared a default namespace, i also created, what i thought was an appropriate serializer to generate a unicode stream, ok, that worked all fine and dandy, but that code took a lot longer to execute, so basically put that stuff back the way it was.

One reason I believe it was slower was that instead of encoding in utf-8 and going to unicode, i was doubling the size of the stream going to the database server, and then that possibly the default namespace declaration was slowing things up a bit, makes sense to me i suppose, but I (and so is the client) kind of miffed that that didn't work out.

I'm also thinking, shouldn't I be able to get directly to the xml without creating this fully qualified object? Shouldn't I also be able to do this directly on sql server too? I'm pretty sure I can, but the x509 cert may pose a problem for me. The client is saying it's taking something like a minute to load a book of 300 people, that seems pretty slow to me, of course, they're making the request from who know where so there is probably some network latency involved as well.

If anyone can shoot my some resources, I'd appreciate it a lot, I've got at least one thing to look at right now, but it's from an old course book that i got a couple years ago.

Thanks,
Dave

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2008-01-16 : 10:15:36
hmm... could you please post the process with bullet points and where exatcly is each point executed?

_______________________________________________
Causing trouble since 1980
blog: http://weblogs.sqlteam.com/mladenp
SSMS Add-in that does a few things: www.ssmstoolspack.com <- new version out
Go to Top of Page

djseng
Starting Member

25 Posts

Posted - 2008-01-16 : 11:04:35
ok, but this is c# psuedo code with some names changed to protect the innocent... please don't kill me on it... I inherited it, so I've cleaned up a lot I think, but it's still kinda wishy washy I think.

this is from the c# class.
void LoadBookOfBusiness(string employeeId, string campaignId)
{
// service.xxx classes are auto-generated xml clases from the WSDL
service.CustomerData customer = null;

// GetCustomerDataSOAP inherits from System.Web.Services.Protocols.SoapHttpClientProtocol
using (service.GetCustomerDataSOAP soap = new service.GetCustomerDataSOAP())
{
service.Transaction transaction = new service.Transaction();
service.Header header = new service.Header();
service.SecurityHeaderType securityHeaderType = new service.SecurityHeaderType();
service.BinarySecurityTokenType binarySecurityTokenType = new service.BinarySecurityTokenType();

transaction.Trans_name = "CustomerDataTrans";

binarySecurityTokenType.Value = Guid.NewGuid().ToString();
securityHeaderType.BinarySecurityToken = binarySecurityTokenType;
header.Security = securityHeaderType;
header.Transaction = transaction;

soap.Url = "<service url>";
soap.HeaderValue = header;

// removed cert retrieval procedd
soap.ClientCertificates.Add(new X509Certificate2());

Request request = new Request();
request.EmployeeId = employeeId;
request.CampaignId = campaignId;

soap.Timeout = 300000;

try
{
customer = soap.GetCustomerData(request);
}
catch (Exception e)
{
Log(e);
}
}

if (customer == null)
return;

using (MemoryStream customerDataStream = new MemoryStream())
{
XmlSerializer customerDataSerializer = new XmlSerializer(typeof(service.CustomerData));
XmlTextWriter customerDataWriter = new XmlTextWriter(customerDataStream, Encoding.UTF8);
customerDataSerializer.Serialize(customerDataWriter, customer);

SqlParameter customerDataParameter = new SqlParameter("@CustomerData", SqlDbType.Xml);

string xml = Encoding.UTF8.GetString(customerDataStream.ToArray())
.Replace(" xmlns=\"cusomter_data_namespace\"", "")
.Replace(" encoding=\"utf-8\"", "");

customerDataParameter.Value = xml;

SQL.ExecuteNonQuery("dbo.usp_OrderProcess_LoadBookOfBusiness__test",
new SqlParameter("@EmployeeId", (object)employeeId),
new SqlParameter("@ProductCampaignsID", (object)campaignId),
customerDataParameter
);
}
}


this is the procedure, a lot of it was truncated as well, probably 20 additional fields loaded
ALTER PROCEDURE [dbo].[usp_OrderProcess_LoadBookOfBusiness__test]
@EmployeeID char(32),
@CampaignID char(32),
@CustomerData xml
AS
SET NOCOUNT ON

DECLARE
@mailListID int

EXEC dbo.usp_OrderProcess_LogBookOfBusiness @EmployeeID, @CustomerData

DECLARE
@mailLists table (
EmployeeID char(32),
MailListID int
)

DECLARE
@mailListItems table (
MailListItemID int,
PersonUID varchar(100)
)

DECLARE
@customers table (
MailListItemID int,
EmployeeID char(32),
PersonUID varchar(100),
FirstName varchar(50),
LastName varchar(50),
Salutation varchar(100),
AddrStreet varchar(100),
AddrCity varchar(30),
AddrState char(2),
AddrZipCode varchar(10),
DOB varchar(15)
)

DECLARE
@contracts table (
PersonUID varchar(100),
PolNumber varchar(100)
)

INSERT @customers (
EmployeeID,
PersonUID,
FirstName,
LastName,
Salutation,
AddrStreet,
AddrCity,
AddrState,
AddrZipCode,
DOB
)
SELECT
Customer.value('../../EmployeeId[1]', 'varchar(15)'),
Customer.value('PersonUID[1]', 'varchar(100)'),
Customer.value('FirstName[1]', 'varchar(50)'),
Customer.value('LastName[1]', 'varchar(50)'),
Customer.value('Salutation[1]', 'varchar(100)'),
Customer.value('AddrStreet[1]', 'varchar(100)'),
Customer.value('AddrCity[1]', 'varchar(30)'),
Customer.value('AddrState[1]', 'char(2)'),
Customer.value('AddrZipCode[1]', 'varchar(10)'),
Customer.value('DOB[1]', 'varchar(15)')
FROM @CustomerData.nodes('//Customers/Customer') as Customers(Customer)

INSERT @contracts
SELECT
[Contract].value('../../PersonUID[1]', 'varchar(100)'),
[Contract].value('PolNumber[1]', 'varchar(100)')
FROM @CustomerData.nodes('//Contracts/Contract') as Contracts([Contract]);

INSERT dbo.MailList (
EmployeeID
)
OUTPUT
inserted.EmployeeID,
inserted.MailListID
INTO @mailLists
SELECT
@EmployeeID

SELECT
@mailListID = MailListID
FROM @mailLists


INSERT dbo.MailList_Item (
MailListID,
PersonUID,
FirstName,
LastName,
Address1,
City,
[State],
PostalCode
DateOfBirth
)
OUTPUT
inserted.MailListItemID
INTO @mailListItems
SELECT
MailListID,
PersonUID,
FirstName,
LastName,
AddrStreet,
AddrCity,
AddrState,
AddrZipCode,
CONVERT(datetime, DOB)
FROM @customers c

UPDATE c
SET
c.MailListItemID = mli.MailListItemID
FROM @customers c
INNER JOIN @mailListItems mli
ON c.PersonUID = mli.PersonUID

INSERT dbo.MailList_Item_AdditionalInfo (
MailListItemID,
Salutation
)
SELECT
MailListItemID,
Salutation
FROM @customers

INSERT INTO dbo.MailList_Item_Contracts(
MailListItemID,
PolicyNumber
)
SELECT
c.MailListItemID,
_c.PolNumber
FROM @customers c
INNER JOIN @contracts _c
ON c.PersonUID = _c.PersonUID

UPDATE bi
SET
MailListID = @mailListID
FROM dbo.Basket_Item bi
WHERE EXISTS (
SELECT 1 FROM dbo.Basket b
WHERE bi.BasketID = b.BasketID
AND b.EmployeeID = @EmployeeID
)
AND EXISTS (
SELECT 1 FROM dbo.Campaigns cp
WHERE cp.ProdID = bi.ProdID
AND cp.CampaignID = @CampaignID
)
Go to Top of Page

djseng
Starting Member

25 Posts

Posted - 2008-01-16 : 11:54:15
After some reading, I don't think I'm going to able to do all of this on the SQL server, but I should still at least be able to eliminate some steps like the creation of the object...
Go to Top of Page

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2008-01-16 : 11:58:34
elimiate creation of which object?

_______________________________________________
Causing trouble since 1980
blog: http://weblogs.sqlteam.com/mladenp
SSMS Add-in that does a few things: www.ssmstoolspack.com <- new version out
Go to Top of Page

djseng
Starting Member

25 Posts

Posted - 2008-01-16 : 12:02:06
I would think I would be able to avoid creating and then serializing the service.CustomerData object and getting at the xml stream directly to send to the procedure.
Go to Top of Page
   

- Advertisement -