Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Transact-SQL (2005)
 HOW TO: Import XML Feed from Another Domain

Author  Topic 

kwilliams

194 Posts

Posted - 2009-06-26 : 15:40:03
I have a stored procedure (SP) that imports 2 internal XML feeds into a SQL Server 2005 table using Bulk Load/OPENROWSET without a problem. But I'd also like to import one more external XML feed that's not located on my domain into that same SP. When I enter the URL to that XML feed, I get this error message when I run the job:

Msg 4861, Level 16, State 1, Line 2
Cannot bulk load because the file "http://codeamber.org/a1xl04act/amberalert.xml" could not be opened. Operating system error code 123(The filename, directory name, or volume label syntax is incorrect.).

(1 row(s) affected)

From research, I'm *thinking* this is an issue regarding accessing data from another site/domain, but I can't say for sure. I'm able to do this in ASP.NET/VB.NET code without a problem, so I'm not sure why it wouldn't work within SQL Server 2005 somehow. I'm including the entire SP code below. If anyone can let me know what I'm doing wrong, it would be greatly appreciated. Thanks.

spXMLImport:
USE [DATABASENAME]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

ALTER PROCEDURE [dbo].[spXMLImport]

AS
BEGIN

SET NOCOUNT ON;

DROP TABLE tblXMLFeeds

CREATE TABLE tblXMLFeeds
(
record_id VARCHAR(50),
xmlTitle VARCHAR(100),
xmlFileName VARCHAR(300),
xml_data xml,
datestamp VARCHAR(23)
)

DECLARE @record_id_aa VARCHAR(50)
DECLARE @record_id_bb VARCHAR(50)
DECLARE @record_id_cc VARCHAR(50)
DECLARE @xmlTitle_aa VARCHAR(100)
DECLARE @xmlTitle_bb VARCHAR(100)
DECLARE @xmlTitle_cc VARCHAR(100)
DECLARE @xmlFileName_aa VARCHAR(300)
DECLARE @xmlFileName_bb VARCHAR(300)
DECLARE @xmlFileName_cc VARCHAR(300)
DECLARE @datestamp VARCHAR(23)

SELECT @record_id_aa = 'aaID'
SELECT @record_id_bb = 'bbID'
SELECT @record_id_cc = 'ccID'

SELECT @xmlTitle_aa = 'Internal Feed 1'
SELECT @xmlTitle_bb = 'Internal Feed 2'
SELECT @xmlTitle_cc = 'External Feed'

SELECT @xmlFileName_aa = '\\SERVERNAME\DIRECTORY\docs\xml\internalfeed1.xml'
SELECT @xmlFileName_bb = '\\SERVERNAME\DIRECTORY\docs\xml\internalfeed2.xml'
SELECT @xmlFileName_cc = 'http://codeamber.org/a1xl04act/amberalert.xml' --<<<----EXTERNAL FEED
--******************I'VE ALSO TRIED THE FOLLOWING:******************
--SELECT @xmlFileName_cc = '//codeamber.org/a1xl04act/amberalert.xml'
--SELECT @xmlFileName_cc = '\\codeamber.org\a1xl04act\amberalert.xml'

--SET datestamp as ISO-8601 datetime format
SELECT @datestamp = CONVERT(VARCHAR(23), GETDATE(), 126)

EXEC('
INSERT INTO tblXMLFeeds(record_id, xmlTitle, xmlFileName, xml_data, datestamp)

SELECT ''' + @record_id_aa + ''', ''' + @xmlTitle_aa + ''', ''' + @xmlFileName_aa + ''', xmlData, ''' + @datestamp + '''

FROM
(
SELECT *
FROM OPENROWSET (BULK N''' + @xmlFileName_aa + ''' , SINGLE_BLOB) AS XMLDATA
) AS FileImport (XMLDATA)


INSERT INTO tblXMLFeeds(record_id, xmlTitle, xmlFileName, xml_data, datestamp)

SELECT ''' + @record_id_bb + ''', ''' + @xmlTitle_bb + ''', ''' + @xmlFileName_bb + ''', xmlData, ''' + @datestamp + '''
FROM
(
SELECT *
FROM OPENROWSET (BULK N''' + @xmlFileName_bb + ''' , SINGLE_BLOB) AS XMLDATA
) AS FileImport (XMLDATA)


INSERT INTO tblXMLFeeds(record_id, xmlTitle, xmlFileName, xml_data, datestamp)

SELECT ''' + @record_id_cc + ''', ''' + @xmlTitle_cc + ''', ''' + @xmlFileName_cc + ''', xmlData, ''' + @datestamp + '''
FROM
(
SELECT *
FROM OPENROWSET (BULK N''' + @xmlFileName_cc + ''' , SINGLE_BLOB) AS XMLDATA
) AS FileImport (XMLDATA)
')
END


KWilliams
-------------------
It's the end of the world as we know it...and I feel fine

kwilliams

194 Posts

Posted - 2009-07-07 : 10:40:07
I received this response from someone else:

quote:
Should use SSIS package to do this...1st step, import the file to your domain. Then do a bulk insert.


...but I can't figure out how to import an external XML file. If anyone can point me in the right direction, that would be great. Thanks for any help.

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 10:43:51
Here is an example of how to download a web file.
See if it works out for you.

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=111356



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 10:55:33
That is a really nifty procedure you have there!!!

Do you perhaps have a much simpler one that simply imports a web file, without all of the additional code that I wouldn't need?

Also, is the "IF @hr <> 0" the validation of the file, or is that somewhere else? I'd like to validate the source file in-case it goes down, so I'm trying to see if you already have that implemented in your code. Thanks so much Peso.

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 11:38:40
In trying to customize your already-existing code to do what I need, I just copied and pasted your code into a query window within SQL Server 2005. But this is the error message I received:
Msg 213, Level 16, State 1, Line 18
Insert Error: Column name or number of supplied values does not match table definition.

Do you have any idea on why I'd get this message?

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 12:38:10
It would be something similar to this
DECLARE	@url VARCHAR(300),
@win INT,
@hr INT,
@Text NVARCHAR(MAX)


SET @url = 'http://codeamber.org/a1xl04act/amberalert.xml'

EXEC @hr = sp_OACreate 'WinHttp.WinHttpRequest.5.1', @win OUT
IF @hr <> 0 EXEC sp_OAGetErrorInfo @win

EXEC @hr = sp_OAMethod @win, 'Open', NULL, 'GET', @url, 'false'
IF @hr <> 0 EXEC sp_OAGetErrorInfo @win

EXEC @hr = sp_OAMethod @win, 'Send'
IF @hr <> 0 EXEC sp_OAGetErrorInfo @win

EXEC @hr = sp_OAGetProperty @win, 'ResponseText', @Text OUT
IF @hr <> 0 EXEC sp_OAGetErrorInfo @win

EXEC @hr = sp_OADestroy @win
IF @hr <> 0 EXEC sp_OAGetErrorInfo @win

SELECT CAST(@Text AS XML)



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 15:18:30
Ok, I gave that a try. Although the query executed successfully, the following two results windows showed up:

RESULTS:
Error: 0x8004271A
Source: ODSOLE Extended Procedure
Description: Error in srv_convert.
HelpFile: NULL
HelpID: 0

(No column name)
1 NULL

Do you know what this means?

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 15:35:25
Use VARCHAR(MAX) instead of NVARCHAR(MAX).



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 15:40:04
The same error happened :(

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 15:51:27
It seems there is a limit of 8000 characters for the 'ResponseText' property.
Maybe not SQL Server 2008 adapted yet?

It's either sp_OAGetProperty implementation or WinHttpRequest implementation.


N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 16:02:52
Yes, we're still using SQL Server 2005, and we won't adapt to 2008 for some time. So does that mean that I'm stuck at this point?

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 16:08:16
At least using the WinHttpRequest object.
Have you tried using SSIS?



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 16:11:36
Google gave me this for SSIS and HTTP
quote:
Public Sub Main()

Dim httpConnection As Microsoft.SqlServer.Dts.Runtime.HttpClientConnection

Dim temp As Object

temp = Dts.Connections("JPGToDownload").AcquireConnection(Nothing)

httpConnection = New HttpClientConnection(temp)

httpConnection.DownloadFile(Dts.Variables("DownloadFile").Value.ToString(), True)

Dts.TaskResult = Dts.Results.Success

End Sub




N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 16:56:02
Yeah, I've looked into using SSIS, but I'm not sure how to import a remote XML feed. I would want to do it from a stored procedure, as opposed to a web form. Do you have any suggestions on how I can easily do that?

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2009-07-07 : 17:07:07
You can write your own CLR procedure, which doesn't seem to have the same limits.



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

kwilliams

194 Posts

Posted - 2009-07-07 : 17:52:18
I'm not sure what a CLR procedure is. I was pretty familiar with SQL Server 2000, but I'm pretty much a newbie to 2005. I've read at so many places that I can import XML feeds, but I hear different opinions on whether or not it can be done remotely in 2005. As

As you can see from my original post, I was able to do this with local XML files. But as soon as I put a remote XML feed in the URL's place, I received this error message: Msg 4861, Level 16, State 1, Line 2
Cannot bulk load because the file "http://codeamber.org/a1xl04act/amberalert.xml" could not be opened. Operating system error code 123(The filename, directory name, or volume label syntax is incorrect.).

I was hoping that there was an easy way to import remote files in a similar way that I can import them locally, but this seems to be a real pain in the you know what.

KWilliams
-------------------
It's the end of the world as we know it...and I feel fine
Go to Top of Page
   

- Advertisement -