Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums  General SQL Server Forums  New to SQL Server Programming  Get Web page content into variable taking time

Author  Topic  bitekkie Starting Member 1 Post Posted - 2014-05-21 : 22:10:34 Hi All, I came across a scenario like hitting an URL and loading the returned data into an Variable using SCRIPT task in SSIS. I have used below C# script and getting the data into variable. Now, the problem is with the time it is taking to load the data. I have to hit nearly 1000+ times HTTP and get the data store into variable and then writing the same into an text file on checking the condition in returned output, now it is taking more time into hitting url and loading data into a variable like 6 to 7 hours for 1000 files.Could any one please help me in narrow down the issue? C# script: WebRequest request = HttpWebRequest.Create(URL); --- Hitting the URl WebResponse response = request.GetResponse(); StreamReader reader = new StreamReader(response.GetResponseStream()); string TMP= reader.ReadToEnd(); --- storing the returned data into TMP note: returned data size is of 2 to 10MB. Please help me on this BI TEKKIE