Please start any new threads on our new site at We've got lots of great SQL Server experts to answer whatever question you can come up with.

Our new SQL Server Forums are live! Come on over! We've restricted the ability to create new threads on these forums.

SQL Server Forums
Profile | Active Topics | Members | Search | Forum FAQ
Save Password
Forgot your Password?

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 Getting LinkedIn Contacts into SQL Server
 Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

Aged Yak Warrior

United Kingdom
529 Posts

Posted - 07/17/2013 :  10:41:10  Show Profile  Reply with Quote
How do I extract data from a public website? With a browser, or is there a different tool?

Which type of file do I need to generate to use with SSIS?

I've done similar with HTML emails, because I get mountains of 'em

I've been asked to do this. As any regular boss would want to do something, with no concept of waht's possible... wanting a 'can do' attitude...

I tried viewing my home page as raw source or save as... HTML

I have 124 contacts

They want an email marketing camapaign. They say 140k or 150k contacts in LinkedIn. Where did they get those figures??

Overall, I think of my own contacts and can see some names in the HTML whis is a bit messy and needs some automated de-tagging of the text, which I've tested. Not sure what a marketing campaigner would do - their own private contact list... I think what you get from a website is very limited.

I think they need to pay for a signed agreement with LinkedIn, as they have value in their database and will only reveal specific details.

Edited by - mikebird on 07/17/2013 11:28:36

Professor Frink

1421 Posts

Posted - 07/17/2013 :  15:58:57  Show Profile  Visit mfemenel's Homepage  Reply with Quote
I'm not sure what format you get from Linked in, but here is the Main() body of a script task that will grab an xml file from an HTTP request and then I have a data flow with an xml source to rip the xml.
public void Main()
//For each loop through the ForLoop container, download a file.
// Logging start of download
bool fireAgain = true;

//Append file_min value to url to get file. Filemin is adjusted with each loop through the forloop container.

string getfile = "" + Dts.Variables["User::file_min"].Value.ToString();

//Set the url of the HTTP connection manager to getFile(above)
Dts.Connections["mySSISConnection"].ConnectionString = getfile;

//Output to the execution status what is happening.
Dts.Events.FireInformation(0, "Download File", "Start downloading " + Dts.Connections["mySSISConnection"].ConnectionString, string.Empty, 0, ref fireAgain);

// Get your newly added HTTP Connection Manager
Object mySSISConnection = Dts.Connections["mySSISConnection"].AcquireConnection(null);

// Create a new connection
HttpClientConnection myConnection = new HttpClientConnection(mySSISConnection);

// Download file and use the Flat File Connectionstring (D:\SourceFiles\Products.csv)
// to save the file (and replace the existing file)

//Set the file name and location.
string filename = "c:\\eventxml_" + Dts.Variables["User::file_min"].Value.ToString();
Dts.Connections["eventxml"].ConnectionString = filename;

//Execute the DownloadFile method and grabe the file.
myConnection.DownloadFile(Dts.Connections["eventxml"].ConnectionString, true);
Dts.Variables["User::xml_file_name"].Value = filename;
// Logging end of download
Dts.Events.FireInformation(0, "Download File", "Finished downloading " + Dts.Connections["eventxml"].ConnectionString, string.Empty, 0, ref fireAgain);

// Quit Script Task succesful
Dts.TaskResult = (int)ScriptResults.Success;
catch (Exception ex)
// Logging why download failed
Dts.Events.FireError(0, "Download File", "Download failed: " + ex.Message, string.Empty, 0);

// Quit Script Task unsuccesful
Dts.TaskResult = (int)ScriptResults.Failure;

"oh, that monkey is going to pay"
Go to Top of Page
  Previous Topic Topic Next Topic  
 Reply to Topic
 Printer Friendly
Jump To:
SQL Server Forums © 2000-2009 SQLTeam Publishing, LLC Go To Top Of Page
This page was generated in 0.03 seconds. Powered By: Snitz Forums 2000