SQL Server Forums
Profile | Register | Active Topics | Members | Search | Forum FAQ
 
Register Now and get your question answered!
Username:
Password:
Save Password
Forgot your Password?

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 Getting LinkedIn Contacts into SQL Server
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

mikebird
Aged Yak Warrior

United Kingdom
529 Posts

Posted - 07/17/2013 :  10:41:10  Show Profile  Reply with Quote
How do I extract data from a public website? With a browser, or is there a different tool?

Which type of file do I need to generate to use with SSIS?

I've done similar with HTML emails, because I get mountains of 'em


I've been asked to do this. As any regular boss would want to do something, with no concept of waht's possible... wanting a 'can do' attitude...

I tried viewing my home page as raw source or save as... HTML

I have 124 contacts

They want an email marketing camapaign. They say 140k or 150k contacts in LinkedIn. Where did they get those figures??

Overall, I think of my own contacts and can see some names in the HTML whis is a bit messy and needs some automated de-tagging of the text, which I've tested. Not sure what a marketing campaigner would do - their own private contact list... I think what you get from a website is very limited.

I think they need to pay for a signed agreement with LinkedIn, as they have value in their database and will only reveal specific details.


Edited by - mikebird on 07/17/2013 11:28:36

mfemenel
Professor Frink

USA
1421 Posts

Posted - 07/17/2013 :  15:58:57  Show Profile  Visit mfemenel's Homepage  Reply with Quote
I'm not sure what format you get from Linked in, but here is the Main() body of a script task that will grab an xml file from an HTTP request and then I have a data flow with an xml source to rip the xml.
public void Main()
{
//For each loop through the ForLoop container, download a file.
try
{
// Logging start of download
bool fireAgain = true;

//Append file_min value to url to get file. Filemin is adjusted with each loop through the forloop container.

string getfile = "http://www.sqlsaturday.com/eventxml.aspx?sat=" + Dts.Variables["User::file_min"].Value.ToString();

//Set the url of the HTTP connection manager to getFile(above)
Dts.Connections["mySSISConnection"].ConnectionString = getfile;

//Output to the execution status what is happening.
Dts.Events.FireInformation(0, "Download File", "Start downloading " + Dts.Connections["mySSISConnection"].ConnectionString, string.Empty, 0, ref fireAgain);

// Get your newly added HTTP Connection Manager
Object mySSISConnection = Dts.Connections["mySSISConnection"].AcquireConnection(null);

// Create a new connection
HttpClientConnection myConnection = new HttpClientConnection(mySSISConnection);

// Download file and use the Flat File Connectionstring (D:\SourceFiles\Products.csv)
// to save the file (and replace the existing file)

//Set the file name and location.
string filename = "c:\\eventxml_" + Dts.Variables["User::file_min"].Value.ToString();
Dts.Connections["eventxml"].ConnectionString = filename;

//Execute the DownloadFile method and grabe the file.
myConnection.DownloadFile(Dts.Connections["eventxml"].ConnectionString, true);
Dts.Variables["User::xml_file_name"].Value = filename;
// Logging end of download
Dts.Events.FireInformation(0, "Download File", "Finished downloading " + Dts.Connections["eventxml"].ConnectionString, string.Empty, 0, ref fireAgain);

// Quit Script Task succesful
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception ex)
{
// Logging why download failed
Dts.Events.FireError(0, "Download File", "Download failed: " + ex.Message, string.Empty, 0);

// Quit Script Task unsuccesful
Dts.TaskResult = (int)ScriptResults.Failure;
}
}

Mike
"oh, that monkey is going to pay"
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
SQL Server Forums © 2000-2009 SQLTeam Publishing, LLC Go To Top Of Page
This page was generated in 0.03 seconds. Powered By: Snitz Forums 2000