It's not the OLDEDB command that is the problem here. It's the table and its indexes. SQL has to find rows matching your criteria:
where ID like 'Excel Column 1%' AND CITY = 'excel column 4' AND Country = 'excel column 5'
If the columns ID, City and Country are not indexed, SQL will have to scan the whole table to find them. If the table is big, this will take some time. If you can build an index on these three columns, the execution time should be dramatically reduced.
Try looking at the execution plan for the query using SSMS. Likely you will see table scans or index scans instead of index seeks.