you need to join the stock field as well:
select b.client
,b.manager
,b.year
,b.quarter
,s.stock
,s.date
from big as b
left outer join small as s
on s.client=b.client
and s.manager=b.manager
and DATEPART(year,s.date)=b.year
and DATEPART(quarter,s.date)=b.quarter
and s.stock=b.stock
where s.stock is NULL
Edit: ohh I forgot to mention, that you should expect:
485 13630 1999 1 18091
485 13630 2003 2 12345