Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I was working at a recruiting startup years ago and we got a cease & desist letter from LI asking us to stop crawling their public data.

Initially crawling them was very easy (we had ~100m profiles). With time crawling became harder and harder on the technical front. They obviously invested millions of $ in technology to stop being crawled.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: