I want create a web crawler that can search the products from websites like amazon from their search engine, I mean I want to get the same benefit as the API have. As in India many websites don’t have APIs so I can’t rely on APIs.
You can use Python for web scraping.
Also possible to do this with other server-side scripting languages… C#, PHP, even the ancient Classic ASP can do this.
When I need to scrape a website I use php and curl, old school but it works.
I found a web scraping tool that can directly scrape data from Amazon without coding. Maybe you could try it and tell me whether it’s good or not.
Web Scraping Template Take Away!
thanks for sharing with us, that looks great!
Hope it would be a useful suggestion for you. 