Reply
cancel
Showing results for 
Search instead for 
Did you mean: 
Reply

Scraping Forums & blogs: what's a reasonable price?

Active Member
Michael B Member Since: Mar 11, 2016
1 of 4

Hi Guys,

 

can somebody provide help me to decide on what's a reasonable price for scraping blogs/forums like the following using python scrapy :

https://www.solarpaneltalk.com/

 

Every post (Date, text, user, title, categories) is supposed to be saved as an item and then into a MongoDB.

The offer I got is 30$ per blog/forum. Is that a fair price?

 

Thank you very much,

Michael

Active Member
Michael B Member Since: Mar 11, 2016
2 of 4

I should also mention that I'm on the client side. Is 30$/site a reasonable plece to pay?

Thank you!

Ace Contributor
Probuddha N Member Since: Aug 29, 2015
3 of 4

There isn't a standard price for web scraping since every website present a different set of challenges when it comes to scraping data from its pages. Some websites are easier to scrape while others aren't. The presence of JS, Ajax, Flash, CAPTCHA etc makes scraping data all the more difficult.

 

Also, you need to consider the security measures some sites apply to prevent bots and scripts from scraping their pages. In such cases, the developer needs to adopt advanced measures for bypassing such filters.

 

So the pricing can only be ascertained once the website that needs to be scraped has been reviewed and analyzed thoroughly, taking all the above points in consideration.

 

Hope this helps!

Active Member
Michael B Member Since: Mar 11, 2016
4 of 4

Thank you so much for your answer! Smiley Happy

The pages we selected have none of the mentioned filters, so in that sense they are very easy to scrape...

 

Kind regards,

Michael