Friday, May 10, 2024
 Popular · Latest · Hot · Upcoming
116
rated 0 times [  117] [ 1]  / answers: 1 / hits: 18888  / 9 Years ago, thu, july 23, 2015, 12:00:00

I wonder if its possible to scrape an external (cross-domain) page through the user's IP?



For a shopping comparison site, I need to scrape pages of an e-com site but several requests from the server would get me banned, so I'm looking for ways to do client-side scraping — that is, request pages from the user's IP and send to server for processing.


More From » php

 Answers
6

No, you won't be able to use the browser of your clients to scrape content from other websites using JavaScript because of a security measure called Same-origin policy.



There should be no way to circumvent this policy and that's for a good reason. Imagine you could instruct the browser of your visitors to do anything on any website. That's not something you want to happen automatically.



However, you could create a browser extension to do that. JavaScript browser extensions can be equipped with more privileges than regular JavaScript.



Adobe Flash has similar security features but I guess you could use Java (not JavaScript) to create a web-scraper that uses your user's IP address. Then again, you probably don't want to do that as Java plugins are considered insecure (and slow to load!) and not all users will even have it installed.



So now back to your problem:




I need to scrape pages of an e-com site but several requests from the server would get me banned.




If the owner of that website doesn't want you to use his service in that way, you probably shouldn't do it. Otherwise you would risk legal implications (look here for details).



If you are on the dark side of the law and don't care if that's illegal or not, you could use something like http://luminati.io/ to use IP adresses of real people.


[#65715] Monday, July 20, 2015, 9 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
mireyag

Total Points: 73
Total Questions: 107
Total Answers: 85

Location: Ukraine
Member since Sun, Dec 13, 2020
3 Years ago
mireyag questions
Sun, Aug 15, 21, 00:00, 3 Years ago
Wed, Dec 16, 20, 00:00, 3 Years ago
Tue, Sep 1, 20, 00:00, 4 Years ago
Sun, Jul 5, 20, 00:00, 4 Years ago
;