Question

48

CasperJS/PhantomJS doesn't load https page

rated 0 times [ 51] [ 3] / answers: 1 / hits: 19303 / 10 Years ago, thu, october 16, 2014, 12:00:00

I know there are certain web pages PhantomJS/CasperJS can't open, and I was wondering if this one was one of them: https://maizepages.umich.edu. CasperJS gives an error: PhantomJS failed to open page status=fail.

I tried ignoring-ssl-errors and changing my user agent but I'm not sure how to determine which ones to use.

All I'm doing right now is the basic casper setup with casper.start(url, function () { ... }) where url=https://maizepages.umich.edu;

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

aubriechantalr

Add To Favorites

Follow

Total Points: 380

Total Questions: 95

Total Answers: 86

Location: Guadeloupe

Member since Sat, Aug 22, 2020

4 Years ago

answered 10 Years ago cartersinceren · Accepted Answer

The problem may be related to the recent discovery of a SSLv3 vulnerability (POODLE). Website owners were forced to remove SSLv3 support from their websites. Since PhantomJS < v1.9.8 uses SSLv3 by default, you should use TLSv1:

casperjs --ssl-protocol=tlsv1 yourScript.js

The catchall solution would be to use any for when newer PhantomJS versions come along with other SSL protocols. But this would make the POODLE vulnerability exploitable on sites which haven't yet disabled SSLv3.

casperjs --ssl-protocol=any yourScript.js

Alternative method: Update to PhantomJS 1.9.8 or higher. Note that updating to PhantomJS 1.9.8 leads to a new bug, which is especially annoying for CasperJS.

How to verify: Add a resource.error event handler like this at the beginning of your script:

casper.on(resource.error, function(resourceError){

    console.log('Unable to load resource (#' + resourceError.id + 'URL:' + resourceError.url + ')');

    console.log('Error code: ' + resourceError.errorCode + '. Description: ' + resourceError.errorString);

});

If it is indeed a problem with SSLv3 the error will be something like:

Error code: 6. Description: SSL handshake failed

As an aside, you also might want to run with the --ignore-ssl-errors=true commandline option, when there is something wrong with the certificate.