Question

59

Error Executing javascript using HtmlUnit

rated 0 times [ 66] [ 7] / answers: 1 / hits: 8835 / 11 Years ago, tue, february 4, 2014, 12:00:00

I am trying to execute a JavaScript on the following URL: http://steamcommunity.com/id/bobcatchris/inventory#730. I can do it using google chrome by opening console by pressing ctrl+shift+j and pasting the following script:

var list = []; 

var size = Object.keys(g_ActiveInventory.rgInventory).size();    

var counter = 0;

while (counter < size) {

 list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name);

 counter +=1;

}

It returns: 150
Then in the next line, i write: >list
and it returns an array of length 150.

When i try to do it in HtmlUnit as:

 public static void main(String[] args) throws IOException {

        WebClient webClient=new WebClient(BrowserVersion.FIREFOX_17);

        HtmlPage page=webClient.getPage(http://steamcommunity.com/id/bobcatchris/inventory#730);

        String script=var list = [];n +

                n +

                n +

                var size = Object.keys(g_ActiveInventory.rgInventory).size();n +

                n +

                n +

                n +

                var counter = 0;n +

                n +

                while (counter < size) {n +

                 list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name);n +

                 counter +=1;n +

                };

        Object result = page.executeJavaScript(script).getJavaScriptResult();

        System.out.println(result);

    }

I am getting the following exception:

Exception in thread main ======= EXCEPTION START ========

EcmaError: lineNumber=[4] column=[0] lineSource=[<no source>] name=[TypeError] sourceName=[injected script] message=[TypeError: Expected argument of type object, but instead had type object (injected script#4)]

com.gargoylesoftware.htmlunit.ScriptException: TypeError: Expected argument of type object, but instead had type object (injected script#4)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:684)

    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:602)

    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:570)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:545)

    at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptIfPossible(HtmlPage.java:959)

    at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScript(HtmlPage.java:927)

    at Scraper.main(Scraper.java:32)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:601)

    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Caused by: net.sourceforge.htmlunit.corejs.javascript.EcmaError: TypeError: Expected argument of type object, but instead had type object (injected script#4)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3603)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3587)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3608)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError1(ScriptRuntime.java:3618)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.ensureScriptable(ScriptableObject.java:2095)

    at net.sourceforge.htmlunit.corejs.javascript.NativeObject.execIdCall(NativeObject.java:287)

    at net.sourceforge.htmlunit.corejs.javascript.IdFunctionObject.call(IdFunctionObject.java:89)

    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1531)

    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)

    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)

    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:405)

    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3031)

    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:561)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669)

    ... 12 more

Enclosed exception: 

net.sourceforge.htmlunit.corejs.javascript.EcmaError: TypeError: Expected argument of type object, but instead had type object (injected script#4)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3603)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3587)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3608)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.typeError1(ScriptRuntime.java:3618)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.ensureScriptable(ScriptableObject.java:2095)

    at net.sourceforge.htmlunit.corejs.javascript.NativeObject.execIdCall(NativeObject.java:287)

    at net.sourceforge.htmlunit.corejs.javascript.IdFunctionObject.call(IdFunctionObject.java:89)

    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1531)

    at script(injected script:4)

    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)

    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)

    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:405)

    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:309)

    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3031)

    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:561)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669)

    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:602)

    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:570)

    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:545)

    at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptIfPossible(HtmlPage.java:959)

    at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScript(HtmlPage.java:927)

    at Scraper.main(Scraper.java:32)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:601)

    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

======= EXCEPTION END ========

The line causeing exception is:

Object result = page.executeJavaScript(script).getJavaScriptResult();

So how should I execute this script from HtmlUnit as I do from google chrome console?

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

jarettajb

Add To Favorites

Follow

Total Points: 678

Total Questions: 94

Total Answers: 90

Location: Guernsey

Member since Tue, Jul 6, 2021

3 Years ago

jarettajb questions

1 Remove unused Javascript base.js (Youtube iframe api)

Thu, Dec 17, 20, 00:00, 3 Years ago

1 how to define variable google when using google one tap javascript API

Thu, Dec 10, 20, 00:00, 4 Years ago

1 How do I fix this issue: node_modules/expo/AppEntry.js: [BABEL]?

Mon, Nov 9, 20, 00:00, 4 Years ago

1 node js, read a specific column from csv file and convert it to an array to be used in other function

Tue, May 12, 20, 00:00, 4 Years ago

1 Check If Array Has Value in Angular

Tue, May 12, 20, 00:00, 4 Years ago

View All

answered 11 Years ago richardaydenc · Accepted Answer

These are your javascript script errors translated to Java exceptions by your framework.

That's because you have not explicitly set the setThrowExceptionOnScriptError option of your WebClient to false.

Unless there are absolutely no errors in the js, it is useful to always set this value as false unless it interferes with the results your are looking for.

Typically, these are my webclient settings when dealing with js and ajax through HtmlUnit.

final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17,

                PROXY_HOST, PROXY_PORT);



WebRequest request = new WebRequest(new URL(

                http://steamcommunity.com/id/bobcatchris/inventory#730));



webClient.getOptions().setThrowExceptionOnScriptError(false);

webClient.setJavaScriptTimeout(10000);

webClient.getOptions().setJavaScriptEnabled(true);

webClient.setAjaxController(new NicelyResynchronizingAjaxController());

webClient.getOptions().setTimeout(10000);



HtmlPage page = webClient.getPage(request);



          String script=var list = [];n +

                    n +

                    n +

                    var size = Object.keys(g_ActiveInventory.rgInventory).size();n +

                    n +

                    n +

                    n +

                    var counter = 0;n +

                    n +

                    while (counter < size) {n +

                     list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name);n +

                     counter +=1;n +

                    };

            Object result = page.executeJavaScript(script).getJavaScriptResult();

            System.out.println(result);

If I try your code, with the above settings I get 150.0 printed to the console which I assume is working as expected.

EDIT:

To traverse over the complete array list:

String script=var list = [];n +

                    n +

                    n +

                    var size = Object.keys(g_ActiveInventory.rgInventory).size();n +

                    n +

                    n +

                    n +

                    var counter = 0;n +

                    n +

                    while (counter < size) {n +

                     list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name);n +

                     counter +=1;n +

                    }

                    + list;



            Object result = page.executeJavaScript(script).getJavaScriptResult();

            if (result instanceof NativeArray) {

                for (Object obj : (NativeArray)result) {

                    System.out.println(obj);    

                }



            }

Above, I have change the js to include list as the return parameter and iterate over NativeArray to get each element.

Output:

P2000 | Scorpion (Factory New)

AK-47 | Black Laminate (Field-Tested)

★ StatTrak™ Karambit | Case Hardened (Minimal Wear)

CS:GO Case Key

CS:GO Case Key

You can read much on their ajax settings at their FAQs - here.