So i'm using this URL because i want so scrape html using axios and cheerio:
https://www.mgewholesale.com/ecommerce/Bags%20and%20Cases.cfm?cat_id=876
I tested a get request in postman and it works fine with status 200
using this code works with status 200 as well but response.data is empty
update, so with this code i got the actual response with the data object filled, but when im trying to access to response.data it shows me this error:
const axios = require('axios');
const cheerio = require('cheerio');
const https = require('https');
let fs = require('fs');
const httpsAgent = new https.Agent({ keepAlive: true });
axios
.get('https://www.mgewholesale.com/ecommerce/Bags%20and%20Cases.cfm', {
httpsAgent,
params: {
cat_id: '876',
},
headers: {
'Accept-Encoding': 'gzip, deflate, br',
},
//is the same as set the entire url
})
.then((res) => {
console.log(res.data)
//this triggers the error
// let status = res.status;
console.log(status);
//Status 200
console.log(response)
//This brings the entire response with data object filled
});
ERROR:
(node:9068) UnhandledPromiseRejectionWarning: Error: read ECONNRESET
at TLSWrap.onStreamRead (internal/stream_base_commons.js:205:27)
(node:9068) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:9068) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
I tried using the entire url and the url with its params it brings me empty data, but if i try with other url like: https://www.google.com it brings me the actual html.