Question

39

Can someone explain an ENOBUFS error?

rated 0 times [ 43] [ 4] / answers: 1 / hits: 30255 / 7 Years ago, wed, june 21, 2017, 12:00:00

I'm making a bunch calls to a database that contains a large amount of data on a Windows 7 64 bit OS. As the calls are queuing up I get the error (for ever HTTP call after the first error):

Error: connect ENOBUFS *omitted* - Local (undefined:undefined)

From my google searching I've learned that this error means that my buffer has grown too large and my system's memory can no longer handle the buffer's size.

But I don't really understand what this means. I'm using node.js to with an HTTPS library to handle my requests. When the requests are getting queued and the sockets are opening is the buffer's size allocated in RAM? What will allow the buffer to expand to a greater size? Is this simply a hardware limitation?

I've also read that some OS are able to handle the size of the buffer better than other OS's. Is this the case? If so which OS would be better suited for running a node script that needs to fetch a lot of data via HTTPS requests?

Here's how I'm doing my requests.

for (let j = 0; j < dataQueries; j++) {

 getData(function())

}



function getData(callback){

 axios.get(url, config)

   .then((res) => {

      // parse res 

      callback(parsedRes(res))

   }).catch(function(err) {

      console.log(Spooky problem alert! :  + err);

   })

}

I've omitted some code for brevity, but this is generally how I'm doing my requests. I have a for loop that for every iteration launches a GET request via axios.

I know there is an axios.all command that is used for storing the promise the axios.HTTPMethod gives you, but I saw no change in my code when I set it up to store promises and then iterate over the promises via axios.all

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

kaylinshayd

Add To Favorites

Follow

Total Points: 443

Total Questions: 104

Total Answers: 111

Location: Nauru

Member since Wed, Mar 29, 2023

1 Year ago

answered 7 Years ago chase · Accepted Answer

Thanks @Jonasw for your help, but there is a very simple solution to this problem.
I used the small library throttled-queue to get the job done. (If you look at the source code it would be pretty easy to implement your own queue based on this package.

My code changed to:

const throttledQueue = require('throttled-queue')



let throttle = throttledQueue(15, 1000) // 15 times per second



for (let j = 0; j < dataQueries; j++) {

 throttle(function(){

   getData(function(res){

     // do parsing

   })

 }



}



function getData(callback){

 axios.get(url, config)

   .then((res) => {

      // parse res 

      callback(parsedRes(res))

   }).catch(function(err) {

      console.log(Spooky problem alert! :  + err);

   })

}