Monday, June 3, 2024
 Popular · Latest · Hot · Upcoming
40
rated 0 times [  45] [ 5]  / answers: 1 / hits: 34073  / 10 Years ago, sat, june 28, 2014, 12:00:00

I am trying to bulk insert documents into MongoDB (so bypassing Mongoose and using the native driver instead as Mongoose doesn't support bulk insert of an array of documents). The reason I'm doing this is to improve the speed of writing.



I am receiving the error RangeError: Maximum Call Stack Size Exceeded at console.log(err) in the code below:



function _fillResponses(globalSurvey, optionsToSelectRegular, optionsToSelectPiped, responseIds, callback) {
Response.find({'_id': {$in: responseIds}}).exec(function(err, responses) {
if (err) { return callback(err); }

if (globalSurvey.questions.length) {
responses.forEach(function(response) {
console.log(Filling response: + response._id);
response.answers = [];
globalAnswers = {};
globalSurvey.questions.forEach(function(question) {
ans = _getAnswer(question, optionsToSelectRegular, optionsToSelectPiped, response);
globalAnswers[question._id] = ans;
response.answers.push(ans);
});
});
Response.collection.insert(responses, function(err, responsesResult) {
console.log(err);
callback()
});
} else {
callback();
}
});
}


So similar to: https://stackoverflow.com/questions/24356859/mongoose-maximum-call-stack-size-exceeded



Perhaps it's something about the format of the responses array that Mongoose returns that means I can't directly insert using MongoDB natively? I've tried .toJSON() on each response but no luck.



I still get the error even with a very small amount of data but looping through and calling the Mongoose save on each document individually works fine.



EDIT: I think it is related to this issue: http://howtosjava.blogspot.com.au/2012/05/nodejs-mongoose-rangeerror-maximum-call.html



My schema for responses is:



var ResponseSchema = new Schema({
user: {
type: Schema.ObjectId,
ref: 'User'
},
randomUUID: String,
status: String,
submitted: Date,
initialEmailId: String,
survey: String,
answers: [AnswerSchema]
});


So, answers are a sub-document within responses. Not sure how to fix it though....


More From » node.js

 Answers
6

I was having this same issue and I started digging through the mongoose source code (version 3.8.14). Eventually it led me to this line within




  • mongoose/node_modules/mongodb/lib/mongodb/collection/core.js -> insert(...) -> insertWithWriteCommands(...) ->

  • mongoose/node_modules/mongodb/lib/mongodb/collection/batch/ordered.js -> bulk.insert(docs[i]) -> addToOperationsList(...) -> bson.calculateObjectSize(document, false);



    var bsonSize = bson.calculateObjectSize(document, false);




Apparently, this calls BSON.calculateObjectSize, which calls calculateObjectSize which then infinitely recurses. I wasn't able to dig that far in to what caused it, but figured that it may have something to do with the mongoose wrapper binding functions to the Schema. Since I was inserting raw data into mongoDB, once I decided to change the bulk insert in mongoose to a standard javascript object, the problem went away and bulk inserts happened correctly. You might be able to do something similar.



Essentially, my code went from



//EDIT: mongoose.model needs lowercase 'm' for getter method

var myModel = mongoose.model('MyCollection');
var toInsert = myModel();
var array = [toInsert];
myModel.collection.insert(array, {}, function(err, docs) {});


to



//EDIT: mongoose.model needs lowercase 'm' for getter method

var myModel = mongoose.model('MyCollection');
var toInsert = { //stuff in here
name: 'john',
date: new Date()
};
var array = [toInsert];
myModel.collection.insert(array, {}, function(err, docs) {});

[#70392] Thursday, June 26, 2014, 10 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
makiyac

Total Points: 470
Total Questions: 100
Total Answers: 115

Location: Botswana
Member since Sat, Jan 7, 2023
1 Year ago
makiyac questions
Sun, Jan 16, 22, 00:00, 2 Years ago
Thu, Dec 17, 20, 00:00, 4 Years ago
Fri, Dec 4, 20, 00:00, 4 Years ago
;