Monday, June 3, 2024
 Popular · Latest · Hot · Upcoming
22
rated 0 times [  26] [ 4]  / answers: 1 / hits: 70994  / 12 Years ago, mon, september 17, 2012, 12:00:00

I have large text files, which range between 30MB and 10GB. How can I count the number of lines in a file using Node.js?



I have these limitations:




  • The entire file does not need to be written to memory

  • A child process is not required to perform the task


More From » node.js

 Answers
49

solution without using wc:



var i;
var count = 0;
require('fs').createReadStream(process.argv[2])
.on('data', function(chunk) {
for (i=0; i < chunk.length; ++i)
if (chunk[i] == 10) count++;
})
.on('end', function() {
console.log(count);
});


it's slower, but not that much you might expect - 0.6s for 140M+ file including node.js loading & startup time



>time node countlines.js video.mp4 
619643

real 0m0.614s
user 0m0.489s
sys 0m0.132s

>time wc -l video.mp4
619643 video.mp4
real 0m0.133s
user 0m0.108s
sys 0m0.024s

>wc -c video.mp4
144681406 video.mp4

[#83058] Friday, September 14, 2012, 12 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
katianatasham

Total Points: 293
Total Questions: 110
Total Answers: 103

Location: Saint Helena
Member since Mon, Jun 28, 2021
3 Years ago
katianatasham questions
Tue, Jul 20, 21, 00:00, 3 Years ago
Thu, Mar 18, 21, 00:00, 3 Years ago
Wed, Nov 25, 20, 00:00, 4 Years ago
Wed, Jun 24, 20, 00:00, 4 Years ago
Fri, May 15, 20, 00:00, 4 Years ago
;