Processing files larger than RAM in python

Posted on Tue 11 September 2012 in General

I got a log file of 131 GB from Mongo-DB that has been collected in last 6 months. So I just thought lets calculate the number of lines.

Here files size is greater than RAM, so after searching over internet; i got solution on http://stackoverflow.com.

I have modified it a little bit! But work is done.

It took around 75 min. and total number of lines was around 2.1 Billion.