Processing files larger than RAM in python
Posted on Tue 11 September 2012 in General
I got a log file of 131 GB from Mongo-DB that has been collected in last 6 months. So I just thought lets calculate the number of lines.
Here files size is greater than RAM, so after searching over internet; i got solution on http://stackoverflow.com.
I have modified it a little bit! But work is done.
It took around 75 min. and total number of lines was around 2.1 Billion.