Python 2.6 GC appears to cleanup objects, but memory is not released -


i have program written in python 2.6 creates large number of short lived instances (it classic producer-consumer problem). noticed memory usage reported top , pmap seems increase when these instances created , never goes down. concerned python module using might leaking memory isolated problem in code. proceeded reproduce in short example possible. came this:

class leaksmemory(list):     timesdelcalled = 0      def __del__(self):         leaksmemory.timesdelcalled +=1   def leaksomememory():     l = []     in range(0,500000):         ml = leaksmemory()         ml.append(float(i))         ml.append(float(i*2))         ml.append(float(i*3))         l.append(ml)  import gc import os   leaksomememory()  print("__del__ called " + str(leaksmemory.timesdelcalled) + " times") print(str(gc.collect())  +" objects collected") print("__del__ called " + str(leaksmemory.timesdelcalled) + " times") print(str(os.getpid()) + " : check memory usage pmap or top") 

if run 'python2.6 -i memoryleak.py' halt , can use pmap -x pid check memory usage. added del method verify gc occuring. not there in actual program , not appear make functional difference. each call leaksomememory() increases amount of memory consumed program. fear making simple error , references getting kept accident, cannot identify it.

python release objects, not release memory operating system immediately. instead, re-use same segments future allocations within same interpreter.

here's blog post issue: http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm

update: tested myself python 2.6.4 , didn't notice persistent increases in memory usage. invocations of leaksomememory() caused memory footprint of python process increase, , made decrease again. depends on how allocator re-using memory.


Comments

Popular posts from this blog

python - Scipy curvefit RuntimeError:Optimal parameters not found: Number of calls to function has reached maxfev = 1000 -

c# - How to add a new treeview at the selected node? -

java - netbeans "Please wait - classpath scanning in progress..." -