Python Slice Assignment Memory Usage -


i read in comment here on stack overflow more memory efficient slice assignment when changing lists. example,

a[:] = [i + 6 in a] 

should more memory efficient than

a = [i + 6 in a] 

because former replaces elements in existing list, while latter creates new list , rebinds a new list, leaving old a in memory until can garbage collected. benchmarking 2 speed, latter quicker:

$ python -mtimeit -s 'a = [1, 2, 3]' 'a[:] = [i + 6 in a]' 1000000 loops, best of 3: 1.53 usec per loop $ python -mtimeit -s 'a = [1, 2, 3]' 'a = [i + 6 in a]' 1000000 loops, best of 3: 1.37 usec per loop 

that i'd expect, rebinding variable should faster replacing elements in list. however, can't find official documentation supports memory usage claim, , i'm not sure how benchmark that.

on face of it, memory usage claim makes sense me. however, giving more thought, expect in former method, interpreter create new list list comprehension , then copy values list a, leaving anonymous list in floating around until garbage collected. if that's case, former method use same amount of memory while being slower.

can show definitively (with benchmark or official documentation) of 2 methods more memory efficient/which preferred method?

thanks in advance.

the line

a[:] = [i + 6 in a] 

would not save memory. python evaluate right hand side first, stated in language documentation:

an assignment statement evaluates expression list (remember can single expression or comma-separated list, latter yielding tuple) , assigns single resulting object each of target lists, left right.

in case @ hand, single resulting object new list, , single target in target list a[:].

we replace list comprehension generator expression:

a[:] = (i + 6 in a) 

now, right hand side evaluates generator instead of list. benchmarking shows still slower naive

a = [i + 6 in a] 

so generator expression save memory? @ first glance, might think does. delving in source code of function list_ass_slice() shows not. line

v_as_sf = pysequence_fast(v, "can assign iterable"); 

uses pysequence_fast() convert iterable (in case generator) tuple first, copied old list. tuple uses same amount of memory list, using generator expression same using list comprehension in case. during last copy, items of original list reused.

the moral seems simplest approach best 1 in regard.


Comments

Popular posts from this blog

python - Scipy curvefit RuntimeError:Optimal parameters not found: Number of calls to function has reached maxfev = 1000 -

c# - How to add a new treeview at the selected node? -

java - netbeans "Please wait - classpath scanning in progress..." -