bash - Remove all occurrences of a duplicate line -


if want remove lines fields duplicated use sort -u -k n,n. keeps 1 occurrence. if want remove occurrences of duplicate there quick bash or awk way this?

eg have:

1   apple   30 2   banana   21 3   apple   9 4   mango   2 

i want:

2 banana   21 4 mango   2 

i presort , use hash in perl v. large files going slow.

try sort -k <your fields> | awk '{print $3, $1, $2}' | uniq -f2 -u | awk '{print $2, $3, $1}' remove lines duplicated (without keeping copies). if don't need last field, change first awk command cut -f 1-5 -d ' ', change -f2 in uniq -f1, , remove second awk command.


Comments

Popular posts from this blog

python - Scipy curvefit RuntimeError:Optimal parameters not found: Number of calls to function has reached maxfev = 1000 -

c# - How to add a new treeview at the selected node? -

java - netbeans "Please wait - classpath scanning in progress..." -