bash - Remove all occurrences of a duplicate line -
if want remove lines fields duplicated use sort -u -k n,n. keeps 1 occurrence. if want remove occurrences of duplicate there quick bash or awk way this?
eg have:
1 apple 30 2 banana 21 3 apple 9 4 mango 2
i want:
2 banana 21 4 mango 2
i presort , use hash in perl v. large files going slow.
try sort -k <your fields> | awk '{print $3, $1, $2}' | uniq -f2 -u | awk '{print $2, $3, $1}'
remove lines duplicated (without keeping copies). if don't need last field, change first awk
command cut -f 1-5 -d ' '
, change -f2
in uniq
-f1
, , remove second awk
command.
Comments
Post a Comment