Perl Code: Remove Duplicate Lines from a File - A quick and easy code snippet to remove duplicate lines from a file.

Quick Search:

Perl Code: Remove Duplicate Lines from a File

Jump to:

Category: >> Perl Code >> Remove Duplicate Lines from a File

<< last

next >>

Snippet Name: Remove Duplicate Lines from a File

Description: A quick and easy code snippet to remove duplicate lines from a file.

Comment: (none)

Language: PERL
Highlight Mode: PERL
Last Modified: March 05th, 2009

#!/usr/bin/perl
 
USE strict;
USE warnings;
 
MY $file = '/path/to/file.txt';
MY %seen = ();
{
   LOCAL @ARGV = ($file);
   LOCAL $^I = '.bac';
   WHILE(<>){
      $seen{$_}++;
      NEXT IF $seen{$_} > 1;
      PRINT;
   }
}
PRINT "finished processing file.";