Remove Consecutive Duplicate Lines

Find, replace, find in files, replace in files, regular expressions

Remove Consecutive Duplicate Lines

Postby peterob » Wed Feb 15, 2006 3:25 pm

I need to remove duplicates only from consecutive lines, preferably not
through a macro because it's a few thousand files.

Basically I need to change this:

a = a * 2
a = a * 2
a = a + b
a = a * 2

to this:

a = a * 2
a = a + b
a = a * 2

I tried this as a Unix style find/replace:

Find: ^(.*)(\p\1)+$
Replace: \1

which should work imo, but it returns no matches.

Please let me know if I'm doing anything wrong (I enabled Unix style find).
User avatar
peterob
Newbie
 
Posts: 1
Joined: Wed Feb 15, 2006 12:00 am

Re: Remove Consecutive Duplicate Lines

Postby Mofi » Thu Feb 16, 2006 7:37 am

The Unix regular expression engine of UltraEdit does not support back referencing in search string. This is only supported by the Perl regular expression engine introduced with UltraEdit v12.00 on 2006-03-15. With the Perl regular expression engine a Replace All with the search string ^(.*)(?:\r\n\1)+$ and the replace string \1 would do the job.

But without the power of the Perl regular expression engine a macro must be used for this job. The character » is a character which should not exist anywhere in your file. You could use any other character if necessary or a whole string with a little adaption (more KEY DEL according to length of the string). Consecutive blank lines without trailing spaces are not deleted by this macro (IfCharIs 13 == IfCharIs CR).

And don't forget to enable the macro property Continue if a Find with Replace not found which is required for this macro.

InsertMode
ColumnModeOff
HexOff
Bottom
IfColNum 1
Else
"
"
EndIf
Top
Clipboard 9
StartSelect
Key END
Copy
EndSelect
Key DOWN ARROW
Key HOME
IfColNumGt 1
Key HOME
EndIf
Loop
IfEof
ExitLoop
EndIf
IfCharIs 13
Key DOWN ARROW
Else
"»"
Key HOME
Find "»^c^p"
IfFound
Delete
Else
Key DEL
StartSelect
Key END
Copy
EndSelect
Key DOWN ARROW
Key HOME
IfColNumGt 1
Key HOME
EndIf
EndIf
EndIf
EndLoop
ClearClipboard
Clipboard 0
Top
User avatar
Mofi
Grand Master
Grand Master
 
Posts: 4064
Joined: Thu Jul 29, 2004 11:00 pm
Location: Vienna


Return to Find/Replace/Regular Expressions

cron