How to delete all lines NOT containing specific word or string or expression?

[marino]

How to delete all lines not containing a specific word or a specific string or where a specific regular expression cannot find a string in the line?

Unix style if possible.
Thanks

[mrainey56]

This has been discussed before - Find one word and other not present in the line.

[marino]

Thanks

[EllE]

And where is code for macro ?

I need example delete all lines which do not contain [images]

Code: Select all

[images]http://...[/images]
Reply: [quotes]bla bla[/quotes]
[images]http://...[/images]
[quotes]bla bla[/quotes]

= in this example I need to remove line 2 and 4

[Mofi]

The second macro I have posted yesterday at Delete all lines not starting with a special phrase will also work for your example without modification.

With a little modification of the first macro posted at the forum article above it will work also if [images] is not at start of the line.

InsertMode
ColumnModeOff
HexOff
Clipboard 9
ClearClipboard
Bottom
IfColNum 1
Else
"
"
EndIf
Top
Loop
Find "[images]"
IfFound
SelectLine
CopyAppend
Else
ExitLoop
EndIf
EndLoop
SelectAll
Paste
Top
ClearClipboard
Clipboard 0

[EllE]

Thanks

[CBruce]

I know how to use the Find/Replace with %*string*^p to find or replace entire lines (replace with ""), but how would I go about removing every line that doesn't contain a certain string?

I'm trying to parse through multiple chat logs and only keep the lines with a very particular system message.

[Mofi]

A macro solution is posted above.

A quick manual solution is to simply search for the string of interest and check the advanced search option List Lines Containing String before executing the search. Then press button Clipboard to copy the lines of interest to the clipboard and close the dialog. Open a new file with Ctrl+N or select everything in current file with Ctrl+A and paste the clipboard content with Ctrl+V.

You can also use Find In Files to get into an edit window or the output window all lines of interest of all your logs.

[pietzcker]

A Perl regular expression (courtesy of http://www.regular-expressions.info/completelines.html) that does the job is:

^(?:(?!%REGEX%).)*$\r\n

Substitute %REGEX% with the string you want to find.

Replace all with nothing.

[CBruce]

A quick manual solution is to simply search for the string of interest and check the advanced search option List Lines Containing String before executing the search. Then press button Clipboard to copy the lines of interest to the clipboard and close the dialog. Open a new file with Ctrl+N or select everything in current file with Ctrl+A and paste the clipboard content with Ctrl+V.

This worked perfect for what I needed. Thanks.

A Perl regular expression (courtesy of http://www.regular-expressions.info/completelines.html) that does the job is:

^(?:(?!%REGEX%).)*$\r\n

Replace all with nothing.

Also works. Two awesome ways to parse out the information I want. Thanks guys.

[M@Cr0MaN]

I've been thinking about this problem for a couple of weeks now and the other day, the solution finally came to me!!! GrandMaster Mofi, your macro is the best and fastest solution if the number of lines containing the keyword, phrase or symbol is less than the number of lines that are missing the keyword, phrase or symbol. However, in most cases I've dealt with so far, unfortunately the lines containing the keyword, phrase or symbol far outnumber the lines that are missing the keyword, phrase or symbol.

That's the reason why I've been trying to figure out how to target and remove lines that do not have a colon. In my example, I had thousands of lines that had colons and less than 20 that were missing colons. But the big problem for me was that there were no specific symbols for me to search for.

For example, for the problem in this thread: Find one word and other not present in the line

If I was making a macro, I would include find and replace all td tags with rt class and then Find td to get the desired result, but I've been using a wordlist for input and there is no specific symbol or word that I can use so then one day, all of a sudden, I had the thought, why not make my own symbol by using column insert, and then use a find and replace RegExp to remove that particular symbol from the lines that do contain colons, then take and slightly modify the code that you posted above and cut and append all lines with that particular symbol!!!?

So this is the macro I've created. It's designed to cut and append all lines that don't have a colon in them and when it's done running, just in case you wanted to use the data that was cut out, I left it on clipboard 9 and did NOT clear the clipboard so if you want to use that data, you simply paste and presto, all the words that were cut out in one neat list! So I hope everyone interested in this subject, will feel free to try my macro and if you want to use it for something other than cutting out lines without colons, all you have to do is go to:

Find RegExp "½^(*^):"
Replace All "^1:"

and replace the colons with something else like the word "images" And it should be the fastest method for removing the lines that don't contain this word, provided that the number of lines that are missing this word is less than the number of lines containing the word!

To be more specific, change it to:

Find RegExp "½^(*^)images"
Replace All "^1images"

I've tested this on my UE and it's working great!!! My version of UE is 14.10.0.1024

If this macro works for you, please let me know and even more importantly, if it does NOT work for you, please be sure to post a message in this thread. Thanks for listening!

MM

Edit: Macro code replaced by a new version, see my next post. The example was also removed.

[Mofi]

To delete all lines not containing a colon, the best method is to use the perl regular expression replace all as posted by pietzcker.

InsertMode
ColumnModeOff
HexOff
PerlReOn
Bottom
IfColNumGt 1
InsertLine
IfColNumGt 1
DeleteToStartofLine
EndIf
EndIf
Top
Find RegExp "^(?:(?!:).)*$\r\n"
Replace All ""

Of course that macro does not record the deleted lines. But it is very easy to create a copy of the source file and replace in the copy all lines containing a colon to get the opposite result.

And users of UE v16.00 or later don't need a macro for that job at all. Simply running non regular expression Find searching for : executed with advanced option button Show Lines (show just lines with a colon, hide all other), or Hide Lines (hide all lines with a colon, only show all other) and using Edit - Delete - Delete All Hidden Lines produce the same result without the need to use a macro which must be modified for every search, or using a Perl regular expression replace without knowing how it works.

Your macro could be improved a little bit, see below (removed later). But macro above is much faster.

[M@Cr0MaN]

First of all, I apologize for taking so long to reply. Real life got very hectic all of a sudden and once again, you gave me quite a lot to think about, Mofi. Thank you very much for reposting pietzckers' Perl code for my convenience, and of course, thank you pietzcker for posting it originally. It's AMAZING!!!!!!!!!! I tested it on 10,000 lines and it was finished running in less than a second!!!!!!!!!!!!!!! Mofi, I took your advice and hopefully the code that I posted in the 1st code box will accomplish what you suggested. It seems to be working on my end!

It's lightning fast and it is superior to my original macro. However, my macro has one advantage to it. If you want to change your search from lines that don't contain colons to lines that don't contain something else like a @ symbol, all you have to do is change the code

Find RegExp "½^(*^):"
Replace All "^1:"

to

Find RegExp "½^(*^)@"
Replace All "^1@"

I tried that with the Perl Expression and got a bunch of errors and it was still trying to find lines without colons. That's why I have continued work on the original macro I made and posted it in the 2nd code box.

I thought about it and decided that my original macro would not work well on a doc that had a lot of blank lines so I added some code to remove the blank lines first. I also noticed that when I did modify the code to find lines not containing an @ symbol, after 5000 lines it seemed to miss some of those lines so I added some code that would perform the operation one more time in order to take care of the lines that were missed the first time.

I really appreciate you improving my macro! I've never used Perl expressions until now so in my opinion, your biggest improvement was adding the UnixReOff. After running pietzckers' Perl macro, a lot of the macro's I've made would not work anymore , so I guess I need to add that UnixReOff to all of my macros, just in case I or someone else was using something like Perl Expressions previously.

I also noticed you had a 0 after the word Loop. Why did you do that? I pasted your code in my UE and after I saved it, the 0 just dissappeared. I ran your version of the macro and worked well in my UE with one exception.

I know you're probably wondering why it was, everytime I used NewFile, I followed it with PreviousDocument and then NextDocument. Well the reason is, that little bit of extra code appears to fix the double click bug that my version of UE has. If I don't include it, I'm almost assured that I will have problems if I double click the macro. I was thrilled when I made that discovery a couple of weeks ago!

I also noticed you added ColumnModeOn over ColumnInsert. Could this be another reason why a lot of my macros work for me but not other people? I definitely want to make my macro's compatible with most versions of UE and all versions if possible so I appreciate the advice!

I'd have to say my most embarassing mistake was when I used:
Find RegExp "*½*^p"

I just remember thinking that part would be easy to code and then I ended up pulling my hair out trying to find the proper code for it and by the time I was finally finished, I was just thrilled that it was working and I wasn't about to mess with it anymore!

Well I think that's about it. I would very much appreciate it if you would test both macros, and let me know what you think. Thanks again for all your help, Mofi. I look forward to your reply!
MM


pietzckers' perl macro, slightly modified

Code: Select all

InsertMode
ColumnModeOff
HexOff
SelectAll
Copy
NewFile
PreviousDocument
NextDocument
Paste
EndSelect
UnixReOff
Top
Find RegExp "%
"
Replace All ""
Bottom
Key END
IfColNum 1
Key BACKSPACE
EndIf
Bottom
IfColNumGt 1
InsertLine
IfColNumGt 1
DeleteToStartofLine
EndIf
EndIf
Top
Find RegExp "%*:*
"
Replace All ""
SelectAll
Clipboard 8
Copy
PreviousDocument
PerlReOn
Bottom
IfColNumGt 1
InsertLine
IfColNumGt 1
DeleteToStartofLine
EndIf
EndIf
Top
Find RegExp "^(?:(?!:).)*$\r\n"
Replace All ""
NextDocument
CloseFile NoSave

My original macro, slightly modified

Code: Select all

InsertMode
ColumnModeOff
HexOff
UnixReOff
Top
Find RegExp "%
"
Replace All ""
Bottom
Key END
IfColNum 1
Key BACKSPACE
EndIf
Bottom
IfColNumGt 1
InsertLine
IfColNumGt 1
DeleteToStartofLine
EndIf
EndIf
Top
ColumnModeOn
ColumnInsert "½"
ColumnModeOff
Find RegExp "½^(*^):"
Replace All "^1:"
TrimTrailingSpaces
Top
Clipboard 9
ClearClipboard
Loop
Find RegExp "%½*^p"
IfFound
SelectLine
CutAppend
Else
ExitLoop
EndIf
EndLoop
NewFile
PreviousDocument
NextDocument
Paste
Top
Find "½"
Replace All ""
SelectAll
Cut
CloseFile NoSave
Else
Bottom
NextDocument
CloseFile NoSave
EndIf
Top
Clipboard 8
ClearClipboard
Loop
Find RegExp "%½*^p"
IfFound
SelectLine
CutAppend
Else
ExitLoop
EndIf
EndLoop
NewFile
PreviousDocument
NextDocument
Clipboard 8
Paste
Top
"
"
Top
Clipboard 9
Paste
Top
Find "½"
Replace All ""
SelectAll
Cut
CloseFile NoSave

[Mofi]

I tried that with the Perl Expression and got a bunch of errors and it was still trying to find lines without colons.

You have replaced obviously both colons in the Perl regular expression string by @ instead of just the second formatted bold red in my previous post. The first colon has a special meaning in combination with the question mark. ?: immediately after and opening round bracket means: Do not mark (tag) the string found by the expression inside the round brackets. If you have replaced that colon also by @, you have created an invalid Perl regular expression.

Find RegExp "^(?:(?![@).)*$\r\n"

works fine for deleting all (DOS terminated) lines (including blank lines) not containing character @.

After running pietzckers' Perl macro, a lot of the macro's I've made would not work anymore.

That's the reason why all my macros in Mofis_Macro_Examples.uem contain at end (or above ExitMacro command) the comment lines.

Code: Select all

// Activate UnixReOn or PerlReOn (v12+ of UE) here at the end of the macro
// if you do not use UltraEdit style regular expressions by default - see
// search configuration. Macro command UnixReOff sets the regular expression
// option to UltraEdit style.
//  UnixReOn
//  PerlReOn

Users of newer versions of UltraEdit than you don't have to take care about setting regular expression engine on macro exit because UltraEdit remembers current active regular expression engine on start of a macro and reselects this engine automatically on exit of a macro.

I also noticed you had a 0 after the word Loop. Why did you do that?

From help of UltraEdit about macro command Loop: A value of 0 indicates Loop forever.

Your version of UltraEdit always omits loop count value zero on decompilation of a macro into the macro editor dialog. Newer versions of UltraEdit always decompile infinite loops with value zero. On compilation all versions of UltraEdit support infinite loops breaked by the commands ExitLoop or ExitMacro either with value zero after command Loop or without any value.

There is no difference whether in compiled macro code nor on compilation of a macro source code in macro editor dialog to the binary macro code in any version of UltraEdit. It simply does not matter if the text version of a macro code contains Loop 0 or just Loop without a loop count value. Older versions of UltraEdit just decompile the binary macro code of an infinite loop to just Loop while newer versions of UltraEdit decompile the same binary macro code to Loop 0. But in the reverse direction for compilation of text source to binary code it does not make a difference in any version of UltraEdit.

I also noticed you added ColumnModeOn over ColumnInsert

As you can see in the Column menu the menu item Insert/Fill Columns is disabled if Column Mode is not active as most other commands in the column menu. Therefore no special column command in a macro should be used without making sure that column mode is turned on during macro execution before using one of the special column commands.

From your macro with the Perl regular expression:

SelectAll
Copy
NewFile
PreviousDocument
NextDocument
Paste
EndSelect

EndSelect after pasting something into a new file is unnecessary. Pasting something always results in disabling the selection mode. It looks like you have recorded that macro. Well, you see here that a paste results in disabling the selection mode. Better would be to use EndSelect command at top of the macro after the commands SelectAll and Copy before opening the new file. That would stop the selection mode in the right file where this mode was enabled by command SelectAll.

[M@Cr0MaN]

You have obviously replace both colons in the Perl regular expression string by @ instead of just the second formatted bold red in my previous post.

You're absolutely right, mofi........That's exactly what I did!

Find RegExp "^(?:(?![@).)*$\r\n"

Right again.......this works GREAT!!! It will end up being a big time saver for quite a few of my slow macros!!! Many thanks for explaining this to me and giving me this sample! I feel kind of silly since I didn't continue to experiment and try just changing only one of the colons to @. Or actually I think I did try that but I changed the wrong one!

That's the reason why all my macros in Mofis_Macro_Examples.uem contain at end (or above ExitMacro command) the comment lines...

Oh, that's good to know. Thanks for the info, Mofi!

From help of UltraEdit about macro command Loop: A value of 0 indicates Loop forever...

Oh, I see. That's kind of a relief to know.

As you can see in the Column menu the menu item Insert/Fill Columns is disabled if Column Mode is not active ...

More good info! Thanks for that! :0 ) I guess it's easy to neglect stuff like this when it still works for you. But like I already mentioned, I want to make my macros compatible with all versions of UE, so I'll try to make sure I add column mode on to all the macros I have that use column insert, from now on! I think I already did that with most of my macros, but when I noticed it didn't really make any difference for my version of UE, I guess I got lazy. I know one thing for certain, if you forget to use columnModeOff, it can really screw up your Macro! I've made that mistake plenty of times in the past! but then again I've learned a lot through trial and error!

EndSelect after pasting something into a new file is unnecessary...

This is good to know for certain. I have been unsure about this till now because of the double click bug that I have. It would automatically select lines even if I had used the command EndSelect and I guess that made me a little paranoid about making sure there was no selection after the paste command. But now that I know for certain that this is unneccessary, I won't use that command after the paste command anymore. And I totally agree with you about where I should have used EndSelect! Thanks Mofi, for taking the time to explain all this to me.......You're lessons are much appreciated!!!

MM