macro involving captured data from other files

[Valaya]

Hello,

I have seen similar things discussed in this forum but nothing addressing exactly what I'm trying to do...

I'm trying to replace certain text in one file with captured-to-the-clipboard text from another file but cannot get certain parts to function. Here's what I have so far:

InsertMode
ColumnModeOff
HexOff
UnixReOff
Find RegExp "xref callout="task"
Find RegExp "wpid="
StartSelect
Find RegExp ""*""
Copy
EndSelect
FindInFiles RegExp PreserveCase Recursive OutputWin Log "C:\XML\Maintenance Manuals\Files\" "*.xml" "wpno=right here I want the contents of the clipboard to be pasted in the find-in-files window after wpno"
Open "I want to open the file result from the above find-in-files"
Find RegExp "wpno=right here I want the contents of the clipboard to be pasted"
Find RegExp "<title>*</title>"
StartSelect
Copy
PreviousWindow
Find RegExp Up "xref callout="task *""
EndSelect
"xref callout=""
Paste
"""
Find RegExp Up "</title>"
Key DEL
Find RegExp Up "<title>"
Key DEL

The contents of the clipboard will be ever-changing as will the files where the find-in-files results will change.

Any help would be greatly appreciated as I'm about to pull out my hair here.

[Mofi]

You can use the special code ^c (see help of UE about Find) in search or replace string to search for clipboard content or replace with clipboard content. But you cannot use ^c in a regular expression search, except in an UltraEdit style regular expression search but the clipboard content is then interpreted as regular expression search string.

Well, your finds are not really regular expression finds so remove the RegExp option from the Find commands where not a real regular expression is needed and use instead of right here I want the contents of the clipboard to be pasted the code ^c.

For your open files problem look at the first macro at Send all open file names to tool.
This macro reformats a FindInFiles result edit window to a simple list of filenames with their paths. Replace the FindInFiles command in this macro with your FindInFiles command.

Instead of the commands SaveAs and UnixReOn at bottom of this macro use following code to open all files and close the modified result file (^s is also explained at the find help topic).

Top
Loop
IfEof
ExitLoop
EndIf
StartSelect
Key END
Open "^s"
NextWindow
Delete
EndSelect
Key DEL
EndLoop
CloseFile NoSave

[Valaya]

To clarify, I have 1 file open only, searching for certain text, copying to the clipboard, searching in a directory of files that are not open for certain text and the text in the clipboard, want to open that file, find some different text, copy that to the clipboard, close that file, go back to the originally open file, find some text, take the current clipboard text (from the now closed file), and insert it into the appropriate place in the open file.

Ok. While testing this I've come across a little problem. Sometimes, the text the macro is looking for is in the currently open file so I edited the macro to this but there must be something I'm overlooking. The macro is stopping if it doesn't find the text in the current open file when it should be continue to the FindInFiles portion of the macro.

Ok. Now I'm at the point of losing my mind.

I've decided to do 2 separate macros since I can't seem to make the above one work correctly in all instances.

One macro will check for text within the open file (and "seems" to be working in my test file):

InsertMode
ColumnModeOff
HexOff
UnixReOff
Find "xref callout="task"
Find "wpid="
StartSelect
Find RegExp ""*""
Copy
EndSelect
ToggleBookmark
Find RegExp "wpno=^c"
IfFound
Find RegExp "<title>*</title>"
StartSelect
Copy
EndSelect
PreviousBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL
ToggleBookmark
EndIf

the next macro will check for text in all files in the directory:

InsertMode
ColumnModeOff
HexOff
UnixReOff
Find "xref callout="task"
Find "wpid="
StartSelect
Find RegExp ""*""
Copy
EndSelect
FindInFiles PreserveCase Recursive Log "C:\XML\Maintenance Manuals\Files\" "*.xml" "wpno=^c"
Loop
Find RegExp Up "%Search complete, found *^p"
IfFound
Delete
ExitLoop
Else
NextWindow
EndIf
EndLoop
Top
Find "----------------------------------------^p"
Replace All ""
Find RegExp "%Find *^p"
Replace All ""
Find RegExp "%Found *^p"
Replace All ""
Find RegExp "/[0-9]+:*$"
Replace All ""
SortAsc IgnoreCase RemoveDup 1 -1 0 0 0 0 0 0
Top
Loop
IfEof
ExitLoop
EndIf
StartSelect
Key END
Open "^s"
NextWindow
Delete
EndSelect
Key DEL
EndLoop
CloseFile NoSave
NextWindow
Find RegExp "wpno=^c"
Find RegExp "<title>*</title>"
StartSelect
Copy
EndSelect
PreviousWindow
Find RegExp Up "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL

The problem is it also checks the open file as well. What can I add to the macro to make it exclude the currently open file from the FindInFiles find?

Please help if anyone can. I'm on a deadline and this is making me crazy.

[Mofi]

I was not online since I have written my first reply. Today I looked again here and I hope I have now the macro code you need. I put the macros again together into a single macro. But first I explain what you have done wrong.

You used ^c in a regular expression search. I have already written in my first post that the clipboard content will be interpreted as regular expression string in this case and not as normal string.

For example if you have wpid="Jim+Tom$" the macro command Find RegExp "wpno=^c" will search for a string which starts with wpno="Ji, followed by the character m or M 1 or more times (the + is a regular expression character), followed by the string Tom. This string must be at the end of a line (the $ is also a regular expression character).

I think, you don't want that. It was hard to find out what you want without example files. Next time post the interesting parts of the files, so we (I) can test the macro.

I have assumed that "xref callout=" is only once in the current open file and the <title></title> line is below that line. Your macro always searches down for the interesting part (except to remove the unwanted <title></title>) with no Top command.

Because you are a newbie I comment the macro code. Remove the green lines before copying the macro code to the edit macro dialog. First the macro searches for "xref callout="task" and if this string is not found, the macro execution is terminated because you run this macro on a wrong file or the cursor is not positioned correct.
InsertMode
ColumnModeOff
HexOff
UnixReOff
Find "xref callout="task"
IfNotFound
ExitMacro
EndIf
If found this string a bookmark is set at this line. Next the ID string from the wpid tag is copied to the user clipboard 8.
ToggleBookmark
Find "wpid="
Find RegExp ""*""
Clipboard 8
Copy
Now the macro checks, if this ID is present in the current file at the wpno tag. If it is, this file contains the title of interest which is copied to the user clipboard 8.
Find "wpno=^c"
IfFound
Find RegExp "<title>*</title>"
Copy
Else
The ID from the wpid tag is not present in the current file. Copy the full filename with path to user clipboard 9. This is part 1 of the new part you asked for in your last post. Then run the FindInFiles command to get the file which contains it.
Clipboard 9
CopyFilePath
Clipboard 8
FindInFiles PreserveCase Recursive Log "C:\XML\Maintenance Manuals\Files\" "*.xml" "wpno=^c"
As you already know from the find result everything except the filenames with path are removed to get a list of filenames.
Loop
Find RegExp Up "%Search complete, found *^p"
IfFound
Delete
ExitLoop
Else
NextWindow
EndIf
EndLoop
Top
Find "----------------------------------------^p"
Replace All ""
Find RegExp "%Find *^p"
Replace All ""
Find RegExp "%Found *^p"
Replace All ""
Find RegExp "/[0-9]+:*$"
Replace All ""
SortAsc IgnoreCase RemoveDup 1 -1 0 0 0 0 0 0
Here is part 2 for the new part you asked. The macro now searches in the list of filenames with path for the current open file and if it finds it, delete this filename from the list.
Top
Clipboard 9
Find "^c"
IfFound
Delete
Key DEL
Top
EndIf
Clipboard 9 is not used anymore and so we can clear this clipboard and switch back to clipboard 8 which still contains the ID of interest. If the edit result window is completely empty now, the ID was not found in any file and the macro must exit here with an appropriate information for you.
ClearClipboard
Clipboard 8
IfEof
Paste
ClearClipboard
Clipboard 0
" was not found in any file !!!"
ExitMacro
EndIf
Now all files found should be opened. Because you have written that only 1 file should be found, I removed the loop and the macro opens only the first file. The rest of the macro is not written for execution on several files. The edit result window is closed without saving it.
StartSelect
Key END
Open "^s"
NextWindow
CloseFile NoSave
NextWindow
Back at the opened file from the FindInFiles result the macro searches also for the ID and copies the title string into clipboard 8.
Find "wpno=^c"
Find RegExp "<title>*</title>"
Copy
PreviousWindow
EndIf
Back at the first or still only open file. Move the cursor to bookmarked line insert the title string here and reformat it.
GotoBookMark
ToggleBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL
Last clipboard 8 is cleared and the windows clipboard is selected again.
ClearClipboard
Clipboard 0

[Valaya]

Thank you so much for being so helpful and so patient with someone that is just learning UltraEdit macro syntax. I realllly appreciate it. I will be testing this in a few minutes and will let you know how it goes.'

Forgot to mention... yes, I remembered about removing RegExp from some of the finds but we lost power a few times while I was in the middle of all this last week and I must have forgotten to re-remove them when I lost my last save.

[Valaya]

Find "wpno=^c"
Find RegExp "<title>*</title>"
Copy
PreviousWindow
EndIf
Back at the first or still only open file. Move the cursor to bookmarked line insert the title string here and reformat it.
should this file be closing to make sure only the original open file is the only file open? I am asking because sometimes more than one file is staying open and then the wrong <title> tags are being deleted. For example, file1.xml is the original open file, then file10.xml was opened to copy a title but file10.xml did not close and then file3.xml opened for another title and now 3 files are open and, for some reason, the macro deleted title tags in the originally open file (file1.xml).
GotoBookMark
ToggleBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL

I have asked some questions about the above code and put them in orange. I would send a few files for you to test this but I don't know where to send them. Thank you once again for your help. Wanted to mention also... each file that I will be running this macro against may have as few as 1 instance of 'xref callout="task' or 1,000 instances of it. It varies from file to file. So I am trying to run it through the end of the file and that is when some of these weird things are happening (as mentioned in orange above).

Should "continue if a find with replace..." be checked for this macro? I thought it was checked, now it's not and I can't remember if it should be.

[Mofi]

To your orange question: Yes, in your situation you better replace the command PreviousWindow before the EndIf with CloseFile to close the file opened from the FindInFiles result.

And better add following bold commands to the section below:

IfEof
Paste
ClearClipboard
Clipboard 0
PreviousWindow
GotoBookmark
ToggleBookmark
Bottom
NextWindow
" was not found in any file !!!"
ExitMacro
EndIf

I have forgotten to remove the bookmark in the main open file before this macro exit.

You cannot send me the files because I have not made my email address visible. I have done this once and got 5 emails within the first our asking for help with UE. I'm not the IDM support, so I have made my email address immediatelly invisible again. Pack your files with ZIP or RAR and upload it anywhere and post the link to the archive here. You cannot upload the archive to this forum. The upload feature of the forum software is not enabled. But there many free web spaces available in internet.

Yes, "continue if a find with replace..." must be checked for this macro.

If you run this macro till EOF you should add the bold command to following section at top of the macro to make sure that the cursor will reach the end of the file when no "xref callout="task" string is found anymore.

Find "xref callout="task"
IfNotFound
Bottom
ExitMacro
EndIf

[Valaya]

Eeek... I can imagine the tons of emails you would get if your email was visible!

I've made the code changes... when it gets to an 'xref callout="task' that is not in any file, it goes to the "find results" file and the macro stops. I need it to continue checking the original open file for 'xref callout="task' as there will more than likely be more.

I have some webspace where I can upload a zip.

Thank you!!

files are here: aya.tecort.net/test_files_script.zip

It's just a random sampling of files, not all of them but should work ok for testing.

[Mofi]

That the macro stops when a wpid is not in any file was by design. I thought you would like to see that something is missing and add it manually in the opened XML file before run the macro again. But because you don't like it, I have modified the macro now a lot to create or append an error info line to a report file.

This report file is created or opened at "C:\XML\Maintenance Manuals\Files\Report.txt". Modify the name and the path 3 times in the macros if you want the report file in a different directory or with a different name. You will get a small file open error notification during macro execution when the report file does not exist. This is no problem. Press OK and the macro will create it. But I suggest to look at the report after the macros have finished on a XML file and then delete the content of the report but not the report file itself before executing the main macro on the next XML file.

Also I detected during execution of the macro on your test files, that although the cursor was set with command Bottom to the end of the file, the macro is still executed in an endless loop until I pressed the ESC key. I detected that macro command IfEof and also the option "Play macro to End of File" in the Macro Play dialog fails on your UTF-8 DOS files with UE v11.20a at Win98SE. I have to look into this UE bug issue deeper.

To solve this UE problem, I have created a second macro which is now the main macro you have to call and run 1 time. Do not run the main macro with option "Play macro to End of File". This main macro will also need the macro property Continue if a Find with Replace not found enabled.

The source code of the main macro:

InsertMode
ColumnModeOff
HexOff
UnixReOff
Top
Clipboard 8
Loop
Find "xref callout="task"
IfNotFound
ExitLoop
Else
PlayMacro 1 "Valaya"
EndIf
EndLoop
Clipboard 9
ClearClipboard
Clipboard 8
ClearClipboard
Clipboard 0
Open "C:\XML\Maintenance Manuals\Files\Report.txt"

You can see that the main macro calls in a loop the submacro with case-sensitive name "Valaya" until no "xref callout="task" is found anymore. Then it clears the used user clipboards, switches back to the windows clipboard and opens the report file. You will get a small error dialog, if there is no report file.

The source code of the macro Valaya:

ToggleBookmark
Find "wpid="
Find RegExp ""*""
Clipboard 8
Copy
Find "wpno=^c"
IfFound
Find RegExp "<title>*</title>"
Copy
Else
Clipboard 9
CopyFilePath
Clipboard 8
FindInFiles PreserveCase Recursive Log "C:\XML\Maintenance Manuals\Files\" "*.xml" "wpno=^c"
Loop
Find RegExp Up "%Search complete, found *^p"
IfFound
Delete
ExitLoop
Else
NextWindow
EndIf
EndLoop
Top
Find "----------------------------------------^p"
Replace All ""
Find RegExp "%Find *^p"
Replace All ""
Find RegExp "%Found *^p"
Replace All ""
Find RegExp "/[0-9]+:*$"
Replace All ""
SortAsc IgnoreCase RemoveDup 1 -1 0 0 0 0 0 0
Top
Clipboard 9
Find "^c"
IfFound
Delete
Key DEL
Top
EndIf
Clipboard 8
IfEof
"wpid="
Paste
Clipboard 9
" in file ""
Paste
"" was not found in any file !!!
"
SelectAll
Clipboard 8
Copy
CloseFile NoSave
GotoBookMark
ToggleBookmark
Find "xref callout="task"
EndSelect
Key RIGHT ARROW
Open "C:\XML\Maintenance Manuals\Files\Report.txt"
" "
Key BACKSPACE
IfNameIs "Report"
Bottom
Paste
CloseFile Save
Else
NewFile
Paste
SaveAs "C:\XML\Maintenance Manuals\Files\Report.txt"
CloseFile
EndIf
ExitMacro
EndIf
StartSelect
Key END
Open "^s"
NextWindow
CloseFile NoSave
NextWindow
Find "wpno=^c"
Find RegExp "<title>*</title>"
Copy
CloseFile
EndIf
GotoBookMark
ToggleBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL

Both macros must be saved in the main macro file. And never run the submacro Valaya manually. Always use the main macro. Add at the Loop command of the main macro the number 1 and run the main macro, if you want to run the submacro Valaya once.

[Valaya]

I'm using UE 10.20d if that makes any difference in what you have written... the reason I'm not concerned with missing wpno's (at the moment the macro is running) is that these files have been edited by users and it is entirely possible that the referenced wpno has been deleted. The report will be wonderful though as it will let me know what links I need to worry about fixing/removing.

Well, great. I just made a discovery of why some of these "finds" of wpno are not being found. Example, wpno="M00177..." is at the top of mim7_1.xml and the macro was starting the find from the cursor position in the original open file which was below the wpno. So, of course, it could not find M00177. A "TOP" needs to be inserted at some point when the search is being performed in the original open file.

This is getting so frustrating and you don't know how much I appreciate your help. If I could I would send you a cake in appreciation.

Edit: just ran the master macro and everything was working ok (except it not finding a wpno above where the cursor was as mentioned earlier in this post). Then, I got an "invalid path in mim2.xml..." error and mim7_1.xml closed on it's own. The find results file had this: "search complete, found 'wpno=C:\XML\Maintenance Manuals\Files\mim2.xml' 0 time(s). (0 files)."

[Valaya]

Still having the same 2 main issues stated above...
I'm so burned out on this macro that I'm putting it aside for a couple of hours and working on another one. The whole situation wouldn't be so bad if I didn't have to start running all my macros on these files sometime today because of the deadline.

[Mofi]

I'm getting so frustrating too. I spent now several hours on your macro and I have now given up this approach to fulfill your task. There are so many FindInFile and focus bugs in UE. You can read about many "FindInFiles issues fixed" in the readme.txt of v12.00. There are also some new bugs with v12.00 because the result edit window is now an Unicode file and executing the Valaya macro above with v12 produces a very bad file list because of the replaces and the sort on the result edit window. I have to write several bug reports to IDM for all the problems I found with UE v12.00.

Also I have the feeling that you do not carefully read my posts and have not understood what the macros do. To solve the problem with the wpno above the current line, a single Top at top of the Valaya macro is enough as shown below:

ToggleBookmark
Find "wpid="
Find RegExp ""*""
Clipboard 8
Copy
Top
Find "wpno=^c"

However, this is not important anymore because I have a new approach. Please carefully read the following and do it.

Backup all your XML files, open UltraEdit and open ALL your XML files in UltraEdit. Then run following macro. The macro properties are not important for this macro because it contains no Find command. This macro merges the content of all XML files with the file name+path together into a single Unicode file. After this macro has finished, you will have only one big file open which is not saved. You should save this large file after the macro execution. Then you can later use the File - Revert to Saved function if the second macro fails.

InsertMode
ColumnModeOff
HexOff
Clipboard 9
NewFile
ASCIIToUnicode
NextWindow
Loop
IfNameIs ""
ExitLoop
Else
CopyFilePath
Top
Key END
Paste
SelectAll
Copy
CloseFile NoSave
PreviousWindow
Paste
NextWindow
EndIf
EndLoop
ClearClipboard
Clipboard 0

The next macro now does what you really want on the single file which contains all XML contents. This macro needs Continue if a Find with Replace not found checked. I hope this macro really works because I canceled the macro execution after a few minutes. There are thousands of searches on the single file which takes a long time.

But before this macro really starts, it checks if there are lines containing 2 "xref callout="Task" strings. If it finds such a line the macro exits before really starting because this line must be splitted manually into 2 or more lines or the macro will run in an endless loop. The file mim7_1.XML contains such a line with 2 "xref callout="Task" strings. This problem was also not detected by the previous macro solutions.

InsertMode
ColumnModeOff
HexOff
UnixReOff
Top
Find RegExp "xref callout="Task*xref callout="Task"
IfFound
ExitMacro
EndIf
Top
Clipboard 9
Loop
Find "xref callout="task"
IfNotFound
ExitLoop
Else
ToggleBookmark
Find "wpid="
Find RegExp ""*""
Copy
Top
Find "wpno=^c"
IfFound
Find RegExp "<title>*</title>"
Copy
GotoBookMark
ToggleBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL
Else
GotoBookMark
ToggleBookmark
Find "xref callout="task"
EndSelect
Key LEFT ARROW
EndIf
EndIf
EndLoop
ClearClipboard
Clipboard 0

The last macro needs also Continue if a Find with Replace not found checked. This macro splits the single file up to the existing XML files and overwrites the content of the existing XML files with the new content from the large single file.

InsertMode
ColumnModeOff
HexOff
Top
Loop
Find "<?xml version="1.0" encoding="utf-8"?>"
IfNotFound
ExitLoop
EndIf
Key LEFT ARROW
Key RIGHT ARROW
StartSelect
Key END
Clipboard 9
Cut
Find "<?xml version="1.0" encoding="utf-8"?>"
IfFound
Key HOME
SelectToTop
Else
SelectAll
EndIf
Clipboard 8
Cut
Clipboard 9
Open "^c"
SelectAll
Clipboard 8
Paste
EndSelect
Save
CloseFile
EndLoop
Clipboard 8
ClearClipboard
Clipboard 9
ClearClipboard
Clipboard 0

It's possible to merge all 3 macros to a single macro with macro property Continue if a Find with Replace not found checked. But for testing and because of the 2 "xref callout="Task" strings in a single line problem it is better to let it splitted into 3 macros.

Note: No report is generated for not found wpno anymore.

[Valaya]

No, I do read your posts carefully and even mentioned that the Top was needed but there have been so many changes and iterations of this macro (not to mention the other 20 macros I'm working on as well) that I could have overlooked something. I'm sorry. I really do appreciate everything you have done and I feel very bad for even involving anyone in such an extensive macro.

I have about 600 files so I don't know if opening them all will be an option however, I had at one point an early stage of this whole thing that worked but had to be run 1 run at a time. I could go back to that. I'll just have to test.

Thank you again and again, my sincerest apologies for using so much of your time.

[Valaya]

For anyone that may ever need to do anything similar, here is the code I ended up using for this issue. I'm opening each file individually that I need to run the macro against. Ridiculous as it may be, I'm running it 1 at a time (not play to end of file) and have done many tests and it's working. The macro deletes the last <title> and </title> tags in the file in error but I'm undoing those 2 and continuing on to the next file. Even though it's not an ideal macro, it will save the other people in my department from having to manually find all this stuff, copy, paste, etc.

InsertMode
ColumnModeOff
HexOff
UnixReOff
ClearClipboard
Find "xref callout="task"
Find "wpid="
StartSelect
Find RegExp ""*""
Copy
EndSelect
ToggleBookmark
Top
Find RegExp "wpno=^c"
IfFound
Find RegExp "<title>*</title>"
StartSelect
Copy
EndSelect
PreviousBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL
ToggleBookmark
Else
FindInFiles PreserveCase Recursive Log "C:\XML\Maintenance Manuals\Files\" "*.xml" "wpno=^c"
Loop
Find RegExp Up "%Search complete, found *^p"
IfFound
Delete
ExitLoop
Else
NextWindow
EndIf
EndLoop
Top
Find "----------------------------------------^p"
Replace All ""
Find RegExp "%Find *^p"
Replace All ""
Find RegExp "%Found *^p"
Replace All ""
Find RegExp "/[0-9]+:*$"
Replace All ""
SortAsc IgnoreCase RemoveDup 1 -1 0 0 0 0 0 0
Top
Loop
IfEof
ExitLoop
EndIf
StartSelect
Key END
Open "^s"
NextWindow
Delete
EndSelect
Key DEL
EndLoop
CloseFile NoSave
NextWindow
Find RegExp "wpno=^c"
Find RegExp "<title>*</title>"
StartSelect
Copy
EndSelect
PreviousWindow
PreviousBookmark
Find RegExp "xref callout="task *""
"xref callout=""
Paste
"""
Find Up "</title>"
Key DEL
Find Up "<title>"
Key DEL
ToggleBookmark
NextWindow
CloseFile
EndIf