TestDuplicate user tool

[rhapdog]

Test Duplicate version 1.00

Download Here

For help on usage, type TestDuplicate.exe without any parameters, or with -h, -help, /h, /help, or --help.

This program is designed to give a report of "words" that are duplicated in multiple color groups in an IDM wordfile for UltraEdit or UEStudio. It is intended to be configured as a user tool from within UltraEdit in the Advanced menu. This is also intended to replace the TestForDuplicate macro included in Mofi's SyntaxTools macro, with a much faster operation time.

In the Tool Configuration Dialog, Insert a new user tool, and give it the name "TestDuplicate". Give the Command line as the full path to the tool with executable name, followed by "%f", and include the double quotes around %f. On the Options tab, check DOS program and "Save active file". On the Output tab, check "Capture output" and "Output to list box". Also check "No replace" in the Replace selected text with section.

With a wordfile that you wish to test opened as the active file, launch the tool. It will give a report of any duplicates found in the Output Window, with full file name followed by /line#/column# so that the cursor can be positioned at the word in question when you double click on the line in the report. This is quite handy, as it makes editing the wordfile in question to remove the duplicates much easier.

The one thing it doesn't do, which Mofi's macro also does not do, is to determine if there is a "conflict" between substrings when 2 substrings are different but may overlap. And example would be:

Code: Select all

/C1
** abcd
/C2
** abcde


The "abcde" in /C2 is superseded by the "abcd" in /C1, but neither this tool nor Mofi's macro will check for it.

Discussion
I would like input from anyone who is interested in this tool on whether or not to expand the capabilities of this tool any further.

Possibilities:
1. Automatic removal of identified duplicates via command line option. Now implemented with the -fix option
2. Add the extra coding for "intelligent" checking of substrings to catch substring conflicts as in the above example.

The only way it will modify your files is if you use the -fix option, as described in the program's help.

[rhapdog]

After further testing, the "location" listed for the duplicate in a group is not always accurate. For example:

Code: Select all

/C1
abac
def
ghi
/C2
abq
bedef
def


In the above example, it will locate the "def" as a duplicate in /C2, however, it will point to the first occurrence of "def" inside the word "bedef" as its location. If "def" would not be included in the second group, it would not mistake "bedef" as the duplicate.

I have the "find duplicate" part correct, but the "extract location" needs work. The utility is still helpful, though not as much as it currently is only accurate to within the group, not line and column like I want, though it is in most cases.

I'll get a fix up when I can, but I've got a lot of packing to do for my trip. I'll be taking a computer with me, and will have internet access, so perhaps I may be able to fix it before Monday, if I get time while I'm on my trip.

[rhapdog]

The bug mentioned in the previous post has been corrected.

In addition, I was able to go ahead and add new functionality. There are now 2 command line switches you can use with the program.

Code: Select all

USAGE:
   TestDuplicate [options] "filename"

Where "filename" is a fully qualified filename including the path if not in
the working directory.  "filename" must include the file extension, and must
be enclosed in double quotes if the path or filename contains spaces.
Only one filename may be passed to the program at a time.

Options:
-fix
  Removes all duplicates found that would normally be ignored according to
  the way IDM wordfiles are parsed by UltraEdit. This will ignore substrings
  that conflict, but will remove substrings that are exact duplicates.
-quiet
  Suppresses the normal report given of what duplicate words are found.  Useful
  only if the -fix option is specified.  If -fix is not specified, this option
  is ignored.

Default action with no options given will generate a report of the duplicates
found, without modifying the wordfile.


Now it will be easy to automatically remove any duplicates that would normally be ignored by UltraEdit. This makes the loading for UE faster and frees up memory resources for wordfiles that contain the duplicates.

Be careful with the -fix option, as it makes changes to your wordfile. Although I have been unable to find any issues with it, I have not tested it under every condition possible, though as far as I can tell the formula I used should work with every wordfile.

Please report any issues you may find with this tool.

[rhapdog]

All known bugs have been handled and the program is now out of beta and is considered stable.

This program is now considered production ready.