Replacing KEdit by UltraEdit - any thoughts?

[Mofi]

The display updates during script and macro execution is a problem and makes them slower. But there are no commands to disable/enable the display updates. That's the reason why I try to write scripts in a manner to avoid display updates as much as possible by doing as much as possible with core objects and methods in memory and write to document as less as possible. Using Replace All command avoids also lots of display updates.

Another common method to avoid display updates on script execution is to deteremine at script start the index of active document in the documents array, then open a new file which becomes active document, but work with the script on the doument with the before determined document index. At end of script this document is set active again and the new file is closed without saving it. If the document windows are maximized, this workaround helps to avoid unnecessary display updates during script execution.


You have modified the text of the dialog displayed at startup, so I can't really help you. I suppose the dialog is showing you that the installer is running to repair the installation of UltraEdit. UltraEdit is installed with MSI (Microsoft Installer). And the MSI service running in background checks for every application installed with MSI if all files installed by the installer (all files in program files directory of UltraEdit) are present and runs a repair if a file is missing. So I think you have manually deleted a file in the program files directory and its subdirectories and the installer wants to repair it. Let MSI do the repair or run manually a repair of UltraEdit installation via Control Panel and you should not see this dialog again.

BTW: You can upload *.png (best for screen shots) and *.jpg files directly to the board. Only other file types must be packed with ZIP or RAR.

[rhapdog]

Seriously, we should all write in a feature request that there be made available both macro and script commands to pause/resume display updates. This is actually one reason I use PHP as a scripting language that I execute against a file, configured as a UE tool, where it prompts me for the PHP script to use. I can only do that if I don't need the UE application object. I can write to the file directly to the disk, and have UE "update" the file when it detects it has been changed on disk. So much faster for me. But I'd probably still use PHP where I can, since it is my primary programming language that I use the most often right now.

[billdehaan]

The display updates during script and macro execution is a problem and makes them slower. But there are no commands to disable/enable the display updates. That's the reason why I try to write scripts in a manner to avoid display updates as much as possible by doing as much as possible with core objects and methods in memory and write to document as less as possible. Using Replace All command avoids also lots of display updates.

Sure, but keyboard macros don't really have that option that I can see. In Kedit, there was no real distinction between macros and scripts; macros were simply scripts that called the "hit" command, ie. "hit U & hit l & hit t & hit r & hit a" would be a keystroke macro that would type the letters U,l,t,r,a in 5 separate operations. If that were to be done in a macro, it would be recoded as "text Ultra", but usually, keystroke macros were for one-off operations. Some could get rather involved, with hundred of keystrokes. If then rerun a couple of hundred times (there's a similar repeat command in Ctrl-L in UltraEdit), it would run without updating the display to the end.

Another common method to avoid display updates on script execution is to determine at script start the index of active document in the documents array, then open a new file which becomes active document, but work with the script on the document with the before determined document index. At end of script this document is set active again and the new file is closed without saving it. If the document windows are maximized, this workaround helps to avoid unnecessary display updates during script execution.

Hmm. I hadn't considered that as a workaround. It seems kludgy, but I'll see if that works. Of course, that would then introduce at least two focus changes in tabbing between two edit files.

You have modified the text of the dialog displayed at startup, so I can't really help you. I suppose the dialog is showing you that the installer is running to repair the installation of UltraEdit. UltraEdit is installed with MSI (Microsoft Installer). And the MSI service running in background checks for every application installed with MSI if all files installed by the installer (all files in program files directory of UltraEdit) are present and runs a repair if a file is missing. So I think you have manually deleted a file in the program files directory and its subdirectories and the installer wants to repair it. Let MSI do the repair or run manually a repair of UltraEdit installation via Control Panel and you should not see this dialog again.

I'm not exactly sure what happened. After rerunning UE without parameters, it seemed to correct itself. I probably won't be using that much any more anyways. I was testing UE at home and UE Portable at work. I found that in the course of my normal workday, the need to be able to take the editor with me to lab machines to outweighs the benefits of the installed application, so if I do go with UE, it will probably be the portable version.

BTW: You can upload *.png (best for screen shots) and *.jpg files directly to the board. Only other file types must be packed with ZIP or RAR.

Okay. I read the board notes, and they said to convert to zip/rar; I didn't see mention of an exclusion for screenshots, so I went with the compressed form. Hopefully I won't need to make use of that feature again for a while anyway

[billdehaan]

Seriously, we should all write in a feature request that there be made available both macro and script commands to pause/resume display updates. This is actually one reason I use PHP as a scripting language that I execute against a file, configured as a UE tool, where it prompts me for the PHP script to use. I can only do that if I don't need the UE application object. I can write to the file directly to the disk, and have UE "update" the file when it detects it has been changed on disk. So much faster for me. But I'd probably still use PHP where I can, since it is my primary programming language that I use the most often right now.

I do that a few times in my Kedit environment, usually for things like checking a file out of source code control (I've written a similar tool for UltraEdit), but generally, any macro is going to need access to the document object.

[johnnyc]

Can you guys give me a quick hand on Kedit equivalent commands? I am totally new to UES and am a bit confused. (By the way, I think you guys are probably "spring chickens" compared to me, since I worked on my first computer in 1957).

UEStudio '11 Development Environment
Version 11.20.0.1006

I used Kedit for massaging massive amounts of data. (Usually somewhat corrupted data from customers that I am trying to make some sense of. These are truely one time jobs and not worth the time to even write a Perl script for.

Most of the time in Kedit I used ALL, MORE, LESS, RANGE amd varopis regex expressions to limit/edit the domain of records I was editing. I cannot find anywhere equivalent commands. Basically, I don't want anything "Hidden" to be within the scope of Find, Search, Replace and Delete commands.

Any thoughts? Thanks, John

[rhapdog]

First, you might want to update to 11.20.0.1010, as it is the latest version and fixes a few bugs.

[Warning: Reminiscing ahead]
Second, I don't mind being called a spring chicken at any time. I didn't repair my first computer until 1972, and I didn't start programming computers until 1974. So, compared to 1957, I suppose I can be called a spring chicken. My father was working for IBM back when you worked on your first computer, and I learned from him at home in the beginning until we were able to build our own computer from parts. Fun stuff. Try computer programming by short-circuiting contacts, which is how we programmed our first programmable circuit because we couldn't afford the proper equipment. It worked and we got going with it. Fun.
[END Reminiscing]

Second, you need to change your mindset. That will be required to update to any modern editor or IDE. The way Kedit had to work back in its day is completely different from how modern editors work, simply because of the hardware limitations of the day. ALL, MORE, LESS, RANGE were all implemented because data often would not fit into memory, so only a piece of it could be worked on at a time. If I said this to some younger than us, they wouldn't understand that, but you should remember those days of that particular limitation. I remember tape reels spinning back and forth reading a piece of data at a time in order to keep it all within the 2KB limit of the mainframe I was working with. That's really sad, considering my coffee pot has more memory now. The additional reason ALL, MORE, LESS, and RANGE were implemented was as a make-shift "selection" area of the data when you only want to work with a specific section. Kedit called them RANGEs, other editors called them other things, like blocks. Working with tools provided from IBM, the editor I started with used blocks instead of ranges. Same thing.

I used Kedit for massaging massive amoounts of data. (Usually somewhat corrupted data from customers that I am trying to make some sense of. These are truely one time jobs and not worth the time to even write a Perl script for.

Let's not start getting confused. Perl Scripts, while they can technically be used, are not a part of the integration of UltraEdit as UE's scripting language. It uses JavaScript. But that's beside the point, since it wouldn't be worth writing any script from what you are saying. UE can use Perl Regular expressions, however, which might be what you are looking for in find/replace situations.

Most of the time in Kedit I used ALL, MORE, LESS, RANGE amd varopis regex expressions to limit/edit the domain of records I was editing. I cannot find anywhere equivalent commands. Basically, I don't want anything "Hidden" to be within the scope of Find, Search, Replace and Delete commands.

ALL, MORE, LESS, and RANGE are obsolete, because you don't need to "load" part of the file for viewing, because the entire file is available for editing and find/replace at once. If you decide you want to do a find/replace on a sub-set of the file, then "select" a range, and tell the find/replace to operate on that selection only. RANGE has been replaced by Selected Text.

Now, since you say you don't want anything "Hidden" to be within the scope of your operations, then you can easily perform those operations on "selected" text. When you open up your find or replace dialog, there is a box labeled "Replace Where:" or "Find Where:". There you can check "Selected Text" which will limit the operations to the range of text you have selected.

I hope this helps.

[billdehaan]

Can you guys give me a quick hand on Kedit equivalent commands? I am totally new to UES and am a bit confused. (By the way, I think you guys are probably "spring chickens" compared to me, since I worked on my first computer in 1957).

Yeah, you've got me there.

Mind, given that I still have 8 days left in my evaluation period (which is for v17, so I can probably extend it another month by restarting with v18 ), I'm not exactly an old hand at UE myself. I'm not using UES, since portability is more important to me, and there's no portable UES, but the editor functions are the same.

I used Kedit for massaging massive amounts of data. (Usually somewhat corrupted data from customers that I am trying to make some sense of. These are truely one time jobs and not worth the time to even write a Perl script for.

Most of the time in Kedit I used ALL, MORE, LESS, RANGE and various regex expressions to limit/edit the domain of records I was editing. I cannot find anywhere equivalent commands. Basically, I don't want anything "Hidden" to be within the scope of Find, Search, Replace and Delete commands.

Yeah, "welcome to my world". That is exactly what I do a lot of the time: edit 300+MB files of timestamped telegram data between different network devices using proprietary formats. In addition of ALL/MORE/LESS and RANGE, I use ZONE a lot to do field restriction. Thank the elder gods that at least the telegrams are (for the most part) fixed format.

To answer your question: no, UE doesn't do that. You can use Find->Show Lines to give an approximation of the ALL command, but the minute you do another search/replace operation, UE will expand the document to the next match it finds. In short, there's no way (that I've found) to approximate Kedit's ALL/RANGE commands.

I looked into the macros, and outside of the delAllHiddenLines and hideOrShowLines, there's nothing.

While UE is arguably a better code editor than Kedit, I think Kedit still beats it as a text editor.

The basic difference is that in UE, the code folding feature is simply a display issue; the underlying file content is unchanged. It's not like the Kedit SCOPE idea, where hidden lines were effectively removed from the file, and all commands would ignore them.

The lack of Kedit-style folding support has really been the only major disappointment I've had with UE. It's been the only major Kedit feature that I haven't been able to find a workaround to, or alternative for.

[billdehaan]

Second, you need to change your mindset. That will be required to update to any modern editor or IDE. The way Kedit had to work back in its day is completely different from how modern editors work, simply because of the hardware limitations of the day. ALL, MORE, LESS, RANGE were all implemented because data often would not fit into memory, so only a piece of it could be worked on at a time.

I agree about the mindset change. UE has provided a lot of ways to address problems differently than Kedit did. While it lacks the same functions, I usually can address the issue in a different way. I can't do solve X using Kedit feature Y, there's no way to do Y, but UE feature Z solves problem X, so the lack of feature Y isn't a big deal.

The additional reason ALL, MORE, LESS, and RANGE were implemented was as a make-shift "selection" area of the data when you only want to work with a specific section. Kedit called them RANGEs, other editors called them other things, like blocks. Working with tools provided from IBM, the editor I started with used blocks instead of ranges. Same thing.

However, I disagree with this. The reason why ALL and the like were implemented doesn't really matter. The facility that it provides is far beyond that. In Kedit, I had a keystroke that would set the RANGE to the current C++ method, so I could then make global changes that would be restricted to that method only. Ah, in UE you can select that method and do the same thing, you say. True. But another reason I use RANGE and ALL is to editor formatted data.

Take a file with a few hundred thousand lines of timestamped data like the following:

Code: Select all

2012-03-01 12:00:00 SRC1 DST1 29 19 88 19 A7 89 98 23 23 98 38 CRC=0xf2ad
2012-03-01 12:00:20 SRC1 DST1 29 19 88 19 A7 90 9A 23 24 98 38 CRC=0xd801
2012-03-01 12:00:20 SRC4 DST3 29 19 88 19 A7 90 9A 23 24 98 38 CRC=0xa9b1
2012-03-01 12:00:40 SRC1 DST1 29 19 88 19 A7 91 9A 23 24 98 38 CRC=0xc871
...
2012-03-14 20:00:00 SRC1 DST1 29 19 88 19 A7 33 98 23 23 98 38 CRC=0x092c
2012-03-14 20:00:20 SRC1 DST1 29 19 88 19 A7 34 9A 23 24 98 38 CRC=0xd8d1
2012-03-14 20:00:40 SRC1 DST1 29 19 88 19 A7 35 9A 23 24 98 38 CRC=0x2138

Now, the problem is that a problem was reported on the 8th of the month "sometime between 2pm and 3pm with the lights off", and again on the 11th, at "about 5:30 in the morning, when the lights were on". With lights being byte 9 in the stream (on==0x23, off==0x24), it's a simple matter to say:

Code: Select all

ALL /2012-03-08 02/
MORE /2012-03-08 03:0/
ZONE 55 56
LESS ~/23/
ZONE 1 *
MORE /2012-03-11 05:/
ZONE 55 56
LESS ~/24/

And instead of looking at 300MB of data over millions of lines, I'm now looking at about ~2000 lines of data for patterns. And of course, it can then be further refined.

While ALL/MORE/LESS/RANGE may have their historical roots due to low-memory systems of the past, the abilities they provide today go far beyond merely paging to and from disk. They provide the capability of filtering data very efficiently. And while you can write perl or tcl/tk scripts to simulate this by processing the above into an output file, try changing a file where you want to patch the input byte on all SRC1 to DST9 machines where bit 13 is 0 and bit 51 is 1, so you have a file that can be used as a test input.

Let's not start getting confused. Perl Scripts, while they can technically be used, are not a part of the integration of UltraEdit as UE's scripting language. It uses JavaScript. But that's beside the point, since it wouldn't be worth writing any script from what you are saying. UE can use Perl Regular expressions, however, which might be what you are looking for in find/replace situations.

I don't think Johnny's confusing Perl with UE. I think he means that it's easier to do it in the editor (Kedit) interactively than going to the effort of writing a perl script to do the same string processing that Kedit does.

ALL, MORE, LESS, and RANGE are obsolete, because you don't need to "load" part of the file for viewing, because the entire file is available for editing and find/replace at once. If you decide you want to do a find/replace on a sub-set of the file, then "select" a range, and tell the find/replace to operate on that selection only. RANGE has been replaced by Selected Text.

I think that Johnny and I use the ALL and RANGE differently than what you're thinking off. It's not an issue of loading a file that's in question, it's the ability to provide tailored views of a subset of lines within a file, and then to perform operations on that subset.

Now, since you say you don't want anything "Hidden" to be within the scope of your operations, then you can easily perform those operations on "selected" text.

Not really. I've tried. Selected text is a single block of contiguous lines. A selection approximates the RANGE command, but not the ALL/MORE/LESS. I routinely have a scope set which for example could show lines 1,2,15-20,91,200-310,1800-1802, and 4399-5102 (a "show all times where the lights were off" example using the criteria above).

[rhapdog]

Ah, yes. Well, as I have always used the editors only for computer programming, I have never used these features in such a way as you describe. I see where it can definitely be useful, however.

There have been times when I needed to process data like that. Back in 1995 - 1998. I would simply write a small Pascal program to manipulate the data as was needed. As I was proficient in the language, it was very quick for me to do it in this manner. It then output a new file with the desired data in the desired format.

I suppose that would be the same as writing a script to handle it now, although in 1995, I began using UltraEdit, and it could not handle scripting to do such a task like that at that time. That's why I would write a program to do it for me.

Oh, well. I got paid by the hour.

In UE/UES, while you can "hide" the lines in multiple sections by multiple regex searches, you can't perform the operations on just the "showing" lines, as it would effect the hidden lines as well. Perhaps Mofi has a suggestion on how to implement this? Perhaps a feature request may be in order to IDM to enable a feature to ignore hidden lines within the search/replace operations.

[billdehaan]

Ah, yes. Well, as I have always used the editors only for computer programming, I have never used these features in such a way as you describe. I see where it can definitely be useful, however.

I'd go so far as to say it's essential. Back in the 1990s, Kedit had folds, but no regular expressions. Other editors had regular expressions, but no folds. It was a real toss-up as to which was more useful. Of course, now Kedit has regular expressions and folds (and can use them together), so it's incredibly powerful. A lot of the features that UE has that Kedit lacks, such as the function list window, I implemented using folds.

And it's not just for data. I've got a ~40 line KEXX script that shows the outline of a C program, by only showing lines with {,},while,for,if,switch,etc. as well as the function header, and optionally the function comment header. It's similar to the "collapse all" command in UE, except it shows the control flow of a function. Oh, and I wrote it originally in 1984

Folds are like regular expressions: if you don't use them, and you just hear about them, you don't appreciate how powerful they are. But after you get used to them, you realize how much you've come to rely on them when they're gone.

There have been times when I needed to process data like that. Back in 1995 - 1998. I would simply write a small Pascal program to manipulate the data as was needed. As I was proficient in the language, it was very quick for me to do it in this manner. It then output a new file with the desired data in the desired format.

Sure, but typing in 5 lines into the command line is a hell of a lot faster. Also, in my case, I'm usually analyzing log files either in a lab setting, or a customer site, and don't have access to a compiler. More importantly, we don't often know what the right filter criteria are; we're sifting through tons of data trying to determine what the criteria is. Sure, once we know, a perl/pascal program is an option, but in the editor, it's immediate visual feedback. Of course, once we have the criteria in the editor, saving it is a simple "ssave as" operation; who needs a program?

In UE/UES, while you can "hide" the lines in multiple sections by multiple regex searches, you can't perform the operations on just the "showing" lines, as it would effect the hidden lines as well. Perhaps Mofi has a suggestion on how to implement this? Perhaps a feature request may be in order to IDM to enable a feature to ignore hidden lines within the search/replace operations.

That would be a start. But implementing things like MORE and LESS (never mind ZONE) would most likely require a fundamental change to the way UE handles the document object. Kedit has a SCOPE associated with each line, where each line has a value between 0 and 255. The MORE/RANGE/LESS/MORE commands modify the scope of selected lines, and the ALL command presents all the lines of a certain scope value as being part of the file. The powerful thing is that every command in the editor uses scope intrinsically, it's not just search and replace. It's a core attribute of the document object.

I'd love to see it happen, but I don't know that it's realistic. Especially since the number of users who would really appreciate such a feature is probably pretty small.

To be fair, that line orientation is one of the reasons that Kedit has nothing like "ViewHighlightAllSelected". The underlying document model is all line based. I really doubt that UE's is.

[Mofi]

There is also the advanced find option List Lines Containing String. With this option enabled and running a normal or a regular expression search, the lines found are output to a window. You can now look on the lines in this window and jump in the file to the active line in the window. Also possible is to copy the lines in the window to clipboard and paste them into a new file. That gives you the possibility to work only on those lines. Of course if you edit them and finally want them back in the large file, the lines must be copied back into the large file replacing the lines there. If the lines are from a coherent block, this is an easy task.

And if you are working with folded blocks, you might want to uncheck setting Automatically unfold hidden areas on Find and Goto at Advanced - Configuration - Editor Display - Code Folding.

[rhapdog]

And if you are working with folded blocks, you might want to uncheck setting Automatically unfold hidden areas on Find and Goto at Advanced - Configuration - Editor Display - Code Folding.

I thought of suggesting that, but decided against it since it was already stated above that he didn't want "hidden" lines to be processed. Even when you don't show those hidden, folded lines, they are still processed and changed. Kind of defeats the purpose. I'd like to see it implemented where hidden lines are "ignored" as an option when doing search/replace and other operations.

For example, I can see where it would be very handy to "hide" lines, then do column operations where hidden lines in between are not affected. This would really increase UE's power.

[billdehaan]

There is also the advanced find option List Lines Containing String.

Oh, I know that one well. I use it all the time.

Of course if you edit them and finally want them back in the large file, the lines must be copied back into the large file replacing the lines there. If the lines are from a coherent block, this is an easy task.

Unfortunately, it rarely is a coherent block. That's where the real power of the ALL command comes in; it gives the ability to work with wildly disconnected regions. I often look at a file with a week's worth of device telegrams that are sent every 20ms, so files routinely have hundreds of thousands, or often millions, of lines. A command like "show all lines with telegrams from device A,B, or C to devices X or Z, where the 17th field is 0x08" will often find the needle in the haystack by finding a group of lines isolated from the others by over 10,000 lines.

In situations like that, copying them back is not an easy task. Worse is when some of the operations involve sorting, which resequences the file

Don't misunderstand, I think UE's handling of code folding is okay, for what it is. I just wanted to clear up the misconception that it replaces the functionality of the Kedit ALL commands; it doesn't. Outside of Xedit (or SPF) based editors, I don't know of many, or any, that support this type of functionality.

[MickRC3]

Outside of Xedit (or SPF) based editors, I don't know of many, or any, that support this type of functionality.

I wondered if SPF was going to be mentioned. I did a lot of the things mentioned using ISFP on files 31 years ago on an IBM mainframe, 20 years ago on a VMS comm computer and then again using SPF on an OpenVMS system a decade ago.

Like rhapdog, I am somewhat a "Spring Chicken" since my first lines of code were written in 1971 in high school and bought my first computer in 1979 (TRS-80 Model 1 with a serial number of 52xx). I didn't start coding for money until 1981, when I changed jobs in the US Air Force from maintaining a flight simulator to writing databases for airborne signal analysis. It's hard to explain how we coded back then to the younger set. Punched cards. Paper tape. 8-track reel tapes (9-track if they had a parity bit). Desk checking. Code sheets. Dumb terminals. Line Editors. No random access. Nothing was visual. No bloated services. No WYSIWYG anything. PC processors ran at a blistering 1Mhz. Workstation processors ran at 4Mhz while highend Mainframe processors were breaking 16Mhz. Floppy disk drives cost $300-$800 depending on capacity. Floppy diskettes were $100 a box of ten. A 5meg Hard Drive would set you back $3900 without the controller. RAM was $200 per 16Kbyte memory board and you mounted the chips yourself. If you wanted your code to run fast you assembled it yourself and avoided compilers because compilers were always middle of the road between speed and usability. Everything was memory constrained, often all of your code and data had to fit in a 16K segment and you had to learn to make it all count. Oh yeah, and listen to the real old-timers talk about working on systems with a few hundred words of memory.

[billdehaan]

I wondered if SPF was going to be mentioned. I did a lot of the things mentioned using ISFP on files 31 years ago on an IBM mainframe, 20 years ago on a VMS comm computer

You have my sympathies

While I'm a big Xedit fan, I never took to SPF, although I had co-workers who adored it. I remember in 1985 having a contract where SPF/PC was mandated, and it was painful to use.

and then again using SPF on an OpenVMS system a decade ago.

There was an OpenVMS port? I didn't know that.