Very simple code folding does not work on large file

[Jack_II]

Hi
I'm testing trial version of UltraEdit V18.20 and found a problem with 'self-made' UEW files :
The document I would like to parse is a 160Mb length file in which I would like to collapse/expand part of text encapsulated in braces.
The blocks of this document likes this :

Code: Select all

Some text
{
    Some text
    Some text
    {
        Some text
        Some text
    }
}

So, in a new UEW file, I have use this (only theses three lines) :

Code: Select all

/L99"Fhx Perso" Block Comment On = /* Block Comment Off = */ String Chars='" File Extensions = fhx
/Open Fold Strings = "{"
/Close Fold Strings = "}"

The result is :
In a tiny file, the "+" and "-" symbols appear in the margin.
In my large file, nothing works, only a highlight of the close parent brace when I place the cursor on the open brace.

What's happend ?

[rhapdog]

One issue I see with the wordfile right away is /L99. The language number should be between 1 and 20.

I had that happen once with a large file. Turns out my large file mixed different types of line endings, both Linux and DOS in the same file.

Another thing that can cause unexpected results is having "unbalanced" open/close strings.

Also, make sure the wordfile has DOS line terminators.

[Mofi]

First, there are space characters missing in String Chars='" which should be String Chars = '".

Second, code folding feature is only applied to files opened by default with usage of a temporary file. For large files opened without usage of a temporary file the code folding feature is turned off. Also if a large file is opened with file size exceeding the threshold value in KB as configured at Advanced - Configuration - File Handling - Temporary Files and the question to open this file without usage of a temp file is answered with No to open the file with usage of a temporary file, the code folding engine is not enabled for this file.

I don't recommend to enable code folding for such large files. But if you want code folding on a file with 160 MB, you have to set the threshold value to 167936 for 164 MB. Be aware that this can result in very slow behavior of UltraEdit on such large files as using code folding, syntax highlighting, undo, etc. are designed for small files with some KB to a few MB. Those features should not be used for large and huge files with several hundred MB or even GB.

[Jack_II]

Thanx for these quick responses !

@Rhapdog :
L99 : Yes, I have choose this number to not interfere with other UEW files that are present in the list.
Unbalanced braces : No, all is correct.
DOS terminator : The file is correctly displayed.

@Mofi :
Ok for the ommited spaces. Added. The strings parse was correct, even so.
Temporary files options :
- This question was asked to me at the first UE launch and I choose to keep temporary file.
Information I found also in the configuration form "Use temporary file for edit (normal use)".
I've use 300,000 kb threshold value, and, after a not so long time, the "+" and "-" appear.
Great!

(few minutes later)
New problems :
-1- The folding detection stop working after the line #100
If I click on a "+" sign to show the block contents, the "-" sign do not appears instead of "+" sign and no more folding detection is done.
-2- The file is not completely displayed :
- the last line displayed is #19211
- the vertical slide bar, at the right, is approximatly at 5% of the beginning
- 10 minutes later, nothing change ...
I've tried to use a 1Gb threshold value, but no change.

Subsidiary question : Where is stored the temporary file ?
Is it on the Windows defined temporary folder ?
Because I use to place this kind of file on a dedicated (single) hard disk, instead of the main (SSD Raid) disk.

Thank you for taking the time to solve my problem.

[Mofi]

Please use only /L1 to /L20. The language number is ignored by UltraEdit nowadays when using *.uew files in a wordfiles directory each containing only 1 syntax highlighting language. Therefore it is no problem if multiple wordfiles have for example /L20. But other tools like my macros or the tools coded by rhapdog for evalutating wordfiles expect a language number in range 1 to 20 as well as UltraEdit when multiple syntax highlighting languages are stored within a wordfile.

UltraEdit searches for String Chars = and if not found, use the built-in default definition for string characters which is "'. This is the reason why string highlighting worked although your wrong definition was not read by UE.

The temporary files are created in directory as stored in Windows environment variable TEMP. There are usually 2 TEMP variables, one for current user and one for Windows system. The current user TEMP is used for all applications started with current user account privileges like UltraEdit. The system TEMP is used by services and processes started as local system or network service (or when there is no current user TEMP like on my computers). You can define both in the Windows environment settings of the system control panel. I have explained at Moving temp file directory from drive C: to drive D: how to define a directory for UltraEdit temporary files different to what is defined for variable TEMP.

I have no experience with code folding on very large files. And I think you are the first one which tries that as code folding is as a matter of priority a feature for source code files of programming languages which are usually small files. It is not designed for large files.

UltraEdit does not load entire file into RAM as many other text editors do. UltraEdit loads the content of a file in blocks depending on current display position. I don't know the block size. Code folding feature could have problems if begin and end of a folding section are too many lines (= KB = blocks) apart.

The code folding scanning routine runs as background thread to be able to edit the file while the search for the fold strings is running. This is also the reason why it works only for files opened with using a temporary file from beginning as without creating a temporay file, edits are made directly on original file (therefore no undo). Therefore it is not possible that main UE task works on one part of the file while the code folding thread scans the entire file. Every modification of file opened without temporary file is done directly on storage media and the code folding thread would immediately fail to finish the task correct because of file offset failure. UltraEdit opens a large file opened without creating a copy in temp folder and editing this copy with write lock for other applications.

I cannot help on your display issues with code folding as I simply never tried what happens when using this feature on large files.

[Jack_II]

Ok Mofi and thank you for this long explanation.

L1/L20 : Corrected
It seems to have corrected the 'fold option', but file is not fully displayed.
Testing :
- Load the file with no language file association - File is fully displayed (more than 3 million lines).
- Associate the UEW file after loaded.
I let UE some minutes to perform fold parsing and ... it works correctly : I can collapse part of text anywhere in the file and search text all around.

I don't know exactly the max number of lines that a fold section can contain in my files, average 100 lines, and big section of 500 or 600 lines = very similar than real programing code section/functions, except the number of sections.

String Chars : I understand now why it works nevertheless.

Windows TEMP directories have been moved previously.
I just wanted to know if UE use them or use special own temp folder, to prevent the main disk saturation.

Mofi : Many thanks again for these long and detailed explanations.
If you have privileged links with IDM, it would be useful to explain them the way that I manually use to get a correct display (because "what one can do manually, a program would be able to reproduce it").
Best regards.
Jack

[Mofi]

I report issues like everybody else by email to IDM support and I have to wait as everybody else for a fix of a reported issue. What I report is not fixed with a higher priority. The fixing priority is determined by number of users (including license count) reporting an issue. Many issues I have reported are waiting for a fix, some minor issues already several years. Well, most of the issues I reported were fixed in the past. I usually don't report issues of other users. The only exception is when I am affected also by a problem. In this case with code folding on large files, I'm not affected and therefore I will not report anything to IDM support by email.

You should do that if you can reproduce the problem by writing and sending an email to IDM support (click on email us directly at top of this page) explaining step by step how to reproduce the problem and attaching your uedit32.ini, the wordfile and an example file in a ZIP or RAR archive. As a large file must be attached it should be a file which contains same block thousands of times to be able to compress it with ZIP or RAR to a very small archive file. It does not matter if you use UltraEdit already registered or if you still evaluate it.

[Jack_II]

Thank you for the advice, Mofi.