Selecting Blocks Of Text

[keithjca]

Hello,

I am trying to create a macro that will parse a web servers log files for certain events.
On occasion, some Http sessions fail from one client in particular and I would like to copy all occurences of this and place them in a seperate file.

My source data looks like this:

0-14 23:14:59.77] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent - turnOnSSL
[2008-10-14 23:14:59.77] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: ReceivedNumber of bytes: 768
[2008-10-14 23:14:59.785] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: ReceivedNumber of bytes: 1024
[2008-10-14 23:14:59.879] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: ReceivedNumber of bytes: 2048
[2008-10-14 23:14:59.973] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: ReceivedNumber of bytes: 4096
[2008-10-14 23:15:00.067] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: ReceivedNumber of bytes: 2966
[2008-10-14 23:15:00.551] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: SendingNumber of bytes: 357
[2008-10-14 23:15:00.551] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: SendingNumber of bytes: 768
[2008-10-14 23:15:00.551] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: SendingNumber of bytes: 1024
[2008-10-14 23:15:00.551] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: SendingNumber of bytes: 2048
[2008-10-14 23:15:00.551] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: SendingNumber of bytes: 4096
[2008-10-14 23:15:00.551] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: SendingNumber of bytes: 2966
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: ReceivedNumber of bytes: 1217
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[HTTP/1.1 202 Accepted]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[Date: Wed, 15 Oct 2008 06:15:00 GMT]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[Server: Jetty/4.2.24 (Windows 2003/5.2 x86 java/1.4.2_06)]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[Connection: close]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[Content-Type: Application/xml]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[Content-Length: 6124]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Received from Server:[]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Sending to Client:[
HTTP/1.1 202 Accepted
Date:Wed, 15 Oct 2008 06:15:00 GMT
Server:SSP 2.0.00
Connection:close
Content-Type:Application/xml
Content-Length:6124


]
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: SendingNumber of bytes: 149
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: SendingNumber of bytes: 1024
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: ReceivedNumber of bytes: 3072
[2008-10-14 23:15:01.192] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: SendingNumber of bytes: 3072
[2008-10-14 23:15:01.207] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ServerAgent: ReceivedNumber of bytes: 2028
[2008-10-14 23:15:01.207] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent: SendingNumber of bytes: 2028
[2008-10-14 23:15:01.207] INFO 999999999999 SSP.Session.SSP102I Session: 56834 - Control:ServerAgent Connection closed (CloseCode.EOF): Bytes Received: 6317
Bytes Sent: 11259

[2008-10-14 23:15:01.207] DEBUG 000000000000 GLOBAL_SCOPE Session :56834 (IN: Public_Inbound_HTTPS_Node_3, ON: Public_Outbound_HTTPS_Node) - Control:ClientAgent - sending conduit.close() code: CloseCode.NORMAL_CLOSE
[2008-10-14 23:15:01.207] INFO 999999999999 SSP.Session.SSP102I Session: 56834 - Control:ClientAgent Connection closed (CloseCode.FULL_CLOSE): Bytes Received: 11246
Bytes Sent: 6273

[2008-10-14 23:16:35.832] INFO 999999999999 SSP.Session.SSP104I Session: 56841 - Session Proceeding after Node match: Public_Inbound_HTTPS_Node_3
[2008-10-14 23:16:35.832] DEBUG 000000000000 GLOBAL_SCOPE Session :56841 (IN: Public_Inbound_HTTPS_Node_3, ON: ) - Using StubAuthenticator for Authentication.
[2008-10-14 23:16:35.832] DEBUG 000000000000 GLOBAL_SCOPE Session :56841 (IN: Public_Inbound_HTTPS_Node_3, ON: ) - Out bound port range: 0
[2008-10-14 23:16:35.832] DEBUG 000000000000 GLOBAL_SCOPE Session :56841 (IN: Public_Inbound_HTTPS_Node_3, ON: ) - Control:ClientAgent - turnOnSSL
[2008-10-14 23:16:36.144] INFO 999999999999 SSP.Session.SSP102I Session: 56841 - Control:ClientAgent Connection closed (CloseCode.CONNECTION_RESET): Bytes Received: 0
Bytes Sent: 0

In particular, I need to isolate out the failed sessions. "Bytes Sent : 0" is the end of these failed sessions and the start is the first "[" after the preceeding CRLF.

My macro looks like this:

InsertMode
ColumnModeOff
HexOff
UnixReOn
Clipboard 9
Bottom
IfColNum 1
Else
"
"
EndIf
Top
ClearClipboard
Loop
Find "Bytes Sent: 0"
IfFound
StartSelect
Find Up Select "^p"
EndSelect
CopyAppend
Else
ExitLoop
EndIf
EndLoop
NewFile
Paste
SaveAs "C:\UETmp.txt"
EndIf
ClearClipboard
Clipboard 0

A new output file is created but the first line is always blank. The rest of of the lines are only populated with "Bytes Sent : 0" but not the rest of of data below the preceeding CRLF.

Any suggestions would be appreciated.

Keith

[Mofi]

You were on the right way. You just made some small mistakes. Here is the macro which worked on your small example. It contains some extra code lines just for in case the first block of the file is a block of interest and there are no 2 pairs of CRLF above. What I don't understand is why no second Find "Bytes Sent: 0" after appending a found block is necessary to move the cursor down from the beginning of the block to the end of the block. That's really strange, maybe a problem of UE with current cursor position. Well, it is the first time that I used find parameter Select in combination with Up. Before UE v14.00 this was not possible, at least for a manual find with selecting.

InsertMode
ColumnModeOff
HexOff
UnixReOn
Clipboard 9
Bottom
IfColNum 1
Else
"
"
EndIf
Top
ClearClipboard
Loop
Find "Bytes Sent: 0"
IfFound
EndSelect
Key LEFT ARROW
Key RIGHT ARROW
StartSelect
Find Up Select "^p^p"
IfNotFound
SelectToTop
EndIf
CopyAppend
EndSelect
Else
ExitLoop
EndIf
EndLoop
NewFile
Paste
"
"
Top
Key END
IfColNum 1
DeleteLine
Else
Key HOME
EndIf
ClearClipboard
Clipboard 0

[keithjca]

Hello Mofi
Thanks for the correction, the macro works correctly and gave me the exact output I needed.

I am working to become more knowledgeble with UE macros and sure that I will have more questions in the future. I have already downloaded your zip examples and begun to work through them, they are quite informative.

Moving forward, does Perl or Java Script have more of an advantage over the other?
I have a development background in other languages and personally have no preference for these tasks.


Regards,
Keith

[pietzcker]

Perl and JavaScript (and Python, Ruby, etc. etc.) are powerful scripting languages that provide a lot more control and flexibility than macros which are primarily designed to automate tasks that you could otherwise do manually as a series of (menu/keyboard) commands in UltraEdit.

In macros, you don't have variables, subroutines/functions, looping constructs (except for simple infinite loops) and many other features of real programming languages. But for many small, straightforward tasks, macros are just fine. And if you're a macro expert like Mofi, you can do some incredible tricks with them.

The good thing about JavaScript is that it's integrated into UE and provides direct access to its functions. So that's the best of both worlds right now.

Native JavaScript/Python/Ruby/Perl etc. will be faster than UE'S integrated JavaScript, but whether that's a problem for you, only you can decide...

[Mofi]

Moving forward, does Perl or Java Script have more of an advantage over the other?

When to use Scripts over Macros might be helpful to answer your question.