Regular Expressions: Using Lookaheads to Group and Grab Exactly the Text You Want

My previous post used zero-width lookahead and lookbehind assertions to grab some text from a gnarly-looking string, so I thought I'd follow up with a quick post on how that works.  It's not as complicated as the name sounds.

I had this string, from which I wanted to extract the domain and username:

\\SERVER\root\cimv2:Win32_Group.Domain="MYDOMAIN",Name="adminuser"

I know that I want the text between the double-quotes immediately following the words "Domain" and "Name".  I decided on this approach:

$string -match '(?<=Domain\=")(?<domain>[^"]+).*(?<=Name\=")(?<name>[^"]+)'

The characters in blue are, as described in the previous post, named groups, which will be captured and assigned in the automatic variable $matches with those names (Eg. $matches.domain).  The characters in red are the zero-width lookbehind assertions.

So what are they good for?  You can use lookaheads and lookbehinds if you want to make sure that a specific pattern comes before or after the pattern you want to capture, but don't actually want that pattern to be captured.  They look like groups, but will not be added to $matches.

A lookbehind assertion looks like this:

(?<=YOUR_PATTERN_HERE)

A lookahead assertion looks like this:

(?=YOUR_PATTERN_HERE)

Ah, but what if I want to make sure that a certain pattern does not follow my group?  Just replace the equality sign with an exclamation point, like so:

(?<!YOUR_PATTERN_HERE)

(?!YOUR_PATTERN_HERE)

So let's break down what my regex does:

# Check that the pattern 'Domain\="' is in the string, 

# but do not capture this group.

(?<=Domain\=") 

# Immediately following it, capture one or more characters that are not the 

# double-quote character and name this group "domain"

(?<domain>[^"]+)

# Match zero or more of any character.

.*

# Check that the pattern 'Name\="' is in the string, 

# but do not capture this group.

(?<=Name\=")

# Immediately following it, capture one or more characters that are not the 

# double-quote character and name this group "name"

(?<name>[^"]+)

Named Groups in Regular Expressions

I don't know how I went this long without discovering named groups in regular expressions, but I'm genuinely excited about them (yes, I'm a nerd).

A quick recap of the most common way to use regular expressions in PowerShell. Let's say I have a string like the one below (sorry it isn't a more simple example, but this is literally something I ran into today).  I got it by querying the local administrators of a system using SCCM.  The problem is, I want it in domain\user format.

\\SERVER\root\cimv2:Win32_Group.Domain="MYDOMAIN",Name="adminuser"

My first thought was to do something like this:

$string -match '(?<=domain\=")([^"]+).*(?<=name\=")([^"]+)'

It evaluates to True on my test string, so I go look at $matches:

$matches


Name                           Value
----                           -----
2                              adminuser
1                              MYDOMAIN
0                              MYDOMAIN",Name="adminuser

Okay, I've captured my groups, but I notice something strange.  Why is $matches a hashtable instead of an array?  Because of named groups, that's why.

To create a named group, you put the parentheses around it just like normal, but you add
'?<groupname>' to the beginning of the capture.  This stores the group under the name 'groupname'.  Let's try that with the above example:

$string -match '(?<=domain\=")(?<domain>[^"]+).*(?<=name\=")(?<name>[^"]+)'


$matches


Name                           Value
----                           -----
name                           adminuser
domain                         MYDOMAIN
0                              MYDOMAIN",Name="adminuser

It makes my regex a little longer, but it is so much easier now when I go to use the values I've collected to remember $matches.domain and $matches.name instead of $matches[1] and $matches[2].

Harnessing the Power of PowerShell to Load-balance Sophos Servers

At work we have a decent-sized Sophos installation.  This means that we have to use message relays to manage the status traffic back and forth between the Enterprise Console and the clients.  I recently discovered that although I could use groups to point client updates to their local server for updating, the message routers weren't affected.  As a result almost all clients ended up using the same server as a message relay.  I confirmed with my TAM that this feature is by design, so I set out to fix it with a script.  What I ended up with is basically what you see below.

A few things worthy of note:

• I've pretty much standardized on using that logging boilerplate for most of my scripts.  It makes it easy to log errors and insert debug statements at the code as I'm writing so that I can always set -loglevel to 'debug' later when troubleshooting.
• I made the caller pass the name of the mrinit.conf file so that I could create one small SCCM package for the script with all five different mrinit.conf files.
• If you decide to do this, don't use the mrinit.conf file from the root of the package directory on the Update server.  There should be an mrinit.conf file in the rms subfolder.  Use that one.  If it isn't there, then you might not be configured to use a message relay, and this script won't help you until you are.

WARNING!  ACHTUNG!  AVISO!  LUU Y!
I am doing the QA and testing for my organization.  I make no guarantees that this script will work for yours.  Sophos is a temperamental beast, and you should do the due diligence to test and do the QA and do whatever modifications it takes to make it work for yours.  You may also wish to consult with your Sophos TAM before undertaking a project like this.

InnerException: We have to go deeper.

I helped a co-worker with an interesting issue today.  He was writing a PowerShell script that downloads a file.  He wanted to catch any errors with the download, so he had some code like this:

$client = New-Object System.Net.WebClient
try {
  $client.DownloadFile('http://www/files/file.txt', file.txt)
catch [System.Net.WebException] {
  # $_ is set to the ErrorRecord of the exception
  Out-Log $_.Exception.Message
}

This usually works, returning the text of the error, but this time he was getting back:

An exception occurred during a WebClient request.

That's not a very informative error.  Removing the try/catch blocks, he got this on the screen:


System.Net.WebException: An exception occurred during a WebClient request. ---> System.UnauthorizedAccessException: Access to the path 'C:\Users\tojo2000\hosts.txt' is denied.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean 
...


So obviously the information is in there somewhere.  It turns out that WebException usually returns the error code of the request, but if it runs into an error that is not related to the actual download, it will add the second exception to a property of the WebException object called InnerException.  

So the updated code looks something like this:


$client = New-Object System.Net.WebClient
try {
  $client.DownloadFile('http://www/files/file.txt', file.txt)
catch [System.Net.WebException] {
  # $_ is set to the ErrorRecord of the exception
  if ($_.Exception.InnerException) {
    Out-Log $_.Exception.InnerException.Message
  } else {
    Out-Log $_.Exception.Message
  }
}


Now we get the correct output:


Access to the path 'C:\Users\tojo2000\hosts.txt' is denied.

When Is a String Not a String?

I came across a really difficult-to-troubleshoot bug today, a real brain-teaser.

A co-worker asked me to help him figure out why his PowerShell script was running a certain piece of code even though it shouldn't.  He had a piece of code to detect a bad Windows Installer exit code like this:

$result = InstallSomeSoftware
if ($result -eq "1603") {
  #do something...

The "do something part was being executed no matter what value he returned from the InstallSomeSoftware function.  To troubleshoot, he added some Write-Host statements to display the return value on the screen before returning it inside the function.  Sure enough, the expected value was printed on the screen, and it was not 1603.

We both scratched our heads for a while, stepped through the function, and still weren't getting anywhere, when I noticed a few random lines of output.  We realized that there were lines in his function that were returning values and not being captured by any variable or thrown away, so they were also being returned along with the expected value.

So that leaves one last piece of the puzzle.  Why would everything always evaluate to True when compared to the string "1603"?  After a little more digging we had the answer.  The first value being returned had the value True.  -eq, when attempting to determine equality, saw that the two object types weren't the same, so it did what it was supposed to do:  it cast the string "1603" as a System.Boolean and then checked to see if True was equal to  True (which is what the string "1603" evaluates to as a Boolean).

Many hairs were lost in this battle, but at least in the end we had our sanity.

The 2011 Scripting Games Start Today!

Just a reminder to get scripting.  I'm going to try to make the time to participate in each event this year, and then I'll be posting my solutions here after the submissions are closed.

If you're interested in the Scripting Games at all, I suggest you bookmark this URL:  All 2011 Scripting Games links on one page.

One Approach to Logging in PowerShell

I was writing a script recently, and I realized that I was missing the ability to use the standard logging module from Python.  I decided to make a stripped-down logging function that would make logging easy, and that could be re-used.  I'd be interested to hear what approaches others have taken to solving this problem.  Here's what I came up with:



# Set severity constants
$MSG_INFORMATION = 0
$MSG_WARNING = 1
$MSG_ERROR = 2
$MSG_SEVERITY = @('Information', 'Warning', 'Error')


# Set configurable settings for logging
$LOG_LEVEL = $MSG_INFORMATION
$LOG_FILE = 'my_logfile.log'
$SMTP_TO = 'devnull@tojo2000.com'
$SMTP_SERVER = 'smtp.tojo2000.com'
$SMTP_SUBJECT = 'Script error!'


function Write-Log {
  <#
  .SYNOPSIS
     Writes a message to the Log.
  .DESCRIPTION
    Logs a message to the logfile if the severity is higher than $LOG_LEVEL.
  .PARAMETER severity
     The severity of the message.  Can be Information, Warning, or Error.
     Use the $MSG_XXXX constants.
     Note that Error will halt the script and send an email.
  .PARAMETER message
     A string to be printed to the log.
  .EXAMPLE
     Log $MSG_ERROR "Something has gone terribly wrong!"
  #>
  param([Parameter(Mandatory=$true)][int]$severity,
        [Parameter(Mandatory=$true)][string]$message)


  if ($severity -ge $LOG_LEVEL) {
    $timestamp = Get-Date -Format 'yyyy-MM-dd hh:mm:ss'
    $output = "$timestamp`t$($MSG_SEVERITY[$severity])`t$message"
    Write-Output $output >> $LOG_FILE


    if ($severity -ge $MSG_ERROR) {
      Send-MailMessage -To $SMTP_TO `
                       -SmtpServer $SMTP_SERVER `
                       -Subject $SMTP_SUBJECT`
                       -Body $output `
                       -From $SMTP_FROM
      exit 1
    }
  }
}