{article Dive into Python}{title} {text} {/article}

Regular Expressions

Strings have methods for searching (index, find, and count), replacing (replace), and parsing (split), but
they are limited to the simplest of cases. The search methods look for a single, hard−coded substring, and they are
always case−sensitive. To do case−insensitive searches of a string s, you must call s.lower() or s.upper() and
make sure your search strings are the appropriate case to match. The replace and split methods have the same
limitations.

If what you're trying to do can be accomplished with string functions, you should use them. They're fast and simple
and easy to read, and there's a lot to be said for fast, simple, readable code. But if you find yourself using a lot of
different string functions with if statements to handle special cases, or if you're combining them with split and
join and list comprehensions in weird unreadable ways, you may need to move up to regular expressions.

Although the regular expression syntax is tight and unlike normal code, the result can end up being more readable
than a hand−rolled solution that uses a long chain of string functions. There are even ways of embedding comments
within regular expressions to make them practically self−documenting.

Street Addresses

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> s = '100 NORTH MAIN ROAD'
>>> s.replace('ROAD', 'RD.')
'100 NORTH MAIN RD.'
>>> s = '100 NORTH BROAD ROAD'
>>> s.replace('ROAD', 'RD.')
'100 NORTH BRD. RD.'
>>> s[:-4] + s[-4:].replace('ROAD', 'RD.')
'100 NORTH BROAD RD.'
>>> import re
>>> re.sub('ROAD$', 'RD.', s)
'100 NORTH BROAD RD.'
>>>

My goal is to standardize a street address so that 'ROAD' is always abbreviated as 'RD.' . At first glance, I thought this was simple enough that I could just use the string method replace. After all, all the data was already uppercase, so case mismatches would not be a problem. And the search string, 'ROAD' , was a constant. And in this deceptively simple example, s.replace does indeed work.

Life, unfortunately, is full of counterexamples, and I quickly discovered this one. The problem here is that 'ROAD' appears twice in the address, once as part of the street name 'BROAD' and once as its own word. The replace method sees these two occurrences and blindly replaces both of them; meanwhile, I see my addresses getting destroyed.

To solve the problem of addresses with more than one 'ROAD' substring, you could resort to something like this: only search and replace 'ROAD' in the last four characters of the address (s[−4:] ), and leave the string alone (s[:−4] ). But you can see that this is already getting unwieldy. For example, the pattern is dependent on the length of the string you're replacing (if you were replacing 'STREET' with 'ST.' , you would need to use s[:−6] and s[−6:].replace(...) ). Would you like to come back in six months and debug this? I know I wouldn't.

It's time to move up to regular expressions. In Python, all functionality related to regular expressions is contained in the re module.

Take a look at the first parameter: 'ROAD$' . This is a simple regular expression that matches 'ROAD' only when it occurs at the end of a string. The $ means "end of the string". (There is a corresponding character, the caret ^, which means "beginning of the string".)

Using the re.sub function, you search the string s for the regular expression 'ROAD$' and replace it with 'RD.' . This matches the ROAD at the end of the string s, but does not match the ROAD that's part of the word BROAD, because that's in the middle of s.

{source}
<!-- You can place html anywhere within the source tags -->
<pre class="brush:py;">

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> s = '100 NORTH MAIN ROAD'
>>> s.replace('ROAD', 'RD.')
'100 NORTH MAIN RD.'
>>> s = '100 NORTH BROAD ROAD'
>>> s.replace('ROAD', 'RD.')
'100 NORTH BRD. RD.'
>>> s[:-4] + s[-4:].replace('ROAD', 'RD.')
'100 NORTH BROAD RD.'
>>> import re
>>> re.sub('ROAD$', 'RD.', s)
'100 NORTH BROAD RD.'
>>>

</pre>

<script language="javascript" type="text/javascript">
// You can place JavaScript like this

</script>
<?php
// You can place PHP like this

?>
{/source}

Matching Whole Words

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.>>> s = '100 BROAD'
>>> re.sub('ROAD$', 'RD.', s)
'100 BRD.'
>>> re.sub('\\bROAD$', 'RD.', s)
'100 BROAD'
>>> re.sub(r'\bROAD$', 'RD.', s)
'100 BROAD'
>>> s = '100 BROAD ROAD APT. 3'
>>> re.sub(r'\bROAD$', 'RD.', s)
'100 BROAD ROAD APT. 3'
>>> re.sub(r'\bROAD\b', 'RD.', s)
'100 BROAD RD. APT 3'

What I really wanted was to match 'ROAD' when it was at the end of the string and it was its own whole word, not a part of some larger word. To express this in a regular expression, you use \b, which means "a word boundary must occur right here". In Python, this is complicated by the fact that the '\' character in a string must itself be escaped. This is sometimes referred to as the backslash plague, and it is one reason why regular expressions are easier in Perl than in Python. On the down side, Perl mixes regular expressions with other syntax, so if you have a bug, it may be hard to tell whether it's a bug in syntax or a bug in your regular expression.

To work around the backslash plague, you can use what is called a raw string, by prefixing the string with the letter r. This tells Python that nothing in this string should be escaped; '\t' is a tab character, but r'\t' is really the backslash character \ followed by the letter t. I recommend always using raw strings when dealing with regular expressions; otherwise, things get too confusing too quickly (and regular expressions get confusing quickly enough all by themselves).

*sigh* Unfortunately, I soon found more cases that contradicted my logic. In this case, the street address contained the word 'ROAD' as a whole word by itself, but it wasn't at the end, because the address had an apartment number after the street designation. Because 'ROAD' isn't at the very end of the string, it doesn't match, so the entire call to re.sub ends up replacing nothing at all, and you get the original string back, which is not what you want.

To solve this problem, I removed the $ character and added another \b. Now the regular expression reads "match 'ROAD' when it's a whole word by itself anywhere in the string," whether at the end, the beginning, or somewhere in the middle.

{source}
<!-- You can place html anywhere within the source tags -->
<pre class="brush:py;">

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.>>> s = '100 BROAD'
>>> re.sub('ROAD$', 'RD.',
'100 BRD.'
>>> re.sub('\\bROAD$', 'RD.',
'100 BROAD'
>>> re.sub(r'\bROAD$', 'RD.',
'100 BROAD'
>>> s = '100 BROAD ROAD APT. 3'
>>> re.sub(r'\bROAD$', 'RD.',
'100 BROAD ROAD APT. 3'
>>> re.sub(r'\bROAD\b', 'RD.',
'100 BROAD RD. APT 3'

</pre>

<script language="javascript" type="text/javascript">
// You can place JavaScript like this

</script>
<?php
// You can place PHP like this

?>
{/source}