I love mjd's It came from... the HOLD SPACE.
This article describes some of the culture that I grew up in.
Slide 18 in particular....wow, that takes me back... I can remember cutting my teeth on some sed/shell-scripts back when I was younger. I had work that needed to be done, and these seemed like the logical tools to do it with.
I made progress on this work too. For example, in one frenetic afternoon/evening I wrote a script that my boss (later) claimed saved the company....a huge sum of money. It was nice to be able to solve some big problems with these tools.
However, I never did really fall in love with these tools. The tools themselves had some downsides: not portable enough ; subject to arbitrary internal buffer limitations ; and, most importantly, for some of the problems I was trying to solve I felt like I was working with my arms tied behind my back.
In mjd's slides, you can see him grapple with a couple of problems in the "sed" section. When I saw this, I got this awful case of déjà vu...
I have a friend who is 100 times as good of a sed programmer as I am. I have seen him whip together phenomenal sed programs that accomplish some totally neato and useful work. I never really got to be a good sed programmer, and I always admired his skills in this area.
However....I never really felt like I needed to put in a ton of time into sharpening my sed skills. I can remember on one specific day, fighting like crazy to get sed to do The Right Thing in a certain text transformation. I must have fought for two hours to try to get things right. But the problem I was running into was a lot like the problem mjd brought up in his article -- what to do about that last line?
Eventually, I decided that it wasn't a good use of my time to spend a huge amount of time with tools like this. There was this neato new language called Perl....and it looked very similar to what I already knew....but it really didn't have any of the limitations that I regularly found myself dealing with with the old tools. So, I spent quite a long time with Perl, and that proved to be pretty fruitful.
Nowadays, I pretty regularly work with some pretty huge datasets with my trusted tool: Perl. I still find the old tools to be useful, but when the problems get serious, I reach for more powerful tools.