You will understand why Sed is concise the first time you write a lexer in C by hand. To lex a single character token you don't need any FSMs; the input character = the lookahead = all you need to recognize that token. You can use getc and ungetc to move forth and back in the input stream; the whole thing can be done with one tidy switch statement.

Lex and Yacc where non-existent, or at least a luxury at one point. Volumes where devoted to tweaking grammars so the internal data structures used by lexers and parsers didn't exceed the minimum core available at the time.

The following page explains "Famous Sed One-Liners", including the one you mention (it's #36, though you'll likely want to work your way down from the top):

http://www.catonmat.net/blog/sed-one-liners-explained-part-o...

(CatOnMat.net is the site of one Peteris Krumins; he also has pages about AWK and Perl one-liners and he also has some other interesting stuff, e.g. he's gone through MIT's Introduction to Algorithms course videos and blogged each one, breaking the videos down by topics, with timestamps, as well as scans of his handwritten notes, etc.).

I actually still use sed once in a while. I sometimes show up at a corporate client and need to cleanup files on a unix or cygwin machine that is completely locked down (no install of anything allowed ever) and doesn't have anything easier to use (perl/python). At least you can always fall back to it and perform regexp replaces, fix unix/dos newlines, etc.

If you think sed is bad try to make sense of apl:

(~v∊v∘.×v)/v←1↓⍳10000

For all the primes up to 10000

Isn't the basic idea that sed is essentially typed-out keyboard shortcuts/commands for ed?

hmmm HN looks like is susceptible to Code injection! Work on it guys