The one that I find repeatedly useful:

find . -name <some pattern> | { while read F; do <do something to file F>; done; }

This lets you do things of almost any level of complexity to files or data listed as input and it doesn't require temporary files so you can process a lot of things.

e.g. you might use sed -i to alter the files in some way or you might find media files of some kind into ffmpeg to convert them.

> ps -ef | grep VBoxHeadless | awk '{print $2}' | xargs kill -9

There's pkill for that :)

  pkill -KILL VBoxHeadless

> Here’s a real-world example I used recently

    gci -rec | ? name -match zabb | ? Extension -eq .deb | % Name  

    gci -rec | ? name -match zabb.+\.deb$ | % name  

    Mode                : --r--  
    BaseName            : zabbix-release_5.0-1+bionic_all  
    Length              : 4240  
    DirectoryName       : /home/appliance  
    FullName            : /home/appliance/zabbix-release_5.0-1+bionic_all.deb  
    Extension           : .deb  
    Name                : zabbix-release_5.0-1+bionic_all.deb  
    CreationTime        : 5/11/2020 12:07:35 PM  

    Get-Process apach* | Stop-Process -Force  

> Find Files Ending With…

    gci / -re | ? name -match \.deb$  

> $ env | grep -w PATH | tr ':' '\n'

    PS /home> ( env ) -match '^PATH' -replace ':',"`n"  
    PATH=/opt/microsoft/powershell/7  
    /home/appliance/bin  
    /home/appliance/.local/bin  
    [stripped]  
    

As usual - there is a ton of useful and nice QOL utils in *nix world, but it always feels like scratching left ear with the right hand.

> Quick Infinite Loop

I use this a lot for polling type commands:

  do_every_n_secs "ps -eaf | grep blablabla" 5


  function do_every_n_secs (){
    cmd=$1
    secs=$2
    if [[ $secs -eq "" ]]; then
      secs=10
    fi
    while true; do
      echo "################################################################";
      date;
      eval $cmd
      sleep $secs;
    done
  }

Interesting stuff. One small correction that I noticed:

   $ sudo locate cfg | grep \.cfg$

   $ sudo locate cfg | grep '\.cfg$'

But locate(1) supports globs, so I think this could be simplified to just:

   $ sudo locate '*.cfg'

   $ find -name '*.cfg'

bash 4 supports `|&` as an alternative to `2>&1` which looks better in pipelines e.g.

  $ docker logs container |& grep word

Can anyone recommend a modern explanation of sed's more advanced features?

I'm ashamed that, despite having learned many other ostensibly more complex systems and languages, sed continues to elude me. Of course I get the usual 's/foo/bar/g' stuff with regular expressions and placeholders, but I still don't really understand most of the other commands or, more usefully, the addresses.

Perhaps I need to spend quality time with the manpage in a cabin in the woods, away from modern distractions, but so far it has proven impermeable.

Perhaps I'm missing a proper understanding of its 'pattern space' vs 'hold space'.

I try to use "smaller" commands instead of more versatile but complex alternatives. I'm not sure if I get better cpu/io/etc usage by using cut or tr instead of sed or awk (or perl and so on) but it is a pattern I followed by decades, at least for simple enough commands. And I may have something less to remember if I do i.e. rev | cut | rev instead of awk to get the last field.

And having something less to remember, and lowering the complexity for the reader (that not always be me, or if I am, may not be in the same mind state as when I wrote that) is usually good. But it also is having predictable/consistent patterns.

As an aside, "zwischenzug" is a great name. A "zwischenzug" is an "inbetween move" in chess that might happen in the middle of a sequence, and can often lead to unexpected results if one side hasn't been precise in their move order.

https://www.chessjournal.com/zwischenzug/ for example has some more info if you're curious.

One feature I really wish for in shells is something similar to Perl's unquoted strings / single line concise HEREDOC syntax / raw literal syntax. e.g.

  $ echo q(no ' escaping ` required " here)
  no ' escaping ` required " here

Going to echo: Bash is bad. Bash is a feature-impoverished language and belongs in the dustbin. I don't understand why we script with one hand tied behind our back.

With Python I can install "click" and get great argument parsing and multi-command support with a simple install. Bash's argument parsing is much more verbose and arcane.

Sure, it has nice ways to combine tools and parse strings and such. However: You could also implement those nice abstractions in a higher quality language. So a good thing about bash does not cancel out all the bad.

I would LOVE a toolkit that is:

- A single binary

- That parses a single file like make ( but NOT a makefile )

- That uses a well known, powerful language.

- That lets me declare tasks, validations ( required arguments, etc ), and workflows as a first-class citizen

- With out of the box `--help` support

- That lets me import other makefile-like files to improve code re-use ( both remote and local )

As much as I like these, these are also how you blow both feet off.

As long as you're just exploring (everything is read only), you're okay.

The moment "kill" and "rm" (anything mutable) come into the picture on a long shell pipeline, stop. You probably don't want to do that.

You need to be thinking very hard to make sure that an errant glob or shell substitution isn't going to wipe out your system. Make a complicated pipeline that creates a simple file, examine the simple file by hand, and then feed that file to "kill"/"rm"/"whatever" as close to unmodified as possible.

You may still blow your system away. However, at that point, you've given yourself the best chances to not do so.

Being able to write a file directly in the shell with heredocs is so cool, but I know I'll never use it because it's far quicker to just open vim and type with all the commands and verbs and motions that I'm used to.

I know there are many ways the same thing can be done in the shell but there are so many problems here. Please take this as feedback and not harsh criticism (I know there's comment section on the blog but I'd rather not give my e-mail to yet another website).

    > cat /etc/passwd | sed 's/\([^:]*\):.*:\(.*\)/user: \1 shell: \2/'

    awk '{FS=":"} {print "user: " $1, "shell: " $NF}' /etc/passwd

    > $ ls | xargs -t ls -l

    > find . | grep azurerm | grep tf$ | xargs -n1 dirname | sed 's/^.\///'
    >
    > ...
    >
    > for each of those files, removes the leading dot and forward slash, leaving the final bare filename (eg somefile.tf)

    > $ ls | grep Aug | xargs -IXXX mv XXX aug_folder

   mv *Aug* aug_folder/

    > $ ps -ef | grep VBoxHeadless | awk '{print $2}' | xargs kill -9

    pkill VBoxHeadless

    > $ sudo updatedb
    > $ sudo locate cfg | grep \.cfg$

    locate '*.cfg'

    > $ grep ^.https doc.md | sed 's/^.(h[^])]).*/open \1/' | sh

    > This example looks for https links at the start of lines in the doc.md file

The sed then matches start of the line, followed by a character, starts a capture group of the letter "h" followed by a single character that is not "]" or ")", closes the capture group and continues to match anything.

The example given will always result in "open ht" for any hit.

Then there's the piping to sh. You mention like to drop this to review the list. I'd suggest focusing on extracting the list of links then pipe to xargs open. If you get a pipeline wrong you could unintentionally blast something into a shell executing anything which might be very bad.

    > Give Me All The Output With 2>&1

    &> stdout-and-stderr

    > env | grep -w PATH | tr ':' '\n'

   echo $PATH | tr ':' '\n'

    tr ':' '\n' <<< $PATH

No mention of my pet peeve! You don't need a subshell/seq to count.

The shell is smart, it can count for you - and even respect digits. These examples are subtle, but different in result:

    for i in {00..10} ; do echo $i ; done
    for i in {0..10} ; do echo $i ; done

I have written thousands locs in bash but every time I have to write another one my teeth start to ache. Not sure why

After almost 30 years and tens of thousands of LOC, possibly even into 6 figures, I've now thrown in the towel and just use Python for all my scripting needs. I've yet to have a single shell script that hasn't turned into a maintenance nightmare, regardless of "bash best practices" (such as they are).

Bash is a terrible language, and needs to die.