17.2 Blank Line and Empty Line Removal
20190204 A task we sometimes find useful is to remove all of the blank lines from a file. Suppose we have empty lines separating sentences and want to bring them together into a single paragraph:
## Linux has many powerful command line tools dating from the 1970's. ## ## Unix and Linux tools focus on a specific task. ## ## Tools then freely collaborate together to accomplish the task.
We can remove blank lines using awk:
awk NF < myfile_blank_lines.txt
## Linux has many powerful command line tools dating from the 1970's. ## Unix and Linux tools focus on a specific task. ## Tools then freely collaborate together to accomplish the task.
This very simple use of awk is powerful and elegant in its
simplicity. The awk variable
NF is the number of fields in
the input record. For blank lines this is zero. The default
awk action is to print lines for which the pattern is
true. Here the pattern is
NF. Zero is interpreted as false
and so nothing is printed.
We can save the result into a new file:
$ awk NF < myfile.txt > noblanks.txt
We could also use sed.
$ sed '/^$/d' myfile.txt
The pattern here will match any empty line since
^ matches the
$ matches the end of the line and allows for no
other characters in between. For lines that match the pattern the
d operator is utilised to delete the line.
Then we can operate on the file in-place using
-i which is
$ sed -i '/^$/d' myfile.txt
Some further alternatives and interesting variations below and also see Stack Overflow.
$ sed '/^ *$/d' myfile.txt # Blank lines may have space. $ sed '/^[[:space:]]*$/d' myfile.txt # Or they may include tabs. $ sed '/^\s*$/d' myfile.txt # Short version of above. $ grep . myfile.txt # Grep non-empty lines. $ grep '\S' myfile.txt # Grep non-space only lines.
Your donation will support ongoing development and give you access to the PDF version of this book. Desktop Survival Guides include Data Science, GNU/Linux, and MLHub. Books available on Amazon include Data Mining with Rattle and Essentials of Data Science. Popular open source software includes rattle, wajig, and mlhub. Hosted by Togaware, a pioneer of free and open source software since 1984. Copyright © 1995-2021 Graham.Williams@togaware.com Creative Commons Attribution-ShareAlike 4.0.