下面是一段解析脚本，使用了重定向知识。
#!/bin/bash for ipinfo in `cat ./test` do case "$ipinfo" in ip=*) echo $ipinfo for var in ipaddr bootp gateway netmask hostname netdev autoconf do eval read $var echo "$var" done << EOF `echo "$ipinfo" | sed "s/:/ /g" | sed "s/^[ ]*$/-/g"` EOF echo "ipaddr = $ipaddr" ipaddr=`echo "$ipaddr" | cut -d = -f 2` echo "ipaddr = $ipaddr" [ x$ipaddr == x ] && ipaddr=x echo "ipaddr = $ipaddr" ;; esac done echo "$ipaddr" echo "$bootp" echo "$gateway" echo "$netmask" echo "$hostname" echo "$netdev" echo "$autoconf"
用上面的脚本解析如下字符串：
noinitrd root=ubi0:rootfs rw rootflags=sync rootfstype=ubifs ubi.mtd=3 ip=$(ipaddr):$(serverip)::$(netmask)::eth0: console=ttyAMA0,38400n8 mem=192M gpio_careports=0x0F,0x00,0x00,0x00,0x00,0x00,0x00,0x00 mtdparts=xarina_nand1_flash:0x800000(spl),0x1000000(u-boot),0x2000000(ubifs),0x7000000(userland),0x2000000(config),0x12800000(local)
下面是执行结果：
1: ip=$(ipaddr):$(serverip)::$(netmask)::eth0:
2: ipaddr
3: bootp
4: gateway
5: netmask
6: hostname
7: netdev
8: autoconf
9: ipaddr = ip=$(ipaddr)
10: ipaddr = $(ipaddr)
11: ipaddr = $(ipaddr)
12: $(ipaddr)
13: $(serverip)
14: -
15: $(netmask)
16: -
17: eth0
18: -
重点是下面的语句：
1: for var in ipaddr bootp gateway netmask hostname netdev autoconf
2: do
3: eval read $var
4: echo "$var"
5: done << EOF
6: `echo "$ipinfo" | sed "s/:/ /g" | sed "s/^[ ]*$/-/g"`
7: EOF
分析：
当执行这个for循环的时候，${ipinfo}=ip=$(ipaddr):$(serverip)::$(netmask)::eth0:
执行完`echo "$ipinfo" | sed "s/:/ /g" | sed "s/^[ ]*$/-/g"` 的结果作为这个for的标准输入，相当于输入了：
1: ip=$(ipaddr)
2: $(serverip)
3: -
4: $(netmask)
5: -
6: eth0
7: -
sed "s/:/ /g" 的作用是将:替换为 ，而回车符是一次read命令的识别标准
sed "s/^[ ]*$/-/g" 的作用是将空行替换为-，*表示匹配空格至少0次（注意：[ ]中有一个空格），如果去掉*，就无法将空行替换为-了。
最终，对变量ipaddr bootp gateway netmask hostname netdev autoconf进行了一次赋值：
1: ipaddr = $(ipaddr)
2: bootp = $(serverip)
3: gateway = -
4: netmask =$(netmask)
5: hostname =-
6: netdev =eth0
7: autoconf =-
其中还是用了cut命令：http://www.cnblogs.com/pengdonglin137/p/3413696.html
下面是这种重定向用法的一些例子：
参考：http://mywiki.wooledge.org/BashFAQ/001
http://www.linuxjournal.com/content/bash-input-redirection
http://tldp.org/LDP/abs/html/redircb.html
http://stackoverflow.com/questions/12855000/bash-redirecting-eof-into-variable
How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
Don't try to use "for". Use a while loop and the read command:
while IFS= read -r line; do
printf '%s
' "$line"
done < "$file"
The -r option to read prevents backslash interpretation (usually used as a backslash newline pair, to continue over multiple lines). Without this option, any backslashes in the input will be discarded. You should almost always use the -r option with read.
The most common exception to this rule is when -e is used, which uses Readline to obtain the line from an interactive shell. In that case, tab completion will add backslashes to escape spaces and such, and you do not want them to be literally included in the variable. This would never be used when reading anything line-by-line, though, and -r should always be used when doing so.
In the scenario above IFS= prevents trimming of leading and trailing whitespace. Remove it if you want this effect.
line is a variable name, chosen by you. You can use any valid shell variable name there.
The redirection < "$file" tells the while loop to read from the file whose name is in the variable file. If you would prefer to use a literal pathname instead of a variable, you may do that as well. If your input source is the script's standard input, then you don't need any redirection at all.
If your input source is the contents of a variable/parameter, BASH can iterate over its lines using a "here string":
while read -r line; do
printf '%s
' "$line"
done <<< "$var"
The same can be done in any Bourne-type shell by using a "here document" (although read -r is POSIX, not Bourne):
while read -r line; do
printf '%s
' "$line"
done <<EOF
$var
EOF
If avoiding comments starting with # is desired, you can simply skip them inside the loop:
# Bash
while read -r line; do
[[ $line = #* ]] && continue
printf '%s
' "$line"
done < "$file"
If you want to operate on individual fields within each line, you may supply additional variables to read:
# Input file has 3 columns separated by white space.
while read -r first_name last_name phone; do
# Only print the last name (second column)
printf '%s
' "$last_name"
done < "$file"
If the field delimiters are not whitespace, you can set IFS (internal field separator):
# Extract the username and its shell from /etc/passwd:
while IFS=: read -r user pass uid gid gecos home shell; do
printf '%s: %s
' "$user" "$shell"
done < /etc/passwd
For tab-delimited files, use IFS=$' '.
You do not necessarily need to know how many fields each line of input contains. If you supply more variables than there are fields, the extra variables will be empty. If you supply fewer, the last variable gets "all the rest" of the fields after the preceding ones are satisfied. For example,
read -r first last junk <<< 'Bob Smith 123 Main Street Elk Grove Iowa 123-555-6789'
# first will contain "Bob", and last will contain "Smith".
# junk holds everything else.
Some people use the throwaway variable _ as a "junk variable" to ignore fields. It (or indeed any variable) can also be used more than once in a single read command, if we don't care what goes into it:
read -r _ _ first middle last _ <<< "$record"
# We skip the first two fields, then read the next three.
# Remember, the final _ can absorb any number of fields.
# It doesn't need to be repeated there.
The read command modifies each line read; by default it removes all leading and trailing whitespace characters (spaces and tabs, or any whitespace characters present in IFS). If that is not desired, the IFS variable has to be cleared:
# Exact lines, no trimming
while IFS= read -r line; do
printf '%s
' "$line"
done < "$file"
One may also read from a command instead of a regular file:
some command | while read -r line; do
printf '%s
' "$line"
done
This method is especially useful for processing the output of find with a block of commands:
find . -type f -print0 | while IFS= read -r -d '' file; do
mv "$file" "${file// /_}"
done
This reads one filename at a time from the find command and renames the file, replacing spaces with underscores.
Note the usage of -print0 in the find command, which uses NUL bytes as filename delimiters; and -d '' in the read command to instruct it to read all text into the file variable until it finds a NUL byte. By default, find and read delimit their input with newlines; however, since filenames can potentially contain newlines themselves, this default behaviour will split up those filenames at the newlines and cause the loop body to fail. Additionally it is necessary to set IFS to an empty string, because otherwise read would still strip leading and trailing whitespace. See FAQ #20 for more details.
Using a pipe to send find's output into a while loop places the loop in a SubShell and may therefore cause problems later on if the commands inside the body of the loop attempt to set variables which need to be used after the loop; in that case, see FAQ 24, or use a ProcessSubstitution like:
while read -r line; do
printf '%s
' "$line"
done < <(some command)
If you want to read lines from a file into an array, see FAQ 5.
My text files are broken! They lack their final newlines!
If there are some characters after the last line in the file (or to put it differently, if the last line is not terminated by a newline character), then read will read it but return false, leaving the broken partial line in the read variable(s). You can process this after the loop:
# Emulate cat
while IFS= read -r line; do
printf '%s
' "$line"
done < "$file"
[[ -n $line ]] && printf %s "$line"
Or:
# This does not work:
printf 'line 1
truncated line 2' | while read -r line; do echo $line; done
# This does not work either:
printf 'line 1
truncated line 2' | while read -r line; do echo "$line"; done; [[ $line ]] && echo -n "$line"
# This works:
printf 'line 1
truncated line 2' | { while read -r line; do echo "$line"; done; [[ $line ]] && echo "$line"; }
The first example, beyond missing the after-loop test, is also missing quotes. See Quotes or Arguments for an explanation why. The Arguments page is an especially important read.
For a discussion of why the second example above does not work as expected, see FAQ #24.
Alternatively, you can simply add a logical OR to the while test:
while IFS= read -r line || [[ $line ]]; do
printf '%s
' "$line"
done < "$file"
printf 'line 1
truncated line 2' | while read -r line || [[ $line ]]; do echo "$line"; done
How to keep other commands from "eating" the input
Some commands greedily eat up all available data on standard input. The examples above do not take precautions against such programs. For example,
while read -r line; do
cat > ignoredfile
printf '%s
' "$line"
done < "$file"
will only print the contents of the first line, with the remaining contents going to "ignoredfile", as cat slurps up all available input.
One workaround is to use a numeric FileDescriptor rather than standard input:
# Bash
while read -r -u 9 line; do
cat > ignoredfile
printf '%s
' "$line"
done 9< "$file"
# Note that read -u is not portable, and pretty much useless when you can use simple redirections instead:
while read -r line <&9; do
cat > ignoredfile
printf '%s
' "$line"
done 9< "$file"
Or:
# Bourne
exec 9< "$file"
while read -r line <&9; do
cat > ignoredfile
printf '%s
' "$line"
done
exec 9<&-
This example will wait for the user to type something into the file ignoredfile at each iteration instead of eating up the loop input.
You might need this, for example, with mencoder which will accept user input if there is any, but will continue silently if there isn't. Other commands that act this way include ssh and ffmpeg. Additional workarounds for this are discussed in FAQ #89.
Bash Input Redirection
May 17, 2008 By Mitch Frazier
in
If you use the shell you surely know about redirection:
# echo 'hello world' >output # cat <outputThe first line writes "hello world" to the file "output", the second reads it back and writes it to standard output (normally the terminal).
Then there are "here" documents:
# cat <<EOF > hello > world > EOFA "here" document is essentially a temporary, nameless file that is used as input to a command, here the "cat" command.
A less commonly seen form of here document is the "here" string:
# cat <<<'hello world'In this form the string following the "<<<" becomes the content of the "here" document.
Another less commonly seen form of redirection is redirecting to a specific file descriptor:
# echo 'Error: oops' >&2This redirects the output of the "echo" command to file descriptor 2, aka standard error. This is useful if you want to keep the error output of your scripts from contaminating the normal output when the output of your script is redirected.
These features work in bash and may not be available in other shells.
______________________
Mitch Frazier is an Associate Editor for Linux Journal.
Comments
Comment viewing options
Select your preferred way to display the comments and click "Save settings" to activate your changes.
Here is an example of a way
Submitted by Anonymous (not verified) on Fri, 09/04/2009 - 16:53.
Here is an example of a way to redirect command output without using a pipe.
while read line; do echo This file: $line; done <<EOF $(ls) EOF
Very useful. You could, for example, substitute ls for some program to query a database or sed commands for processing some files
I am not sure if this will run on old versions of bash though
It's worth mentioning some
Submitted by Adam Backstrom (not verified) on Tue, 05/20/2008 - 10:48.
It's worth mentioning some more examples of input redirection. For a file matched_files containing a list of files, you could grep the list:
grep mp3 < matched_files
Or do something to each file in the list:
while read line ; do
flac "$line" && rm -f "$line"
done < matched_filesLots more great examples are on the Useless Use of Cat Award page.
swap STDOUT and STDERR
Submitted by Serge van Ginderachter (not verified) on Sat, 05/17/2008 - 13:32.
swap STDOUT and STDERR: "3>&1 1>&2 2>&3"
as in:
(/usr/bin/$COMMAND $PARAM 3>&1 1>&2 2>&3 | grep -v $uninteresting_error ) 3>&1 1>&2 2>&3
Grep stderr
Submitted by Mitch Frazier on Sat, 05/17/2008 - 20:44.
In a bit more detail:
- 3>&1 - moves file descriptor 1 (aka stdout) to file descriptor 3.
- 1>&2 - moves file descriptor 2 (aka stderr) to file descriptor 1.
- 2>&3 - moves file descriptor 3 to file descriptor 2 (aka stderr).
tmp = stdout stdout = stderr stderr = tmpand thereby swapping the standard out and the standard error. Which then allows the stderr (rather than stdout) to be piped into grep.
Mitch Frazier is an Associate Editor for Linux Journal.
To elaborate further
Submitted by rak (not verified) on Tue, 01/19/2010 - 17:15.
3>&1 redirects output sent to 3 to stdout,
1>&2 redirects output sent to stdout to stderr,
and 2>&3 redirects output sent to stderr to 3.
If you just want to include stderr output in your stdout along with the existing stdout output, you can just use 2>&1
20.2. Redirecting Code Blocks
Blocks of code, such as while, until, and for loops, even if/then test blocks can also incorporate redirection of stdin. Even a function may use this form of redirection (see Example 24-11). The < operator at the end of the code block accomplishes this.
Example 20-5. Redirected while loop
#!/bin/bash
# redir2.sh
if [ -z "$1" ]
then
Filename=names.data # Default, if no filename specified.
else
Filename=$1
fi
#+ Filename=${1:-names.data}
# can replace the above test (parameter substitution).
count=0
echo
while [ "$name" != Smith ] # Why is variable $name in quotes?
do
read name # Reads from $Filename, rather than stdin.
echo $name
let "count += 1"
done <"$Filename" # Redirects stdin to file $Filename.
# ^^^^^^^^^^^^
echo; echo "$count names read"; echo
exit 0
# Note that in some older shell scripting languages,
#+ the redirected loop would run as a subshell.
# Therefore, $count would return 0, the initialized value outside the loop.
# Bash and ksh avoid starting a subshell *whenever possible*,
#+ so that this script, for example, runs correctly.
# (Thanks to Heiner Steven for pointing this out.)
# However . . .
# Bash *can* sometimes start a subshell in a PIPED "while-read" loop,
#+ as distinct from a REDIRECTED "while" loop.
abc=hi
echo -e "1
2
3" | while read l
do abc="$l"
echo $abc
done
echo $abc
# Thanks, Bruno de Oliveira Schneider, for demonstrating this
#+ with the above snippet of code.
# And, thanks, Brian Onn, for correcting an annotation error.
Example 20-6. Alternate form of redirected while loop
#!/bin/bash
# This is an alternate form of the preceding script.
# Suggested by Heiner Steven
#+ as a workaround in those situations when a redirect loop
#+ runs as a subshell, and therefore variables inside the loop
# +do not keep their values upon loop termination.
if [ -z "$1" ]
then
Filename=names.data # Default, if no filename specified.
else
Filename=$1
fi
exec 3<&0 # Save stdin to file descriptor 3.
exec 0<"$Filename" # Redirect standard input.
count=0
echo
while [ "$name" != Smith ]
do
read name # Reads from redirected stdin ($Filename).
echo $name
let "count += 1"
done # Loop reads from file $Filename
#+ because of line 20.
# The original version of this script terminated the "while" loop with
#+ done <"$Filename"
# Exercise:
# Why is this unnecessary?
exec 0<&3 # Restore old stdin.
exec 3<&- # Close temporary fd 3.
echo; echo "$count names read"; echo
exit 0
Example 20-7. Redirected until loop
#!/bin/bash # Same as previous example, but with "until" loop. if [ -z "$1" ] then Filename=names.data # Default, if no filename specified. else Filename=$1 fi # while [ "$name" != Smith ] until [ "$name" = Smith ] # Change != to =. do read name # Reads from $Filename, rather than stdin. echo $name done <"$Filename" # Redirects stdin to file $Filename. # ^^^^^^^^^^^^ # Same results as with "while" loop in previous example. exit 0
Example 20-8. Redirected for loop
#!/bin/bash
if [ -z "$1" ]
then
Filename=names.data # Default, if no filename specified.
else
Filename=$1
fi
line_count=`wc $Filename | awk '{ print $1 }'`
# Number of lines in target file.
#
# Very contrived and kludgy, nevertheless shows that
#+ it's possible to redirect stdin within a "for" loop...
#+ if you're clever enough.
#
# More concise is line_count=$(wc -l < "$Filename")
for name in `seq $line_count` # Recall that "seq" prints sequence of numbers.
# while [ "$name" != Smith ] -- more complicated than a "while" loop --
do
read name # Reads from $Filename, rather than stdin.
echo $name
if [ "$name" = Smith ] # Need all this extra baggage here.
then
break
fi
done <"$Filename" # Redirects stdin to file $Filename.
# ^^^^^^^^^^^^
exit 0
We can modify the previous example to also redirect the output of the loop.
Example 20-9. Redirected for loop (both stdin and stdout redirected)
#!/bin/bash
if [ -z "$1" ]
then
Filename=names.data # Default, if no filename specified.
else
Filename=$1
fi
Savefile=$Filename.new # Filename to save results in.
FinalName=Jonah # Name to terminate "read" on.
line_count=`wc $Filename | awk '{ print $1 }'` # Number of lines in target file.
for name in `seq $line_count`
do
read name
echo "$name"
if [ "$name" = "$FinalName" ]
then
break
fi
done < "$Filename" > "$Savefile" # Redirects stdin to file $Filename,
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^ and saves it to backup file.
exit 0
Example 20-10. Redirected if/then test
#!/bin/bash if [ -z "$1" ] then Filename=names.data # Default, if no filename specified. else Filename=$1 fi TRUE=1 if [ "$TRUE" ] # if true and if : also work. then read name echo $name fi <"$Filename" # ^^^^^^^^^^^^ # Reads only first line of file. # An "if/then" test has no way of iterating unless embedded in a loop. exit 0
Example 20-11. Data file names.data for above examples
Aristotle Arrhenius Belisarius Capablanca Dickens Euler Goethe Hegel Jonah Laplace Maroczy Purcell Schmidt Schopenhauer Semmelweiss Smith Steinmetz Tukhashevsky Turing Venn Warshawski Znosko-Borowski # This is a data file for #+ "redir2.sh", "redir3.sh", "redir4.sh", "redir4a.sh", "redir5.sh".
Redirecting the stdout of a code block has the effect of saving its output to a file. See Example 3-2.
Here documents are a special case of redirected code blocks. That being the case, it should be possible to feed the output of a here document into the stdin for a while loop.
# This example by Albert Siersema
# Used with permission (thanks!).
function doesOutput()
# Could be an external command too, of course.
# Here we show you can use a function as well.
{
ls -al *.jpg | awk '{print $5,$9}'
}
nr=0 # We want the while loop to be able to manipulate these and
totalSize=0 #+ to be able to see the changes after the 'while' finished.
while read fileSize fileName ; do
echo "$fileName is $fileSize bytes"
let nr++
totalSize=$((totalSize+fileSize)) # Or: "let totalSize+=fileSize"
done<<EOF
$(doesOutput)
EOF
echo "$nr files totaling $totalSize bytes"
bash redirecting EOF into variable
this is my file perl5lib.sh:
export PERL5LIB=`cat |tr '
' ':'`<<EOF
/home/vul/repository/projects/implatform/daemon/trunk/lib/
/home/vul/repository/projects/platformlib/tool/trunk/cpan_lib
/home/projects/libtrololoI want to start file as
. perl5lib.shto populate PERL5LIB variable, but it hangs. What is wrong? my goal is to left folder names at the end of file, so I can add new simply:
echo dirname >> myscriptI have tried and export PERL5LIB=$(echo blabla) and cat<<EOF both work separately, but not together.
=================== THE SOLUTION ============================
function do the trick!
function fun
{
export PERL5LIB=`cat|tr '
' ':'`
}
fun<<EOF
/dir1/lib
/dir2/lib
/dir3/libasked Oct 12 '12 at 8:24
xoid
1207
Please work on your accept rate. – Shahbaz Oct 12 '12 at 8:30
This should not work. It should complain about a missing here-document terminator. And there's a useless cat as well. – Jens Oct 12 '12 at 13:13
your "function do the trick!" section spawns two extra child processes – bobah Oct 12 '12 at 15:30
1
Please remove the "does the trick" section and accept one of the answers (or make it an answer and accept it). – einpoklum Apr 19 '13 at 6:27
3 Answers
cat is useless here. Provide EOF inside the subshell:
#! /bin/bash
export PERL5LIB=$(tr '
' ':'<<EOF
/home/vul/repository/projects/implatform/daemon/trunk/lib/
/home/vul/repository/projects/platformlib/tool/trunk/cpan_lib
/home/projects/libtrololo
EOF
)answered Oct 12 '12 at 8:34
choroba
42k32760
What you call "EOF" can be googled as Here Document. Here Document can only be used to feed the standard input of a command. The below example does what you want without spawning child processes
#!/bin/bash
multilineconcat=
while read line; do
#echo $line
multilineconcat+=$line
multilineconcat+=":"
done << EOF
path1
path2
EOF
echo $multilineconcatanswered Oct 12 '12 at 8:41
bobah
6,2721223
Isn't it waiting for the EOF in your heredoc ?
I'd expect you to say
$ mycommand <<EOF
input1
input2
...
EOFNote that EOF isn't a magic keyword. It's just a marker, and could be anything (ABC etc.). It indicates the end-of-file, but people simply write EOF as convention.
answered Oct 12 '12 at 8:26
Brian Agnew
133k11150249
No, you are wrong. End of the file is EOF itself – xoid Oct 12 '12 at 8:29
No, EOF here is just a literal string. – chepner Oct 12 '12 at 12:12