Most of the tools do not actually read a single line in from a file at a time, rather they use a buffer in memory to store chunks of lines. The tools operate a line at a time on the data in this buffer.
NOTE: By "line" I mean split by a , in grep's case, or whatever character is denoted as the "delimiter" when the "tool" is invoked.
Here's a example to illustrate this effect.
Sample data
Create a file with 100,000 lines in it. The file is called afile100k.
$ for i in $(seq 100000);do echo "file$i" >> afile100k; done
Using strace
We can utilize strace to peak inside a running process, in this case the grep command.
$ strace -s 2000 -o log100k grep 5 afile100k
This will log the output from strace, up to 2000 characters per line of output to the file log100k. The command we'll be tracing is grep 5 afile100k.
Here's some key output from the log:
ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff8bf73b20) = -1 ENOTTY (Inappropriate ioctl for device)
read(3, "file1 file2 file3 file4 file5 file6 file7 file8 file9 file10 file11 file12 file13 file14 file15 file16 file17 file18 file19 file20 file21 file22 file23 file24 file25 file26 file27 file28 file29 file30 file31 file32 file33 file34 file35 file36 file37 file38 file39 file40 file41 file42 file43 file44 file45 file46 file47 file48 file49 file50 file51 file52 file53 file54 file55 file56 file57 file58 file59 file60 file61 file62 file63 file64 file65 file66 file67 file68 file69 file70 file71 file72 file73 file74 file75 file76 file77 file78 file79 file80 file81 file82 file83 file84 file85 file86 file87 file88 file89 file90 file91 file92 file93 file94 file95 file96 file97 file98 file99 file100 file101 file102 file103 file104 file105 file106 file107 file108 file109 file110 file111 file112 file113 file114 file115 file116 file117 file118 file119 file120 file121 file122 file123 file124 file125 file126 file127 file128 file129 file130 file131 file132 file133 file134 file135 file136 file137 file138 file139 file140 file141 file142 file143 file144 file145 file146 file147 file148 file149 file150 file151 file152 file153 file154 file155 file156 file157 file158 file159 file160 file161 file162 file163 file164 file165 file166 file167 file168 file169 file170 file171 file172 file173 file174 file175 file176 file177 file178 file179 file180 file181 file182 file183 file184 file185 file186 file187 file188 file189 file190 file191 file192 file193 file194 file195 file196 file197 file198 file199 file200 file201 file202 file203 file204 file205 file206 file207 file208 file209 file210 file211 file212 file213 file214 file215 file216 file217 file218 file219 file220 file221 file222 file223 file224 file225 file226 file227 file228 file229 file230 file231 file232 file233 file234 file235 file236 file237 file238 file239 file240 file241 file242 file243 file244 file245 file246 file247 file248 file249 file250 file251 file252 file253 file254 file255 file256 file257 file258 file259 file260 file261 file262 file263 file"..., 32768) = 32768
lseek(3, 32768, SEEK_HOLE) = 988895
lseek(3, 32768, SEEK_SET) = 32768
Notice grep is reading in 32k (32768) at a time. NOTE: I've tried to break up the log a bit so that it's easier to read here on SE.
Now it starts writing out results:
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fcafcfff000
write(1, "file5
", 6) = 6
write(1, "file15
", 7) = 7
write(1, "file25
", 7) = 7
write(1, "file35
", 7) = 7
write(1, "file45
", 7) = 7
write(1, "file50
", 7) = 7
After exhausting out the contents of this buffer it will re-read the next 32k (32768) chunk from the file.
write(1, "file3759
", 9) = 9
read(3, "
file3765
file3766
file3767
file3768
file3769
file3770
file3771
file3772
file3773....
file3986
f"..., 32768) = 32768
Followed by more writes:
write(1, "file3765
", 9) = 9
write(1, "file3775
", 9) = 9
Grep continues to do this until it's completely exhausted the contents of the file, at which point it ends.
write(1, "file99995
", 10) = 10
read(3, "", 24576) = 0
close(3) = 0
close(1) = 0
munmap(0x7fcafcfff000, 4096) = 0
close(2) = 0
exit_group(0) = ?
+++ exited with 0 +++
工作中遇到grep -w比较慢，从而strace -p跟踪进程过程，google了下比较通俗易懂，就不翻译了，记录下
原文link