如何使用临时文件?

需求:
某项目中,我们传感器采集数据,每收集到1G数据后,做数据分析,最终只保存分析结果。这样很大的临时数据如果常驻内存,将消耗大量内存资源。我们可以使用临时文件来存储这些临时数据(外部存储:硬盘)
临时文件不用命名,县城关闭后会自动删除。

思路:
使用标准库中的tempfile下的TemporaryFile,NamedTemporaryFile

代码:

from tempfile import TemporaryFile , NamedTemporaryFile # 这两个都是创建临时文件,只是named这个可以对临时文件进行命名
# 创建一个临时文件的对象,在文件系统中是找不到这个临时文件的,只能由对象f来访问
f = TemporaryFile()
# 生成临时数据并放入临时文件中去
f.write(b'abcdef'* 100000)  # 注意python3中要加b,写入到磁盘中去
f.seek(0)  # 把文件的指针移到文件的开头
# 根据具体的需求,把临时文件中的数据读入到内存中
f.read(100) # b'abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd'

# 创建一个可以在文件系统中找到的临时文件,使用NamedTemporaryFile
ntf = NamedTemporaryFile()  # 返回的不直接是一个文件对象
print(ntf.name)  # 描述了在文件系统下的name属性
ntf = NamedTemporaryFile(delete=False) # 这样在临时文件关闭后,不会进行自动删除

==============================================================================

>>> from tempfile import TemporaryFile,NamedTemporaryFile

>>> TemporaryFile?
Signature:
TemporaryFile(
    mode='w+b',
    buffering=-1,
    encoding=None,
    newline=None,
    suffix=None,
    prefix=None,
    dir=None,
)
Docstring:
Create and return a temporary file.
Arguments:
'prefix', 'suffix', 'dir' -- as for mkstemp.
'mode' -- the mode argument to io.open (default "w+b").
'buffering' -- the buffer size argument to io.open (default -1).
'encoding' -- the encoding argument to io.open (default None)
'newline' -- the newline argument to io.open (default None)
The file is created as mkstemp() would do it.

Returns an object with a file-like interface.  The file has no
name, and will cease to exist when it is closed.
File:      /usr/lib/python3.6/tempfile.py
Type:      function

>>> tf = TemporaryFile()

>>> tf.write(b'*' * 1024 * 1024)
1048576

>>> tf.write(b'*' * 1024 * 1024)
1048576

>>> tf.write(b'*' * 1024 * 1024)
1048576

>>> tf.write(b'*' * 1024 * 1024)
1048576

>>> tf.seek(0)
0

>>> tf.read(512)
b'********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************'

>>> tf.close()

>>> tf = TemporaryFile()

>>> tf.write(b'*' * 1024 * 1024 * 1024)
1073741824

>>> tf.close()

>>> TemporaryFile
<function tempfile.TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None)>

>>> import os

>>> NamedTemporaryFile?
Signature:
NamedTemporaryFile(
    mode='w+b',
    buffering=-1,
    encoding=None,
    newline=None,
    suffix=None,
    prefix=None,
    dir=None,
    delete=True,
)
Docstring:
Create and return a temporary file.
Arguments:
'prefix', 'suffix', 'dir' -- as for mkstemp.
'mode' -- the mode argument to io.open (default "w+b").
'buffering' -- the buffer size argument to io.open (default -1).
'encoding' -- the encoding argument to io.open (default None)
'newline' -- the newline argument to io.open (default None)
'delete' -- whether the file is deleted on close (default True).
The file is created as mkstemp() would do it.

Returns an object with a file-like interface; the name of the file
is accessible as its 'name' attribute.  The file will be automatically
deleted when it is closed unless the 'delete' argument is set to False.
File:      /usr/lib/python3.6/tempfile.py
Type:      function

>>> ntf = NamedTemporaryFile()

>>> ntf.name
'/tmp/tmph3tgjae_'

>>> ll /tmp/tmph3tgjae_
-rw------- 1 richardo 0 11月  4 09:59 /tmp/tmph3tgjae_

>>> import tempfile

>>> tempfile.gettempdir()
'/tmp'

>>> tempfile.gettempprefix()
'tmp'

>>> ntf.close()

>>> ll /tmp/tmph3tgjae_
ls: 无法访问'/tmp/tmph3tgjae_': 没有那个文件或目录

>>> ntf2 = NamedTemporaryFile(delete=False)

>>> ntf2.name
'/tmp/tmpnpjilyx5'

>>> ll  /tmp/tmpnpjilyx5
-rw------- 1 richardo 0 11月  4 10:01 /tmp/tmpnpjilyx5

>>> ntf2.close()

>>> ll  /tmp/tmpnpjilyx5
-rw------- 1 richardo 0 11月  4 10:01 /tmp/tmpnpjilyx5

原文地址:https://www.cnblogs.com/Richardo-M-Q/p/13326340.html