DiskLruCache 阅读笔记

DiskLruCache使用名为 journal 的日志文件来记录对缓存的操作。
* This cache uses a journal file named "journal". A typical journal file
* looks like this:
*     libcore.io.DiskLruCache
*     1
*     100
*     2
*
*     CLEAN 3400330d1dfc7f3f7f4b8d4d803dfcf6 832 21054
*     DIRTY 335c4c6028171cfddfbaae1a9c313c52
*     CLEAN 335c4c6028171cfddfbaae1a9c313c52 3934 2342
*     REMOVE 335c4c6028171cfddfbaae1a9c313c52
*     DIRTY 1ab96a171faeeee38496d8b330771a7a
*     CLEAN 1ab96a171faeeee38496d8b330771a7a 1600 234
*     READ 335c4c6028171cfddfbaae1a9c313c52
*     READ 3400330d1dfc7f3f7f4b8d4d803dfcf6


libcore.io.DiskLruCache 是一种声明,表示该文件是使用 DiskLruCache 生成的。
1 表示缓存的版本号
100 表示应用的版本号
2 表示每个 key 值对应的缓存文件的个数,通常使用 1 即可。

CLEAN 表示记录了对缓存文件的操作已经完成,表示一个无错误的操作。
DIRTY 表示记录了正在操作缓存文件的操作,该记录后面必须要跟一个 CLEAN 和 REMOVE 记录,用来表示该操作成功(CLEAN)或者失败(REMOVE)了。
READ 表示记录了一个获取缓存文件的操作。
REMOVE 表示记录了一条删除缓存文件的操作。

3400330d1dfc7f3f7f4b8d4d803dfcf6 这样的字符串表示缓存文件的文件名,也是 key 值。CLEAN 记录后面跟着的多个数字表示缓存文件的大小,key 对应多个缓存文件,就有多个数字。


先看构造函数:

private DiskLruCache(File directory, int appVersion, int valueCount, long maxSize, int maxFileCount) {
   this.directory = directory;
   this.appVersion = appVersion;
   this.journalFile = new File(directory, JOURNAL_FILE);
   this.journalFileTmp = new File(directory, JOURNAL_FILE_TEMP);
   this.journalFileBackup = new File(directory, JOURNAL_FILE_BACKUP);
   this.valueCount = valueCount;
   this.maxSize = maxSize;
   this.maxFileCount = maxFileCount;
}

主要是给成员变量赋值,初始化版本号,缓存容量等,构造函数私有,只能通过别的方式来获取实例对象。

使用 DiskLruCache ,只能使用 open() 方法:

	/**
	 * Opens the cache in {@code directory}, creating a cache if none exists
	 * there.
	 *
	 * @param directory a writable directory
	 * @param valueCount the number of values per cache entry. Must be positive.
	 * @param maxSize the maximum number of bytes this cache should use to store
	 * @param maxFileCount the maximum file count this cache should store
	 * @throws IOException if reading or writing the cache directory fails
	 */
	public static DiskLruCache open(File directory, int appVersion, int valueCount, long maxSize, int maxFileCount)
			throws IOException {
		if (maxSize <= 0) {
			throw new IllegalArgumentException("maxSize <= 0");
		}
		if (maxFileCount <= 0) {
			throw new IllegalArgumentException("maxFileCount <= 0");
		}
		if (valueCount <= 0) {
			throw new IllegalArgumentException("valueCount <= 0");
		}

		// If a bkp file exists, use it instead.
		File backupFile = new File(directory, JOURNAL_FILE_BACKUP);
		if (backupFile.exists()) {
			File journalFile = new File(directory, JOURNAL_FILE);
			// If journal file also exists just delete backup file.
			if (journalFile.exists()) {
				backupFile.delete();
			} else {
				renameTo(backupFile, journalFile, false);
			}
		}

		// Prefer to pick up where we left off.
		DiskLruCache cache = new DiskLruCache(directory, appVersion, valueCount, maxSize, maxFileCount);
		if (cache.journalFile.exists()) {
			try {
				cache.readJournal();
				cache.processJournal();
				cache.journalWriter = new BufferedWriter(
						new OutputStreamWriter(new FileOutputStream(cache.journalFile, true), Util.US_ASCII));
				return cache;
			} catch (IOException journalIsCorrupt) {
				System.out
						.println("DiskLruCache "
								+ directory
								+ " is corrupt: "
								+ journalIsCorrupt.getMessage()
								+ ", removing");
				cache.delete();
			}
		}

		// Create a new empty cache.
		directory.mkdirs();
		cache = new DiskLruCache(directory, appVersion, valueCount, maxSize, maxFileCount);
		cache.rebuildJournal();
		return cache;
	}

1.先检查设置是否正常。

2.如果日志备份文件存在,日志文件不存在,将备份文件替重命名为日志文件,日志文件存在,删除备份文件。

3.实例化 DiskLruCache,日志文件存在,则读取并处理日志文件,否则重新创建一个日志文件。

private void readJournal() throws IOException {
   StrictLineReader reader = new StrictLineReader(new FileInputStream(journalFile), Util.US_ASCII);
   try {
      String magic = reader.readLine();
      String version = reader.readLine();
      String appVersionString = reader.readLine();
      String valueCountString = reader.readLine();
      String blank = reader.readLine();
      if (!MAGIC.equals(magic)
            || !VERSION_1.equals(version)
            || !Integer.toString(appVersion).equals(appVersionString)
            || !Integer.toString(valueCount).equals(valueCountString)
            || !"".equals(blank)) {
         throw new IOException("unexpected journal header: [" + magic + ", " + version + ", "
               + valueCountString + ", " + blank + "]");
      }

      int lineCount = 0;
      while (true) {
         try {
            readJournalLine(reader.readLine());
            lineCount++;
         } catch (EOFException endOfJournal) {
            break;
         }
      }
      redundantOpCount = lineCount - lruEntries.size();
   } finally {
      Util.closeQuietly(reader);
   }
}

  

一行一行的读取日志文件:

1.先看日志文件头是否格式匹配,否则抛出异常。

2.读取每一行记录

	private void readJournalLine(String line) throws IOException {
		int firstSpace = line.indexOf(' ');
		if (firstSpace == -1) {
			throw new IOException("unexpected journal line: " + line);
		}

		int keyBegin = firstSpace + 1;
		int secondSpace = line.indexOf(' ', keyBegin);
		final String key;
		if (secondSpace == -1) {
			key = line.substring(keyBegin);
			if (firstSpace == REMOVE.length() && line.startsWith(REMOVE)) {
				lruEntries.remove(key);
				return;
			}
		} else {
			key = line.substring(keyBegin, secondSpace);
		}

		Entry entry = lruEntries.get(key);
		if (entry == null) {
			entry = new Entry(key);
			lruEntries.put(key, entry);
		}

		if (secondSpace != -1 && firstSpace == CLEAN.length() && line.startsWith(CLEAN)) {
			String[] parts = line.substring(secondSpace + 1).split(" ");
			entry.readable = true;
			entry.currentEditor = null;
			entry.setLengths(parts);
		} else if (secondSpace == -1 && firstSpace == DIRTY.length() && line.startsWith(DIRTY)) {
			entry.currentEditor = new Editor(entry);
		} else if (secondSpace == -1 && firstSpace == READ.length() && line.startsWith(READ)) {
			// This work was already done by calling lruEntries.get().
		} else {
			throw new IOException("unexpected journal line: " + line);
		}
	}

1.如果是 REMOVE 记录,从 lruEntries 中删除 key 对应的 Entry。

2.根据 key 获取 Entry,如果为 null,新建一个放入 lruEntries 中。

3.记录为 CLEAN,DIRTY,READ ,对 Entry 做相应的操作。

	/**
	 * Computes the initial size and collects garbage as a part of opening the
	 * cache. Dirty entries are assumed to be inconsistent and will be deleted.
	 */
	private void processJournal() throws IOException {
		deleteIfExists(journalFileTmp);
		for (Iterator<Entry> i = lruEntries.values().iterator(); i.hasNext(); ) {
			Entry entry = i.next();
			if (entry.currentEditor == null) {
				for (int t = 0; t < valueCount; t++) {
					size += entry.lengths[t];
					fileCount++;
				}
			} else {
				entry.currentEditor = null;
				for (int t = 0; t < valueCount; t++) {
					deleteIfExists(entry.getCleanFile(t));
					deleteIfExists(entry.getDirtyFile(t));
				}
				i.remove();
			}
		}
	}

1.删除临时日志文件

2.entry.currentEditor 为 null,表示 CLEAN 或者 READ 记录,可以算出缓存文件的总大小,缓存文件的个数。

3.entry.currentEditor 不为 null,表示 DIRTY 记录,将对应的文件删除。

	/**
	 * Creates a new journal that omits redundant information. This replaces the
	 * current journal if it exists.
	 */
	private synchronized void rebuildJournal() throws IOException {
		if (journalWriter != null) {
			journalWriter.close();
		}

		Writer writer = new BufferedWriter(
				new OutputStreamWriter(new FileOutputStream(journalFileTmp), Util.US_ASCII));
		try {
			writer.write(MAGIC);
			writer.write("
");
			writer.write(VERSION_1);
			writer.write("
");
			writer.write(Integer.toString(appVersion));
			writer.write("
");
			writer.write(Integer.toString(valueCount));
			writer.write("
");
			writer.write("
");

			for (Entry entry : lruEntries.values()) {
				if (entry.currentEditor != null) {
					writer.write(DIRTY + ' ' + entry.key + '
');
				} else {
					writer.write(CLEAN + ' ' + entry.key + entry.getLengths() + '
');
				}
			}
		} finally {
			writer.close();
		}

		if (journalFile.exists()) {
			renameTo(journalFile, journalFileBackup, true);
		}
		renameTo(journalFileTmp, journalFile, false);
		journalFileBackup.delete();

		journalWriter = new BufferedWriter(
				new OutputStreamWriter(new FileOutputStream(journalFile, true), Util.US_ASCII));
	}

根据日志文件的格式,生成一个日志文件。

	/**
	 * Returns a snapshot of the entry named {@code key}, or null if it doesn't
	 * exist is not currently readable. If a value is returned, it is moved to
	 * the head of the LRU queue.
	 */
	public synchronized Snapshot get(String key) throws IOException {
		checkNotClosed();
		validateKey(key);
		Entry entry = lruEntries.get(key);
		if (entry == null) {
			return null;
		}

		if (!entry.readable) {
			return null;
		}

		// Open all streams eagerly to guarantee that we see a single published
		// snapshot. If we opened streams lazily then the streams could come
		// from different edits.
		File[] files = new File[valueCount];
		InputStream[] ins = new InputStream[valueCount];
		try {
			File file;
			for (int i = 0; i < valueCount; i++) {
				file = entry.getCleanFile(i);
				files[i] = file;
				ins[i] = new FileInputStream(file);
			}
		} catch (FileNotFoundException e) {
			// A file must have been deleted manually!
			for (int i = 0; i < valueCount; i++) {
				if (ins[i] != null) {
					Util.closeQuietly(ins[i]);
				} else {
					break;
				}
			}
			return null;
		}

		redundantOpCount++;
		journalWriter.append(READ + ' ' + key + '
');
		if (journalRebuildRequired()) {
			executorService.submit(cleanupCallable);
		}

		return new Snapshot(key, entry.sequenceNumber, files, ins, entry.lengths);
	}

根据 key 值获取缓存文件,返回的是 Snapshot 对象,同时生成一个 READ 记录。

	/** A snapshot of the values for an entry. */
	public final class Snapshot implements Closeable {
		private final String key;
		private final long sequenceNumber;
		private File[] files;
		private final InputStream[] ins;
		private final long[] lengths;

		private Snapshot(String key, long sequenceNumber, File[] files, InputStream[] ins, long[] lengths) {
			this.key = key;
			this.sequenceNumber = sequenceNumber;
			this.files = files;
			this.ins = ins;
			this.lengths = lengths;
		}

		/**
		 * Returns an editor for this snapshot's entry, or null if either the
		 * entry has changed since this snapshot was created or if another edit
		 * is in progress.
		 */
		public Editor edit() throws IOException {
			return DiskLruCache.this.edit(key, sequenceNumber);
		}

		/** Returns file with the value for {@code index}. */
		public File getFile(int index) {
			return files[index];
		}

		/** Returns the unbuffered stream with the value for {@code index}. */
		public InputStream getInputStream(int index) {
			return ins[index];
		}

		/** Returns the string value for {@code index}. */
		public String getString(int index) throws IOException {
			return inputStreamToString(getInputStream(index));
		}

		/** Returns the byte length of the value for {@code index}. */
		public long getLength(int index) {
			return lengths[index];
		}

		public void close() {
			for (InputStream in : ins) {
				Util.closeQuietly(in);
			}
		}
	}

Snapshot 类,封装了 key 和缓存文件的关系。

最重要的方法:

public InputStream getInputStream(int index) {
   return ins[index];
}

根据这个方法,就可以获取缓文件的输入流,参数 index 是由 key 值对应的缓存文件个数决定的,缓存文件个数是1,那么这里显然传0。

	/**
	 * Returns an editor for the entry named {@code key}, or null if another
	 * edit is in progress.
	 */
	public Editor edit(String key) throws IOException {
		return edit(key, ANY_SEQUENCE_NUMBER);
	}

	private synchronized Editor edit(String key, long expectedSequenceNumber) throws IOException {
		checkNotClosed();
		validateKey(key);
		Entry entry = lruEntries.get(key);
		if (expectedSequenceNumber != ANY_SEQUENCE_NUMBER && (entry == null
				|| entry.sequenceNumber != expectedSequenceNumber)) {
			return null; // Snapshot is stale.
		}
		if (entry == null) {
			entry = new Entry(key);
			lruEntries.put(key, entry);
		} else if (entry.currentEditor != null) {
			return null; // Another edit is in progress.
		}

		Editor editor = new Editor(entry);
		entry.currentEditor = editor;

		// Flush the journal before creating files to prevent file leaks.
		journalWriter.write(DIRTY + ' ' + key + '
');
		journalWriter.flush();
		return editor;
	}

写缓存文件,返回一个 Editor 对象,同时生成一个 DIRTY 记录。

	/** Edits the values for an entry. */
	public final class Editor {
		private final Entry entry;
		private final boolean[] written;
		private boolean hasErrors;
		private boolean committed;

		private Editor(Entry entry) {
			this.entry = entry;
			this.written = (entry.readable) ? null : new boolean[valueCount];
		}

		/**
		 * Returns an unbuffered input stream to read the last committed value,
		 * or null if no value has been committed.
		 */
		public InputStream newInputStream(int index) throws IOException {
			synchronized (DiskLruCache.this) {
				if (entry.currentEditor != this) {
					throw new IllegalStateException();
				}
				if (!entry.readable) {
					return null;
				}
				try {
					return new FileInputStream(entry.getCleanFile(index));
				} catch (FileNotFoundException e) {
					return null;
				}
			}
		}

		/**
		 * Returns the last committed value as a string, or null if no value
		 * has been committed.
		 */
		public String getString(int index) throws IOException {
			InputStream in = newInputStream(index);
			return in != null ? inputStreamToString(in) : null;
		}

		/**
		 * Returns a new unbuffered output stream to write the value at
		 * {@code index}. If the underlying output stream encounters errors
		 * when writing to the filesystem, this edit will be aborted when
		 * {@link #commit} is called. The returned output stream does not throw
		 * IOExceptions.
		 */
		public OutputStream newOutputStream(int index) throws IOException {
			synchronized (DiskLruCache.this) {
				if (entry.currentEditor != this) {
					throw new IllegalStateException();
				}
				if (!entry.readable) {
					written[index] = true;
				}
				File dirtyFile = entry.getDirtyFile(index);
				FileOutputStream outputStream;
				try {
					outputStream = new FileOutputStream(dirtyFile);
				} catch (FileNotFoundException e) {
					// Attempt to recreate the cache directory.
					directory.mkdirs();
					try {
						outputStream = new FileOutputStream(dirtyFile);
					} catch (FileNotFoundException e2) {
						// We are unable to recover. Silently eat the writes.
						return NULL_OUTPUT_STREAM;
					}
				}
				return new FaultHidingOutputStream(outputStream);
			}
		}

		/** Sets the value at {@code index} to {@code value}. */
		public void set(int index, String value) throws IOException {
			Writer writer = null;
			try {
				writer = new OutputStreamWriter(newOutputStream(index), Util.UTF_8);
				writer.write(value);
			} finally {
				Util.closeQuietly(writer);
			}
		}

		/**
		 * Commits this edit so it is visible to readers.  This releases the
		 * edit lock so another edit may be started on the same key.
		 */
		public void commit() throws IOException {
			if (hasErrors) {
				completeEdit(this, false);
				remove(entry.key); // The previous entry is stale.
			} else {
				completeEdit(this, true);
			}
			committed = true;
		}

		/**
		 * Aborts this edit. This releases the edit lock so another edit may be
		 * started on the same key.
		 */
		public void abort() throws IOException {
			completeEdit(this, false);
		}

		public void abortUnlessCommitted() {
			if (!committed) {
				try {
					abort();
				} catch (IOException ignored) {
				}
			}
		}

		private class FaultHidingOutputStream extends FilterOutputStream {
			private FaultHidingOutputStream(OutputStream out) {
				super(out);
			}

			@Override public void write(int oneByte) {
				try {
					out.write(oneByte);
				} catch (IOException e) {
					hasErrors = true;
				}
			}

			@Override public void write(byte[] buffer, int offset, int length) {
				try {
					out.write(buffer, offset, length);
				} catch (IOException e) {
					hasErrors = true;
				}
			}

			@Override public void close() {
				try {
					out.close();
				} catch (IOException e) {
					hasErrors = true;
				}
			}

			@Override public void flush() {
				try {
					out.flush();
				} catch (IOException e) {
					hasErrors = true;
				}
			}
		}
	}

Editor 可以看做一个帮助类。拿到 Editor 对象后,就可以获取缓存的输出流:

		/**
		 * Returns a new unbuffered output stream to write the value at
		 * {@code index}. If the underlying output stream encounters errors
		 * when writing to the filesystem, this edit will be aborted when
		 * {@link #commit} is called. The returned output stream does not throw
		 * IOExceptions.
		 */
		public OutputStream newOutputStream(int index) throws IOException {
			synchronized (DiskLruCache.this) {
				if (entry.currentEditor != this) {
					throw new IllegalStateException();
				}
				if (!entry.readable) {
					written[index] = true;
				}
				File dirtyFile = entry.getDirtyFile(index);
				FileOutputStream outputStream;
				try {
					outputStream = new FileOutputStream(dirtyFile);
				} catch (FileNotFoundException e) {
					// Attempt to recreate the cache directory.
					directory.mkdirs();
					try {
						outputStream = new FileOutputStream(dirtyFile);
					} catch (FileNotFoundException e2) {
						// We are unable to recover. Silently eat the writes.
						return NULL_OUTPUT_STREAM;
					}
				}
				return new FaultHidingOutputStream(outputStream);
			}
		}

可以看到是输出到一个 DirtyFile 文件。获取到了输出流,就可以写缓存了,写完缓存之后,需要调用 commit 方法。

		/**
		 * Commits this edit so it is visible to readers.  This releases the
		 * edit lock so another edit may be started on the same key.
		 */
		public void commit() throws IOException {
			if (hasErrors) {
				completeEdit(this, false);
				remove(entry.key); // The previous entry is stale.
			} else {
				completeEdit(this, true);
			}
			committed = true;
		}
	private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
		Entry entry = editor.entry;
		if (entry.currentEditor != editor) {
			throw new IllegalStateException();
		}

		// If this edit is creating the entry for the first time, every index must have a value.
		if (success && !entry.readable) {
			for (int i = 0; i < valueCount; i++) {
				if (!editor.written[i]) {
					editor.abort();
					throw new IllegalStateException("Newly created entry didn't create value for index " + i);
				}
				if (!entry.getDirtyFile(i).exists()) {
					editor.abort();
					return;
				}
			}
		}

		for (int i = 0; i < valueCount; i++) {
			File dirty = entry.getDirtyFile(i);
			if (success) {
				if (dirty.exists()) {
					File clean = entry.getCleanFile(i);
					dirty.renameTo(clean);
					long oldLength = entry.lengths[i];
					long newLength = clean.length();
					entry.lengths[i] = newLength;
					size = size - oldLength + newLength;
					fileCount++;
				}
			} else {
				deleteIfExists(dirty);
			}
		}

		redundantOpCount++;
		entry.currentEditor = null;
		if (entry.readable | success) {
			entry.readable = true;
			journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '
');
			if (success) {
				entry.sequenceNumber = nextSequenceNumber++;
			}
		} else {
			lruEntries.remove(entry.key);
			journalWriter.write(REMOVE + ' ' + entry.key + '
');
		}
		journalWriter.flush();

		if (size > maxSize || fileCount > maxFileCount || journalRebuildRequired()) {
			executorService.submit(cleanupCallable);
		}
	}

completeEdit() 方法将 DirtyFile 重新命名为 CleanFile,如果写入成功,则记录一条 CLEAN,否则记录 REMOVE。

这就是 DIRTY 记录后面一定会跟一条 CLEAN 或者 REMOVE 的原因。

	/**
	 * Drops the entry for {@code key} if it exists and can be removed. Entries
	 * actively being edited cannot be removed.
	 *
	 * @return true if an entry was removed.
	 */
	public synchronized boolean remove(String key) throws IOException {
		checkNotClosed();
		validateKey(key);
		Entry entry = lruEntries.get(key);
		if (entry == null || entry.currentEditor != null) {
			return false;
		}

		for (int i = 0; i < valueCount; i++) {
			File file = entry.getCleanFile(i);
			if (file.exists() && !file.delete()) {
				throw new IOException("failed to delete " + file);
			}
			size -= entry.lengths[i];
			fileCount--;
			entry.lengths[i] = 0;
		}

		redundantOpCount++;
		journalWriter.append(REMOVE + ' ' + key + '
');
		lruEntries.remove(key);

		if (journalRebuildRequired()) {
			executorService.submit(cleanupCallable);
		}

		return true;
	}

删除缓存,记录一条 REMOVE。

大致的方法就是这些了,还有一些琐碎的方法,跟使用到 LruCache 的其他类,都差不多。


原文地址:https://www.cnblogs.com/aprz512/p/5330223.html