《Cracking the Coding Interview》——第10章：可扩展性和存储空间限制—

《Cracking the Coding Interview》——第10章：可扩展性和存储空间限制——题目6

2014-04-24 22:01

题目：你有10亿条url，怎么检测其中时候有重复呢？

解法：Hash，算签名，然后用K-V数据库保存数据查重。

代码：

1 // 10.6 You have 10 billion URLs, how would you do to detect duplicates in them.
2 // Answer:
3 //    1. Use digital sign algorithm to convert string to a number of checksum.
4 //    2. Use this sign as the hash key, if memory allow, use an in-memory hash table to detect duplicates.
5 //    3. If memory won't fit in, use K-V database instead. 10GB scale should be acceptable for one machine, so I won't seek help from another computer.
6 int main()
7 {
8     return 0;
9 }