Python爬虫2----------运用代理访问

为request添加一个代理,及将浏览器头部信息加入,随机从ip列表中拿出一个ip进行访问

注意函数参数的形式,如request.proxyhandler(协议,地址)

 1 import urllib.request
 2 import random
 3 url='http://www.baidu.com'
 4 iplist=['121.226.174.246:8080','210.38.1.142:8080','210.38.1.143:8080']
 5 proxyhandler=urllib.request.ProxyHandler({'http':random.choice(iplist)})
 6 openner=urllib.request.build_opener(proxyhandler)
 7 openner.addheaders=[('User-Agent','Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.221 Safari/537.36 SE 2.X MetaSr 1.0')]
 8 urllib.request.install_opener(openner)
 9 
10 response=urllib.request.urlopen(url)
11 html=response.read().decode('utf-8')
12 
13 print(html)
原文地址:https://www.cnblogs.com/caojunjie/p/6725910.html