【转】利用 selenium 的 webdrive 驱动 headless chrome

1.参考

2.概念

Headless模式解决了什么问题: 自动化工具例如 selenium 利用有头浏览器进行测试,面临效率和稳定性的影响,所以出现了 Headless Browser, 3年前，无头浏览器 PhantomJS 已经如火如荼出现了，紧跟着 NightmareJS 也成为一名巨星。无头浏览器带来巨大便利性：页面爬虫、自动化测试、WebAutomation... 用过PhantomJS的都知道，它的环境是运行在一个封闭的沙盒里面，在环境内外完全不可通信，包括API、变量、全局方法调用等。

So, Chrome59 推出了 headless mode，Chrome59版支持的特性，全部可以利用：
ES2017
ServiceWork(PWA测试随便耍)
无沙盒环境
无痛通讯&API调用
无与伦比的速度

https://developers.google.com/web/updates/2017/04/headless-chrome Getting Started with Headless Chrome

https://jiayi.space/post/zai-ubuntufu-wu-qi-shang-shi-yong-chrome-headless 在ubuntu服务器上使用Chrome Headless 845

UA有所差异：User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/63.0.3239.84 Safari/537.36

解决办法：chrome_options.add_argument("user-agent='Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36'")

https://0x0d.im/archives/headless-browser-detection.html 无头浏览器异闻录

我们如何区分这些无头浏览器和正常的浏览器呢？从 Server Side 分析用户行为进行检测是一劳永逸的方法，但成本和难度都很大。不过通过无头浏览器的一些特性，我们也可以从 Client Side 找出一些不同来。

3.代码

#coding:utf-8
from selenium import webdriver

url = "http://demo.testfire.net"
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
driver = webdriver.Chrome(chrome_options=chrome_options,executable_path='/Users/xxxx/driver/chromedriver')

driver.get('http://demo.testfire.net')
driver.find_element_by_xpath('//*[@id="_ctl0__ctl0_LoginLink"]').click()
driver.find_element_by_xpath('//*[@id="uid"]').clear()
driver.find_element_by_xpath('//*[@id="uid"]').send_keys('admin')
driver.find_element_by_xpath('//*[@id="passw"]').send_keys('admin')
driver.find_element_by_xpath('//*[@id="login"]/table/tbody/tr[3]/td[2]/input').click()

print driver.current_url

最后 print 出登录成功的当前 url:http://demo.testfire.net/bank/main.aspx

20171227 更新：仅设置参数 '--headless' 会报错，而仅设置参数 '--disable-gpu' 则会自动补充 '--headless'

63.0.3239.84（正式版本）（64 位）

In [43]: from selenium import webdriver

In [44]: chrome_options = webdriver.ChromeOptions()

In [45]: chrome_options.add_argument('--headless')

In [46]: driver = webdriver.Chrome(chrome_options=chrome_options)
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:63614/session {"capabilities": {"alwaysMatch": {"platform": "ANY", "browserName": "chrome", "version": "", "chromeOptions": {"ar
gs": ["--headless"], "extensions": []}}, "firstMatch": []}, "desiredCapabilities": {"platform": "ANY", "browserName": "chrome", "version": "", "chromeOptions": {"args": ["--headless"], "extensions": [
]}}}

DevTools listening on ws://127.0.0.1:12481/devtools/browser/aa60bb34-e8a5-4740-909e-aa3e6f315376
[1227/173400.592:ERROR:gpu_main.cc(164)] Exiting GPU process due to errors during initialization
[1227/173400.606:ERROR:browser_gpu_channel_host_factory.cc(107)] Failed to launch GPU process.
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request

In [47]: chrome_options.add_argument('--disable-gpu')

In [48]: driver = webdriver.Chrome(chrome_options=chrome_options)
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:63672/session {"capabilities": {"alwaysMatch": {"platform": "ANY", "browserName": "chrome", "version": "", "chromeOptions": {"ar
gs": ["--headless", "--disable-gpu"], "extensions": []}}, "firstMatch": []}, "desiredCapabilities": {"platform": "ANY", "browserName": "chrome", "version": "", "chromeOptions": {"args": ["--headless",
 "--disable-gpu"], "extensions": []}}}

DevTools listening on ws://127.0.0.1:12452/devtools/browser/c175a776-71be-40c0-aa73-a64c253f1cb0
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request

In [49]: driver.get('http://httpbin.org')
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:63672/session/026f8a02c2cd9bf7c7688c6b2934cd66/url {"url": "http://httpbin.org", "sessionId": "026f8a02c2cd9bf7c7688c6b2934cd66"
}
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request

In [56]: chrome_ops = webdriver.ChromeOptions()

In [57]: chrome_ops.add_argument('--disable-gpu')

In [58]: dri = webdriver.Chrome(chrome_options=chrome_options)
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:64092/session {"capabilities": {"alwaysMatch": {"platform": "ANY", "browserName": "chrome", "version": "", "chromeOptions": {"ar
gs": ["--headless", "--disable-gpu"], "extensions": []}}, "firstMatch": []}, "desiredCapabilities": {"platform": "ANY", "browserName": "chrome", "version": "", "chromeOptions": {"args": ["--headless",
 "--disable-gpu"], "extensions": []}}}

DevTools listening on ws://127.0.0.1:12713/devtools/browser/b873f662-1ca6-4cd0-a6cd-f9bd13d7e236
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request