HttpWebRequest 下载网页Html代码 POST方式(站内使用了form方式) System.Net.WebException (417) Expectation failed

上一篇说了Get方式可以处理一般的站内搜索,假如使用了表单方式,用Get方式就获取不了。

例如:

网站站内搜索界面:

对就源码:

可以看出使用了form的post,action指向search.html,其中表单域是key;

Post的提交数据格式如下:

所以提交请求应该是:key=key值&proClass=&x=26&y=15灰色部分也可以不用。注意key值需要经过编码,不要包含了&等特殊符号。

实现的代码如下:

string searchKey = "搜索内容";
ASCIIEncoding encoding
= new ASCIIEncoding();
string postData = "key=" + System.Uri.EscapeUriString(searchKey);
postData
+= "&proClass=&x=0&y=0";

ServicePointManager.Expect100Continue
= false;
byte[] data = encoding.GetBytes(postData);
HttpWebRequest hwrequest
= (HttpWebRequest)WebRequest.Create("http://www.website.net/search.html");
hwrequest.Method
= "POST";
hwrequest.ContentType
= "application/x-www-form-urlencoded";
hwrequest.ContentLength
= data.Length;
Stream stream
= hwrequest.GetRequestStream();

stream.Write(data,
0, data.Length);
stream.Close();

HttpWebResponse hwresponse
= (HttpWebResponse)hwrequest.GetResponse();
StreamReader sr
= new StreamReader(hwresponse.GetResponseStream(), Encoding.UTF8);
string content = sr.ReadToEnd();

在这里如果不使用ServicePointManager.Expect100Continue = false;这段代码时,以上方法执行至GetResponse()就会产生如下错误信息:

System.Net.WebException: 远程服务器返回错误: (417) Expectation failed。

其中在msdn上有如下说明:

如果 POST 请求需要 100-Continue 响应,则为 true;否则为 false。默认值为 true。详情请进入Expect100Continue查阅!

在上也有如下说明:

Requirements for HTTP/1.1 clients: - If a client will wait for a 100 (Continue) response before sending the request body, it MUST send an Expect request-header field with the "100-continue" expectation.

  - A client MUST NOT send an Expect request-header field with the "100-continue" expectation if it does not intend to send a request body.

详情请进入HTTP/1.1查阅。

原文地址:https://www.cnblogs.com/blackcore/p/2064858.html