原编码问题

在修复原有代码bug时，发现日志里经常抛出Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended，这种提示。查看了原有代码中的逻辑如下：

... ...
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
String respContent = getMethod.getResponseBodyAsString();
... ...

原因就在于使用了getResponseBodyAsString()方法

源码分析

getResponseBodyAsString()源码

	//getResponseBodyAsString()方法源码
    public String getResponseBodyAsString() throws IOException {
        byte[] rawdata = null;
        if (this.responseAvailable()) {
        	//调用了getResponseBody()，容易消耗内存
            rawdata = this.getResponseBody();
        }

        return rawdata != null ? EncodingUtil.getString(rawdata, this.getResponseCharSet()) : null;
    }
    //responseAvailable()方法源码
    private boolean responseAvailable() {
        return this.responseBody != null || this.responseStream != null;
    }
    
    //getResponseBody()方法源码
    public byte[] getResponseBody() throws IOException {
        if (this.responseBody == null) {
            InputStream instream = this.getResponseBodyAsStream();
            if (instream != null) {
                long contentLength = this.getResponseContentLength();
                if (contentLength > 2147483647L) {
                    throw new IOException("Content too large to be buffered: " + contentLength + " bytes");
                }

                int limit = this.getParams().getIntParameter("http.method.response.buffer.warnlimit", 1048576);
                if (contentLength == -1L || contentLength > (long)limit) {
                //这里是warn的原文
                    LOG.warn("Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.");
                }

                LOG.debug("Buffering response body");
                ByteArrayOutputStream outstream = new ByteArrayOutputStream(contentLength > 0L ? (int)contentLength : 4096);
                byte[] buffer = new byte[4096];

                int len;
                while((len = instream.read(buffer)) > 0) {
                    outstream.write(buffer, 0, len);
                }

                outstream.close();
                this.setResponseStream((InputStream)null);
                this.responseBody = outstream.toByteArray();
            }
        }

        return this.responseBody;
    }

从源码中可以看出，warn的条件是(contentLength == -1L || contentLength > (long)limit)，如果http头没有指定contentLength或大于上限值（默认1M），就会抛异常。其实，如果返回的结果比较确定，对程序没有太大影响。而对于返回结果不确定时，源码也建议我们使用下面的getResponseBodyAsStream()方法。

getResponseBodyAsStream()源码

    public InputStream getResponseBodyAsStream() throws IOException {
        if (this.responseStream != null) {
            return this.responseStream;
        } else if (this.responseBody != null) {
            InputStream byteResponseStream = new ByteArrayInputStream(this.responseBody);
            LOG.debug("re-creating response stream from byte array");
            return byteResponseStream;
        } else {
            return null;
        }
    }

从源码中可以看出，getResponseBodyAsStream()内部没有使用getResponseBody()方法，避免了内存耗尽问题，而是使用了InputStream流方式处理。

优化原编码

... ...
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
//String respContent = getMethod.getResponseBodyAsString();
//使用getResponseBodyAsStream()
InputStream inputStream = getMethod.getResponseBodyAsStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
StringBuffer stringBuffer = new StringBuffer();
String str = "";
while ((str = br.readLine()) != null) {
       stringBuffer.append(str);
}
LOGGER.info("respContent: {}", stringBuffer.toString());
... ...

测了一把，哦豁，不再报这个异常了。完美。

烧不死的鸟就是凤凰