Re: bugs found in http.e and news.ex
- Posted by CoJaBo2 Sep 13, 2012
- 1547 views
Some sites do not give a content_length in the http header (hence when news.ex used http.e, yahoo never had a hit on *anything*), some give bad values.
In HTTP/1.1, Content-Length is omitted in some cases. At a minimum, the Transfer-Encoding header (and its Chunked encoding) must be supported to fix this issue. See http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4
(Note: Technically this is could be considered a server bug- Content-Length is always mandatory in HTTP/1.0, so servers should not be sending HTTP/1.1 replies to HTTP/1.0 clients, as they will not be compatible; nonetheless, several major sites [Wikipedia] sometimes do this. Also, servers should not be sending Content-Length along with Transfer-Encoding, yet some do; in these cases, the Content-Length is bogus, as the length computed from the Transfer-Encoding supercedes it).
Ideally, http.e should be fully HTTP/1.1 compliant.
At least one uses 31-bit values which roll over to negative values and are wrong from then on (wikimedia, but others too).
I suspect you are running into the above isue, rather than rollover.
