On Tue, Aug 21, 2012 at 06:25:20PM -0500, Morten Stevens wrote:
Hello,
My name is Morten Stevens and my company offers a new EPEL Mirror.
http://mirror.imt-systems.com/epel/
IP: 81.30.145.134
Bandwidth: 1000 Mbit
Location: Germany
Company: IMT-Systems GmbH
The Mirror Manager log shows:
Directory pub/epel/4/i386/debug/headers is not up-to-date on this
host.
Thanks for the report. Here's the MirrorManager fix I'll need to
deploy to production. It shouldn't have any effect on client
operation though, as up2date doesn't request header files from MM so
deep in the structure I believe.
commit 5d61bb4a1f90ae37efadff590b9eb407ac69ca6b
Author: Matt Domsch <Matt_Domsch(a)dell.com>
Date: Wed Aug 22 09:46:45 2012 -0500
crawler: handle zero-length files
The crawler, when doing HTTP HEAD calls to retrieve info about a file,
gets its content length from the HTTP header. If a file is
zero-length, the HTTP server may not return a Content-Length header.
This was causing the crawler to believe the file (and thus its
directory) was not up-to-date, even though it was up-to-date.
Fix is to check that content-length value is present, and if not
present, don't rely upon it for determining up-to-date-edness.
diff --git a/server/crawler_perhost b/server/crawler_perhost
index b8027d2..81bdab0 100755
--- a/server/crawler_perhost
+++ b/server/crawler_perhost
@@ -275,7 +275,7 @@ def check_head(hoststate, url, filedata, recursion, readable,
retry=0):
if status >= 200 and status < 300:
# fixme should check last_modified too
- if filedata['size'] == content_length:
+ if filedata['size'] == content_length or content_length is None: # handle
no content-length header, streaming/chunked return or zero-length file
return True
else:
return False
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO