Our crawlers should both connect natted from 209.132.181.102
Adrian
On Mon, Oct 09, 2017 at 10:31:30AM -0400, Mike Neir wrote:
> Greetings Adrian,
>
> I'll follow up with my team and see if we can get to the bottom of this
> issue. As far as I know, there shouldn't be any CDN involvement with our
> mirror, just content served from one machine. Can you tell provide
> information on which IP address(es) those queries might be coming from so
> we can search more efficiently?
>
> MN
>
>
> Mike Neir
> Infrastructure Administrator
> Liquid Web, Inc.
>
> On Mon, Oct 9, 2017 at 8:42 AM, Adrian Reber <adrian@lisas.de> wrote:
>
> > Hello mirrors.liquidweb.com admins,
> >
> > Fedora's mirror crawler has trouble crawling your mirror as it does
> > return a unusual status code:
> >
> > # curl http://mirrors.liquidweb.com/fedora-epel/ -v
> > * About to connect() to mirrors.liquidweb.com port 80 (#0)
> > * Trying 69.167.187.144...
> > * Connected to mirrors.liquidweb.com (69.167.187.144) port 80 (#0)
> > > GET /fedora-epel/ HTTP/1.1
> > > User-Agent: curl/7.29.0
> > > Host: mirrors.liquidweb.com
> > > Accept: */*
> > >
> > < HTTP/1.1 000
> > < Server: nginx
> > < Date: Mon, 09 Oct 2017 12:37:13 GMT
> > < Content-Type: text/html
> > < Transfer-Encoding: chunked
> > < Connection: keep-alive
> >
> >
> > The crawler expects the status code 200 if everything works correctly and
> > ignores your mirror because of the status code 000. Any reason for
> > status code 000. It seems that the code 000 is something Akamai uses for
> > requests where the client aborted:
> >
> > """
> > Log Delivery Services (LDS) LDS will show a 000 for any 200 or 206
> > responses with a client abort: the object was served correctly from the
> > origin or edge, but the end-user terminated the connection/transaction
> > before it completed.
> > """
> >
> > Not sure if that is related to your situation. But right now your mirror
> > is disabled as it seems to be inaccessible (at least from the crawlers
> > point of view).
> >
> > Adrian
> >