I thought I would email everyone before touching the Apache server, plus check it was OK to go ahead and make the changes.
I would like to try an optimize Apache before Thursday (Fedora 7 is release) to handle the traffic a bit better. I will be performing some benchmarking on the Apache server using Apache benchmark.
I would also like to deploy mod_evasive which should help with the load on Apache. I will be attending the meeting this week, however I would like any responses via email if possible as the meeting is Thursday, and Fedora 7 is release then.
On Sun, 2007-05-27 at 13:34 +0100, Damian Myerscough wrote:
I thought I would email everyone before touching the Apache server, plus check it was OK to go ahead and make the changes.
I would like to try an optimize Apache before Thursday (Fedora 7 is release) to handle the traffic a bit better. I will be performing some benchmarking on the Apache server using Apache benchmark.
I would also like to deploy mod_evasive which should help with the load on Apache. I will be attending the meeting this week, however I would like any responses via email if possible as the meeting is Thursday, and Fedora 7 is release then.
I would like to recommend that we leave things alone this close to the release. Especially items like mod_evasive which very few people in the infrastructure group are familiar with, so far.
-sv
Ok, I guess we can talk more about it after the release of Fedora 7.
On 27/05/07, seth vidal skvidal@linux.duke.edu wrote:
On Sun, 2007-05-27 at 13:34 +0100, Damian Myerscough wrote:
I thought I would email everyone before touching the Apache server, plus check it was OK to go ahead and make the changes.
I would like to try an optimize Apache before Thursday (Fedora 7 is release) to handle the traffic a bit better. I will be performing some benchmarking on the Apache server using Apache benchmark.
I would also like to deploy mod_evasive which should help with the load on Apache. I will be attending the meeting this week, however I would like any responses via email if possible as the meeting is Thursday, and Fedora 7 is release then.
I would like to recommend that we leave things alone this close to the release. Especially items like mod_evasive which very few people in the infrastructure group are familiar with, so far.
-sv
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
mod_evasive is also recommended by Ryan Barnett who is the author of Preventing Web Attacks with Apache.
Has anyone also thought about deploying mod_security? or is it me just being paranoid.
I think this would make an excellent topic to talk about in the next meeting?
On 27/05/07, Mike McGrath mmcgrath@redhat.com wrote:
Damian Myerscough wrote:
Ok, I guess we can talk more about it after the release of Fedora 7.
I also have a question about mod_evasive. Wouldn't it be better/most efficient to limit that traffic before it even hits the apache instance?
-Mike
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Its always nice to have extra security surounding the architecture and apps, but for that we will need extra resources that we currently dont have right now, i personally would like to have extra RAM on the proxies. mod_security, is not that light, but its really nice and handy :)
Tweaks that we should do right now, is verify the values in MaxClients and everything regarding the threads (childs), and see if unecessary modules are being loaded or not
At least is my opinion... Paulo
On 5/27/07, Damian Myerscough damian.myerscough@gmail.com wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
mod_evasive is also recommended by Ryan Barnett who is the author of Preventing Web Attacks with Apache.
Has anyone also thought about deploying mod_security? or is it me just being paranoid.
I think this would make an excellent topic to talk about in the next meeting?
On 27/05/07, Mike McGrath mmcgrath@redhat.com wrote:
Damian Myerscough wrote:
Ok, I guess we can talk more about it after the release of Fedora 7.
I also have a question about mod_evasive. Wouldn't it be better/most efficient to limit that traffic before it even hits the apache instance?
-Mike
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
-- Regards, Damian
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
- Karsten
Dr. Egon Spengler: Not necessarily. There's definitely a *very slim* chance we'll survive. [pause while they consider this] Dr. Peter Venkman: [slaps Ray] I love this plan! I'm excited it could work! LET'S DO IT!
I guess with the release soo close no one wants to have the date pushed back due to users unable to get to the source. However, we are guaranteed to get large volumes of traffic which might knock the web servers offline.
Karsten Wade, I think another reason for leaving it is due to mod_evasive as Seth said no all the infrastructure people are familiar with it.
Anyways we will see how the web servers stand on Thursday.
On 27/05/07, Karsten Wade kwade@redhat.com wrote:
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
- Karsten
Dr. Egon Spengler: Not necessarily. There's definitely a *very slim* chance we'll survive. [pause while they consider this] Dr. Peter Venkman: [slaps Ray] I love this plan! I'm excited it could work! LET'S DO IT! -- Karsten Wade, 108 Editor ^ Fedora Documentation Project Sr. Developer Relations Mgr. | fedoraproject.org/wiki/DocsProject quaid.108.redhat.com | gpg key: AD0E0C41 ////////////////////////////////// \\\\\\\\\\\\\\\\\\
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
everything will rock! positive thinking :D And also lets not forget that the infrastructure itself is much more stable then in last release.
Paulo
On 5/27/07, Damian Myerscough damian.myerscough@gmail.com wrote:
I guess with the release soo close no one wants to have the date pushed back due to users unable to get to the source. However, we are guaranteed to get large volumes of traffic which might knock the web servers offline.
Karsten Wade, I think another reason for leaving it is due to mod_evasive as Seth said no all the infrastructure people are familiar with it.
Anyways we will see how the web servers stand on Thursday.
On 27/05/07, Karsten Wade kwade@redhat.com wrote:
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
- Karsten
Dr. Egon Spengler: Not necessarily. There's definitely a *very slim* chance we'll survive. [pause while they consider this] Dr. Peter Venkman: [slaps Ray] I love this plan! I'm excited it could work! LET'S DO IT! -- Karsten Wade, 108 Editor ^ Fedora Documentation Project Sr. Developer Relations Mgr. | fedoraproject.org/wiki/DocsProject quaid.108.redhat.com | gpg key: AD0E0C41 ////////////////////////////////// \\\\\\\\\\\\\\\\\\
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
-- Regards, Damian
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Karsten Wade wrote:
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
Actually we've got lots of plans as it is. I've been meaning to send this out but have been busy so here goes (all, comment and become familiar as necessary, the more people that know what is going on, the better)
This is only things related to the F7 launch:
Critical: 1) The static pages 2) The wiki 3) Mirror manager 4) Mirrors (which should be taken care of and synced long before the actual bit flip)
Presently the static pages are fully redundant and the wiki and mirror manager are redundant up to their data layers. (netapp and database respectively)
Plan A) We currently have 2 proxy servers and 4 application servers in place. 2 app servers are FC6, 2 are RHEL. At present the two RHEL boxes load balance the wiki (moin), the two app servers are for mirror manager (TurboGears). The static pages are on the actual proxy servers. On release day we'll track the load trend on these 6 machines. As long as release related content is being served, we hold.
Plan B) If any one of the above pieces begins to fail we can add capacity as required as well as limiting traffic with iptables. FC6 servers run mirror manager. RHEL servers run the wiki (though the wiki could technically be run on FC6 boxes, it just hasn't been done yet, no reason) and the static pages are run directly on the proxy servers. Adding application servers is easy, just kick start the box, have it connect to puppet. Proxy servers are trickier. They exist on a different network, but the same principal applies. We just contact the network guys to switch a port to the correct vlan. I've got xen6 so it has one nic on the app network and one nic on the proxy network.
Plan C) Start redirecting all traffic to the mirrors (those that have our web content). The theory is the most efficient transaction our web servers can do is to take a get and re-direct to a different server. The big 'gotcha' here is mirror manager. Mirror manager won't know which mirrors have Fedora 7 until they have it. So we'll have to serve the public server list at PHX. We're featuring the torrents a bit more this time compared to last time. Hopefully that will help keep load on the mirrors lighter, there's little way for us to track this AFAIK.
Unfortunately we don't have full trends from last year because a different team was running fedora.redhat.com during the last release (and f.rh.c doesn't even exist this release) We'll get a much better idea of what we're facing on release day.
In general I'd like to encourage that we hold this configuration unless there's any simple upgrades or obvious flaws. We're not really in an 'infrastructure change freeze' but I'd encourage everyone needing to make changes to wait until a day or so after the release. Exceptions, of course, would include something like bodhi which is of a high priority. I'll be flying to Germany for Linux Tag tomorrow (Monday) and will remain there for the release. I'll probably be on German time but will try to be around as needed or if any problems arise. My cell phone may be limited though.
Comments?
-Mike
cool plan. Hopefully it should hold against regular release day traffic. However, on FC6 launch, we were deliberately 'attacked', right? flooders might deliberately hit the non static pages, are we prepared for that ? How would everyone feel about limiting the number of connections per /24 network to a reasonable number, a la iptables -p tcp --syn --dport 80 -m connlimit --connlimit-above 16 --connlimit-mask 24 -j REJECT
regards
On 5/28/07, Mike McGrath mmcgrath@redhat.com wrote:
Karsten Wade wrote:
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
Actually we've got lots of plans as it is. I've been meaning to send this out but have been busy so here goes (all, comment and become familiar as necessary, the more people that know what is going on, the better)
This is only things related to the F7 launch:
Critical: 1) The static pages 2) The wiki 3) Mirror manager 4) Mirrors (which should be taken care of and synced long before the actual bit flip)
Presently the static pages are fully redundant and the wiki and mirror manager are redundant up to their data layers. (netapp and database respectively)
Plan A) We currently have 2 proxy servers and 4 application servers in place. 2 app servers are FC6, 2 are RHEL. At present the two RHEL boxes load balance the wiki (moin), the two app servers are for mirror manager (TurboGears). The static pages are on the actual proxy servers. On release day we'll track the load trend on these 6 machines. As long as release related content is being served, we hold.
Plan B) If any one of the above pieces begins to fail we can add capacity as required as well as limiting traffic with iptables. FC6 servers run mirror manager. RHEL servers run the wiki (though the wiki could technically be run on FC6 boxes, it just hasn't been done yet, no reason) and the static pages are run directly on the proxy servers. Adding application servers is easy, just kick start the box, have it connect to puppet. Proxy servers are trickier. They exist on a different network, but the same principal applies. We just contact the network guys to switch a port to the correct vlan. I've got xen6 so it has one nic on the app network and one nic on the proxy network.
Plan C) Start redirecting all traffic to the mirrors (those that have our web content). The theory is the most efficient transaction our web servers can do is to take a get and re-direct to a different server. The big 'gotcha' here is mirror manager. Mirror manager won't know which mirrors have Fedora 7 until they have it. So we'll have to serve the public server list at PHX. We're featuring the torrents a bit more this time compared to last time. Hopefully that will help keep load on the mirrors lighter, there's little way for us to track this AFAIK.
Unfortunately we don't have full trends from last year because a different team was running fedora.redhat.com during the last release (and f.rh.c doesn't even exist this release) We'll get a much better idea of what we're facing on release day.
In general I'd like to encourage that we hold this configuration unless there's any simple upgrades or obvious flaws. We're not really in an 'infrastructure change freeze' but I'd encourage everyone needing to make changes to wait until a day or so after the release. Exceptions, of course, would include something like bodhi which is of a high priority. I'll be flying to Germany for Linux Tag tomorrow (Monday) and will remain there for the release. I'll probably be on German time but will try to be around as needed or if any problems arise. My cell phone may be limited though.
Comments?
-Mike
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Ahmed Kamal wrote:
cool plan. Hopefully it should hold against regular release day traffic. However, on FC6 launch, we were deliberately 'attacked', right? flooders might deliberately hit the non static pages, are we prepared for that ? How would everyone feel about limiting the number of connections per /24 network to a reasonable number, a la iptables -p tcp --syn --dport 80 -m connlimit --connlimit-above 16 --connlimit-mask 24 -j REJECT
The attack thing was never totally confirmed one way or the other and we don't have the logs from last year (we weren't running fedora.redhat.com then) So we're much better prepared this run but it's still difficult to tell exactly what to expect.
-Mike
On Sun, 2007-05-27 at 18:53 -0500, Mike McGrath wrote:
Karsten Wade wrote:
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
<snip>
Unfortunately we don't have full trends from last year because a different team was running fedora.redhat.com during the last release (and f.rh.c doesn't even exist this release) We'll get a much better idea of what we're facing on release day.
<snip>
I'm guessing this has been done, but just in case it's not.
Do we need to make any changes to cacti for monitoring of services before release? To make sure we do have all of the required trends for next year.
Doug
everything is there but app3 and app4 (mirrormanager) ill add it today or tomorrow (since im flying to the US in a few hours)
Paulo
On 5/29/07, Douglas Furlong dfurlong@dark-hill.co.uk wrote:
On Sun, 2007-05-27 at 18:53 -0500, Mike McGrath wrote:
Karsten Wade wrote:
On Sun, 2007-05-27 at 17:41 +0100, Damian Myerscough wrote:
Yea, it would. However mod_easvie is able to detect if users a continuously hitting refresh to consume bandwidth.
Would it make sense to have a few tricks ready to go? Untested or unproven items to pull out in response to what happens. Sort of like what Egon Spengler might pull out in Ghostbusters ...
I know, it could be *worse* than whatever happens, but there is a slim chance we'll survive.
<snip> > Unfortunately we don't have full trends from last year because a > different team was running fedora.redhat.com during the last release > (and f.rh.c doesn't even exist this release) We'll get a much better > idea of what we're facing on release day.
<snip>
I'm guessing this has been done, but just in case it's not.
Do we need to make any changes to cacti for monitoring of services before release? To make sure we do have all of the required trends for next year.
Doug
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
On Tue, 2007-05-29 at 09:25 +0100, Paulo Santos wrote:
everything is there but app3 and app4 (mirrormanager) ill add it today or tomorrow (since im flying to the US in a few hours)
Paulo
Morning Paulo
App3 and App4 are present, and have the standard graphs being done CPU, Disk Space, Load, Memory and Net traffic.
Is there any thing else that needed to be monitored?
I though app3 and app4 werent there yet... oh well :D
Would be nice to have stats from the Loadbalancer, but for that i think its Mike that needs to add it, since i dont even know if the LB belongs to fp.org or not.
Paulo
On 5/29/07, Douglas Furlong dfurlong@dark-hill.co.uk wrote:
On Tue, 2007-05-29 at 09:25 +0100, Paulo Santos wrote:
everything is there but app3 and app4 (mirrormanager) ill add it today or tomorrow (since im flying to the US in a few hours)
Paulo
Morning Paulo
App3 and App4 are present, and have the standard graphs being done CPU, Disk Space, Load, Memory and Net traffic.
Is there any thing else that needed to be monitored?
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/29/2007 10:25 AM, Paulo Santos wrote:
everything is there but app3 and app4 (mirrormanager) ill add it today or tomorrow (since im flying to the US in a few hours)
I don't see a mod_evasive RPM available via cvs... [oliver@pils extras.public]$ cvs up -d mod_evasive cvs update: Updating mod_evasive U mod_evasive/.cvsignore U mod_evasive/Makefile U mod_evasive/sources
Somethin' went wrong? :-)
Best, Oliver
I have built an RPM and it is available in my /home/damian on lockbox.
I have not checked it into the CVS as I was unsure if everyone wanted it deployed.
If you can't get access to my /home directory on lockbox I can email you a copy to have a look over.
On 29/05/07, Oliver Falk oliver@linux-kernel.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/29/2007 10:25 AM, Paulo Santos wrote:
everything is there but app3 and app4 (mirrormanager) ill add it today or tomorrow (since im flying to the US in a few hours)
I don't see a mod_evasive RPM available via cvs... [oliver@pils extras.public]$ cvs up -d mod_evasive cvs update: Updating mod_evasive U mod_evasive/.cvsignore U mod_evasive/Makefile U mod_evasive/sources
Somethin' went wrong? :-)
Best, Oliver -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD8DBQFGW+0KxWN5Ge8lKUMRAkzwAJ94WfADbRecUIFYfRI9VR5nRg+UYACg0POb +ODs9U5JiUB1THpDQ+lX/lY= =qMxW -----END PGP SIGNATURE-----
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
On Sun, May 27, 2007 at 01:34:11PM +0100, Damian Myerscough wrote:
I thought I would email everyone before touching the Apache server, plus check it was OK to go ahead and make the changes.
I would like to try an optimize Apache before Thursday (Fedora 7 is release) to handle the traffic a bit better. I will be performing some benchmarking on the Apache server using Apache benchmark. I would also like to deploy mod_evasive which should help with the load on Apache. I will be attending the meeting this week, however I would like any responses via email if possible as the meeting is Thursday, and Fedora 7 is release then.
I made the (safe) change to app[34] httpd serving up mirrors.fedoraproject.org/publiclist to add mod_expires and a default Expires time of modification time plus 1 hour. These pages are static but refreshed at the top of every hour, and the flow is:
end user -> (end user proxy?) -> proxy[12].fp.o -> app[34]
for every request. By adding the Expires: header, and testing from behind a corporate proxy, the first page load takes 0.5s for a cache miss, but subsequent page loads take 0.05s for a non-expired cache hit, and we never take even a HTTP HEAD on the cache hits. This lets these pages play nicely with those who have their own proxy server.
Would it make sense to add Expires headers to the static fp.o home content being served directly from proxy[12] ? We're not going to change them often (we hope), so a 1-4 hour expiry time, just in case we do have to change them, might be beneficial.
Thoughts?
Thanks, Matt
Sounds good Matt,
I had a little idea for mirrors.fedoraproject.org/publiclist why not add mod_geoip and have the mirrors list broken down so that users connecting from Germany get mirrors located in Germany and not mirrors in England (also give them the opportunity to download from a mirror outside their country). That's just a little thought, it should help with the load slightly.
On 29/05/07, Matt Domsch Matt_Domsch@dell.com wrote:
On Sun, May 27, 2007 at 01:34:11PM +0100, Damian Myerscough wrote:
I thought I would email everyone before touching the Apache server, plus check it was OK to go ahead and make the changes.
I would like to try an optimize Apache before Thursday (Fedora 7 is release) to handle the traffic a bit better. I will be performing some benchmarking on the Apache server using Apache benchmark. I would also like to deploy mod_evasive which should help with the load on Apache. I will be attending the meeting this week, however I would like any responses via email if possible as the meeting is Thursday, and Fedora 7 is release then.
I made the (safe) change to app[34] httpd serving up mirrors.fedoraproject.org/publiclist to add mod_expires and a default Expires time of modification time plus 1 hour. These pages are static but refreshed at the top of every hour, and the flow is:
end user -> (end user proxy?) -> proxy[12].fp.o -> app[34]
for every request. By adding the Expires: header, and testing from behind a corporate proxy, the first page load takes 0.5s for a cache miss, but subsequent page loads take 0.05s for a non-expired cache hit, and we never take even a HTTP HEAD on the cache hits. This lets these pages play nicely with those who have their own proxy server.
Would it make sense to add Expires headers to the static fp.o home content being served directly from proxy[12] ? We're not going to change them often (we hope), so a 1-4 hour expiry time, just in case we do have to change them, might be beneficial.
Thoughts?
Thanks, Matt
-- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
On Tue, May 29, 2007 at 02:41:12PM +0100, Damian Myerscough wrote:
Sounds good Matt,
I had a little idea for mirrors.fedoraproject.org/publiclist why not add mod_geoip and have the mirrors list broken down so that users connecting from Germany get mirrors located in Germany and not mirrors in England (also give them the opportunity to download from a mirror outside their country). That's just a little thought, it should help with the load slightly.
The mirrorlists returned to yum do use geoip plus other heuristics (e.g. if there are fewer than 3 mirrors in a country, return the global list). So we're good there.
The web pages under /publiclist don't though. They do list the country a mirror is in, but they list all the mirrors, currently 194. :-) Maybe it's too long a page for people to crawl through to find their country; I'm open to page design suggestions. But I don't want to keep per-country static pages for these; I do per-country per-repo static text files for the yum mirrorlists as a backup, and there are nearly 4300 such files. I don't want to duplicate all that again for the /publiclist pages...
Thanks, Matt
Matt,
I don't really see any problems with having the complete list of all the mirrors. I usually use the quick find feature in my browser(Ctr F) and that let's me jump directly to the first mirror when I type "US". I suppose you could make the page dynamic and return results when a user indicates a country of origin, but wouldn't this use up more of our scarce computing resources? Just my opinion but I wouldn't worry too much about making /publiclist dynamic. Actually, now that I think about it. Could we not code something to do this for the user in a client side script?
On 5/29/07, Matt Domsch Matt_Domsch@dell.com wrote:
On Tue, May 29, 2007 at 02:41:12PM +0100, Damian Myerscough wrote:
Sounds good Matt,
I had a little idea for mirrors.fedoraproject.org/publiclist why not add mod_geoip and have the mirrors list broken down so that users connecting from Germany get mirrors located in Germany and not mirrors in England (also give them the opportunity to download from a mirror outside their country). That's just a little thought, it should help with the load slightly.
The mirrorlists returned to yum do use geoip plus other heuristics (e.g. if there are fewer than 3 mirrors in a country, return the global list). So we're good there.
The web pages under /publiclist don't though. They do list the country a mirror is in, but they list all the mirrors, currently 194. :-) Maybe it's too long a page for people to crawl through to find their country; I'm open to page design suggestions. But I don't want to keep per-country static pages for these; I do per-country per-repo static text files for the yum mirrorlists as a backup, and there are nearly 4300 such files. I don't want to duplicate all that again for the /publiclist pages...
Thanks, Matt
-- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
On Tue, May 29, 2007 at 08:38:32PM -0700, Freddie Rosario wrote:
Matt,
I don't really see any problems with having the complete list of all the mirrors. I usually use the quick find feature in my browser(Ctr F) and that let's me jump directly to the first mirror when I type "US". I suppose you could make the page dynamic and return results when a user indicates a country of origin, but wouldn't this use up more of our scarce computing resources? Just my opinion but I wouldn't worry too much about making /publiclist dynamic.
Glad we agree. :-)
Actually, now that I think about it. Could we not code something to do this for the user in a client side script?
then your browser coding foo is stronger than mine. I wouldn't spend brain cycles worrying about it at this point, but if it keeps you up at night, I'd take patches.
On Tue, 2007-05-29 at 07:24 -0500, Matt Domsch wrote:
Would it make sense to add Expires headers to the static fp.o home content being served directly from proxy[12] ? We're not going to change them often (we hope), so a 1-4 hour expiry time, just in case we do have to change them, might be beneficial.
Normally, we won't change that content that much. Right now, we do seem to be tweaking still, such is the wont of perfectionists.
Maybe crank back the expiry time if we start having problems? Or at least wait until 0800 UTC tomorrow. :)
- Karsten
infrastructure@lists.fedoraproject.org
-
Ahmed Kamal -
Damian Myerscough -
Douglas Furlong -
Freddie Rosario -
Karsten Wade -
Matt Domsch -
Mike McGrath -
Oliver Falk -
Paulo Santos -
seth vidal