httpd logs
by Kevin Fenzi
So, the recent fas vulnerability made us realize that we were not
collecting and saving httpd logs from staging machines.
I've since added:
app01.stg
app02.stg
proxy01.stg
to have their httpd logs pulled over to log02 and kept.
This however got me thinking. Since we are moving to a model where each
app has it's own server, should we widen the servers we pull httpd logs
from? For example, ask01/02? fedocal? blockerbugs?
Or should we figure out a better way to collect and store the httpd
logs.
Thoughts?
kevin
10 years, 11 months
backups planning
by Kevin Fenzi
Another fun mail to start some discussion. ;)
The topic this time is backups.
Currently, we do backups two ways:
1. We have a backup server in phx2 with a tape drive running bacula. It
backs up machines and spools the backups to tape. It backs up the
following:
ask01
bastion01
bastion02
collab04
db01
db04
db05
db-fas01
fas01
hosted04
hosted-lists01
lockbox01
log02
nfs01
people03
pkgs01
proxy01
proxy02
releng03
releng04
relepel01
It does a subset of data on those, but for the most part all of them.
2. We have a backup server at ibiblio. It backs up much more
selectively. It keeps lvm snapshots of 3 days worth of these backups
(so there's the current one, and 2 older ones available):
db01
db04
db05
db-fas01
(only database backups from those)
pkgs01
(git and lookaside)
lockbox01
(git repos and infra rpms)
We are finally getting some more storage in phx2, and I thought this
might be a good time to look at backups and revamping them.
I'm going to get a backups volume added to backup03 in phx2.
I'd like to install rdiff-backup there and have it run backups to disk
there, and then change the tape backups to just backup the rdiff backup
data to tape. I'd like to also rethink what we backup, and restrict
rdiff-backup to /etc and /home and /actual-data. If we are restoring
an instance we would do a new install, ansible or puppet and then
restore data, so it doesn't make much sense to me to backup system
binaries, etc. With rdiff-backup we could keep a pretty long history if
needed.
Some of our instances don't actually have any data on them, thats all
in database. Logs should hopefully go to log02, so if we back that up
we should have all those.
I'm not sure what we could do better on external backups, but open to
ideas there. Should we add or remove anything there?
Thoughts? Comments?
kevin
10 years, 11 months
RH Bugzilla Upgrade to 4.4-3 on 20th of May
by Róman Joost
Hi,
this is a friendly reminder of the upgrade process for
https://bugzilla.redhat.com.
The outstanding upgrade of Red Hat Bugzilla from 4.2.5-8.4 to 4.4-3 will
take place on:
Monday, May 20 (APAC)
The outages involved will be approximately >4 hours for the disruptive
production sensitive cases.
PS: A previous mail dating the upgrade of https://bugzilla.redhat.com to
happen on Monday, May 13 (APAC) was a mistake on my behalf. I apologize
for any inconvenience this may cause.
Kind Regards,
--
Róman Joost
Product Manager, HSS - Infrastructure Engineering & Development (Brisbane)
email: rjoost(a)redhat.com | tz: UTC+10
irc: rjoost #bugzilla
blog: https://blogs.corp.redhat.com/bugzilla
10 years, 11 months
Fedora 19 Alpha Freeze in effect.
by Kevin Fenzi
Greetings.
we are now in the infrastructure freeze leading up to the Fedora 19
Alpha release. This is a pre-release freeze.
Please see:
https://fedorahosted.org/fedora-infrastructure/browser/architecture/Envir...
Anything in the Pre Release freeze box is frozen until 2013-04-16 (or
later if Alpha slips). This means there should be NO puppet changes to
any hosts in there (including global ones) without signoff of the
change from at least 2 folks in sysadmin-main and/or
release-engineering.
Thanks,
kevin
10 years, 11 months
Re: MirrorManager GEO IP API
by Vratislav Podzimek
On Thu, 2013-05-02 at 11:07 -0600, Kevin Fenzi wrote:
> On Tue, 30 Apr 2013 07:56:31 -0500
> <Matt_Domsch(a)Dell.com> wrote:
>
> > I think it would work best to use a dedicated Apache module, such as
> > http://dev.maxmind.com/geoip/mod_geoip2 rather than add another
> > interface into MirrorManager which otherwise duplicates this. Fedora
> > Infrastructure could easily roll out such a service then, independent
> > of MirrorManager.
>
> We could look at this. Would you folks be willing to repost the below
> on the infrastructure list and we could start a discussion there?
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Absolutely, reposting. I think this list is the right place to discuss
such things. And I just wonder -- anybody thinks this could be deployed
before the F19 is released?
>
> I don't see mod_geoip2 packaged yet, just mod_geoip.
>
> kevin
> --
> >
> > --
> > Matt Domsch
> > Technology Strategist
> > Dell | Office of the CTO
> >
> >
> >
> > -----Original Message-----
> > From: Martin Kolman [mailto:mkolman@redhat.com]
> > Sent: Tuesday, April 30, 2013 7:24 AM
> > To: mdomsch(a)fedoraproject.org
> > Cc: vpodzime(a)redhat.com; dcantrel(a)redhat.com
> > Subject: MirrorManager GEO IP API
> >
> > Hi,
> > we've recently added GeoIP support to Anaconda, so that it can
> > automatically preset the language and timezone for users that are
> > online during the installation.
> >
> > We are currently using the mirror manager API, as it is part of the
> > Fedora infrastructure so users installing fedora are not calling some
> > third-party service and leaking their public addresses all over.
> >
> > Currently, we basically just use the mirror API like this:
> >
> > https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-19&arch=i386
> >
> > And use the first country code. As we don't really need all the other
> > data & only care about GeoIP location data without the closest-mirror
> > tweaks MirrorManager does, we wanted to ask - would it be possible to
> > add some simple API just for GeoIP ?
> >
> > Basically, just an API, that returns the most probable country code
> > for the caller, without the closest mirror tweaks. I think it should
> > also result in less strain on the servers than calling the full API.
> >
> > We are even willing to supply patches adding this functionality. :)
> >
> > Best wishes
> > Martin Kolman
> > from the Anaconda team
> >
>
--
Vratislav Podzimek
Anaconda Rider | Red Hat, Inc. | Brno - Czech Republic
10 years, 11 months
Plan for tomorrow's Fedora Infrastructure meeting (2013-05-09)
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2013-05-09 at 19:00 UTC in #fedora-meeting on the freenode network.
Suggested topics:
#topic New folks introductions and Apprentice tasks.
If any new folks want to give a quick one line bio or any apprentices
would like to ask general questions, they can do so in this part of the
meeting. Don't be shy!
#topic Applications status / discussion
Check in on status of our applications: pkgdb, fas, bodhi, koji,
community, voting, tagger, packager, dpsearch, etc.
If there's new releases, bugs we need to work around or things to note.
#topic Sysadmin status / discussion
Here we talk about sysadmin related happenings from the previous week,
or things that are upcoming.
#topic Private Cloud status update / discussion
#topic Upcoming Tasks/Items
#info 2013-05-14 to 2013-05-28 BETA infrastructure freeze
#info 2013-05-20 - bugzilla upgrade.
#info 2013-05-28 F19 beta release
#info 2013-05-31 end of 1st quarter
#info 2013-06-01 nag fi-apprentices
#info 2013-06-08 drop inactive apprentices
#info 2013-06-18 to 2013-07-02 FINAL infrastructure freeze.
#info 2013-07-01 nag fi-apprentices
#info 2013-07-02 F19 FINAL release
#topic Open Floor
Submit your agenda items, as tickets in the trac instance and send a
note replying to this thread.
More info here:
https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings
Thanks
kevin
10 years, 11 months
Meeting Agenda Item: Introduction Kenny Ayers
by Kenny
Hey folks,
I signed up to learn how the Fedora project works, and I'm hoping to
contribute.
*IRC handle*: kenny or kennyayers
*Skills to offer*: python programming, RHEL system administration, storage
admin, mysql client programming and administration, Atlassian product
admin, Jenkins admin and debugging
*Skills to learn*: Hoping to improve on items from the previous list as
well as learning anything new that comes my way
*I'd like to work on*: python development projects, system and storage
administration, database programming and admin
Thank you,
Kenny
10 years, 11 months
Re: announcing fedocal calendar
by Bruno Wolff III
On Tue, May 07, 2013 at 11:09:45 -0600,
Kevin Fenzi <kevin(a)scrye.com> wrote:
>Greetings.
>
>Fedora Infrastructure is happy to announce the fedocal calendar
>application for all Fedora related calendar needs.
Thank you to all of the people that worked on this.
10 years, 11 months
