Meeting tomorow
by Mike McGrath
So tomorow we normally have a meeting in #fedora-admin. I have a
hunch many people won't be there because of Thanksgiving but in the
interest of being international I'd like to encourage those that can
be there to come and meet and discuss. Take minutes and send them to
the list and add them to the wiki.
http://fedoraproject.org/wiki/Infrastructure/Meetings
-Mike
17 years, 5 months
Possibly Unneeded Database Attributes
by TomLy
While straightening out the fedoraPerson schema I'm running into
attributes in the current accountsDB that aren't used or are all the
same value for all users. My plan is to get rid of these if possible.
We can add attributes later if needed.
Here they are in no particular order:
internal_comments -- this is always blank right now
wiki_prefs -- also always blank
orts_orgid -- always Fedora
Comments, concerns, chairs??
~tom
17 years, 5 months
Indroducing Myself
by Jeffrey Ollie
Hello everyone,
I thought that I'd take a moment to introduce myself. I'm a Linux and
network administrator for a community college in central Iowa. I've
been using various forms of Red Hat/Fedora Linux for 10+ years, and am
proficient in C, Python, shell, some Perl. I currently maintain ~21
packages in Fedora Extras. My primary interest in joining the
infrastructure list is helping with the development of the new version
control system (yes, I'm aware that a dedicated list for that purpose is
in the works). I mostly use Subversion for my own tasks but I'm
interested in helping with Mercurial, Bazzaar-NG, and Git. I may
volunteer for other tasks as time/interest permits.
Jeff
17 years, 5 months
Re: CVS is busted
by Mike McGrath
On 18 Nov 2006 09:31:26 -0600, Jason L Tibbitts III <tibbs(a)math.uh.edu> wrote:
> >>>>> "AK" == Avi Kivity <avi(a)argo.co.il> writes:
>
> AK> Just curious: why the read-only mount? shouldn't the RAID have
> AK> continued in degraded mode?
>
> Probably because something else bad happened that just completely
> screwed up the SCSI bus and corrupted data on multiple disks.
>
> - J<
>
We're talking about multiple failures across multiple drives, possibly
a backplane. Here's the current plan.
1) Move proxy 3-4 into the f.rh.c cluster so we can take our new dells back.
2) Grab one of the new Dells and build the new cvs box. This will
allow us to A) trust the hardware (we're all a little wary about the
current cvs box) and B) build a new box with atleast access to the old
box if we're missing something. It will also allow us greater
capacity with regards to future growth and the whole FC+FC=Fedora
thing.
3) Restore backups to the new cvs box.
4) test test test
5) Release to the wild and fix bugs as needed.
6) Take the old cvs box and run full diagnostics before we rebuild it
(it'll be come one of our db servers, either primary or backup)
Right now mgalgoci is working working on steps 1 and 2. When they are
done I'll be on step 3 and we'll need a few people for 4. We'll
probably discuss in #fedora-extras when the time comes.
Bottom line, this sucks but we're working on it. Should be up and
better than ever by Monday.
-Mike
17 years, 5 months
DRAFT: CVS Restored, Help Needed in Verification
by Warren Togami
Hey folks, this is a draft that I will soon be sending to
fedora-devel-list and fedora-extras-list.
Anything should be corrected?
========= DRAFT BEGINS HERE ==========
Sometime Early Friday, November 17th we suffered some kind of
catastrophic multi-disk or SCSI backplane failure in
cvs.fedora.redhat.com. Thanks to volunteer efforts of Fedora
Infrastructure and tremendous help from Red Hat IS/IT, we are now
nearing restoration of cvs.fedora.redhat.com from a backup on a powerful
new server graciously donated by Dell.
We are restoring from a backup that began November 16th, 2006 at 8:01PM
MST (Friday, November 17, 2006 at 03:01) and finished 100 minutes later.
Because this backup is not an exact snapshot, it is possible that
checkins that happened after 8:01PM are already in the restored data.
https://www.redhat.com/archives/fedora-extras-commits/2006-November/date....
We require your help to verify that all checkins are properly in
/cvs/extras and everything generally is working as expected. Please pay
careful attention to everything that changed in November 16th and 17th
according to the mailing list archive.
My personal records indicate that no CVS imports happened after this
backup, so we are fine in that regard.
PACKAGE OWNERS - especially kevin, corsepiu, remi and qspencer are to
verify and fix their own CVS modules, or explicitly identify someone
else to do it on their behalf.
EVERYONE ELSE - please keep an eye on CVS. SCREAM LOUDLY on
fedora-extras-list if you see anything that is wrong.
Sorry about any inconvenience or disruption that this unexpected problem
may have caused. Everyone owes a beer to the Fedora Infrastructure and
Red Hat IS/IT teams. =)
Thank you for supporting the Fedora Project.
Warren Togami
wtogami(a)redhat.com
17 years, 5 months
cvs update
by Matthew Galgoci
Some time on friday, cvs-int.fedora.phx.redhat.com sustained undetermined
storage problems and resulting filesystem corruption. As best I can figure,
we had a one drive in a raid6 array drop offline, and another disk in that
array emit scsi errors. Now, you're probably thinking, this is raid6, it
should have been able to sustain losing two disks and keep on going.
Well, you're right and you're wrong. If two disks had simply dropped out of
the array, we'd be fine. That wasn't the case however. Somewhere in the
equation is data corruption. raid is great up until your hardware corrupts
the data. To support this claim, all you need to do is realize that we
sustained numerous ext3 errors and had the journal abort, and the root fs
went read-only.
I did my level best to revive the system on friday and saturday. I was able
to get it pxe booted onto rescue media, which helped recovery immensely. I
took numerous screen shots to chronical what I went through as I attempted
to recover the raid6 arrays and the logical volumes.
http://people.redhat.com/~mgalgoci/cvs-int.jpg
http://people.redhat.com/~mgalgoci/cvs-int2.jpg
http://people.redhat.com/~mgalgoci/cvs-int3.jpg
http://people.redhat.com/~mgalgoci/cvs-int4.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs5.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs6.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs8.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs9.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs10.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs11.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs12.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs13.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs14.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs15.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs18.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs17.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs16.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs19.jpg
http://people.redhat.com/~mgalgoci/fedora-cvs20.jpg
After #20, I said the hell with it, time to move on.
We've installed one of the new Dell 2950 machines that Dell was kind enough
to donate to the Fedora Project. Mike McGrath is in the process of updatifying
and restorifying the data from backups.
I have a Dell tech coming on site again today to do some more work on the
old new cvs-int server. I think we know what the issues are on it and we'll
have it usable again in the next day or so.
In the mean time, I think we need to take a look at all the Dell fedora boxes
and check the scsi drives in them. There are known issues with certain drive
firmware that cause drives to go offline and report spurrious errors.
The relevant Dell update is here:
http://support.us.dell.com/support/downloads/download.aspx?c=us&cs=555&l=...
We'll need downtime and hands on site to do this update. I'm sure Stacy will
be able to assist.
--
Matthew Galgoci
GIS Production Operations
Red Hat, Inc
919.754.3700 x44155
17 years, 5 months
Fwd: Host DOWN alert for proxy2!
by Mike McGrath
Buhhh, anyone know what these are from?
-Mike
---------- Forwarded message ----------
From: nagios(a)fedoraproject.org <nagios(a)fedoraproject.org>
Date: Nov 19, 2006 8:53 PM
Subject: Host DOWN alert for proxy2!
To: mmcgrath(a)fedoraproject.org
***** Nagios *****
Notification Type: PROBLEM
Host: proxy2
State: DOWN
Address: proxy2.fedora.phx.redhat.com
Info: CRITICAL - Plugin timed out after 10 seconds
Date/Time: Sun Nov 19 19:53:46 MST 2006
17 years, 5 months
OpenSUSE Build Service?
by Jeroen Janssen
Hi,
I'm not sure if this is the correct Fedora mailinglist, but has anyone
had a look at the OpenSuSe Build Service pages (
http://en.opensuse.org/Build_Service )?
There are two presentations (from FOSDEM) about it and it seems they
have also done some experimenting with building non SuSe distributions
with this build system (like Fedora Core 4).
Is this something that also could benefit the Fedora Project?
Best regards,
Jeroen Janssen
17 years, 5 months
Project hosting thoughts
by Jesse Keating
Last night I decided I wanted more of a project presence for pungi than an hg
web interface. So I started exploring options. 108 was a disaster, I spent
30 minutes trying to find where to start a project, only to find out that
there is no way to find that out, you have to ask somebody and then get
direct URLS that aren't linked anywhere. Plus 108 doesn't support hg. So I
went looking to setup my own software, because damned if I'm going to use
Sourceforge, or anything based on it.
Trac software has been around for a while, and I've stumbled across a few
projects using it. Reading up on their website http://trac.edgewall.org/ the
trac feature list looked pretty neat. More interestingly each project within
a trac environment can have its own source control mechanism, and the have
support for cvs, svn, hg, bzr, darcs, and an experimental git. The git
support isn't all that great, but really, what the SCM support does is just
provide a web view of the files. The more important thing is that you could
get one of these source control repos at project creation time to do with as
you see fit.
Other things I like about trac, the wiki component uses the same markup (or
seems to thus far) as MoinMoin. This will make it very easy to take content
from a project and move it into the Fedora wiki should the need/desire arise.
Also users wouldn't have to learn yet another markup language. The ticketing
system is pretty simple and easy to use, wiki markup language works here too
should you want it. Tickets can easily be set to block Milestones, and a
Milestone view shows you the list of milestones and an easy view to see what
tickets are blocking a milestone, or how many blockers have been fixed (and
thus how close a milestone is to completion). There is also a 'timeline'
view that shows you a running history of project activity, such as wiki
edits, ticket creation/resolution, scm checkins, etc... with rss capability.
This would make it very easy to aggregate and track the progress of a
project.
Some things that I'd like to see investigated:
How decent is the support for multiple projects using one trac install?
Can we use the Fedora account system as a web authentication (trac just uses
web auth as far as I can tell)
How easy is it to create a web 'project creation' page that would handle the
initial setup of a project?
As a proof of concept, is there any way I can make use of a xen guest to setup
a trac instance for pungi? Some of the problems with using Trac would
involve that it has to be installed on the same system as the SCM (at least
for hg and git) or else regular syncs would have to be made with the actual
scm location. But honestly this might not be so bad to separate out
our 'hosted' project SCMs from our Fedora SCMs. Trac would run just fine on
CentOS/RHEL4 but would also work on FC6 for a xen guest (much like
CentOS/RHEL5) If we do setup a xen guest with a public IP, can we also set
it to something like 'hosted.fedoraproject.org' ?
--
Jesse Keating
Release Engineer: Fedora
17 years, 5 months
