3:22 p.m.
I have spent significant portions of the last couple of days trying to
weap my mind around what Fedora is doing. Let me start off by saying
that I think the direction of the project is wonderful and much
needed. As a system administrator managing three boxes, I love the
prospect of being able to use apt-get or yum to continuously upgrade
my systems without having to go through periodic CD shuffles and
reboots. Thus, I've joined fedora-devel to help.
However, at the moment there is at least one serious practical problem
that loses me as a package contributor. That is the use of Bugzilla
(or any other method that requires manual click-and-confirm) for RPM
submission. Most people maintain one or two packages at most; they
can live with a manual submission process. At last count, I
maintained thirty-seven packages -- thus, I can't. (This is also why
I don't publish through SourceForge.)
What I need is to be able to write an "upload" script for each of my
projects *once* (per project) that remote-scripts the RPM submission
process and all other things I need to do to publish a release. So,
what I need from Fedora is for there to be a submission front end
(call it 'fedora-submit') which, given a collection of RPMS as
arguments, does in a batch mode all that is necessary to upload them
and put them on a submission queue at fedora. It is OK if
fedora-submit requires additional metadata as long as it can all be
specified at start of run by command-line switches.
I asked about this on the IRC channel and was told that Bugzilla has
an XML-RPC interface. If so, writing fedora-submit as an XML-RPC
client ought not be too difficult; I would be surprised if it required
more than 150-200 lines of Python.
I am actually willing to write this myself, if anyone can point me at
documentation for the XML-RPC interface to Bugzilla and is willing to
answer questions about places where the documentation is inadequate.
In the process, I would be willing to help improve the (presently
inadequate) documentation on the submission procedure.
In case it's not obvious, I think the result ought to ship as part of
fedora-core in order to reduce the entry barriers for independent
packagers as much as possible.
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
1:36 a.m.
On Sun, 28 Sep 2003, Eric S. Raymond wrote:
[...]
However, at the moment there is at least one serious practical
problem
that loses me as a package contributor. That is the use of Bugzilla
(or any other method that requires manual click-and-confirm) for RPM
submission. Most people maintain one or two packages at most; they
can live with a manual submission process. At last count, I
maintained thirty-seven packages -- thus, I can't. (This is also why
I don't publish through SourceForge.)
[...]
I don't think there is an official "RPM submission procedure" yet. The
unofficial (only!) one is to send mails to Red Hat folks, or add a bug in
Bugzilla.
But I think the final one will be much, much better .. once we get there.
:-)
--
Pekka Savola "You each name yourselves king, yet the
Netcore Oy kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
1:59 a.m.
Pekka Savola <pekkas(a)netcore.fi>:
I don't think there is an official "RPM submission
procedure" yet. The
unofficial (only!) one is to send mails to Red Hat folks, or add a bug in
Bugzilla.
But I think the final one will be much, much better .. once we get there.
:-)
Indeed it will, because I'm already on the problem :-) I am stepping up with
an offer to write, document, and maintain the client software for doing
package submissions.
I had a long IRC chat with some key Bugzilla developers this afternoon.
The conclusion of the chat was that Bugzilla ought to have an XML-RPC
interface for posting bugs. But, pending that, Christian Reis gave me
a scratch Python script for submitting bugs via scripted CGI that I
am in the process of reworking into a production-quality tool. It can be
used with Bugzilla as it is now. It's called `bug-bugzilla'.
Once I get bug-bugzilla done, I'll write a thin wrapper around it called
`fedora-submit' for submitting RPMS through Fedora Bugzilla. If your
channel for RPM submissions changes, I will modify fedora-submit
accordingly. Someday it might be an XML-RPC client, but users
need not care.
Fedora can ship both scripts as RPMs, fedora-submit will depend on
bug-bugzilla until and unless that changes, and everybody should be
happy. Including the Bugzilla people, who have had people clamoring
for a production-quality tool of this kind forever.
Does this sound like a good plan?
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
4:01 a.m.
On Mon, 29 Sep 2003, Eric S. Raymond wrote:
[...]
Fedora can ship both scripts as RPMs, fedora-submit will depend on
bug-bugzilla until and unless that changes, and everybody should be
happy. Including the Bugzilla people, who have had people clamoring
for a production-quality tool of this kind forever.
Does this sound like a good plan?
If a package in Fedora Core is maintained by non-RH person X, having
person X go through Bugzilla to update his package (whether with a tool or
not), implying having a Red Hat person look at the problem and apply it in
a timely fashion, is IMHO *way* too heavy a process.
We need more than that, e.g. CVS commit access to the server housing the
RPM spec files and patch files, etc.
I'm not saying that the tool you propose would not be useful, I just think
that (at least as I see Fedora) it's still too heavy a process...
--
Pekka Savola "You each name yourselves king, yet the
Netcore Oy kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
4:30 a.m.
Pekka Savola <pekkas(a)netcore.fi>:
If a package in Fedora Core is maintained by non-RH person X, having
person X go through Bugzilla to update his package (whether with a tool or
not), implying having a Red Hat person look at the problem and apply it in
a timely fashion, is IMHO *way* too heavy a process.
We need more than that, e.g. CVS commit access to the server housing the
RPM spec files and patch files, etc.
I'm not saying that the tool you propose would not be useful, I just think
that (at least as I see Fedora) it's still too heavy a process...
That's not a philosophical issue I'm equipped to have an opinion on,
at this point. Fundamentally, I don't care (for purposes of designing
this tool) whether the submission channel is Bugzilla or something
else, and whether submissions are filtered by a human or not.
What I care about is that *there be a tool that allows remote-scripting the
submission process*. I don't see that the interface of the tool needs
to be any more complicated than
fedora-submit --tree {stable|testing|...} foo-1.2.3.fdr.0.rpm
regardless of how the process is actually implemented. My present
plan is to implement fedora-submit as a wrapper around a script for
submitting Bugzilla bugs (which script I have just finished writing)
but that is an implementation detail about which the user should not
have to care. If you guys want to change the implementation mechanism,
you just tell me and I'll make fedora-submit use it.
So let's *not* get sidetracked onto whether the submission process
is right or not. Just tell me what it actually *is*.
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
6:24 a.m.
Eric S. Raymond wrote:
regardless of how the process is actually implemented. My present
plan is to implement fedora-submit as a wrapper around a script for
submitting Bugzilla bugs (which script I have just finished writing)
but that is an implementation detail about which the user should not
have to care. If you guys want to change the implementation mechanism,
you just tell me and I'll make fedora-submit use it.
So let's *not* get sidetracked onto whether the submission process
is right or not. Just tell me what it actually *is*.
Fedora.us is not "The Fedora Project". Fedora.us is the older packaging
project for united volunteer packagers around the RH platform.
(Fedora.us continues in operation during the next few months while the
new Fedora is in progress. No packages will be published from the new
Fedora for quite a while because all the infrastructure, policies,
standards and procedures need to be formed. It has been unofficially
suggested by some RH elders to point people at fedora.us for package
submission and approval during these next few months because
fedora.redhat.com is NOT READY. fedora.us is recognized as having some
technical clue and over-paranoid reluctance in accepting stuff, so stuff
that is accepted is likely to be accepted into Fedora Extras much more
readily at a later date .... after legal approvals and stuff... stuff
stuff stuff)
It was our (fedora.us) intention from the beginning to better automate
the submission process with command-line based tools, and eventually
write a (RDBMS) database driven management system. Until the effort
was put forward to make that happen, Fedora.us had used a Bugzilla based
package submission, QA discussion and tracking system. It worked great
for the publication of hundreds of packages, with almost 200 more
currently needing QA approval.
However this being said, there has been almost zero discussion about
whether the new fedora.redhat.com will use anything like fedora.us'
submission and approval process. While we don't know yet
fedora.redhat.com will become, now is the time to experiment with
fedora.us to learn more about "What works" and "What doesn't".
Your suggestion of a fedora-submit script sounds like a good way to
reduce overhead in submitting Bugzilla reports for packages. I am glad
somebody finally wants to implement it. I suggest to you to read
through many of the fedora.us reports to see the steps involved in this
process. Your script would need to (off the top of my head)
1) Check for duplicates.
2) Submit a new report for a package, along with GPG signatures.
3) Submit revised packages along with new GPG signatures after some
discussion points out flaws.
4) Upload packages to *somewhere* which we still haven't worked out. We
obviously cannot give CVS access to everyone, and there may be issues
with a public upload location at an official server. I much prefer the
GPG signatures + submitter controlled URL process that fedora.us
currently uses, but perhaps a future process may have a hybrid. The
more senior trusted packagers have upload access to an official staging
area or CVS, while everyone else uses GPG + URLs.
(Sopwith, the "hostile build" requirement with mach+vserver would be a
boon in the "everyone else" category, because automation greatly reduces
the amount of man-hours needed for package development while reducing
the cost of entry for new-comers.)
Regarding your other comment regarding the over-complexity of package
naming requirements in fedora.us documents: May I warn the feeling of
"too-complex" may be partly from ignorance of the complexity of avoiding
clashes. There are a great many common problems that happen when people
do not THINK when creating their package.
Those requirements for seemingly over-complex naming requirements was
the result of arguing for almost 4 months with no useful packaging
happening. The guidelines were mainly about preventing common cases
where "Epoch" needs to be incremented for poor reasons. Unfortunately a
large part of the other requirements were based upon assumption that Red
Hat would not pay attention to our naming scheme or packages, and we
needed to avoid any chance of conflicting with future RH updates. In
almost all cases the naming scheme has successfully avoided naming and
version clashes with RH released, errata and beta packages.
Fortunately now that we are no longer an unaffilited 3rd party, I
believe we no longer need the many weird corner case naming
requirements. The naming document can be greatly simplified, dropping
the leading "X.fdr." part of the %release tag and all related naming
policies because they are not needed any longer.
(We would need to have some common agreement with RH and fedora.us
packagers in order to make this official, but I believe that will be an
easy sell after I post my upcoming draft for "Fedora Project package
naming conventions" for fedora.redhat.com.)
In the mean time, please submit your packages to fedora.us in any matter
that you wish. Don't worry if you don't understand the overly complex
naming guidelines, because the QA people will quickly point out errors.
You mention that you would like to help in rewriting that documentation.
We would greatly appreciate your help in doing so (and I personally
like your writing skills), but may I highly suggest seeing the actual
old and inefficient process in motion for a while before making any
assumptions.
Also it is my opinion that while our current GPG requirements and manual
Bugzilla usage seems like a "waste of time", that time requirement pales
in comparison to the QA approval process. fedora-submit and improved
command line tools integrated with GPG would speed up the Bugzilla
report communication process would be a plus to that process.
BTW, your fedora-submit tools may be good for our RPM development
toolkit package "fedora-rpmdevtools" which is briefly documented here:
http://www.fedora.us/wiki/FedoraRPMDevTools
Warren Togami
warren(a)togami.com
2:57 p.m.
Warren Togami <warren(a)togami.com>:
Your suggestion of a fedora-submit script sounds like a good way to
reduce overhead in submitting Bugzilla reports for packages. I am glad
somebody finally wants to implement it. I suggest to you to read
through many of the fedora.us reports to see the steps involved in this
process. Your script would need to (off the top of my head)
1) Check for duplicates.
2) Submit a new report for a package, along with GPG signatures.
3) Submit revised packages along with new GPG signatures after some
discussion points out flaws.
4) Upload packages to *somewhere* which we still haven't worked out. We
obviously cannot give CVS access to everyone, and there may be issues
with a public upload location at an official server. I much prefer the
GPG signatures + submitter controlled URL process that fedora.us
currently uses, but perhaps a future process may have a hybrid. The
more senior trusted packagers have upload access to an official staging
area or CVS, while everyone else uses GPG + URLs.
On point 1, the design of fedora-submit should not try to wire in
repository policy. In particular, putting dup checking in there would
probably be a bad idea, as your concept of a dup is likely to evolve
over time. For example, is it a dup when the same-named RPM is
submitted for two different trees? You want to be able to change the
answers to questions like that without replacing every client instance
in the world.
Points 2 and 3 look right to me. As a toolbuilder I'm agnostic on point 4,
but I share your preference. There's no reason for your server to have
to pay storage costs for RPMs not yet accepted when a URL pointer can
be functionally equivalent to having a copy.
Fortunately now that we are no longer an unaffilited 3rd party, I
believe we no longer need the many weird corner case naming
requirements. The naming document can be greatly simplified, dropping
the leading "X.fdr." part of the %release tag and all related naming
policies because they are not needed any longer.
Good. Then by all means nuke 'em.
In the mean time, please submit your packages to fedora.us in any
matter
that you wish. Don't worry if you don't understand the overly complex
naming guidelines, because the QA people will quickly point out errors.
OK. I'll write fedora-submit and do some testing, then. I think I've
essentially finished the bug-bugzilla tool that the first implementation
of fedora-submit will use; it's now in evaluation and testing with Christian
Reis of the Bogzilla project.
BTW, your fedora-submit tools may be good for our RPM development
toolkit package "fedora-rpmdevtools" which is briefly documented here:
http://www.fedora.us/wiki/FedoraRPMDevTools
Ahh, that's good to know. Yes, that is obviously the right place.
I see I even got the naming convention right :-).
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
7520
days inactive
7521
days old
6 comments
3 participants
participants (3)
-
Eric S. Raymond -
Pekka Savola -
Warren Togami