October 2014 - rel-eng - Fedora mailing-lists

#6017: please give kushal permissions to do scratch cloud image builds in koji
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6017: please give kushal permissions to do scratch cloud image builds in koji ----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ If I remember right (and I admit I remember only vaguely) there are special permissions required to build images in koji. Please give these to kushal so he can create scratch images to test things out. Thank you! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6017> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

2 11

#5665: Providing a master CHECKSUMS file for boot.fpo
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5665: Providing a master CHECKSUMS file for boot.fpo -----------------------------+------------------------ Reporter: shaiton | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Alpha | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, as requested to webmasters[1], could we get a CHECKSUMS file (as provided for other Fedora releases) including all checksums? It also needs to be signed of course. You might decide that it is not needed, just forwarded the query. [1] https://lists.fedoraproject.org/pipermail/websites/2013-July/011535.html -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5665> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5939: f21-kde koji tag/target
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5939: f21-kde koji tag/target ---------------------+------------------------ Reporter: rdieter | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: koji Keywords: | Blocked By: Blocking: | ---------------------+------------------------ Please create a f21-kde koji tag/target. kde-sig will continue to have monthly kde4 and kde frameworks 5 updates to work on, so having this till be greatly beneficial. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5939> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#1107: auto-cleanup rawhide trees
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#1107: auto-cleanup rawhide trees -------------------------+-------------------------------------------------- Reporter: notting | Owner: rel-eng(a)lists.fedoraproject.org Type: enhancement | Status: new Milestone: | Component: koji Keywords: | -------------------------+-------------------------------------------------- Rawhide tree expiry is manual at the moment, unless I missed something. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/1107> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 8

#6020: If Workstation tree is being dropped, please do same with Cloud
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#6020: If Workstation tree is being dropped, please do same with Cloud ----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ Based on yesterday's FESCo meeting, the plan is for Fedora Workstation composes to be refactored such that there is not a separate Workstation tree, but that top-level directory on the mirrors will just contain the live ISOs in various architectures. I still think just rm'ing the netinst images from those trees is the lowest-impact workaround, but if this is what we are doing, Cloud should match because the exact same situation applies. So, please drop everything but the Images subdirectory from the Cloud tree. Thank you! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6020> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#5984: Adding metadata to the Fedora Workstation Live CD
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5984: Adding metadata to the Fedora Workstation Live CD ----------------------------+------------------------ Reporter: rhughes | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ Hi Dennis, and other rel-eng dudes! I've hit somewhat of a roadblock with this: http://blogs.gnome.org/hughsie/2014/08/29/putting-packagekit-metadata-on- the-fedora-livecd/ Basically, I need access to the post-mash (i.e. fedora, fedora-updates) from the compose server. This allows us to put the old metadata on the Fedora LiveCD so we don't have to wait the few minutes after install to be able to search for and install new applications. I totally agree we don't want unfettered internet access, or even access to the mirror network, as I'm happy doing something like this in the fedora-live-workstation.ks file: %post --nochroot DESTDIR=$INSTALL_ROOT $INSTALL_ROOT/usr/libexec/packagekit-direct refresh --rewrite-baseurl fedora=http://some.local.ip.address/f21 --rewrite- baseurl updates=http://some.local.ip.address/f21-updates On the assumption that some.local.ip.address is available from inside the LiveCD builder and gives us access to the mash'ed *metadata* only (I don't need packages, and it can even be a few days old with no penalty). The alternative is that I ship the fedora and updates metadata in the PackageKit rpm file, which means doing a -0.01 day release of the RPM for the Live media, which makes everything so very fragile. Other ideas of course welcome. Thanks! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5984> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5894: git branches for SCL packages
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5894: git branches for SCL packages -----------------------------+------------------------ Reporter: mmaslano | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Alpha | Component: git Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ I would prefer SCL packages in branches of the main package (eg ruby). We used the same approach in some internal projects and it worked well with koji, composes, everything... I suggest name of branches: scl-f21 scl-f22 -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5894> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 68

#5805: add index.asc files for cloud images for virt-builder
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5805: add index.asc files for cloud images for virt-builder ----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: enhancement | Status: new Milestone: Fedora 20 Beta | Component: other Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ Please add a GPG-signed 'index.asc' file to the Fedora 20 Images/ directory describing cloud images for virt-builder and other tools. The format is documented at http://libguestfs.org/virt-builder.1.html #creating-your-own-templates, and there's actually a Fedora example there. Thanks! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5805> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 16

#5883: New Cloud Images Process
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5883: New Cloud Images Process -----------------------------+------------------------ Reporter: red | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: git Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ The cloud kickstart file has been moved to the spin-kickstarts Git by rel- eng. When this happened, it seems there was no discussion regarding the future process to create cloud images and the Spins Process doesn't seem appropriate. Therefore, the Cloud SIG would like to know what the intended process should look like to give new contributors access to the cloud image kickstart files and how to make sure new images (i.e. products) are created from new kickstart files if necessary. For now, mattdm (probably already has) and I need access from the Cloud SIG but more will follow soon as we get ready to work on new products. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5883> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 6

#5880: update to pv-grub aki used in EC2
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5880: update to pv-grub aki used in EC2 -----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Alpha | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ * '''What:''' We are currently using old boot images with pv grub 1.02 -- newer versions at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProvidedKernels.html * '''Where:''' https://git.fedorahosted.org/cgit/releng/tree/scripts/cloud/fedora_ec2.py * '''Why:''' new version allows GPT partitions, works with both unpartioned and partitioned, would allow us to use a separate /boot if we want to change the filesystem for / in the future; also new version has bugfixes * '''When:''' F21 alpha; prereq of using whole-disk images -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5880> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5721: Need application icons and application descriptions
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5721: Need application icons and application descriptions ----------------------------+------------------------ Reporter: rhughes | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ Hi, For the application installer preview I need the application icons and localized description data. I had the fedora-app-data package NAKd, but something needs to provide this data. Spot seemed to think it would be okay to include as metadata. The idea is to: * Generate the appstream.xml and appstream.tar files on koji * xmlmerge the .xml files and cat the .tar files when mash'ing the compose together. I've got C code to do the former, although someone familiar with python probably wants to re-write the code before putting it into koji. For F20 we can probably just ship the 15 most popular application icons and descriptions in gnome-software, although this won't scale with all the apps in Fedora. Any questions, I'm hughsie on IRC. Thanks. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5721> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 19

#5886: need method for distributing urgent fixes... urgently
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5886: need method for distributing urgent fixes... urgently -----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: enhancement | Status: new Milestone: Fedora 21 Alpha | Component: mash Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Right now, the time it takes to do a push of F19 and F20 updates is a critical bottleneck in our ability to put out zero-day security updates. We need a solution for this. I understand that Infrastructure is working on a Netapp fix which will give a 3x speedup. I think that's probably still too slow. We need to be able to put out urgent updates on the scale of minutes, not hours. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5886> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 31

#5985: Delete branch created by mistake in dist-git: openstack-glance/f21-patches
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5985: Delete branch created by mistake in dist-git: openstack-glance/f21-patches -----------------------------+------------------------ Reporter: hguemar | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: git Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, someone created a branch by mistake in dist-git: f21-patches in openstack-glance. Could someone delete it ? http://pkgs.fedoraproject.org/cgit/openstack-glance.git/log/?h=f21-patches Thanks -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5985> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5856: tag git commit for successful builds in dist-git
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5856: tag git commit for successful builds in dist-git -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ After a build succeeded, the commit used to build from should be tagged in dist-git. The tag should probably be the build's name-version-release. The information about successful builds could be fetched from fedmsg. Action Items: - Check whether fedmsg contains messages for successful builds - Decide on the name for tags I can work on this, I just want to track this publicly here. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5856> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5986: write script to get all git commits used to build a package
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5986: write script to get all git commits used to build a package -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: other Keywords: easyfix | Blocked By: Blocking: 5856 | -----------------------------+------------------------ Create a mapping from build NVR to commit ID and vice-versa. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5986> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#5843: Create koji plugin to allow building only from whitelisted git branches
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5843: Create koji plugin to allow building only from whitelisted git branches -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ To allow user git branches, koji needs to be able to allow building only from whitelisted git branches such as master or epel-7. This might need to be addressed in a koji plugin. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5843> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 7

#5878: noarch packages corrupted on import on ppc.koji.fp.o
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5878: noarch packages corrupted on import on ppc.koji.fp.o -----------------------------+------------------------ Reporter: dwa | Owner: rel-eng@… Type: defect | Status: new Milestone: Fedora 20 Final | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ (Filing a ticket so other people can weigh in / brainstorm without getting lost in an e-mail thread) Recently (in the past 2 weeks or so) I've seen a number of source RPMs become corrupted when koji-shadow imports them to ppc.koji.fp.o. This was previously a once-in-a-blue-moon occurence but it's now happened multiple times. Most recently corrupted srpm: * moodle-2.5.5-1.fc20.src.rpm Previously: * mingw-gnutls-3.1.21-1.fc20 * elementary-xfce-icon-theme-0.4-1.fc20 * kde-l10n-4.12.3-1.fc20 * oxygen-icon-theme-4.12.3-1.fc20 I've not noticed this happening with any f19 packages, and f21 isn't signed (which is when we typically notice this is when signing packages.) ppc.koji is currently running koji-1.8.0-2.el6. Any ideas for troubleshooting? It's not difficult to fix (I've documented the procedure I'm using at https://fedoraproject.org/wiki/User:Dwa#Corrupt_noarch_RPMs ) but it'd be nice for it to not happen at all. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5878> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 5

#5678: koji package for cloud images so they can be non-scratch, and garbage collection policies for those packages
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5678: koji package for cloud images so they can be non-scratch, and garbage collection policies for those packages -----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Replying to [ticket:3897 mattdm]: > We want to generate non-scratch cloud images. Because of a bug, these currently need to have the name of the architecture in the package. That will be fixed in the Real Soon Future, but in the meantime I'd like to get going with what we have. Could we please create koji packages named > > {{{ > Fedora-19-cloud-x86_64 > Fedora-19-cloud-i386 > Fedora-20-cloud-x86_64 > Fedora-20-cloud-i386 > Fedora-rawhide-cloud-x86_64 > Fedora-rawhide-cloud-i386 > }}} > > And, we would also like to have a custom garbage collection policy for these; even though they are not scratch builds, they don't need to stay around forever. If we can do a nightly builds (ideal), then I think a week is fine; if they are weekly, keeping a month around seems good. Of course, longer is always better. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5678> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 6

#5622: [RFE] Allow user to (create/)delete private branches
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5622: [RFE] Allow user to (create/)delete private branches -----------------------------+------------------------ Reporter: vondruch | Owner: rel-eng@… Type: enhancement | Status: new Milestone: Fedora 19 Alpha | Component: git Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ I'd like to be able to manage my private branches. For example, in preparation for Ruby 2.0 release, I was pushing my changes to .spec file into Ruby's repository in dist git, into ruby-2.0 branch. This branch is already merged into master for some while and I'd like to delete it, but it is forbidden: {{{ $ git push -f origin :ruby-2.0 remote: + refs/heads/ruby-2.0 ruby vondruch DENIED by fallthru remote: error: hook declined to update refs/heads/ruby-2.0 To ssh://vondruch@pkgs.fedoraproject.org/ruby ! [remote rejected] ruby-2.0 (hook declined) error: failed to push some refs to 'ssh://vondruch@pkgs.fedoraproject.org/ruby' }}} Would it be possible to allow some development branches, which can be created and deleted as well, without need of opening rel-eng tickets etc? Thanks -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5622> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 5

#5329: RFE: Send branched report only when there is any change committed
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5329: RFE: Send branched report only when there is any change committed -----------------------------+------------------------ Reporter: pnemade | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 18 Alpha | Component: mash Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, While we are in freeze, can branched report be only sent to the devel/test list when there is any change added for any package? Parag. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5329> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#5218: block httpd-devel.i686, php-devel.i686 from x86_64 compose
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5218: block httpd-devel.i686, php-devel.i686 from x86_64 compose -----------------------------+------------------------ Reporter: jorton | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 18 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Could the .i686 packages for both httpd-devel and php-devel be blocked from inclusion in the x86_64 compose, we don't (can't/won't) support them biarch. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5218> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

#5215: Stage comps and spin-kickstarts during freezes
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5215: Stage comps and spin-kickstarts during freezes -----------------------------+------------------------ Reporter: adamwill | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 18 Alpha | Component: git Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ As noted on the F17 QA retrospective by me and Bruno Wolff: https://fedoraproject.org/wiki/Fedora_17_QA_Retrospective it's a problem that we use git master of both comps and spin-kickstarts for composes - not the latest packaged version - and both these git repos are not frozen during freezes. During the 17 cycle, we had a concrete example of a comps change during a freeze causing considerable trouble: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=807879 - pulling Boxes into comps during the Beta freeze made that issue a serious problem. For spin-kickstarts, the same kind of thing is possible, and we also have a release criterion requiring a spin-kickstarts package which matches the .ks files used for the compose to be present in the release repository; obviously, the lack of a git freeze makes this tricky to ensure. So we would strongly recommend, at minimum, a mechanism for freezing spin- kickstarts and comps at Alpha, Beta and Final freeze. Post-freeze changes could be made following the same blocker/NTH process used for other post- freeze changes. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5215> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#5971: kojipkgs shows a 403 error instead of 404 error for deleted scratch build RPMS
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5971: kojipkgs shows a 403 error instead of 404 error for deleted scratch build RPMS -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ This can currently be seen with https://kojipkgs.fedoraproject.org/work/tasks/507/7190507/libreport- plugin-logger-2.2.3-1.rhbz1122690.1.fc20.x86_64.rpm: {{{$ curl -I https://kojipkgs.fedoraproject.org/work/tasks/507/7190507 /libreport-plugin-logger-2.2.3-1.rhbz1122690.1.fc20.x86_64.rpm HTTP/1.0 403 Forbidden Date: Tue, 19 Aug 2014 17:31:13 GMT Server: Apache/2.2.15 (Red Hat) Content-Type: text/html; charset=iso-8859-1 X-Cache: MISS from kojipkgs02.phx2.fedoraproject.org X-Cache-Lookup: MISS from kojipkgs02.phx2.fedoraproject.org:80 Via: 1.0 kojipkgs02.phx2.fedoraproject.org (squid/3.1.10) Connection: keep-alive }}} -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5971> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#5223: F18 TODO: automate torrent generation
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5223: F18 TODO: automate torrent generation -----------------------------+------------------------ Reporter: ausil | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 18 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Torrent Releases Infrastructure SOP http://torrent.fedoraproject.org/ is our master torrent server for Fedora distribution. It runs out of ServerBeach. Contact Information Owner: Fedora Infrastructure Team Contact: #fedora-admin, sysadmin-torrent group Location: ibiblio Servers: torrent.fedoraproject.org Purpose: Provides the torrent master server for Fedora distribution Torrent Release When you want to add a new torrent to the tracker at http://torrent.fedoraproject.org you need to take the following steps to have it listed correctly: 1. login to torrent.fedoraproject.org. If you are unable to do so please contact the fedora infrastructure group about access. This procedure requires membership in the torrentadmin group. 2. upload the files you want to add to the torrent to torrent.fedoraproject.org:/srv/torrent/new/$yourOrg/ 3. use sha1sum and verify the file you have uploaded matches the source 4. organize the files into subdirs (or not) as you would like 5. run /srv/torrent/new/maketorrent [file-or-dir-to-torrent] ([file-or-dir-to-torrent]) to generate a .torrent file or files 6. copy the .torrent file(s) to: /srv/torrent/www/torrents/$yourOrg/ 7. cd to /srv/torrent/torrent-generator/ or /srv/torrent/spins- generator/ (depending on if it is an official release or spins release) 8. add a .ini file in this directory for the content you'll be torrenting. we use fedora-torrent-ini.py from fedora releng git repo to generate it 9. mv all files from /srv/torrent/new/$yourOrg into /srv/torrent/btholding/ - this includes the files you uploaded as well as the .torrent files you've created. Your files will be linked on the website and available on the tracker after this. $yourOrg is one of fedora or spins this all needs to be setup to be done via a single script. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5223> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#5332: [RFE] Generate .composeinfo for Fedora secondary arches
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5332: [RFE] Generate .composeinfo for Fedora secondary arches -----------------------------+------------------------ Reporter: bpeck | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 18 Alpha | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Please generate a .composeinfo for secondary arches. The secondary arches that we are interested in are: arm, ppc, and s390. Ideally we could get a .composeinfo when *all* these arches are complete. But I understand that they have different time schedules and may update at different times. Worst case we look to see if the datetime on the file changes and we can re-import again with the newly added arch. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5332> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#5866: Create new channel for building Java packages
by Fedora Release Engineering 29 Apr '15

29 Apr '15

#5866: Create new channel for building Java packages ----------------------+------------------------ Reporter: mizdebsk | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: koji Keywords: | Blocked By: Blocking: | ----------------------+------------------------ == Request == Please create a new Koji channel for building Java packages which would exclude ARM builders, or implement an equivalent solution. My goal is to be able to build noarch Java packages skipping any ARM builders. Using custom koji command is not a problem for me. == Justification == The great majority of Java packages are noarch. As such they are most often pisked up by ARM builders. Java performance on ARM is very poor. The situation got even worse after JIT for ARM was disabled in JVM. ARM machines are often slower by one to three orders of magnitude than x86_64. Some tests which execute in a few seconds on x86_64 can take over half an hour on ARM. Current state of Java on ARM combined with Koji configuration makes it difficult to maintain Java packages in Fedora. A substantial amount of human resources is wasted on waiting for builds to complete, debugging tests which fail only on ARM, keeping track on builds and resubmitting them. That resources could be utilised better and I think that justifies my request. == Alternative solutions == I have tried differnt alternative solutions since ARM became a primary architecture and none of them is good enough. One ad-hoc solution is submitting numerous dummy koji tasks for ARM builders until their capacity is exhausted, then submitting Java package build and canceling ARM tasks. This is obviously a hack and should be avoided. If Koji configuration is not improved the only remaining solution will be converting Java packages from noarch to arch-specific and adding ExcludeArch. I still hope that ARM situation will eventually improve, so I would like to avoid doing drastic things like that, if possible. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5866> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 20

#5941: Requesting Koji user certificate for Koschei
by Fedora Release Engineering 27 Apr '15

27 Apr '15

#5941: Requesting Koji user certificate for Koschei -----------------------------+------------------------ Reporter: msimacek | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Koschei is a continuous integration tool for Fedora packages that does scratch-builds of packages in rawhide as their dependencies change. Right now it is using my personal certificate for initiating the scratch-builds, but since we'd like to deploy it in the Fedora Infrastructure, we need a separate Koji account for it. It will be used only for scratch-building, no other permissions are needed. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5941> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 12

#5967: Enable source repo generation
by Fedora Release Engineering 30 Mar '15

30 Mar '15

#5967: Enable source repo generation -------------------------+------------------------ Reporter: mizdebsk | Owner: rel-eng@… Type: enhancement | Status: new Milestone: | Component: koji Keywords: | Blocked By: Blocking: | -------------------------+------------------------ Please enable source repository generation in kojira, at least for rawhide (currently f22-build). Justification: There are many use cases which require access to YUM source repository. One of them is Koschei (a continous rebuild service), which requires SRPM repo for resolving package build-dependencies. Implementation of this feature should be simple - it just involves editing kojira.conf file -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5967> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 6

#5870: rawhide signing
by Fedora Release Engineering 23 Mar '15

23 Mar '15

#5870: rawhide signing -----------------------------+------------------------ Reporter: kevin | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ We have talked a number of times about getting rawhide packages signed, but haven't been able to come up with a solution that is secure and meets our needs. We should try and do so. :) This came up again today because gnome-software has different code paths for signed/unsigned content and they would very much like rawhide to be signed so it tests the same code as for stable releases. * There is a koji plugin to sign all builds, but it's not implemented in a very nice way and stores it's keys/passphrases in clear text files on the hub. * Manually signing with sigul could be done, but since there's no gating, it would mean either large amounts of packages would go out unsigned or composes would fail for unsigned packages often. * Additional space would be taken up by more signed rpms/signatures. * Any solution we come up with could possibly be also used by copr, which also wishes to sign builds in an unattended manner. Ideas welcome. ;) -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5870> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 18

#5931: [Proposal] Move new branch and unretire requests to pkgdb2
by Fedora Release Engineering 23 Mar '15

23 Mar '15

#5931: [Proposal] Move new branch and unretire requests to pkgdb2 -------------------------+------------------------ Reporter: pingou | Owner: rel-eng@… Type: enhancement | Status: new Milestone: | Component: git Keywords: | Blocked By: Blocking: | -------------------------+------------------------ Till suggested me recently that maybe we could port the new-branch requests to pkgdb2, together with the requests that for a package to be unretired. I have been working with these ideas, they are simply a way for users to ask an admin to perform an action (atm: request.branch and request.unretire). The output can be seen for admins via the UI at: http://209.132.184.188/admin/actions/ Or for everyone via the API at: http://209.132.184.188/api/admin/actions/ Note: the logic to change the status of a request isn't there yet but I thought I would first run the idea by you guys to get some early feedback, especially since if you like the idea, some tools will need adjustments. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5931> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 21

#6016: Use fedpkg-minimal in Fedora buildroots
by Fedora Release Engineering 16 Mar '15

16 Mar '15

#6016: Use fedpkg-minimal in Fedora buildroots ----------------------+------------------------ Reporter: pbabinca | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: koji Keywords: meeting | Blocked By: Blocking: | ----------------------+------------------------ Speedup build root readiness for the builds by reducing dependencies pulled into build root by fedpkg python package. Use [http://koji.fedoraproject.org/koji/buildinfo?buildID=485120|fedora- minimal] shell script which fetches sources only. Similar script is currently used in EPEL 7 buildroot. Start with rawhide later to other supported dists. Original idea came in [https://lists.fedoraproject.org/pipermail/devel/2014-July/201154.html|discu… about slow buildSRPMFromSCM]. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6016> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#6008: Fedora 21 Alpha .composeinfo should have family=Fedora
by Fedora Release Engineering 11 Mar '15

11 Mar '15

#6008: Fedora 21 Alpha .composeinfo should have family=Fedora ----------------------+------------------------ Reporter: dcallagh | Owner: rel-eng@… Type: defect | Status: new Milestone: | Component: other Keywords: | Blocked By: Blocking: | ----------------------+------------------------ The .composeinfo file for Fedora 21 Alpha Server currently has this: {{{ [product] family = Fedora-Server name = Fedora-Server-21_Alpha version = 21_Alpha variants = Server }}} For future F21 composes please update it so that the family is just "Fedora", with "Server" as the variant like it is currently. This is consistent with how RHEL has been doing its variant products for many years. It also just makes more sense -- all the new Fedora variants are in the same family, that's the whole point of that field. And it will make our lives easier in Beaker, where we use the family to figure out which harness builds to use and which kickstart syntax to use. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6008> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#5775: switch to partitioned image and "hd00" pvgrub akis for EC2; remove editing of menu.lst from image processing
by Fedora Release Engineering 09 Mar '15

09 Mar '15

#5775: switch to partitioned image and "hd00" pvgrub akis for EC2; remove editing of menu.lst from image processing ----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ The "hd00 aki" is like the pvgrub kernel image we are using now, but meant for a partitioned device. If we switched to doing this, we could avoid editing the menu.lst file, and therefore would have a more-identical image. . See http://www.dowdandassociates.com/content/howto-amazon-ec2-kernel- images-pv-grub/ (or http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProvidedkernels.html, which seems to have vanished today) for more. You can find the images in each region with: euca-describe-images -o amazon --filter "manifest-location=*pv-grub- hd00*" and of course take the newest version. At this point, I doubt we want to change this by alpha, but maybe we could for beta? -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5775> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 6

#4985: Extend test image creation SOP to require notification of delays
by Fedora Release Engineering 09 Mar '15

09 Mar '15

#4985: Extend test image creation SOP to require notification of delays ----------------------+----------------------------------------------------- Reporter: adamwill | Owner: rel-eng(a)lists.fedoraproject.org Type: task | Status: new Milestone: | Component: koji Keywords: | ----------------------+----------------------------------------------------- Working through the F16 QA retrospective here: https://fedoraproject.org/wiki/Fedora_16_QA_Retrospective One of the notes is: "jlaska - Alpha TC1 - Missed the Alpha-TC1 milestone by 1 week, little to no communication regarding status. With only two weeks Alpha ISO test time, losing 50% of time almost certainly results in a slip. See rel-eng ticket#4844 . None of the Alpha rel-eng release tickets have been filed at this time. adamw - perhaps releng image compose SOP should require updating of ticket with progress info, especially when image compose / delivery is delayed?" So: this ticket proposes that the releng 'Composing test images' SOP - https://fedoraproject.org/wiki/Composing_test_images - should be extended to require whoever is creating the images to post a comment on the image request ticket if the compose is delayed, explaining the reasons for the delay (so that any other interested party can attempt to contribute to resolving them) and giving a best estimate of an ETA if possible (so other groups can adjust their scheduling as best they can). Thanks! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/4985> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#5836: Update gitolite to gitolite3
by Fedora Release Engineering 09 Mar '15

09 Mar '15

#5836: Update gitolite to gitolite3 ------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: git Keywords: | Blocked By: Blocking: | ------------------+------------------------ The package gitolite3 in EPEL should be used to update gitolite for dist- git. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5836> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

#5924: CPUID flags on buildvm-20.phx2.fedoraproject.org
by Fedora Release Engineering 09 Mar '15

09 Mar '15

#5924: CPUID flags on buildvm-20.phx2.fedoraproject.org -----------------------------+------------------------ Reporter: jakub | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ In the gcc-4.9.0-12.fc21 build logs I saw a weird error: /builddir/build/BUILD/gcc-4.9.0-20140619/obj-x86_64-redhat-linux/gcc/xgcc -B/builddir/build/BUILD/gcc-4.9.0-20140619/obj-x86_64-redhat-linux/gcc/ /builddir/build/BUILD/gcc-4.9.0-20140619/gcc/testsuite/gcc.target/i386/pr57275.c -fno-diagnostics-show-caret -fdiagnostics-color=never -O3 -march=native -lm -o ./pr57275.exe /builddir/build/BUILD/gcc-4.9.0-20140619/gcc/testsuite/gcc.target/i386/pr57275.c:1:0: error: CPU you selected does not support x86-64 instruction set and I'd like to debug that, but don't know of a way to force e.g. a scratch rpm build to a particular build host. Would it be possible for some release engineer to run gcc -v -march=native -S -xc /dev/null -o /tmp/foo.s cat /proc/cpuinfo in the F21 mock chroot on that box? Perhaps we are missing something in the -m{arch,tune}=native logic, a CPU which clearly does support x86_64 64-bit ISA should not be determined as one that does not. Thanks. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5924> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5959: Enable keep-alive connections for koji (primary and secondaries)
by Fedora Release Engineering 02 Mar '15

02 Mar '15

#5959: Enable keep-alive connections for koji (primary and secondaries) -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ Keeping connections alive will speed up koji acces when multiCall() is not used without a significant speed penalty. Therefore it is a good idea to support this. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5959> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

Lookaside: Move away from md5
by Mathieu Bridon 24 Feb '15

24 Feb '15

We should move to something more secure than md5 for the uploaded sources. This patch series implements the client-side part of this change. We might want to drop the md5 fallback once we have migrated completely, that is when: 1. all archives on the lookaside have been moved to a stronger hash 2. the "sources" file in all git repos has been updated to the same hash https://fedorahosted.org/rel-eng/ticket/5846

5 10

#5846: move away from md5 for look-aside cache
by Fedora Release Engineering 23 Feb '15

23 Feb '15

#5846: move away from md5 for look-aside cache ------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: other Keywords: | Blocked By: Blocking: | ------------------+------------------------ The lookaside cache uses md5, but something more secure like sha-256 or sha-512 should be used instead. Maybe it should even be made to allow easy changes in the future. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5846> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 22

#5992: Create new fedora-ca certificates without SHA1
by Fedora Release Engineering 23 Feb '15

23 Feb '15

#5992: Create new fedora-ca certificates without SHA1 -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Currently .fedora-server-ca.cert and .fedora-upload-ca.cert use SHA1 for their self-signatures. Since SHA1 support is phased out, it might be a good idea to re-create the signatures. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5992> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#6023: allow Peter Robinson to restart sigul bridges
by Fedora Release Engineering 09 Feb '15

09 Feb '15

#6023: allow Peter Robinson to restart sigul bridges -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ The sigul bridges are very unstable, but currently nobody from Europe can restart it, therefore it might be unusable for a long time until Dennis Gilmore or Kevin Fenzi find the time to restart it. Especially to get the autosigner run smoothly it would be better to have at least one more being able to restart the bridges. Peter Robinson would be a good candidate. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6023> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 5

#5654: Report full image creation for quality website updates
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#5654: Report full image creation for quality website updates -----------------------------+------------------------ Reporter: shaiton | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 19 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, we already discussed a bit about this issues with nirik, before the F19 release day. However, yesterday was worse as expected. Please see the number of fixes: https://git.fedorahosted.org/cgit/fedora-web.git/log/ (7 important fixes after the merge into prod aedf487eb). == Summary of major issues: * http://torrent.fedoraproject.org/spins/ changed to http://torrent.fedoraproject.org/torrents and we had no idea about this. * The naming scheme of the file changed which introduced a bug in our spins website build as it is based against the Json file. We had to fix a 2 years old python code. And we did not know about this. * Each release we have the 32 bit arch named i686 or i386, it's always changing. We defined variables in order to help us maintain the websites.. And this does not help us. I know that it depends of the proc instruction set. But where is the need to change that really? Could we avoid it? Sooner or later we will drop 32 bit arch... Couldn't we define which arch to stick with? * We don't know before last minute if the Spin has built for GA and therefore if it is going to be released. We need to check it manually. And really, we can't do it all manually. (as lazy programmers we can't even think about this). * Even the spins name has changed in the past, which break our code and already existing URLs. And then we need to define URL redirect... I hope jam-kde won't be changed to jam-mate-compiz-fusion-dark at some point.. Just wondering... * The secondary (ARM) path changed from Images/arm/ to Images/armhfp/.. I understand the need to tell if it's using FP or not, but again we didn't know about this before testing in prod. That should not happen again. Please, help me define the best way to avoid this. It could be improving SOP, or updating a file after each build.. whatever. It's a probably wider collaboration issues as it is involving primary, secondary, SIGs (spins, cloud).. But starting with Releng we can probably sort this and define the smoother solution for all. What we need in a simple way (script friendly) and easy to generate for you is a way to get: All image full name, path (if possible before release), size (not needed for torrents), format (torrent, spin, dvd, cloud... whatever) and checksum. What can't be available easily can have an easy process to get them or at least we need to know how and when to get them. The most important of course is the image full name. If we don't know that this image exists (or died), we won't be able to update it. Any brillant idea? -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5654> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 9

#5978: Please create a f21-gnome side tag
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#5978: Please create a f21-gnome side tag -----------------------------+------------------------ Reporter: kalev | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, Please create a new f21-gnome side tag and build target. We'll be building GNOME 3.13.91 early next week and this would help ensure it doesn't cause instabilities in F21 proper while we're preparing it. Thanks, Kalev -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5978> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 8

#5922: skip 32-bit arches for fedora-cloud-atomic.ks and fedora-cloud-bigdata.ks
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#5922: skip 32-bit arches for fedora-cloud-atomic.ks and fedora-cloud-bigdata.ks -----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ The Cloud SIG decided that the Atomic and Big Data images will be 64-bit only. (Docker currently only works in 64 bits, and while big data tools might work, the target audience is almost always on 64 bits). -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5922> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

#5859: Releng improvements ideas 2014
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#5859: Releng improvements ideas 2014 -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: other Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ Ideas to improve releng: - Properly announce in advance if meetings are skipped -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5859> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 5

#5891: Deliverables and release engineering changes for Fedora.Next/Fedora 21
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#5891: Deliverables and release engineering changes for Fedora.Next/Fedora 21 -----------------------------+------------------------ Reporter: jreznik | Owner: rel-eng@… Type: enhancement | Status: new Milestone: Fedora 21 Alpha | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Fedora Working Groups are discussing different options how to deliver specific Fedora Products. These deliverables has to be collected (and agreed on) to plan changes in Fedora release engineering process. This ticket should serve to collect requirements from WGs in a centralized way (as there's possible overlap etc). For specific implementation details, new tickets will be created. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5891> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 7

#5944: drop "bigdata" from build-cloud-images for f21 -- feature was postponed
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#5944: drop "bigdata" from build-cloud-images for f21 -- feature was postponed -----------------------------+------------------------ Reporter: mattdm | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Big Data cloud image isn't going to make the milestone, so we should stop trying to make the image. Still hoping to do this for F22, so if we can keep the rawhide one, awesome. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5944> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

#6004: Trigger new Atomic builds based on bash vulnerability
by Fedora Release Engineering 19 Jan '15

19 Jan '15

#6004: Trigger new Atomic builds based on bash vulnerability -----------------------------+------------------------ Reporter: jzb | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Alpha | Component: koji Keywords: security | Blocked By: Blocking: | -----------------------------+------------------------ We have packages out for the bash vulnerability for Fedora 21 alpha, but users cannot "yum update" the Atomic images. We need to spin + push out new images and/or an update tree for users who are on Atomic. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6004> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

#5988: F21 stable push requests
by Fedora Release Engineering 05 Dec '14

05 Dec '14

#5988: F21 stable push requests -----------------------------+------------------------ Reporter: adamwill | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ This ticket will be used for requesting pushes of tested blocker / freeze exception fixes during F21 release freezes. See https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process and https://fedoraproject.org/wiki/QA:SOP_freeze_exception_bug_process. Similar to TC/RC request tickets, this ticket can be closed by releng after each push and re-opened by QA each time a new push is needed. First request, current Alpha blocker / FE fixes with sufficient karma to be pushed: * [https://admin.fedoraproject.org/updates/FEDORA-2014-10184 selinux- policy-3.13.1-78.fc21] for [https://bugzilla.redhat.com/show_bug.cgi?id=1112387 #1112387] * [https://admin.fedoraproject.org/updates/FEDORA-2014-10210 GNOME 3.13.91 mega-update] for [https://bugzilla.redhat.com/show_bug.cgi?id=1135890 #1135890], [https://bugzilla.redhat.com/show_bug.cgi?id=1136959 #1136959] and [https://bugzilla.redhat.com/show_bug.cgi?id=1136953 #1136953] * [https://admin.fedoraproject.org/updates/xorg-x11-drv- intel-2.99.914-3.fc21] for [https://bugzilla.redhat.com/show_bug.cgi?id=1133142 #1133142] -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5988> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 37

[PATCH] Remove requirement on system-release-product
by Stephen Gallagher 17 Nov '14

17 Nov '14

This was added to be overcautious (to ensure that every installation of Fedora has at least one of the fedora-release-* packages installed). However, this was both redundant (all install media includes at least one) and problematic. Since some of the fedora-release-$PRODUCT packages have additional explicit deps, it was causing a circular dependency loop in certain situations. See https://bugzilla.redhat.com/show_bug.cgi?id=1156198 for additional details.

2 1

#6009: Curious about python-sqlalchemy on epel7
by Fedora Release Engineering 11 Nov '14

11 Nov '14

#6009: Curious about python-sqlalchemy on epel7 --------------------+------------------------ Reporter: pingou | Owner: rel-eng@… Type: defect | Status: new Milestone: | Component: epel Keywords: | Blocked By: Blocking: | --------------------+------------------------ According to datagrepper python-sqlalchemy was branched for epel7 9 months ago: https://apps.fedoraproject.org/datagrepper/id?id=2013-b8045ad9-1985-416a- 92ff-3c6630f17642&is_raw=true&size=extra-large Since then, I can find ACL approved/dropped but no orphan nor retirement. Same thing in pkgdb2's logs: https://admin.fedoraproject.org/pkgdb/admin/log/?package=python-sqlalchemy Nothing special in git: http://pkgs.fedoraproject.org/cgit/python- sqlalchemy.git/log/?h=epel7 So, what happened with this branch? How has it end-up being blocked in koji and marked as retired in pkgdb w/o pkgdb or datagrepper knowing about it. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6009> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#5679: merge buildbranched/buildrawhide into a single script
by Fedora Release Engineering 10 Nov '14

10 Nov '14

#5679: merge buildbranched/buildrawhide into a single script -----------------------------+------------------------ Reporter: ausil | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Alpha | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ in order to simplify things it would be good to make a single script to build rawhide and branched. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5679> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

#6011: Request for override tag for tomcatjss-7.1.1-1.fc20 and fc21 in tomcatjss FC20 and FC21 branches
by Fedora Release Engineering 10 Nov '14

10 Nov '14

#6011: Request for override tag for tomcatjss-7.1.1-1.fc20 and fc21 in tomcatjss FC20 and FC21 branches -----------------------------+------------------------ Reporter: cfu | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ we'd like to build tomcatjss-7.1.1-1.fc20 and fc21 which both have a dependency on jss-4.2.6-35 which is now waiting for karma. We'd like to request for a permanent override flag. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6011> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 6

#6024: Changes to the process of unretire a dead package
by Fedora Release Engineering 10 Nov '14

10 Nov '14

#6024: Changes to the process of unretire a dead package -----------------------------+------------------------ Reporter: hannes | Owner: rel-eng@… Type: enhancement | Status: new Milestone: Fedora 20 Final | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi all, at the moment, when a new package was re-reviewed and the git request was processed the original maintainer, who perhaps initially imported the package has still commit access to various branches, most importantly to the devel branch. In this particular case the original maintainer stopped working on the package in 2010, at least that was when he last initiated a build on koji. [1] I then took it over until it was retired by me, because upstream was dead and it wasn't really necessary any more. It was then again picked up by a different upstream, who contacted me and asked me to introduce it to fedora again. I did so and filed a review ticket and did all the necessary steps. [2] And then after it was approved and the git process was done, I noticed in pkgdb, that the original maintainer was still listed there. [3] I mean of course this is not a big problem, but if he really would be interested to me it would seem more natural, that he then requests co- maintainership and don't gets it automatically. -johannes [1] http://koji.fedoraproject.org/koji/packageinfo?packageID=3984 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1153302 [3] https://admin.fedoraproject.org/pkgdb/package/tilda/ -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6024> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 10

#6014: multiple builds in primary koji are lackign arm buildroot info
by Fedora Release Engineering 10 Nov '14

10 Nov '14

#6014: multiple builds in primary koji are lackign arm buildroot info -----------------------------+------------------------ Reporter: sharkcz | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ There are multiple builds in primary koji run during 10/02 that are lacking the content for arm buildroots breaking builds in secondary kojis http://koji.fedoraproject.org/koji/buildrootinfo?buildrootID=2444699 (ice-3.5.1-13.fc21) http://koji.fedoraproject.org/koji/buildrootinfo?buildrootID=2443844 (java-1.8.0-openjdk-1.8.0.20-12.b26.fc21) http://koji.fedoraproject.org/koji/buildrootinfo?buildrootID=2443370 (tracker-0.16.4-4.fc20) and more -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6014> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 7

#6025: Unblock xpa package in current release
by Fedora Release Engineering 07 Nov '14

07 Nov '14

#6025: Unblock xpa package in current release -----------------------------+------------------------ Reporter: sergiopr | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ Hi, this ticket is to request the unblocking of the package xpa in the current release. xpa was retired a while ago, This is the re-review ticket: https://bugzilla.redhat.com/show_bug.cgi?id=1157794 -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6025> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

#6010: Create Fedora 21 Beta test compose (TC) and release candidate (RC)
by Fedora Release Engineering 04 Nov '14

04 Nov '14

#6010: Create Fedora 21 Beta test compose (TC) and release candidate (RC) ----------------------------+------------------------ Reporter: adamwill | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Beta | Component: koji Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ Time flies, so it's time for Beta... Please build a Fedora 21 Beta Test Compose. I don't believe there are any special build requests at this point in time, let's just fire a TC1 and see where we stand. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6010> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 23

#6015: Remove phpMyAdmin3 packages from EPEL-5
by Fedora Release Engineering 01 Nov '14

01 Nov '14

#6015: Remove phpMyAdmin3 packages from EPEL-5 --------------------+------------------------ Reporter: robert | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: epel Keywords: | Blocked By: Blocking: | --------------------+------------------------ I retired the el5 branch of phpMyAdmin3 as it is superseeded by phpMyAdmin4 - however the phpMyAdmin3 packages should be removed from the EPEL repositories if possible (unmaintained, outdated and contain various known security vulnerabilities). A standard "yum update" should upgrade existing phpMyAdmin3 installations to phpMyAdmin4. Thanks, Robert -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6015> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

[PATCH] Don't suppress error messages
by Mathieu Bridon 31 Oct '14

31 Oct '14

If a failure occured when trying to create the cache folder, mash would just silently ignore the error and happily continue cranking along. It might of course fail later on, but the problem is impossible to debug if errors are silently suppressed. However, if the error is merely that the directory already exists (as opposed to a permission issue, for example), then that is fine to ignore. Likewise, if a failure occurs when trying to download, merely saying "can't download" is not helpful. Adding the actual error message makes it easier to figure out what went wrong. --- mash/__init__.py | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/mash/__init__.py b/mash/__init__.py index 566b8c1..27cdc47 100644 --- a/mash/__init__.py +++ b/mash/__init__.py @@ -12,6 +12,7 @@ # with this program; if not, write to the Free Software Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +import errno import glob import os import logging @@ -269,8 +270,11 @@ class Mash: path = os.path.join(koji.pathinfo.build(builds_hash[pkg['build_id']]), koji.pathinfo.rpm(pkg)) try: os.mkdir(os.path.dirname(cachepath)) - except: - pass + except Exception as e: + if e.errno != errno.EEXIST: + # If the directory already exists, that's fine + self.logger.error(e) + try: result = urlgrabber.grabber.urlgrab(path, cachepath) except: @@ -279,8 +283,9 @@ class Mash: path = os.path.join(koji.pathinfo.build(builds_hash[pkg['build_id']]), koji.pathinfo.rpm(pkg)) try: result = urlgrabber.grabber.urlgrab(path, cachepath) - except: - self.logger.error("ERROR: can't download %s from %s" % (nevra(pkg), path)) + except Exception as e: + self.logger.error("ERROR: can't download %s from %s: %s" + % (nevra(pkg), path, e)) return None fd = open(result) -- 2.1.0

4 8

[PATCHES] Sprinting on rpkg during PyCon-Fr
by Mathieu Bridon 31 Oct '14

31 Oct '14

>From Saturday to Tuesday was PyCon-Fr, here in Lyon, so I used the time to hack a bit on rpkg. The result is a pretty big changeset: .gitignore | 1 + setup.cfg | 5 + setup.py | 38 ++----- src/pyrpkg/__init__.py | 357 +++++++++++++++++++++++++++++++------------------------------- src/pyrpkg/cli.py | 976 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------------------------------------------- src/rpkg_man_page.py | 11 +- test/commands/__init__.py | 110 ++++++++++++++++++++ test/commands/test_add_tag.py | 156 ++++++++++++++++++++++++++++ test/commands/test_clone.py | 88 ++++++++++++++++ test/commands/test_delete_tag.py | 51 +++++++++ test/commands/test_list_tag.py | 159 ++++++++++++++++++++++++++++ test/test_gitgnore.py | 83 +++++++++++++++ 12 files changed, 1274 insertions(+), 761 deletions(-) Now, out of these 1274 insertions, 647 are new unit tests. So if you remove that from the count, the overall diff stat is rather: 6 files changed, 627 insertions(+), 761 deletions(-) That's right, keeping the same features, fixing a couple of bugs, and making us almost entirely Flake8-compliant, I'm still removing more 100 lines of code. Not bad. :) Speaking of Flake8 compliance, the remaining problems are all stuff I'd rather not touch right now, in the middle of what are mostly trivial changes. We'll get there at some point, though. To make review easier, here's an outline of what the patches do. * [PATCH 01/20] Some more PEP8 * [PATCH 02/20] Remove unused import Just some trivial changes to the setup.py script, to make the next commit easier to review. * [PATCH 03/20] tests: Use nose This makes use of the awesome 'nose' testing library, removing some home-grown code, and adding a pretty nice feature: a report of the code coverage of the unit tests. * [PATCH 04/20] tests: Ensure proper functioning of GitIgnore * [PATCH 05/20] gitignore: We're not modified any more after we wrote * [PATCH 06/20] gitignore: Make sure each line ends with a \n These add some unit tests for our handling of .gitignore files, which actually found 2 bugs in the code. Hurray for unit tests! * [PATCH 07/20] tests: Ensure functioning of Commands.clone * [PATCH 08/20] tests: Factor out some code This sets up the necessary conditions for easily testing all the interactions with Git repositories, and actually tests the clone command. * [PATCH 09/20] tests: Ensure proper functioning of Commands.add_tag * [PATCH 10/20] add_tag: Run the tag command in the right directory These add some unit tests for the add_tag command, which actually found a bug in the code. One more victory for unit tests! * [PATCH 11/20] tests: Ensure functioning of Commands.delete_tag * [PATCH 12/20] delete_tag: Stop executing a command This adds some unit tests for the delete_tag command. And once it is tested, it is easy to move away from subprocessing the git cli, which makes the code nicer and avoids the overhead of forking child processes. Plus, pyrpkg is a library, and it's generally a terrible idea to subprocess command line tools inside a library. * [PATCH 13/20] list_tags: Fix the docstring * [PATCH 14/20] list_tags: Stop executing a command * [PATCH 15/20] tests: Ensure functioning of Commands.list_tag These greatly improve the list_tag command. The second patch is in fact necessary to make the method testable. * [PATCH 16/20] Fix typo * [PATCH 17/20] Simplify some code Just two trivial changes I came up with while idly reading the source code. * [PATCH 18/20] Fix some flake8 issues * [PATCH 19/20] Fix some more Flake8 issues * [PATCH 20/20] Massive Flake8 fix These are pretty massive patches, but they are worth it: they make the code easier to read and "get into" for newcomers by following standard Python conventions, removing unused variables and imports, etc... All in all, they even remove lines of code. The diffstat for just these 3 changes is: 3 files changed, 577 insertions(+), 709 deletions(-)

2 21

Re: firewall rules on builders (iptables, firewalld, libvirt...)
by Paul W. Frields 31 Oct '14

31 Oct '14

On Tue, Oct 28, 2014 at 08:50:29AM -0600, Stephen John Smoogen wrote: > On 28 October 2014 08:04, Matthew Miller <mattdm(a)fedoraproject.org> wrote: > > > It's my understanding (Dennis please correct if I'm wrong) that the > > problem with cloud image creation was due to libvirt iptables rules > > being lost when iptables was restarted. This is a fundamental known > > issue (see last paragraph of <http://libvirt.org/firewall.html>), and > > one of the things firewalld was meant to solve. > > > > Dennis says that there are lot of complicated rules on the builders > > making switching to firewalld difficult. One possibility might be to > > move those complicated rules from the builders to a network firewall, > > and keep the host rules simple and functional. But that's probably a > > big undertaking. > > > > > It would be.. It would be creating a new network for these boxes, putting > the hardware behind such a firewall, setting up routing for such devices > etc etc. [Plus a budget needed for that hardware.] > > > > In the meantime, any time iptables is restarted or reloaded, libvirt > > needs a SIGHUP. (I suppose this means: ansible playbooks and also added > > to any manual procedures.) > > > > That actually would be 'easier' to set up even if it is a cron job which > checks to see if a marker is in iptables and if not sends a sighup to > libvirt The firewalld rich language is probably also worth looking into -- if for no other reason than to determine whether it is capable of handling these use cases. If not, we should file RFEs upstream because we I'm betting we're not *that* special. :-) -- Paul W. Frields http://paul.frields.org/ gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717 http://redhat.com/ - - - - http://pfrields.fedorapeople.org/ The open source story continues to grow: http://opensource.com

2 1

[patch] Choose URL in build-cloud-images based on new command line argument
by Kushal Das 31 Oct '14

31 Oct '14

Hi, The attached patch adds a new command line argument to the build-cloud-images script, if that value is mash then it will use a different URL else it will use standard URL to build cloud images. I go the mash URL from the git history as suggested by Dennis. Note: The patch is not tested. Kushal -- Fedora Cloud Engineer CPython Core Developer http://kushaldas.in

3 3

[releng] remove unneeded header infor for licensing, the SPDX header defines the license entirely
by Dennis Gilmore 30 Oct '14

30 Oct '14

commit cf18b3ebaa74c1c40af31b8e71112a692f82cb50 Author: Dennis Gilmore <dennis(a)ausil.us> Date: Thu Oct 30 22:13:48 2014 -0500 remove unneeded header infor for licensing, the SPDX header defines the license entirely scripts/autosigner.py | 14 -------------- scripts/block_retired.py | 14 -------------- scripts/check_sigul.py | 14 -------------- scripts/fedretire | 14 -------------- 4 files changed, 0 insertions(+), 56 deletions(-) --- diff --git a/scripts/autosigner.py b/scripts/autosigner.py index d2cc120..7752f13 100755 --- a/scripts/autosigner.py +++ b/scripts/autosigner.py @@ -1,20 +1,6 @@ #!/usr/bin/python -tt # vim: fileencoding=utf8 foldmethod=marker # SPDX-License-Identifier: GPL-2.0+ -#{{{ License header: GPLv2+ -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see <https://www.gnu.org/licenses/>. -#}}} import argparse import datetime diff --git a/scripts/block_retired.py b/scripts/block_retired.py index 7685719..690ea7a 100755 --- a/scripts/block_retired.py +++ b/scripts/block_retired.py @@ -1,20 +1,6 @@ #!/usr/bin/python -tt # vim: fileencoding=utf8 foldmethod=marker # SPDX-License-Identifier: GPL-2.0+ -# {{{ License header: GPLv2+ -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see <https://www.gnu.org/licenses/>. -# }}} import argparse import datetime diff --git a/scripts/check_sigul.py b/scripts/check_sigul.py index 63bbf2b..47568dd 100755 --- a/scripts/check_sigul.py +++ b/scripts/check_sigul.py @@ -1,20 +1,6 @@ #!/usr/bin/python -tt # vim: fileencoding=utf8 # SPDX-License-Identifier: GPL-2.0+ -# {{{ License header: GPLv2+ -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see <https://www.gnu.org/licenses/>. -# }}} import getpass import logging diff --git a/scripts/fedretire b/scripts/fedretire index c277f77..243c1f5 100755 --- a/scripts/fedretire +++ b/scripts/fedretire @@ -1,20 +1,6 @@ #!/usr/bin/python -tt # vim: fileencoding=utf8 # SPDX-License-Identifier: GPL-2.0+ -# {{{ License header: GPLv2+ -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see <https://www.gnu.org/licenses/>. -# }}} import argparse import os

1 0

[releng] Updated tag rawhide-stable
by Dennis Gilmore 30 Oct '14

30 Oct '14

The lightweight tag 'rawhide-stable' was updated to point to: 46a1fd5... adjust the url based on if its a nightly compose or TC/RC o It previously pointed to: 8703091... adjust the url based on if its a nightly compose or TC/RC o NOTE: People pulling from the repository will not get the new tag. For more information, please see: http://live.gnome.org/Git/Help/TagUpdates

1 0

[releng] adjust the url based on if its a nightly compose or TC/RC one
by Dennis Gilmore 30 Oct '14

30 Oct '14

commit 46a1fd5667b93bd936dd28e591f14578c8e213cd Author: Dennis Gilmore <dennis(a)ausil.us> Date: Thu Oct 30 02:17:41 2014 -0500 adjust the url based on if its a nightly compose or TC/RC one scripts/build-cloud-images | 9 ++++++--- scripts/buildbranched | 2 +- scripts/buildrawhide | 2 +- 3 files changed, 8 insertions(+), 5 deletions(-) --- diff --git a/scripts/build-cloud-images b/scripts/build-cloud-images index 01b6bc5..1a9582b 100755 --- a/scripts/build-cloud-images +++ b/scripts/build-cloud-images @@ -22,11 +22,16 @@ TARGET=f$VERSION-candidate BRANCH=branched fi +if [ "$COMPOSE" == "nightly" ]; then + url=http://kojipkgs.fedoraproject.org/mash/$BRANCH-$BUILD/$VERSION/\$arch/o… +else + url=http://compose-x86-02.phx2.fedoraproject.org/compose/$VERSION$COMPOSE/$… +fi + for spin in Base do declare -l lspin lspin=$spin - url=http://compose-x86-02.phx2.fedoraproject.org/compose/$VERSION$COMPOSE/$… kickstart=fedora-cloud-$lspin-$GITHASH.ks ksflatten -c fedora-cloud-$lspin.ks -o $kickstart >& /dev/null echo "url --url=$url"|sed -e 's|$arch|$basearch|g' >> $kickstart @@ -38,7 +43,6 @@ for spin in Atomic do declare -l lspin lspin=$spin - url=http://compose-x86-02.phx2.fedoraproject.org/compose/$VERSION$COMPOSE/$… kickstart=fedora-cloud-$lspin-$GITHASH.ks ksflatten -c fedora-cloud-$lspin.ks -o $kickstart >& /dev/null echo "url --url=$url"|sed -e 's|$arch|$basearch|g' >> $kickstart @@ -50,7 +54,6 @@ for spin in Base do declare -l lspin lspin=$spin - url=http://compose-x86-02.phx2.fedoraproject.org/compose/$VERSION$COMPOSE/$… kickstart=fedora-docker-$lspin-$GITHASH.ks ksflatten -c fedora-docker-$lspin.ks -o $kickstart >& /dev/null echo "url --url=$url"|sed -e 's|$arch|$basearch|g' >> $kickstart diff --git a/scripts/buildbranched b/scripts/buildbranched index fd6ef49..d34b3e6 100755 --- a/scripts/buildbranched +++ b/scripts/buildbranched @@ -214,7 +214,7 @@ log "finished checking out spin-kickstarts" log "started building live/arm/cloud images" ../releng/scripts/build-livecds $BRANCHED $DATE $BRANCHED ../releng/scripts/build-arm-images $BRANCHED $DATE $BRANCHED -../releng/scripts/build-cloud-images $BRANCHED $DATE $BRANCHED +../releng/scripts/build-cloud-images $BRANCHED $DATE $BRANCHED nightly log "finished starting building live/arm/cloud images" popd popd diff --git a/scripts/buildrawhide b/scripts/buildrawhide index 31c789f..0ab6ad3 100755 --- a/scripts/buildrawhide +++ b/scripts/buildrawhide @@ -206,7 +206,7 @@ log "finished checking out spin-kickstarts" log "started building live/arm/cloud images" ../releng/scripts/build-livecds $BRANCHED $DATE $BRANCHED ../releng/scripts/build-arm-images $BRANCHED $DATE $BRANCHED -../releng/scripts/build-cloud-images $BRANCHED $DATE $BRANCHED +../releng/scripts/build-cloud-images $BRANCHED $DATE $BRANCHED nightly log "finished starting building live/arm/cloud images" popd popd

1 0

[releng] Updated tag rawhide-stable
by Dennis Gilmore 30 Oct '14

30 Oct '14

The lightweight tag 'rawhide-stable' was updated to point to: 8703091... adjust the url based on if its a nightly compose or TC/RC o It previously pointed to: b3c72ca... add sigul check script NOTE: People pulling from the repository will not get the new tag. For more information, please see: http://live.gnome.org/Git/Help/TagUpdates

1 0

[releng] check_sigul: Add debug info, fix it
by Till Maas 29 Oct '14

29 Oct '14

commit 4b4b26aab2f2fa39c877a39f5f8f26b76fd66457 Author: Till Maas <opensource(a)till.name> Date: Wed Oct 29 20:45:03 2014 +0100 check_sigul: Add debug info, fix it scripts/check_sigul.py | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) --- diff --git a/scripts/check_sigul.py b/scripts/check_sigul.py index 73c3dd3..63bbf2b 100755 --- a/scripts/check_sigul.py +++ b/scripts/check_sigul.py @@ -44,7 +44,7 @@ if __name__ == "__main__": mail_logger = SubjectSMTPHandler( "127.0.0.1", fedora_user, [fedora_user], "Sigul check log event") mail_logger.subject_prefix = "Sigul check: " - mail_logger.setLevel(logging.DEBUG) + mail_logger.setLevel(logging.WARNING) mail_logger.setFormatter(formatter) log.addHandler(mail_logger) @@ -68,7 +68,13 @@ if __name__ == "__main__": for key, helper in helpers.items(): res = helper.get_public_key() ret, pubkey, errors = res - if status.setdefault(key, res) != res: + if ret != 0 or errors: + log.debug("Key '{}' not working {}:{}".format( + key, ret, errors)) + else: + log.debug("Key '{}' working".format(key)) + + if status.get(key, res) != res: if ret != 0 or errors: log.error( "Sigul for key '{}' stopped working: {}:{}".format( @@ -77,4 +83,6 @@ if __name__ == "__main__": log.warning( "Sigul for key '{}' resumed working".format(key)) + status[key] = res + time.sleep(600)

1 0

[PATCH] Replace bare 'mock' with the full path.
by Ralph Bean 29 Oct '14

29 Oct '14

I hit issues on composer.stg where the $PATH was configured to have /usr/sbin/mock precede /usr/bin/mock. This patch makes it explicit. --- scripts/buildbranched | 25 +++++++++++++------------ scripts/buildrawhide | 19 ++++++++++--------- 2 files changed, 23 insertions(+), 21 deletions(-) diff --git a/scripts/buildbranched b/scripts/buildbranched index 450bf43..a5bfd00 100755 --- a/scripts/buildbranched +++ b/scripts/buildbranched @@ -20,6 +20,7 @@ MASHDIR="/mnt/koji/mash/${DIST}-$DATE" EXPANDARCH="" DEPOPTS="--treename F-$BRANCHED" MASHOPTS="" +MOCK="/usr/bin/mock" MOCKCONFIG="fedora-${DIST}-compose-i386" TOMAIL="devel(a)lists.fedoraproject.org test(a)lists.fedoraproject.org" SUBJECT='F-'$BRANCHED' Branched report: '$DATE' changes' @@ -109,43 +110,43 @@ log "git clone of comps finished" [ -f "$logdir/${COMPSFILE}" ] || exit 1 log "mock init" -mock -r $MOCKCONFIG --uniqueext=$DATE --init +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --init log "mock install base packages" -mock -r $MOCKCONFIG --uniqueext=$DATE --no-clean --install koji yum createrepo cvs make intltool findutils mash yum-utils rsync repoview hardlink +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --no-clean --install koji yum createrepo cvs make intltool findutils mash yum-utils rsync repoview hardlink # until we move to bodhi lets not be strict about the gpg keys -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "sed -i -e 's|strict_keys = True|strict_keys = False|g' /etc/mash/${DIST}.mash" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "sed -i -e 's|strict_keys = True|strict_keys = False|g' /etc/mash/${DIST}.mash" #disable delta close to release as we do not want them in the final trees -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "sed -i -e 's|delta = True|delta = False|g' /etc/mash/${DIST}.mash" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "sed -i -e 's|delta = True|delta = False|g' /etc/mash/${DIST}.mash" # secondary arches are a bit harder to make sure everything is signed lets not be too strict, but actual release compsoes need to be. [ -n "$ARCH" ] && { -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "sed -i -e 's|strict_keys = True|strict_keys = False|g' /etc/mash/${DIST}.$ARCH.mash" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "sed -i -e 's|strict_keys = True|strict_keys = False|g' /etc/mash/${DIST}.$ARCH.mash" } # Copy in the hosts file so that we get the right address for koji log "mock setup /etc/hosts" -mock -r $MOCKCONFIG --uniqueext=$DATE --copyin /etc/hosts /etc/hosts >/dev/null 2>&1 # this reports to fail, but actually works +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --copyin /etc/hosts /etc/hosts >/dev/null 2>&1 # this reports to fail, but actually works send_fedmsg start mash.start log "starting mash" # Drop privs here so that we run as the masher UID -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 send_fedmsg done mash.complete log "finished mash" log "starting hardlink" # hardlink the noarch deltarpms between x86_64 and i386 -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "hardlink -v -c ${MASHDIR}/$BRANCHED$EXPANDARCH" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "hardlink -v -c ${MASHDIR}/$BRANCHED$EXPANDARCH" log "finished hardlink" log "starting repodiff" -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "rm -f /var/lib/rpm/__db*" -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/bin/repodiff -s -q --new=file://${MASHDIR}/$BRANCHED$EXPANDARCH/source/SRPMS --old=file://$TREEPREFIX/development/$BRANCHED/source/SRPMS > $logdir/repodiff" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "rm -f /var/lib/rpm/__db*" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/bin/repodiff -s -q --new=file://${MASHDIR}/$BRANCHED$EXPANDARCH/source/SRPMS --old=file://$TREEPREFIX/development/$BRANCHED/source/SRPMS > $logdir/repodiff" log "finished repodiff" log "starting spam-o-matic" -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/share/mash/spam-o-matic $DEPOPTS ${MASHDIR}/$BRANCHED$EXPANDARCH >$logdir/depcheck" & +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/share/mash/spam-o-matic $DEPOPTS ${MASHDIR}/$BRANCHED$EXPANDARCH >$logdir/depcheck" & log "finished spam-o-matic" send_fedmsg start pungify.start @@ -173,7 +174,7 @@ echo "Running build_composeinfo" log "finished build_composeinfo" log "starting mock clean" -mock -r $MOCKCONFIG --uniqueext=$DATE --clean +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --clean log "finished mock clean" [ -n "$NOSYNC" ] && exit $rc diff --git a/scripts/buildrawhide b/scripts/buildrawhide index 96cb74a..d547a6e 100755 --- a/scripts/buildrawhide +++ b/scripts/buildrawhide @@ -20,6 +20,7 @@ MASHDIR="/mnt/koji/mash/${DIST}-$DATE" EXPANDARCH="" DEPOPTS="" MASHOPTS="" +MOCK="/usr/bin/mock" MOCKCONFIG="fedora-${DIST}-compose-i386" TOMAIL="devel(a)lists.fedoraproject.org test(a)lists.fedoraproject.org" SUBJECT='rawhide report: '$DATE' changes' @@ -109,34 +110,34 @@ log "git clone of comps finished" [ -f "$logdir/${COMPSFILE}" ] || exit 1 log "mock init" -mock -r $MOCKCONFIG --uniqueext=$DATE --init +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --init log "mock install base packages" -mock -r $MOCKCONFIG --uniqueext=$DATE --no-clean --install koji yum createrepo cvs make intltool findutils mash yum-utils rsync repoview hardlink +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --no-clean --install koji yum createrepo cvs make intltool findutils mash yum-utils rsync repoview hardlink # Copy in the hosts file so that we get the right address for koji log "mock setup /etc/hosts" -mock -r $MOCKCONFIG --uniqueext=$DATE --copyin /etc/hosts /etc/hosts >/dev/null 2>&1 # this reports to fail, but actually works +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --copyin /etc/hosts /etc/hosts >/dev/null 2>&1 # this reports to fail, but actually works send_fedmsg start mash.start log "starting mash" # Drop privs here so that we run as the masher UID -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 send_fedmsg done mash.complete log "finished mash" log "starting hardlink" # hardlink the noarch deltarpms between x86_64 and i386 -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "hardlink -v -c ${MASHDIR}/$BRANCHED$EXPANDARCH/*/os/drpms/" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "hardlink -v -c ${MASHDIR}/$BRANCHED$EXPANDARCH/*/os/drpms/" log "finished hardlink" log "starting repodiff" -mock -r $MOCKCONFIG --uniqueext=$DATE --chroot "rm -f /var/lib/rpm/__db*" -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/bin/repodiff -s -q --new=file://${MASHDIR}/$BRANCHED$EXPANDARCH/source/SRPMS --old=file://$TREEPREFIX/development/$BRANCHED/source/SRPMS > $logdir/repodiff" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --chroot "rm -f /var/lib/rpm/__db*" +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/bin/repodiff -s -q --new=file://${MASHDIR}/$BRANCHED$EXPANDARCH/source/SRPMS --old=file://$TREEPREFIX/development/$BRANCHED/source/SRPMS > $logdir/repodiff" log "finished repodiff" log "starting spam-o-matic" -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/share/mash/spam-o-matic $DEPOPTS ${MASHDIR}/$BRANCHED$EXPANDARCH >$logdir/depcheck" & +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "/usr/share/mash/spam-o-matic $DEPOPTS ${MASHDIR}/$BRANCHED$EXPANDARCH >$logdir/depcheck" & log "finished spam-o-matic" send_fedmsg start pungify.start @@ -165,7 +166,7 @@ echo "Running build_composeinfo" wait log "finished build_composeinfo" log "starting mock clean" -mock -r $MOCKCONFIG --uniqueext=$DATE --clean +$MOCK -r $MOCKCONFIG --uniqueext=$DATE --clean log "finished mock clean" [ -n "$NOSYNC" ] && exit $rc -- 1.9.3

1 0

[PATCH] Make compose scripts accept an environment argument.
by Ralph Bean 29 Oct '14

29 Oct '14

We can use this argument as a general switch to toggle behaviors in the subprocesses. An example of how to use this with scripts/block_retired.py is present, but we'll need to make more changes like it before composes in staging are possible start-to-finish. --- scripts/buildbranched | 12 +++++++++++- scripts/buildrawhide | 12 +++++++++++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/scripts/buildbranched b/scripts/buildbranched index fd6ef49..450bf43 100755 --- a/scripts/buildbranched +++ b/scripts/buildbranched @@ -7,6 +7,7 @@ DATE=$1 ARCH=$2 +ENVIRONMENT=$3 BRANCHED="21" DIST=branched @@ -39,6 +40,10 @@ RSYNCPREFIX="" MOCKCONFIG="fedora-${DIST}-compose-ppc64" } + +[ -n "$ENVIRONMENT" ] && { +ENVIRONMENT='production'; +} RSYNC_OPTS="-rlptDHhv --delay-updates" DESTPATH="$TREEPREFIX/development/$BRANCHED/" @@ -83,7 +88,12 @@ log "started" send_fedmsg start start log "blocking retired packages" -./scripts/block_retired.py +if [ "$ENVIRONMENT" == "production" ]; then + ./scripts/block_retired.py +else + ./scripts/block_retired.py --staging +fi + log "git clone of comps started" pushd $TMPDIR diff --git a/scripts/buildrawhide b/scripts/buildrawhide index 31c789f..96cb74a 100755 --- a/scripts/buildrawhide +++ b/scripts/buildrawhide @@ -7,6 +7,7 @@ DATE=$1 ARCH=$2 +ENVIRONMENT=$3 BRANCHED=rawhide DIST=rawhide @@ -39,6 +40,10 @@ RSYNCPREFIX="" MOCKCONFIG="fedora-${DIST}-compose-ppc64" } + +[ -n "$ENVIRONMENT" ] && { +ENVIRONMENT='production'; +} RSYNC_OPTS="-rlptDHhv --delay-updates" DESTPATH="$TREEPREFIX/development/$BRANCHED/" @@ -83,7 +88,12 @@ log "started" send_fedmsg start start log "blocking retired packages" -./scripts/block_retired.py +if [ "$ENVIRONMENT" == "production" ]; then + ./scripts/block_retired.py +else + ./scripts/block_retired.py --staging +fi + log "git clone of comps started" pushd $TMPDIR -- 1.9.3

1 0

[PATCH] Update block_retired.py to work against staging.
by Ralph Bean 29 Oct '14

29 Oct '14

A --staging option is added that does 2 things: - Points the script at staging services like koji.stg.fp.o and admin.stg.fp.o/pkgdb. - Limits the branches on which packages get retired in staging by default. This is necessary only because, for instance, production koji seems to have el5 defined while staging koji does not. --- scripts/block_retired.py | 82 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 55 insertions(+), 27 deletions(-) diff --git a/scripts/block_retired.py b/scripts/block_retired.py index 2556980..2c87a0b 100755 --- a/scripts/block_retired.py +++ b/scripts/block_retired.py @@ -29,6 +29,13 @@ import pkgdb2client log = logging.getLogger(__name__) RETIRING_BRANCHES = ["el5", "el6", "epel7", "f21", "master"] +PROD_ONLY_BRANCHES = ["el5", "el6", "epel7", "master"] + +PRODUCTION_PKGDB = "https://admin.fedoraproject.org/pkgdb" +STAGING_PKGDB = "https://admin.stg.fedoraproject.org/pkgdb" + +PRODUCTION_KOJI = "https://koji.fedoraproject.org/kojihub" +STAGING_KOJI = "https://koji.stg.fedoraproject.org/kojihub" class ReleaseMapper(object): @@ -36,7 +43,7 @@ class ReleaseMapper(object): KOJI_TAG = 1 EPEL_BUILD_TAG = 2 - def __init__(self): + def __init__(self, staging=False): # git branchname, koji tag, pkgdb version self.mapping = ( @@ -45,10 +52,13 @@ class ReleaseMapper(object): ("f20", "f20", ""), ("f19", "f19", ""), ("f18", "f18", ""), - ("epel7", "epel7", "epel7-build"), - ("el6", "dist-6E-epel", "dist-6E-epel-build"), - ("el5", "dist-5E-epel", "dist-5E-epel-build"), ) + if not staging: + self.mapping = self.mapping + ( + ("epel7", "epel7", "epel7-build"), + ("el6", "dist-6E-epel", "dist-6E-epel-build"), + ("el5", "dist-5E-epel", "dist-5E-epel-build"), + ) def branchname(self, key=""): return self.lookup(key, self.BRANCHNAME) @@ -71,32 +81,42 @@ class ReleaseMapper(object): return None -def blocked_packages(branch="master"): - mapper = ReleaseMapper() +def blocked_packages(branch="master", staging=False): + mapper = ReleaseMapper(staging=staging) tag = mapper.koji_tag(branch) - kojisession = koji.ClientSession('https://koji.fedoraproject.org/kojihub') + url = PRODUCTION_KOJI if not staging else STAGING_KOJI + kojisession = koji.ClientSession(url) pkglist = kojisession.listPackages(tagID=tag, inherited=True) - blocked = [p["package_name"] for p in pkglist if p["blocked"]] + blocked = [p["package_name"] for p in pkglist if p.get("blocked")] return blocked -def get_retired_packages(branch="master"): - pkgdb = pkgdb2client.PkgDB() - retiredresponse = pkgdb.get_packages( - "", branches=branch, page="all", status="Retired") +def get_retired_packages(branch="master", staging=False): + url = PRODUCTION_PKGDB if not staging else STAGING_PKGDB + pkgdb = pkgdb2client.PkgDB(url) + + try: + retiredresponse = pkgdb.get_packages( + "", branches=branch, page="all", status="Retired") + except pkgdb2client.PkgDBException as e: + if not "No packages found for these parameters" in str(e): + raise + return [] + retiredinfo = retiredresponse["packages"] retiredpkgs = [p["name"] for p in retiredinfo] return retiredpkgs -def pkgdb_retirement_status(package, branch="master"): +def pkgdb_retirement_status(package, branch="master", staging=False): """ Returns retirement info for `package` in `branch` :returns: dict: retired: True - if retired, False if not, None if there was an error, status_change: last status change as datetime object """ - pkgdb = pkgdb2client.PkgDB() + url = PRODUCTION_PKGDB if not staging else STAGING_PKGDB + pkgdb = pkgdb2client.PkgDB(url) retired = None status_change = None try: @@ -141,14 +161,16 @@ def get_retirement_info(message): return None -def block_package(packages, branch="master"): +def block_package(packages, branch="master", staging=False): if isinstance(packages, basestring): packages = [packages] if len(packages) == 0: return None - mapper = ReleaseMapper() + url = PRODUCTION_KOJI if not staging else STAGING_KOJI + + mapper = ReleaseMapper(staging=staging) tag = mapper.koji_tag(branch) cmd = ["koji", "block-pkg", tag] + packages log.debug("Running: %s", " ".join(cmd)) @@ -157,16 +179,16 @@ def block_package(packages, branch="master"): epel_build_tag = mapper.epel_build_tag(branch) if epel_build_tag: - cmd = ["koji", "untag-build", "--all", tag] + packages + cmd = ["koji", "-s", url, "untag-build", "--all", tag] + packages log.debug("Running: %s", " ".join(cmd)) subprocess.check_call(cmd) - cmd = ["koji", "unblock-pkg", epel_build_tag] + packages + cmd = ["koji", "-s", url, "unblock-pkg", epel_build_tag] + packages log.debug("Running: %s", " ".join(cmd)) subprocess.check_call(cmd) -def handle_message(message, retiring_branches=RETIRING_BRANCHES): +def handle_message(message, retiring_branches=RETIRING_BRANCHES, staging=False): messageinfo = get_retirement_info(message) msg_id = message["msg_id"] if messageinfo is None: @@ -183,21 +205,24 @@ def handle_message(message, retiring_branches=RETIRING_BRANCHES): package = messageinfo["name"] - pkgdbinfo = pkgdb_retirement_status(package, branch) + pkgdbinfo = pkgdb_retirement_status(package, branch, staging) if pkgdbinfo["retired"] is not True: log.error("Processing '%s', package '%s' not retired", msg_id, package) log.debug("'%s' retired on '%s'", package, pkgdbinfo["status_change"]) - return block_package(package, branch) + return block_package(package, branch, staging=staging) -def block_all_retired(branches=RETIRING_BRANCHES): +def block_all_retired(branches=RETIRING_BRANCHES, staging=False): for branch in branches: log.debug("Processing branch %s", branch) - retired = get_retired_packages(branch) - blocked = blocked_packages(branch) + if staging and branch in PROD_ONLY_BRANCHES: + log.warning('%s not handled in staging..' % branch) + continue + retired = get_retired_packages(branch, staging) + blocked = blocked_packages(branch, staging) unblocked = [] for pkg in retired: @@ -206,7 +231,7 @@ def block_all_retired(branches=RETIRING_BRANCHES): if unblocked: log.info("Blocked packages %s on %s", unblocked, branch) - block_package(unblocked, branch) + block_package(unblocked, branch, staging=staging) class SubjectSMTPHandler(logging.handlers.SMTPHandler): @@ -260,11 +285,14 @@ if __name__ == "__main__": parser.add_argument( "--branch", default="master", help="Branch to retire specified packages on, default: %(default)s") + parser.add_argument( + "--staging", default=False, action="store_true", + help="Talk to staging services (pkgdb), instead of production") args = parser.parse_args() setup_logging(args.debug) if not args.packages: - block_all_retired() + block_all_retired(staging=args.staging) else: - block_package(args.packages, args.branch) + block_package(args.packages, args.branch, staging=args.staging) -- 1.9.3

1 0

firewall rules on builders (iptables, firewalld, libvirt...)
by Matthew Miller 28 Oct '14

28 Oct '14

It's my understanding (Dennis please correct if I'm wrong) that the problem with cloud image creation was due to libvirt iptables rules being lost when iptables was restarted. This is a fundamental known issue (see last paragraph of <http://libvirt.org/firewall.html>), and one of the things firewalld was meant to solve. Dennis says that there are lot of complicated rules on the builders making switching to firewalld difficult. One possibility might be to move those complicated rules from the builders to a network firewall, and keep the host rules simple and functional. But that's probably a big undertaking. In the meantime, any time iptables is restarted or reloaded, libvirt needs a SIGHUP. (I suppose this means: ansible playbooks and also added to any manual procedures.) [cc rel-eng, reply-to infrastructure] -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

1 0

[releng] check_sigul: Fix mail parameters
by Till Maas 28 Oct '14

28 Oct '14

commit 8b150dad3b0a0154608a337741144c8cf7110997 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 28 08:00:18 2014 +0100 check_sigul: Fix mail parameters scripts/check_sigul.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) --- diff --git a/scripts/check_sigul.py b/scripts/check_sigul.py index 9f2b01c..73c3dd3 100755 --- a/scripts/check_sigul.py +++ b/scripts/check_sigul.py @@ -42,8 +42,8 @@ if __name__ == "__main__": # FIXME: Make this a config option fedora_user = getpass.getuser() mail_logger = SubjectSMTPHandler( - "127.0.0.1", fedora_user, [fedora_user], "Autosigner log event") - mail_logger.subject_prefix = "Autosigner: " + "127.0.0.1", fedora_user, [fedora_user], "Sigul check log event") + mail_logger.subject_prefix = "Sigul check: " mail_logger.setLevel(logging.DEBUG) mail_logger.setFormatter(formatter) log.addHandler(mail_logger)

1 0

[releng] check_sigul: Fix comments
by Till Maas 28 Oct '14

28 Oct '14

commit 9a3926c2eb9dde1814cbcfde5d514175c2e42642 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 28 07:59:22 2014 +0100 check_sigul: Fix comments scripts/check_sigul.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) --- diff --git a/scripts/check_sigul.py b/scripts/check_sigul.py index baa4202..9f2b01c 100755 --- a/scripts/check_sigul.py +++ b/scripts/check_sigul.py @@ -1,7 +1,7 @@ #!/usr/bin/python -tt # vim: fileencoding=utf8 # SPDX-License-Identifier: GPL-2.0+ -#{{{ License header: GPLv2+ +# {{{ License header: GPLv2+ # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or @@ -14,7 +14,7 @@ # # You should have received a copy of the GNU General Public License # along with this program. If not, see <https://www.gnu.org/licenses/>. -#}}} +# }}} import getpass import logging

1 0

[releng] Updated tag rawhide-stable
by Till Maas 27 Oct '14

27 Oct '14

The lightweight tag 'rawhide-stable' was updated to point to: b3c72ca... add sigul check script It previously pointed to: 2729264... buildbranched/rawhide: Fix mash calls NOTE: People pulling from the repository will not get the new tag. For more information, please see: http://live.gnome.org/Git/Help/TagUpdates

1 0

[releng] add sigul check script
by Till Maas 27 Oct '14

27 Oct '14

commit b3c72ca48b955d233a0d5e60a51f52f0a44618bf Author: Till Maas <opensource(a)till.name> Date: Mon Oct 27 22:14:00 2014 +0100 add sigul check script scripts/check_sigul.py | 80 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 80 insertions(+), 0 deletions(-) --- diff --git a/scripts/check_sigul.py b/scripts/check_sigul.py new file mode 100755 index 0000000..baa4202 --- /dev/null +++ b/scripts/check_sigul.py @@ -0,0 +1,80 @@ +#!/usr/bin/python -tt +# vim: fileencoding=utf8 +# SPDX-License-Identifier: GPL-2.0+ +#{{{ License header: GPLv2+ +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +#}}} + +import getpass +import logging +log = logging.getLogger(__name__) +import os +import time + +from sigulsign_unsigned import SigulHelper +from autosigner import SubjectSMTPHandler + + +if __name__ == "__main__": + formatter = logging.Formatter( + '%(asctime)s: %(levelname)s: %(message)s', + ) + # Log in UTC + formatter.converter = time.gmtime + + console_logger = logging.StreamHandler() + console_logger.setLevel(logging.DEBUG) + console_logger.setFormatter(formatter) + log.setLevel(logging.DEBUG) + log.addHandler(console_logger) + + # FIXME: Make this a config option + fedora_user = getpass.getuser() + mail_logger = SubjectSMTPHandler( + "127.0.0.1", fedora_user, [fedora_user], "Autosigner log event") + mail_logger.subject_prefix = "Autosigner: " + mail_logger.setLevel(logging.DEBUG) + mail_logger.setFormatter(formatter) + log.addHandler(mail_logger) + + keys = ["fedora-22", "fedora-22-secondary"] + helpers = {} + + for k in keys: + log.debug("Monitoring key " + k) + if k.endswith("-secondary"): + config_file = os.path.expanduser("~/.sigul/client-secondary.conf") + else: + config_file = None + + helpers[k] = SigulHelper(k, config_file=config_file, + ask_with_agent=True, ask=True) + + status = {} + + log.debug("Starting checking...") + while True: + for key, helper in helpers.items(): + res = helper.get_public_key() + ret, pubkey, errors = res + if status.setdefault(key, res) != res: + if ret != 0 or errors: + log.error( + "Sigul for key '{}' stopped working: {}:{}".format( + key, ret, errors)) + else: + log.warning( + "Sigul for key '{}' resumed working".format(key)) + + time.sleep(600)

1 0

[releng] block_retired: Import SubjectSMTPHandler
by Till Maas 27 Oct '14

27 Oct '14

commit 4c9c1301605849e502613aabe1c46eae0549fa56 Author: Till Maas <opensource(a)till.name> Date: Mon Oct 27 22:03:56 2014 +0100 block_retired: Import SubjectSMTPHandler scripts/block_retired.py | 13 ++----------- 1 files changed, 2 insertions(+), 11 deletions(-) --- diff --git a/scripts/block_retired.py b/scripts/block_retired.py index 2556980..7685719 100755 --- a/scripts/block_retired.py +++ b/scripts/block_retired.py @@ -26,6 +26,8 @@ import time import koji import pkgdb2client +from autosigner import SubjectSMTPHandler + log = logging.getLogger(__name__) RETIRING_BRANCHES = ["el5", "el6", "epel7", "f21", "master"] @@ -209,17 +211,6 @@ def block_all_retired(branches=RETIRING_BRANCHES): block_package(unblocked, branch) -class SubjectSMTPHandler(logging.handlers.SMTPHandler): - # Class copied from autosigner.py - subject_prefix = "" - - def getSubject(self, record): - first_line = record.message.split("\n")[0] - fmt = self.subject_prefix + "{0.levelname}: {first_line}" - - return fmt.format(record, first_line=first_line) - - def setup_logging(debug=False, mail=False): if debug: log.setLevel(logging.DEBUG)

1 0

[releng] sigulsign_unsigned: add get_public_key()
by Till Maas 27 Oct '14

27 Oct '14

commit 5da8d841b1dc77f3fbae9ee9a854d2babb5e42d4 Author: Till Maas <opensource(a)till.name> Date: Mon Oct 27 22:00:18 2014 +0100 sigulsign_unsigned: add get_public_key() scripts/sigulsign_unsigned.py | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) --- diff --git a/scripts/sigulsign_unsigned.py b/scripts/sigulsign_unsigned.py index b401d89..3a9b185 100755 --- a/scripts/sigulsign_unsigned.py +++ b/scripts/sigulsign_unsigned.py @@ -358,12 +358,16 @@ class SigulHelper(object): self.config_file = config_file self.arch = arch - command = self.build_cmdline('get-public-key', self.key) - ret, pubkey, stderr = self.run_command(command) + ret, pubkey, stderr = self.get_public_key() if ret != 0: raise ValueError("Invalid key or password: " + stderr) self.keyid, self.v3 = get_key_info(pubkey) + def get_public_key(self): + command = self.build_cmdline('get-public-key', self.key) + ret, stdout, stderr = self.run_command(command) + return ret, stdout, stderr + def build_cmdline(self, *args): cmdline = ['sigul', '--batch'] if self.config_file:

1 0

last minute f21 change -- workstation dropping installable tree
by Matthew Miller 27 Oct '14

27 Oct '14

So in today's FESCo meeting, it was decided that since the workaround for making the netinstall image isn't all... worked around... rel-eng wants to just drop making a separate installable Workstation tree. On the mirrors, Workstation will just contain the live images. (Netinstall will still be possible, from the server netinstall, but not the default.) I'm thinking that if that's being done, we should do the same with Cloud, and just have Cloud images directly under Cloud (with no repo or rpms there). What do you think? This will all need to be revisited for F22 in any case. -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

2 5

[Fedocal] Reminder meeting : Fedora Release Engineering
by Dennis Gilmore 26 Oct '14

26 Oct '14

Dear all, You are kindly invited to the meeting: Fedora Release Engineering on 2014-10-27 from 15:30:00 to 16:30:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Fedora Release Engineering weekly meeting Source: https://apps.fedoraproject.org/calendar//meeting/162/

1 0

[releng] sigulsign_unsigned: Fix PEP8 compliance
by Till Maas 26 Oct '14

26 Oct '14

commit a5788fdbf8c30b72e364bf84e30538dbad8f9720 Author: Till Maas <opensource(a)till.name> Date: Sun Oct 26 11:45:16 2014 +0100 sigulsign_unsigned: Fix PEP8 compliance scripts/sigulsign_unsigned.py | 18 ++++++++++-------- 1 files changed, 10 insertions(+), 8 deletions(-) --- diff --git a/scripts/sigulsign_unsigned.py b/scripts/sigulsign_unsigned.py index a49d465..b401d89 100755 --- a/scripts/sigulsign_unsigned.py +++ b/scripts/sigulsign_unsigned.py @@ -138,10 +138,10 @@ def get_key_info(source, filename=False): return keyid, v3 -def get_gpg_agent_passphrase(cache_id, ask=False, error_message="X", prompt="X", - description=None): +def get_gpg_agent_passphrase(cache_id, ask=False, error_message="X", + prompt="X", description=None): gpg_agent = subprocess.Popen(["gpg-connect-agent"], stdin=subprocess.PIPE, - stdout=subprocess.PIPE) + stdout=subprocess.PIPE) cmdline = ["GET_PASSPHRASE"] if not ask: @@ -160,8 +160,8 @@ def get_gpg_agent_passphrase(cache_id, ask=False, error_message="X", prompt="X", response, error = gpg_agent.communicate(cmdline) if gpg_agent.returncode != 0: - raise RuntimeError("gpg-agent-connect error: {}:{}".format( - gpg_agent.returncode, error)) + raise RuntimeError("gpg-agent-connect error: {}:{}".format( + gpg_agent.returncode, error)) elif not response.startswith("OK"): raise RuntimeError("gpg-agent-connect error message: {}".format( response)) @@ -343,12 +343,14 @@ class SigulHelper(object): cache_id = "sigul:{}:{}".format(fas_username, key) try: - self.password = get_gpg_agent_passphrase(cache_id, ask=ask_with_agent) + self.password = get_gpg_agent_passphrase(cache_id, + ask=ask_with_agent) except: self.password = None if self.password is None and ask and not ask_with_agent: - self.password = getpass.getpass(prompt='Passphrase for %s: ' % key) + self.password = getpass.getpass( + prompt='Passphrase for %s: ' % key) else: self.password = password if self.password is None: @@ -458,7 +460,7 @@ if __name__ == "__main__": key = args[0] logging.debug('Using %s for key %s' % (KEYS[key]['id'], key)) - if not key in KEYS.keys(): + if key not in KEYS.keys(): logging.error('Unknown key %s' % key) parser.print_help() sys.exit(1)

1 0

[releng] sigulsign_unsigned: Add details to error message
by Till Maas 26 Oct '14

26 Oct '14

commit dbf026112fd5d339916381d1827490be999dd787 Author: Till Maas <opensource(a)till.name> Date: Sun Oct 26 11:42:39 2014 +0100 sigulsign_unsigned: Add details to error message scripts/sigulsign_unsigned.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) --- diff --git a/scripts/sigulsign_unsigned.py b/scripts/sigulsign_unsigned.py index ad7cf3f..a49d465 100755 --- a/scripts/sigulsign_unsigned.py +++ b/scripts/sigulsign_unsigned.py @@ -357,9 +357,9 @@ class SigulHelper(object): self.arch = arch command = self.build_cmdline('get-public-key', self.key) - ret, pubkey = self.run_command(command)[0:2] + ret, pubkey, stderr = self.run_command(command) if ret != 0: - raise ValueError("Invalid key or password") + raise ValueError("Invalid key or password: " + stderr) self.keyid, self.v3 = get_key_info(pubkey) def build_cmdline(self, *args): @@ -487,8 +487,9 @@ if __name__ == "__main__": config_file=opts.sigul_config_file, arch=opts.arch, ask=True, ask_with_agent=opts.gpg_agent) - except ValueError: - logging.error('Error validating passphrase for key %s' % key) + except ValueError as error: + logging.error('Error validating passphrase for key %s: %s', key, + error) sys.exit(1) # setup the koji session

1 0

#6022: Unblock perl-ExtUtils-CBuilder in rawhide
by Fedora Release Engineering 24 Oct '14

24 Oct '14

#6022: Unblock perl-ExtUtils-CBuilder in rawhide --------------------+------------------------ Reporter: ppisar | Owner: rel-eng@… Type: task | Status: new Milestone: | Component: koji Keywords: | Blocked By: Blocking: | --------------------+------------------------ Please unblock ''perl-ExtUtils-CBuilder'' package in F22. See <https://bugzilla.redhat.com/show_bug.cgi?id=1144033>. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6022> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

[releng] autosigner: Fix exception reporting
by Till Maas 24 Oct '14

24 Oct '14

commit d2e2f60ea8d46128344795c48d88b3b4be9bf1cf Author: Till Maas <opensource(a)till.name> Date: Fri Oct 24 18:26:07 2014 +0200 autosigner: Fix exception reporting scripts/autosigner.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/autosigner.py b/scripts/autosigner.py index d8388f0..d2cc120 100755 --- a/scripts/autosigner.py +++ b/scripts/autosigner.py @@ -490,6 +490,6 @@ if __name__ == "__main__": exc_type, exc_value, exc_traceback = sys.exc_info() tb = traceback.format_exception(exc_type, exc_value, exc_traceback) - log.error("Exception: %s\nmessage was: %s", ("".join(tb), msg)) + log.error("Exception: %s\nmessage was: %s", "".join(tb), msg) except Exception, ee: log.error("Exception fallback %s", e)

1 0

#6021: Please unblock python-cjson package
by Fedora Release Engineering 24 Oct '14

24 Oct '14

#6021: Please unblock python-cjson package -----------------------------+------------------------ Reporter: hubbitus | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ By some reason it did not happened automatically: https://bugzilla.redhat.com/show_bug.cgi?id=1130097 -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6021> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

#6019: unblock the "bashmount" package for the fedora/epel
by Fedora Release Engineering 23 Oct '14

23 Oct '14

#6019: unblock the "bashmount" package for the fedora/epel -----------------------------+------------------------ Reporter: jamielinux | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: git Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ https://bugzilla.redhat.com/show_bug.cgi?id=1089493 Retired package now un-retired across all branches (f22, f21, f20, f19, epel7, el6). Can it please be unblocked? Thanks! -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6019> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

#6018: Please unblock tilda package
by Fedora Release Engineering 23 Oct '14

23 Oct '14

#6018: Please unblock tilda package -----------------------------+------------------------ Reporter: hannes | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ It was re-reviewed and it was unretired, but it's still blocked in koji for f20, f21 and rawhide. https://bugzilla.redhat.com/show_bug.cgi?id=1153302 -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6018> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 1

[releng] do not make the Workstation install tree per FESCo
by Dennis Gilmore 23 Oct '14

23 Oct '14

commit bf1c146406be6bb7ffc2ca994938848d7208fdec Author: Dennis Gilmore <dennis(a)ausil.us> Date: Thu Oct 23 09:53:55 2014 -0500 do not make the Workstation install tree per FESCo scripts/run-pungi | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) --- diff --git a/scripts/run-pungi b/scripts/run-pungi index 123c328..3cdbc94 100755 --- a/scripts/run-pungi +++ b/scripts/run-pungi @@ -43,7 +43,7 @@ popd for arch in armhfp i386 x86_64 do mock -r fedora-branched-compose-i386 --init || exit 1 - mock -r fedora-branched-compose-i386 --shell "pungi -c $KICKSTARTS/fedora-install-workstation.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ --arch=$arch -G -C --ver $VERSION --workdirbase=/tmp/work --name Fedora-Workstation --flavor Workstation" || exit 1 + #mock -r fedora-branched-compose-i386 --shell "pungi -c $KICKSTARTS/fedora-install-workstation.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ --arch=$arch -G -C --ver $VERSION --workdirbase=/tmp/work --name Fedora-Workstation --flavor Workstation" || exit 1 mock -r fedora-branched-compose-i386 --shell "pungi -c $KICKSTARTS/fedora-install-server.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ --arch=$arch -G -C --ver $VERSION --workdirbase=/tmp/work --name Fedora-Server --flavor Server" || exit 1 mock -r fedora-branched-compose-i386 --shell "pungi -c $KICKSTARTS/fedora-install-cloud.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ --arch=$arch -G -C --ver $VERSION --workdirbase=/tmp/work --name Fedora-Cloud --flavor Cloud" || exit 1 mock -r fedora-branched-compose-i386 --clean || exit 1 @@ -66,7 +66,7 @@ do set -x umask 002 mock -r fedora-branched-compose-$arch --init || exit 1 - mock -r fedora-branched-compose-$arch --shell "pungi -c $KICKSTARTS/fedora-install-workstation.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ -B -I --no-dvd --ver $VERSION --workdirbase=/tmp/work --flavor Workstation --name=Fedora-Workstation" + #mock -r fedora-branched-compose-$arch --shell "pungi -c $KICKSTARTS/fedora-install-workstation.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ -B -I --no-dvd --ver $VERSION --workdirbase=/tmp/work --flavor Workstation --name=Fedora-Workstation" mock -r fedora-branched-compose-$arch --shell "pungi -c $KICKSTARTS/fedora-install-server.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ -B -I --ver $VERSION --workdirbase=/tmp/work --flavor Server --name=Fedora-Server" mock -r fedora-branched-compose-$arch --shell "pungi -c $KICKSTARTS/fedora-install-cloud.ks --destdir $DESTDIR --cachedir /srv/pungi/cache/ -B -I --no-dvd --ver $VERSION --workdirbase=/tmp/work --flavor Cloud --name=Fedora-Cloud" mock -r fedora-branched-compose-$arch --clean || exit 1 @@ -83,7 +83,7 @@ popd hardlink -vvc $FINALDEST/$VERSION/Workstation $FINALDEST/$VERSION/Server $FINALDEST/$VERSION/Cloud -sg releng "./build_composeinfo $FINALDEST/$VERSION/Workstation" +#sg releng "./build_composeinfo $FINALDEST/$VERSION/Workstation" sg releng "./build_composeinfo $FINALDEST/$VERSION/Server" sg releng "./build_composeinfo $FINALDEST/$VERSION/Cloud" @@ -101,7 +101,7 @@ done pushd $FINALDEST/$VERSION/ -sg releng "mkdir -p Images/armhfp Cloud/Images/i386 Cloud/Images/x86_64 Spins/i386 Spins/x86_64 Live/i386 Live/x86_64 Docker/x86_64 Workstation/armhfp/Images/" +sg releng "mkdir -p Images/armhfp Cloud/Images/i386 Cloud/Images/x86_64 Spins/i386 Spins/x86_64 Live/i386 Live/x86_64 Docker/x86_64 Workstation/armhfp/Images/ Workstation/x86_64/iso Workstation/i386/iso" popd

1 0

Re: last minute f21 change -- workstation dropping installable tree
by Matthew Miller 23 Oct '14

23 Oct '14

On Thu, Oct 23, 2014 at 08:46:00AM +0200, Haïkel wrote: > > This will all need to be revisited for F22 in any case. > If that will be revisited afterwards, then we should just drop it. > As long as documentation reflects that, I'm fine with delaying to F22. Request to releng: https://fedorahosted.org/rel-eng/ticket/6020 And yes, this needs to all be different for f22. -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

1 0

okay so.... tc4 is unbootable
by Matthew Miller 20 Oct '14

20 Oct '14

I thought the most likely culprit might be my hack for the growpart/sfdisk/mbr problem, but it seems like the kickstart file is actually truncated -- see http://koji.fedoraproject.org/koji/taskinfo?taskID=7900887 (and click on https://kojipkgs.fedoraproject.org//work/tasks/887/7900887/fedora-cloud-bas… Something is not right.... -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

1 0

Summary/Minutes from today's Release Engineering Meeting (2014-10-20)
by Kevin Fenzi 20 Oct '14

20 Oct '14

====================================== #fedora-meeting-1: RELENG (2014-10-20) ====================================== Meeting started by nirik at 14:30:20 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting-1/2014-10-20/releng.2014-10… . Meeting summary --------------- * init process (nirik, 14:30:20) * Secondary Architectures update - s390 (nirik, 14:34:06) * Secondary Architectures update - ppc (nirik, 14:35:20) * Secondary Architectures update - arm (nirik, 14:37:58) * tickets (nirik, 14:41:46) * LINK: https://fedorahosted.org/rel-eng/report/10 (nirik, 14:41:48) * LINK: https://fedorahosted.org/rel-eng/ticket/6014 (nirik, 14:54:15) * Open Floor (nirik, 14:58:09) * LINK: https://bugzilla.redhat.com/show_bug.cgi?id=1081622 (nirik, 15:05:27) Meeting ended at 15:06:24 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (55) * sharkcz (13) * bochecha (9) * pbrobinson (8) * pbabinca (5) * zodbot (4) * masta (2) * janeznemanic (1) * tyll (0) * dgilmore (0) -- 14:30:20 <nirik> #startmeeting RELENG (2014-10-20) 14:30:20 <zodbot> Meeting started Mon Oct 20 14:30:20 2014 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:30:20 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 14:30:20 <nirik> #meetingname releng 14:30:20 <nirik> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson 14:30:20 <nirik> #topic init process 14:30:20 <zodbot> The meeting name has been set to 'releng' 14:30:20 <zodbot> Current chairs: bochecha dgilmore masta nirik pbrobinson sharkcz tyll 14:30:28 <nirik> who all is around for a releng meeting 14:30:36 * sharkcz is here 14:30:42 <bochecha> I'm here 14:30:43 <janeznemanic> hi 14:32:10 <nirik> morning. I didn't send an agenda, because I forgot to. ;) 14:32:17 <nirik> dgilmore-bne is traveling today. 14:33:31 * masta is here 14:33:35 <masta> hiya 14:34:00 <nirik> cool. lets start with secondary arches? 14:34:06 <nirik> #topic Secondary Architectures update - s390 14:34:18 <nirik> anything new and exciting? 14:34:36 <sharkcz> I was on PTO last week, so nothing new to report 14:34:54 <sharkcz> except Peter signed f21 rpms 14:35:12 <nirik> cool. 14:35:20 <nirik> #topic Secondary Architectures update - ppc 14:35:26 <nirik> anything with ppc? 14:35:41 <sharkcz> no Peter, so Ill 14:35:53 <nirik> yeah. 14:36:01 <sharkcz> stil lworking on Alpha, one last blocker in blivet should be resolved today, so we are close 14:36:20 <nirik> cool. 14:36:30 <nirik> how are thinking looking for beta there? or hard to say until alpha is out? 14:36:58 <sharkcz> actually the ppc alpha is very close to beta in primary 14:37:29 <nirik> great 14:37:46 <nirik> anything else on ppc? 14:37:52 <sharkcz> nope 14:37:58 <nirik> #topic Secondary Architectures update - arm 14:39:18 <nirik> anything for arm? I guess no peter here either. ;) 14:40:00 <sharkcz> yes, out of my scope :-) 14:40:30 <nirik> One thing I'll note here: we may be moving arm and ppc stuff to their own racks at some point. Consolidate them in one place and such... 14:40:47 <nirik> probibly will need to be short downtimes to move things, but then they will all be on the same serial/etc 14:41:13 <nirik> it's still in the planning stages... just a early headsup 14:41:29 <sharkcz> ok, thanks for update 14:41:46 <nirik> #topic tickets 14:41:48 <nirik> https://fedorahosted.org/rel-eng/report/10 14:41:58 <nirik> is there any specific tickets folks wanted to talk about? 14:42:35 <nirik> some of those are waiting on more data... 14:43:31 <pbabinca> #6016 - do you think it does make a sense to proceed with the ticket or decide gather more info if that's good idea? 14:44:17 <bochecha> pbabinca, can you maybe provide a list of packages that don't get installed with this change? 14:44:20 <nirik> I don't see a problem with doing it... I guess I'd like dgilmore to chime in before we implement, but seems ok 14:44:30 <nirik> how much time does it save? 14:45:01 <pbabinca> nirik, I don't have the data yet. 14:45:09 <bochecha> nirik, not just time, but also, the odds of breaing the buildroot are lower if less packages get installed 14:45:12 <sharkcz> or how many packages? 14:45:16 <pbabinca> bochecha, true that. 14:45:24 <nirik> we already do this for rhel5 I think build roots? 14:46:01 <pbabinca> We implement new features in fedpkg and that means change of build root package. 14:46:03 <sharkcz> can it affect the minimal buildroot content mentioned in the guuidelines? 14:46:54 <nirik> this would only be for srpm right? 14:47:09 <pbabinca> nirik, yes. But that's all what fedpkg is used for there. 14:47:14 <nirik> yeah 14:47:34 <bochecha> sharkcz, no it wouldn't, this is only for the srpm-build group, not the build one 14:47:35 <sharkcz> ah, then https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines… won't be affected 14:49:02 <nirik> right. 14:50:17 <nirik> so, lets ping dgilmore-bne when he's back on line about it... see if he sees any problems... 14:50:25 <nirik> but seems fine to me off hand. 14:51:08 <nirik> anything else on this? or other tickets folks would like to discuss? 14:52:49 * pbrobinson is here 14:52:56 <nirik> hey pbrobinson 14:53:08 <nirik> anything you want to note or mention? 14:53:51 <pbrobinson> nope, all looks good here 14:54:05 <nirik> sharkcz: on 6014 did you get a list of builds? did we sort that out? 14:54:15 <nirik> https://fedorahosted.org/rel-eng/ticket/6014 14:54:17 <pbrobinson> I did primary tc4 Friday night and it mostly went to plan from a rel-eng and my access POV 14:54:27 <nirik> cool. 14:56:01 <sharkcz> nirik: I don't have the list, sorry, maybe pbrobinson has one from arm koji? 14:56:52 <pbrobinson> no, unfortunately I don't 14:57:01 <nirik> ok. 14:57:58 <nirik> I guess we update that as we can... 14:58:09 <nirik> #topic Open Floor 14:58:17 <nirik> anything for open floor? 14:58:35 <pbrobinson> not from me 14:59:23 <pbrobinson> I'm hoping we can close out ppc alpha this week, aarch64 alpha is done, announce tomorrow and we'll then move straight into beta at basically the same cnadince as mainline 14:59:35 <pbrobinson> candice ... 14:59:49 <nirik> :) ok. cool. 15:00:16 <nirik> ok, if nothing else will close out in a minute... 15:00:59 <bochecha> I have a couple of mash patches still awaiting for reviewx 15:01:12 <bochecha> if anyone with commit access has time, they are on the mailing-list :) 15:02:03 <nirik> bochecha: cool. 15:02:11 <pbrobinson> bochecha: mailing list or BZ?? 15:02:23 <nirik> BTW, threebean is working on getting a staging compose box... so we can test mash changes. 15:02:27 <bochecha> pbrobinson, rel-eng mailing-list 15:03:03 <bochecha> pbrobinson, https://lists.fedoraproject.org/pipermail/rel-eng/2014-September/018424.html and https://lists.fedoraproject.org/pipermail/rel-eng/2014-September/018425.html 15:04:41 <nirik> bochecha: cool. BTW, if you have time/desire we are having problems with a max delta rpm patch we made to mash... 15:04:53 <nirik> thats what threebean wanted to work on debugging. 15:05:12 <bochecha> nirik, sure, what can I do? 15:05:27 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=1081622 15:05:43 <nirik> if you see anything wrong with the patch or why it wouldn't be working, chime in. ;) 15:06:21 <nirik> anyhow, thanks for coming everyone! 15:06:24 <nirik> #endmeeting

1 0

[Fedocal] Reminder meeting : Fedora Release Engineering
by Dennis Gilmore 19 Oct '14

19 Oct '14

Dear all, You are kindly invited to the meeting: Fedora Release Engineering on 2014-10-20 from 15:30:00 to 16:30:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Fedora Release Engineering weekly meeting Source: https://apps.fedoraproject.org/calendar//meeting/162/

1 0

[releng] Add helper script to retire packages
by Till Maas 18 Oct '14

18 Oct '14

commit 5ea1e5250b9635d40598222cd1ef219ed1bea5e9 Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 23:26:55 2014 +0200 Add helper script to retire packages scripts/fedretire | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 78 insertions(+), 0 deletions(-) --- diff --git a/scripts/fedretire b/scripts/fedretire new file mode 100755 index 0000000..c277f77 --- /dev/null +++ b/scripts/fedretire @@ -0,0 +1,78 @@ +#!/usr/bin/python -tt +# vim: fileencoding=utf8 +# SPDX-License-Identifier: GPL-2.0+ +# {{{ License header: GPLv2+ +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# }}} + +import argparse +import os +import shutil +import subprocess +import tempfile + + +def retire(pkg, branch, reason, dryrun=False): + tempdir = tempfile.mkdtemp(pkg, branch) + try: + cmd = ["fedpkg", "clone", pkg] + if dryrun: + print(cmd) + else: + subprocess.check_call(cmd, cwd=tempdir) + pkgdir = os.path.join(tempdir, pkg) + + cmd = ["fedpkg", "switch-branch", branch] + if dryrun: + print(cmd) + else: + subprocess.check_call(cmd, cwd=pkgdir) + + cmd = ["fedpkg", "retire", reason] + if dryrun: + print(cmd) + else: + subprocess.check_call(cmd, cwd=pkgdir) + finally: + shutil.rmtree(tempdir) + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Helper to retire packages") + parser.add_argument("--branch", default="master", + choices=["master", "f21", "epel7", "el6", "el5"]) + parser.add_argument("--reasonfile", default=None) + parser.add_argument("--reason", default=None) + parser.add_argument("pkg", nargs="+") + args = parser.parse_args() + + if args.reasonfile is None and args.reason is None: + reasonfile = tempfile.NamedTemporaryFile(delete=False) + filename = reasonfile.name + childpid = os.fork() + if childpid == 0: + os.execlp("vim", "vim", filename) + else: + os.waitpid(childpid, 0) + reason = reasonfile.read() + os.unlink(filename) + elif args.reasonfile is not None: + with open(args.reasonfile, "rb") as reasonfile: + reason = reasonfile.read() + else: + reason = args.reason + if reason[-1] != "\n": + reason += "\n" + + for pkg in args.pkg: + retire(pkg, args.branch, reason, dryrun=False)

1 0

[releng] find_unblocked_orphans: Highlight unshipped packages
by Till Maas 18 Oct '14

18 Oct '14

commit b750c92a780bdb0198c92e9d01f344b09b33b1aa Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 23:23:39 2014 +0200 find_unblocked_orphans: Highlight unshipped packages scripts/find_unblocked_orphans.py | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 688b30f..02807c8 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -315,6 +315,7 @@ class DepChecker(object): self.pkgdbinfo_queue = Queue() self.pkgdb_cache = "orphans-pkgdb-{}.pickle".format(release) self.pkgdb_dict = get_cache(self.pkgdb_cache, default={}) + self.not_in_repo = [] def create_mapping(self): src_by_bin = {} # Dict of source pkg objects by binary package objects @@ -368,6 +369,7 @@ class DepChecker(object): # If we don't have a package in the repo, there is nothing to do sys.stderr.write( "Package {0} not found in repo\n".format(srpmname)) + self.not_in_repo.append(srpmname) rpms = [] # provides of all packages built from ``srpmname`` @@ -587,9 +589,10 @@ def maintainer_info(affected_people): return info -def package_info(unblocked, dep_map, pkgdb_dict, orphans=None, failed=None, +def package_info(unblocked, dep_map, depchecker, orphans=None, failed=None, week_limit=6): info = "" + pkgdb_dict = depchecker.pkgdb_dict table, affected_people = maintainer_table(unblocked, pkgdb_dict) info += table @@ -671,6 +674,10 @@ def package_info(unblocked, dep_map, pkgdb_dict, orphans=None, failed=None, ftbfs_not_breaking_deps) info += "\n\n" + if depchecker.not_in_repo: + info += wrap_and_format("Not found in repo", + sorted(depchecker.not_in_repo)) + addresses = ["{0}(a)fedoraproject.org".format(p) for p in affected_people.keys() if p != ORPHAN_UID] return info, addresses @@ -716,7 +723,7 @@ def main(): # TODO: add app args to either depsolve or not dep_map = depchecker.recursive_deps(unblocked) sys.stderr.write('done\n') - info, addresses = package_info(unblocked, dep_map, depchecker.pkgdb_dict, + info, addresses = package_info(unblocked, dep_map, depchecker, orphans=orphans, failed=failed) text += "\n" text += info

1 0

[releng] find_unblocked_orphans: Show stale dependent packages
by Till Maas 18 Oct '14

18 Oct '14

commit 853bf9c5eb91e41a9e68e7737869e073a865db84 Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 14:19:45 2014 +0200 find_unblocked_orphans: Show stale dependent packages scripts/find_unblocked_orphans.py | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 5807c6a..688b30f 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -649,6 +649,15 @@ def package_info(unblocked, dep_map, pkgdb_dict, orphans=None, failed=None, if breaking: info += wrap_and_format("Depending packages", sorted(breaking)) + if orphans: + stale_breaking = set() + for package in orphans_breaking_deps_stale: + stale_breaking = stale_breaking.union( + set(dep_map[package].keys())) + info += wrap_and_format( + "Packages depending on packages orphaned for more than " + "{} weeks".format(week_limit), sorted(stale_breaking)) + if failed: info += "\nFTBFS: " + " ".join(failed) info += "\n"

1 0

[releng] find_unblocked_orphans: Mention retirement deadline
by Till Maas 18 Oct '14

18 Oct '14

commit b34df01a35c3e61eb56734cd563611cf646a577f Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 12:51:22 2014 +0200 find_unblocked_orphans: Mention retirement deadline scripts/find_unblocked_orphans.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 7a41d0c..5807c6a 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -99,14 +99,15 @@ According to https://fedoraproject.org/wiki/Schedule branching will occur not earlier than 2014-07-08. The packages will be retired shortly before. """ -HEADER = """The following packages are orphaned and might be retired -eventually, unless someone adopts them. If you know for sure that the package -should be retired, please do so now with a proper reason: +HEADER = """The following packages are orphaned and will be retired when they +are orphaned for six weeks, unless someone adopts them. If you know for sure +that the package should be retired, please do so now with a proper reason: https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life Note: If you received this mail directly you (co)maintain one of the affected packages or a package that depends on one. Please adopt the affected package or -retire your depending package to avoid broken dependencies. +retire your depending package to avoid broken dependencies, otherwise your +package will be retired when the affected package gets retired. """ FOOTER = """-- \nThe script creating this output is run and developed by Fedora

1 0

[releng] find_unblocked_orphans: Rename age column
by Till Maas 18 Oct '14

18 Oct '14

commit 2489d4fcb0ba556066b5de6afd57d554e10f75fc Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 12:41:42 2014 +0200 find_unblocked_orphans: Rename age column scripts/find_unblocked_orphans.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index e2889e4..7a41d0c 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -528,7 +528,7 @@ def maintainer_table(packages, pkgdb_dict): if with_table: table = texttable.Texttable(max_width=80) - table.header(["Package", "(co)maintainers", "age"]) + table.header(["Package", "(co)maintainers", "Status Change"]) table.set_cols_align(["l", "l", "l"]) table.set_deco(table.HEADER) else: @@ -542,8 +542,8 @@ def maintainer_table(packages, pkgdb_dict): p = ', '.join(people) status_change = pkginfo.status_change age = pkginfo.age - agestr = "{} ({} weeks)".format(status_change.strftime("%Y-%m-%d"), - age.days / 7) + agestr = "{} ({} weeks ago)".format(status_change.strftime("%Y-%m-%d"), + age.days / 7) if with_table: table.add_row([package_name, p, agestr])

1 0

[releng] find_unblocked_orphans: Show stale packages
by Till Maas 18 Oct '14

18 Oct '14

commit d70220425e61126fd9e5cef3c7459e9a1fa8b793 Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 12:28:17 2014 +0200 find_unblocked_orphans: Show stale packages scripts/find_unblocked_orphans.py | 22 ++++++++++++++++++++-- 1 files changed, 20 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index d8eb073..e2889e4 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -586,10 +586,11 @@ def maintainer_info(affected_people): return info -def package_info(packages, dep_map, pkgdb_dict, orphans=None, failed=None): +def package_info(unblocked, dep_map, pkgdb_dict, orphans=None, failed=None, + week_limit=6): info = "" - table, affected_people = maintainer_table(packages, pkgdb_dict) + table, affected_people = maintainer_table(unblocked, pkgdb_dict) info += table info += "\n\nThe following packages require above mentioned packages:\n" info += dependency_info(dep_map, affected_people, pkgdb_dict) @@ -617,12 +618,29 @@ def package_info(packages, dep_map, pkgdb_dict, orphans=None, failed=None): info += wrap_and_format("Orphans (dependend on)", orphans_breaking_deps) + orphans_breaking_deps_stale = [ + o for o in orphans_breaking_deps if + (pkgdb_dict[o].age.days / 7) >= week_limit] + + info += wrap_and_format( + "Orphans for at least {} weeks (dependend on)".format(week_limit), + orphans_breaking_deps_stale) + orphans_not_breaking_deps = [o for o in orphans if o not in dep_map or not dep_map[o]] info += wrap_and_format("Orphans (not depended on)", orphans_not_breaking_deps) + orphans_not_breaking_deps_stale = [ + o for o in orphans_not_breaking_deps if + (pkgdb_dict[o].age.days / 7) >= week_limit] + + info += wrap_and_format( + "Orphans for at least {} weeks (not dependend on)".format( + week_limit), + orphans_not_breaking_deps_stale) + breaking = set() for package, deps in dep_map.items(): breaking = breaking.union(set(deps.keys()))

1 0

[releng] find_unblocked_orphans: Limit to unblocked pkgs
by Till Maas 18 Oct '14

18 Oct '14

commit 8656f465b5769ce5f64e3d0969d4c99182fc44a8 Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 12:27:18 2014 +0200 find_unblocked_orphans: Limit to unblocked pkgs - Limit short reports to unblocked pkgs scripts/find_unblocked_orphans.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index b128e00..d8eb073 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -609,6 +609,7 @@ def package_info(packages, dep_map, pkgdb_dict, orphans=None, failed=None): return wrappedtext if orphans: + orphans = [o for o in orphans if o in unblocked] info += wrap_and_format("Orphans", orphans) orphans_breaking_deps = [o for o in orphans if

1 0

[releng] find_unblocked_orphans: Check packages after sorting
by Till Maas 18 Oct '14

18 Oct '14

commit e5ba69594362e20662763337285d77e04e307d3f Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 12:25:37 2014 +0200 find_unblocked_orphans: Check packages after sorting scripts/find_unblocked_orphans.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index bfd8f24..b128e00 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -447,7 +447,7 @@ class DepChecker(object): # dict for all dependent packages for each to-be-removed package dep_map = OrderedDict() - for name in packages: + for name in sorted(packages): sys.stderr.write("Checking: {0}\n".format(name)) ignore = rpm_pkg_names dep_map[name] = OrderedDict()

1 0

[releng] find_unblocked_orphans: Display orphan's age
by Till Maas 18 Oct '14

18 Oct '14

commit 666e88334c73683b8a5dacca26a1bd8fc9548537 Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 11:34:57 2014 +0200 find_unblocked_orphans: Display orphan's age scripts/find_unblocked_orphans.py | 40 +++++++++++++++++++++++++++--------- 1 files changed, 30 insertions(+), 10 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index bf94aea..bfd8f24 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -30,9 +30,9 @@ import yum try: import texttable - with_texttable = True + with_table = True except ImportError: - with_texttable = False + with_table = False EPEL5_RELEASE = dict( @@ -220,6 +220,21 @@ class PKGDBInfo(object): else: return [] + @property + def age(self): + now = datetime.datetime.utcnow() + age = now - self.status_change + return age + + @property + def status_change(self): + status_change = self.pkginfo["status_change"] + status_change = datetime.datetime.utcfromtimestamp(status_change) + return status_change + + def __getitem__(self, *args, **kwargs): + return self.pkginfo.__getitem__(*args, **kwargs) + def setup_yum(repo=RAWHIDE_RELEASE["repo"], source_repo=RAWHIDE_RELEASE["source_repo"]): @@ -511,26 +526,31 @@ class DepChecker(object): def maintainer_table(packages, pkgdb_dict): affected_people = {} - if with_texttable: + if with_table: table = texttable.Texttable(max_width=80) - table.header(["Package", "(co)maintainers"]) - table.set_cols_align(["l", "l"]) + table.header(["Package", "(co)maintainers", "age"]) + table.set_cols_align(["l", "l", "l"]) table.set_deco(table.HEADER) else: table = "" for package_name in packages: - people = pkgdb_dict[package_name].get_people() + pkginfo = pkgdb_dict[package_name] + people = pkginfo.get_people() for p in people: affected_people.setdefault(p, set()).add(package_name) p = ', '.join(people) + status_change = pkginfo.status_change + age = pkginfo.age + agestr = "{} ({} weeks)".format(status_change.strftime("%Y-%m-%d"), + age.days / 7) - if with_texttable: - table.add_row([package_name, p]) + if with_table: + table.add_row([package_name, p, agestr]) else: - table += "{0} {1}\n".format(package_name, p) + table += "{} {} {}\n".format(package_name, p, agestr) - if with_texttable: + if with_table: table = table.draw() return table, affected_people

1 0

[releng] find_unblocked_orphans: Cache complete pkgdb info
by Till Maas 18 Oct '14

18 Oct '14

commit e6f7b5f47d4019b095d23781fe251d618385544b Author: Till Maas <opensource(a)till.name> Date: Sat Oct 18 10:49:24 2014 +0200 find_unblocked_orphans: Cache complete pkgdb info scripts/find_unblocked_orphans.py | 111 ++++++++++++++++++++----------------- 1 files changed, 61 insertions(+), 50 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index d485d6d..bf94aea 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -179,35 +179,46 @@ def write_cache(data, filename, cachedir='~/.cache'): pickle.dump(data, pickle_file) -def get_people(package, branch=RAWHIDE_RELEASE["branch"]): - def associated(pkginfo, exclude=None): - """ - - :param exclude: People to exclude, e.g. the point of contact. - :type exclude: list - """ - other_people = set() - for acl in pkginfo.get("acls", []): - if acl["status"] == "Approved": - fas_name = acl["fas_name"] - if fas_name != "group::provenpackager" and \ - fas_name not in exclude: - other_people.add(fas_name) - return sorted(other_people) +class PKGDBInfo(object): + def __init__(self, package, branch=RAWHIDE_RELEASE["branch"]): + self.package = package + self.branch = branch - try: - pkginfo = pkgdb.get_package(package, branches=branch) - except Exception as e: - sys.stderr.write( - "Error getting maintainer of package {} on branch {}\n".format( - package, branch)) - # FIXME: Write proper traceback - sys.stderr.write(str(e)) - return [] - pkginfo = pkginfo["packages"][0] - people_ = [pkginfo["point_of_contact"]] - people_.extend(associated(pkginfo, exclude=people_)) - return people_ + try: + pkginfo = pkgdb.get_package(package, branches=branch) + except Exception as e: + sys.stderr.write( + "Error getting pkgdb info for {} on {}\n".format( + package, branch)) + # FIXME: Write proper traceback + sys.stderr.write(str(e)) + self.pkginfo = None + return + + self.pkginfo = pkginfo["packages"][0] + + def get_people(self): + def associated(pkginfo, exclude=None): + """ + + :param exclude: People to exclude, e.g. the point of contact. + :type exclude: list + """ + other_people = set() + for acl in pkginfo.get("acls", []): + if acl["status"] == "Approved": + fas_name = acl["fas_name"] + if fas_name != "group::provenpackager" and \ + fas_name not in exclude: + other_people.add(fas_name) + return sorted(other_people) + + if self.pkginfo is not None: + people_ = [self.pkginfo["point_of_contact"]] + people_.extend(associated(self.pkginfo, exclude=people_)) + return people_ + else: + return [] def setup_yum(repo=RAWHIDE_RELEASE["repo"], @@ -285,9 +296,9 @@ class DepChecker(object): yumbase = setup_yum(repo=RELEASES[release]["repo"], source_repo=RELEASES[release]["source_repo"]) self.yumbase = yumbase - self.people_queue = Queue() - self.people_cache = "orphans-people-{}.pickle".format(release) - self.people_dict = get_cache(self.people_cache, default={}) + self.pkgdbinfo_queue = Queue() + self.pkgdb_cache = "orphans-pkgdb-{}.pickle".format(release) + self.pkgdb_dict = get_cache(self.pkgdb_cache, default={}) def create_mapping(self): src_by_bin = {} # Dict of source pkg objects by binary package objects @@ -394,19 +405,19 @@ class DepChecker(object): prov) return OrderedDict(sorted(dependent_packages.items())) - def people_worker(self): + def pkgdb_worker(self): branch = RELEASES[self.release]["branch"] while True: - package = self.people_queue.get() - if package not in self.people_dict: - people_ = get_people(package, branch) - self.people_dict[package] = people_ - self.people_queue.task_done() + package = self.pkgdbinfo_queue.get() + if package not in self.pkgdb_dict: + pkginfo = PKGDBInfo(package, branch) + self.pkgdb_dict[package] = pkginfo + self.pkgdbinfo_queue.task_done() def recursive_deps(self, packages, max_deps=20): # Start threads to get information about (co)maintainers for packages for i in range(0, 2): - people_thread = Thread(target=self.people_worker) + people_thread = Thread(target=self.pkgdb_worker) people_thread.daemon = True people_thread.start() # keep pylint silent @@ -414,7 +425,7 @@ class DepChecker(object): # get a list of all rpm_pkgs that are to be removed rpm_pkg_names = [] for name in packages: - self.people_queue.put(name) + self.pkgdbinfo_queue.put(name) # Empty list if pkg is only for a different arch bin_pkgs = self.by_src.get(name, []) rpm_pkg_names.extend([p.name for p in bin_pkgs]) @@ -456,7 +467,7 @@ class DepChecker(object): ).setdefault(pkg, set()).add(dep) for srpm_name in new_srpm_names: - self.people_queue.put(srpm_name) + self.pkgdbinfo_queue.put(srpm_name) ignore.extend(new_names) if allow_more: @@ -473,9 +484,9 @@ class DepChecker(object): " completed\n".format(max_deps, name)) sys.stderr.write("Waiting for (co)maintainer information...") - self.people_queue.join() + self.pkgdbinfo_queue.join() sys.stderr.write("done\n") - write_cache(self.people_dict, self.people_cache) + write_cache(self.pkgdb_dict, self.pkgdb_cache) return dep_map # This function was stolen from pungi @@ -497,7 +508,7 @@ class DepChecker(object): sys.exit(1) -def maintainer_table(packages, people_dict): +def maintainer_table(packages, pkgdb_dict): affected_people = {} if with_texttable: @@ -509,7 +520,7 @@ def maintainer_table(packages, people_dict): table = "" for package_name in packages: - people = people_dict[package_name] + people = pkgdb_dict[package_name].get_people() for p in people: affected_people.setdefault(p, set()).add(package_name) p = ', '.join(people) @@ -524,14 +535,14 @@ def maintainer_table(packages, people_dict): return table, affected_people -def dependency_info(dep_map, affected_people, people_dict): +def dependency_info(dep_map, affected_people, pkgdb_dict): info = "" for package_name, subdict in dep_map.items(): if subdict: info += "Depending on: {} ({})\n".format(package_name, len(subdict.keys())) for fedora_package, dependent_packages in subdict.items(): - people = people_dict[fedora_package] + people = pkgdb_dict[fedora_package].get_people() for p in people: affected_people.setdefault(p, set()).add(package_name) p = ", ".join(people) @@ -555,13 +566,13 @@ def maintainer_info(affected_people): return info -def package_info(packages, dep_map, people_dict, orphans=None, failed=None): +def package_info(packages, dep_map, pkgdb_dict, orphans=None, failed=None): info = "" - table, affected_people = maintainer_table(packages, people_dict) + table, affected_people = maintainer_table(packages, pkgdb_dict) info += table info += "\n\nThe following packages require above mentioned packages:\n" - info += dependency_info(dep_map, affected_people, people_dict) + info += dependency_info(dep_map, affected_people, pkgdb_dict) info += "Affected (co)maintainers\n" info += maintainer_info(affected_people) @@ -656,7 +667,7 @@ def main(): # TODO: add app args to either depsolve or not dep_map = depchecker.recursive_deps(unblocked) sys.stderr.write('done\n') - info, addresses = package_info(unblocked, dep_map, depchecker.people_dict, + info, addresses = package_info(unblocked, dep_map, depchecker.pkgdb_dict, orphans=orphans, failed=failed) text += "\n" text += info

1 0

while true; do sleep 1h; rpm-ostree compose tree...; done
by Colin Walters 17 Oct '14

17 Oct '14

Hi, right now the Atomic host is kind of missing its most important part - the ability to receive updates. I am actually kind of confused as to what the blockers are, because it seems like it'd be basically a two line shell script in $SUBJECT run as a daemon. I've been doing this on the atomic01.qa.fedoraproject.org and pushing it out to alt.fp.org, but I'd like to figure out how we move forward on the official compose. I think Dennis said something about filing a ticket for a location? Where is that? Are there any other blockers? Storage? Tool documentation?

6 14

[PATCHES] system-release-product virtual provides and rename of -standard to -nonproduct
by Stephen Gallagher 16 Oct '14

16 Oct '14

Patches are for F21 branch, master will need a merge. We need to ensure that at least one of the fedora-release-[nonproduct| $PRODUCT] packages is always installed. Also, based on feedback on and off-list, we've just decided to run with -nonproduct rather than -standard (which everyone agreed was not good).

2 4

[releng] find_unblocked_orphans: Clarify instructions
by Till Maas 16 Oct '14

16 Oct '14

commit 7c7bc5bb25269e6e59a72bf51e1815f3d91d09d8 Author: Till Maas <opensource(a)till.name> Date: Thu Oct 16 18:30:26 2014 +0200 find_unblocked_orphans: Clarify instructions scripts/find_unblocked_orphans.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index b47ef2f..d485d6d 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -105,8 +105,8 @@ should be retired, please do so now with a proper reason: https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life Note: If you received this mail directly you (co)maintain one of the affected -packages or a package that depends on one. Please orphan the affected package -or retire your package to avoid broken dependencies. +packages or a package that depends on one. Please adopt the affected package or +retire your depending package to avoid broken dependencies. """ FOOTER = """-- \nThe script creating this output is run and developed by Fedora

1 0

Fedora Release Engineering meeting summary for 2014-10-13
by Dennis Gilmore 13 Oct '14

13 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 ====================================== #fedora-meeting-1: RELENG (2014-10-13) ====================================== Meeting started by dgilmore-bne at 14:34:27 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting-1/2014-10-13/releng.2014-10… . Meeting summary - --------------- * init process (dgilmore-bne, 14:34:47) * Secondary Architectures updates (dgilmore-bne, 14:41:06) * Secondary Architectures update - ppc (dgilmore-bne, 14:41:11) * Secondary Architectures update - s390 (dgilmore-bne, 14:48:41) * Secondary Architectures update - arm (dgilmore-bne, 14:50:25) * #5931 [Proposal] Move new branch and unretire requests to pkgdb2 (dgilmore-bne, 14:58:59) * LINK: https://fedorahosted.org/rel-eng/ticket/5931 (dgilmore-bne, 14:59:20) * open floor (dgilmore-bne, 15:28:34) Meeting ended at 15:33:57 UTC. Action Items - ------------ Action Items, by person - ----------------------- * **UNASSIGNED** * (none) People Present (lines said) - --------------------------- * dgilmore-bne (66) * pingou (29) * pbrobinson (22) * nirik (8) * zodbot (4) * sharkcz (4) * pbabinca (3) * bochecha (1) * tyll (0) * masta (0) * dgilmore (0) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJUPHcrAAoJEH7ltONmPFDR/cUP/28Yw2H4e80/NSGa2j+oAeMg BOqANdkt2aV98gz6Ut/p7a8SAYUpSbzD7+CILomSBfdEnLeswlwSzvKF2VizjvOD jNOK1teKmhCogyy5c3b4puuFZ5Kp1MwXtmjTotj7dBu/DSUL8VRQf6twJwpBwUn3 6Rtx4IVzqdgTestGeT0706HDDiLAE+sQbPa1w9/tfqoYp6L5ZgcGrqEe9DQdjIwF 0LukTjGyRrEC0PgTMYsQOd2sS5K55a/5Xvix+JqTn71psdNPU0HomlBf/QuWcuon KzosN6+/HxY5OAIXPiNEUJnq2bC8E+ClNQ+UMXRQ1rEp9MFSyTeTp5bk5g5HiisP 0vpNBdPUoPjn/fdnMlY+b1Ers6HD3aDNGjCAoEoqmsVG2tPMyhXmLthNKZX6MugP MPITb3sfSFuHHo1750LHIAhV+IQh5c93OtrypV/XeHKFLQ5R//9T5B3nEzKNdE8H I5jvx9yj4wwbUJQesqHHxiyDbNDadZdY7Slzf6yD6Cco4KekD/5klEp/ysoN3iQj zlr7phaJPz5Q4fwJgROBuSDGDnq64kA4FcS08WowNshlRxbf6+Ck7d9sbCsbe1kN KWGOkZ0SDBn9yynlUtLVxy1IWDOFuQCGUVlwEBHS56VzvV7sbaMcWhCo4hUVLIQd 8LVwX9E14Z5IxcF/+euP =YQmS -----END PGP SIGNATURE-----

1 0

[Fedocal] Reminder meeting : Fedora Release Engineering
by Dennis Gilmore 12 Oct '14

12 Oct '14

Dear all, You are kindly invited to the meeting: Fedora Release Engineering on 2014-10-13 from 15:30:00 to 16:30:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Fedora Release Engineering weekly meeting Source: https://apps.fedoraproject.org/calendar//meeting/162/

1 0

[releng] Fix line widths
by Till Maas 10 Oct '14

10 Oct '14

commit 45be26a7d7308840abeaef942822591a202a8860 Author: Till Maas <opensource(a)till.name> Date: Fri Oct 10 20:18:54 2014 +0200 Fix line widths scripts/find_unblocked_orphans.py | 22 +++++++++++++--------- 1 files changed, 13 insertions(+), 9 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 9687377..b47ef2f 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -71,7 +71,8 @@ RAWHIDE_RELEASE = dict( BRANCHED_RELEASE = dict( repo='https://kojipkgs.fedoraproject.org/mash/branched/i386/os', - source_repo='https://kojipkgs.fedoraproject.org/mash/branched/source/SRPMS', + source_repo='https://kojipkgs.fedoraproject.org/mash/branched/source/' + 'SRPMS', tag='f21', branch='f21', mailto='devel(a)lists.fedoraproject.org', @@ -98,14 +99,14 @@ According to https://fedoraproject.org/wiki/Schedule branching will occur not earlier than 2014-07-08. The packages will be retired shortly before. """ -HEADER = """The following packages are orphaned and might be retired eventually, -unless someone adopts them. If you know for sure that the package should be -retired, please do so now with a proper reason: +HEADER = """The following packages are orphaned and might be retired +eventually, unless someone adopts them. If you know for sure that the package +should be retired, please do so now with a proper reason: https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life Note: If you received this mail directly you (co)maintain one of the affected -packages or a package that depends on one. Please orphan the affected package or -retire your package to avoid broken dependencies. +packages or a package that depends on one. Please orphan the affected package +or retire your package to avoid broken dependencies. """ FOOTER = """-- \nThe script creating this output is run and developed by Fedora @@ -338,7 +339,8 @@ class DepChecker(object): rpms = self.by_src[srpmname] except KeyError: # If we don't have a package in the repo, there is nothing to do - sys.stderr.write("Package {0} not found in repo\n".format(srpmname)) + sys.stderr.write( + "Package {0} not found in repo\n".format(srpmname)) rpms = [] # provides of all packages built from ``srpmname`` @@ -490,7 +492,8 @@ class DepChecker(object): arch='src')[0] return srpmpo except IndexError: - print >> sys.stderr, "Error: Cannot find a source rpm for %s" % srpm + sys.stderr.write( + "Error: Cannot find a source rpm for {}\n".format(srpm)) sys.exit(1) @@ -579,7 +582,8 @@ def package_info(packages, dep_map, people_dict, orphans=None, failed=None): orphans_breaking_deps = [o for o in orphans if o in dep_map and dep_map[o]] - info += wrap_and_format("Orphans (dependend on)", orphans_breaking_deps) + info += wrap_and_format("Orphans (dependend on)", + orphans_breaking_deps) orphans_not_breaking_deps = [o for o in orphans if o not in dep_map or not dep_map[o]]

1 0

discussion on Fedora Council Engineering representative
by Matthew Miller 10 Oct '14

10 Oct '14

Spamming many lists to make sure this doesn't go unnoticed. (In case my accidentally posting to both devel and devel-announce separately doesn't do it.) Please see the thread titled "Engineering Representiatve for the new Fedora Council" on the devel list — this affects and should have the input of everyone involved in the broad category of "building and shipping the distro" efforts in Fedora. Thanks! -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader

1 0

Pkgdb2 - Update on new branch/package process
by Pierre-Yves Chibon 10 Oct '14

10 Oct '14

Dear all, Sorry for the long silence about this. Here is where we are now (in git, nothing merged/in prod atm): * user can request new packages on pkgdb * user can request a new branch for a package on pkgdb * these actions are recorded and stored in pkgdb Available at: http://209.132.184.188/admin/actions/ and http://209.132.184.188/api/admin/actions/ * there is a new pkgdb-admin part of pkgdb-cli allowing to list actions and act of them (approve (ie: process)/deny) -> the pkgdb UI only updates the status of the action, while the cli pkgdb-admin actually process the request (as well as updating the status of the action at the end) -> pkgdb-admin would be the place where we would add all the checks we want to do before processing a package/branch request fedmsg messages will be broadcasted for new branch. This part is all there and basically ready :) The second part is about listening to fedmsg message about new branches and adjusting the git repos accordingly. This is where things are a little more hairy. Currently, we have fedmsg-pkgdb-sync-git [1] that basically listens for pkgdb messages on the bus and run the ansible playbook: run_pkgdb_sync_git.yml [2] (which in turn runs pkgdb_sync_git_branches.py) when needed. The problem which I am facing is in the case of new EPEL branch creation. These branches are created from another one. For example in the case of the epel7 branch we used (are we still?) to branch either from either el6 or f19. With that in mind, I allowed in pkgdb to specify from which branch one would like to get the new branch. The problem is that this information is not available to fedmsg-pkgdb-sync-git and even if it was, we would not be able to pass it along to the ansible playbook. So afaics the options are: 1/ We hard-code somewhere from which branch each branch should come (ie: f* comes from master, el6 comes from ?, epel7 comes from f19...) 2/ We manage to create the branch in such a way that they point to the first commit in the repo (suggestion from Till), from there the maintainers should be able to fast-forward to the branch they think should apply 3/ We manage to just create the branch in gitolite and just let the maintainer populate it by merging the branch they think should apply I have a preference for #3, but our tests with bochecha for it didn't quite succeed. But in all 3 options, I basically can drop the `branch_from` field when requesting a new branch in pkgdb. So what is our preference and how do we want to proceed from here? Thanks, Pierre [1] https://github.com/fedora-infra/fedmsg-pkgdb-sync-git [2] https://infrastructure.fedoraproject.org/infra/ansible/playbooks/run_pkgdb_…

1 1

[releng] find_unblocked_orphans: refactor a little
by Till Maas 09 Oct '14

09 Oct '14

commit ff5c84609f0d8f8b03850d64d08dc2323a0eecfc Author: Till Maas <opensource(a)till.name> Date: Thu Oct 9 19:17:41 2014 +0200 find_unblocked_orphans: refactor a little scripts/find_unblocked_orphans.py | 32 +++++++++++++++++--------------- 1 files changed, 17 insertions(+), 15 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 2760397..9687377 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -552,27 +552,20 @@ def maintainer_info(affected_people): return info -def package_info(packages, release, orphans=None, failed=None): - sys.stderr.write("Setting up dependency checker...") - depchecker = DepChecker(release) - sys.stderr.write("done\n") +def package_info(packages, dep_map, people_dict, orphans=None, failed=None): info = "" - sys.stderr.write('Calculating dependencies...') - # Create yum object and depsolve out if requested. - # TODO: add app args to either depsolve or not - dep_map = depchecker.recursive_deps(packages) - sys.stderr.write('done\n') - table, affected_people = maintainer_table(packages, depchecker.people_dict) + table, affected_people = maintainer_table(packages, people_dict) info += table info += "\n\nThe following packages require above mentioned packages:\n" - info += dependency_info(dep_map, affected_people, depchecker.people_dict) + info += dependency_info(dep_map, affected_people, people_dict) info += "Affected (co)maintainers\n" info += maintainer_info(affected_people) wrapper = textwrap.TextWrapper( - break_long_words=False, subsequent_indent=" ", break_on_hyphens=False + break_long_words=False, subsequent_indent=" ", + break_on_hyphens=False ) def wrap_and_format(label, pkgs): @@ -624,7 +617,8 @@ def main(): parser.add_argument("--skip-orphans", dest="skip_orphans", help="Do not look for orphans", default=False, action="store_true") - parser.add_argument("--release", choices=RELEASES.keys(), default="rawhide") + parser.add_argument("--release", choices=RELEASES.keys(), + default="rawhide") parser.add_argument("--mailto", default=None, help="Send mail to this address (for testing)") parser.add_argument( @@ -650,8 +644,16 @@ def main(): sys.stderr.write('done\n') text = HEADER.format(RELEASES[args.release]["tag"].upper()) - info, addresses = package_info(unblocked, args.release, orphans=orphans, - failed=failed) + sys.stderr.write("Setting up dependency checker...") + depchecker = DepChecker(args.release) + sys.stderr.write("done\n") + sys.stderr.write('Calculating dependencies...') + # Create yum object and depsolve out if requested. + # TODO: add app args to either depsolve or not + dep_map = depchecker.recursive_deps(unblocked) + sys.stderr.write('done\n') + info, addresses = package_info(unblocked, dep_map, depchecker.people_dict, + orphans=orphans, failed=failed) text += "\n" text += info text += FOOTER

1 0

#6013: Cannot update dead.package file
by Fedora Release Engineering 08 Oct '14

08 Oct '14

#6013: Cannot update dead.package file ----------------------------+------------------------ Reporter: orion | Owner: rel-eng@… Type: defect | Status: new Milestone: Fedora 21 Beta | Component: git Keywords: | Blocked By: Blocking: | ----------------------------+------------------------ I was trying to update the dead.package file for dkim-milter, but get: {{{ Total 3 (delta 0), reused 0 (delta 0) remote: W refs/heads/master dkim-milter orion DENIED by fallthru remote: error: hook declined to update refs/heads/master To ssh://orion@pkgs.fedoraproject.org/dkim-milter ! [remote rejected] master -> master (hook declined) error: failed to push some refs to 'ssh://orion@pkgs.fedoraproject.org /dkim-milter' Could not execute commit: Command '['git', 'push']' returned non-zero exit status 1 }}} This is still needed at times to put better information into dead.package files. I am a proven packager. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6013> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

[releng] find_unblocked_orphans: Reword label
by Till Maas 07 Oct '14

07 Oct '14

commit 3c657271b37c136727eb29cbc06b83f7070ac70c Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 23:23:18 2014 +0200 find_unblocked_orphans: Reword label scripts/find_unblocked_orphans.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 071f7fe..2760397 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -599,7 +599,7 @@ def package_info(packages, release, orphans=None, failed=None): breaking = breaking.union(set(deps.keys())) if breaking: - info += wrap_and_format("Affected packages", sorted(breaking)) + info += wrap_and_format("Depending packages", sorted(breaking)) if failed: info += "\nFTBFS: " + " ".join(failed)

1 0

[releng] find_unblocked_orphans: Fix spelling
by Till Maas 07 Oct '14

07 Oct '14

commit c9275fc589ce0cc7d663da1721f234ebd774fddb Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 22:42:25 2014 +0200 find_unblocked_orphans: Fix spelling scripts/find_unblocked_orphans.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 63f5ad4..071f7fe 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -658,7 +658,7 @@ def main(): print text if args.mailto or args.send: - subject = "Orphan packages in " + args.release + subject = "Orphaned packages in " + args.release if args.mailto: mailto = args.mailto else:

1 0

[releng] find_unblocked_orphans: Show address count
by Till Maas 07 Oct '14

07 Oct '14

commit dbadea6b3a7bf0bdaa0bbc379518ae34337692df Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 22:36:55 2014 +0200 find_unblocked_orphans: Show address count scripts/find_unblocked_orphans.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 09a866c..63f5ad4 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -671,7 +671,8 @@ def main(): if mail_errors: sys.stderr.write("mail errors: " + repr(mail_errors) + "\n") - sys.stderr.write("Addresses: " + ", ".join(addresses) + "\n") + sys.stderr.write("Addresses ({}): {}\n".format(len(addresses), + ", ".join(addresses))) if __name__ == "__main__": main()

1 0

[releng] find_unblocked_orphans: Add more length details
by Till Maas 07 Oct '14

07 Oct '14

commit e9182fafeb32891913f9519ecab877fb8b093a33 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 22:34:45 2014 +0200 find_unblocked_orphans: Add more length details scripts/find_unblocked_orphans.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index ad633dc..09a866c 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -427,7 +427,8 @@ class DepChecker(object): allow_more = True seen = [] while True: - sys.stderr.write("to_check: {0}\n".format(repr(to_check))) + sys.stderr.write("to_check ({}): {}\n".format(len(to_check), + repr(to_check))) check_next = to_check.pop() seen.append(check_next) dependent_packages = self.find_dependent_packages(check_next, @@ -524,7 +525,8 @@ def dependency_info(dep_map, affected_people, people_dict): info = "" for package_name, subdict in dep_map.items(): if subdict: - info += "Depending on: %s\n" % package_name + info += "Depending on: {} ({})\n".format(package_name, + len(subdict.keys())) for fedora_package, dependent_packages in subdict.items(): people = people_dict[fedora_package] for p in people:

1 0

[releng] find_unblocked_orphans: Output currently checked package
by Till Maas 07 Oct '14

07 Oct '14

commit 844e47b2464bd03debb635396eb4f491f707619d Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 22:12:22 2014 +0200 find_unblocked_orphans: Output currently checked package scripts/find_unblocked_orphans.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 5964d1f..ad633dc 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -420,6 +420,7 @@ class DepChecker(object): # dict for all dependent packages for each to-be-removed package dep_map = OrderedDict() for name in packages: + sys.stderr.write("Checking: {0}\n".format(name)) ignore = rpm_pkg_names dep_map[name] = OrderedDict() to_check = [name]

1 0

[releng] find_unblocked_orphans: Output list of breaking packages
by Till Maas 07 Oct '14

07 Oct '14

commit 090d26b00bedcb257aa3040193c80374442ceb79 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 22:12:02 2014 +0200 find_unblocked_orphans: Output list of breaking packages scripts/find_unblocked_orphans.py | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 486b6f4..5964d1f 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -590,6 +590,14 @@ def package_info(packages, release, orphans=None, failed=None): info += wrap_and_format("Orphans (not depended on)", orphans_not_breaking_deps) + + breaking = set() + for package, deps in dep_map.items(): + breaking = breaking.union(set(deps.keys())) + + if breaking: + info += wrap_and_format("Affected packages", sorted(breaking)) + if failed: info += "\nFTBFS: " + " ".join(failed) info += "\n"

1 0

[releng] find_unblocked_orphans: Indent wrapped lines
by Till Maas 07 Oct '14

07 Oct '14

commit 7e434930c7d1c6b0e62c57d10c01ec5108caa376 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 22:11:43 2014 +0200 find_unblocked_orphans: Indent wrapped lines scripts/find_unblocked_orphans.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 0de345d..486b6f4 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -568,8 +568,9 @@ def package_info(packages, release, orphans=None, failed=None): info += "Affected (co)maintainers\n" info += maintainer_info(affected_people) - wrapper = textwrap.TextWrapper(break_long_words=False, - break_on_hyphens=False) + wrapper = textwrap.TextWrapper( + break_long_words=False, subsequent_indent=" ", break_on_hyphens=False + ) def wrap_and_format(label, pkgs): count = len(pkgs)

1 0

[releng] find_unblocked_orphans: Wrap list of orphans
by Till Maas 07 Oct '14

07 Oct '14

commit d4fb31b8b642948324641b3058ae7e223e0647a3 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 21:45:08 2014 +0200 find_unblocked_orphans: Wrap list of orphans scripts/find_unblocked_orphans.py | 27 +++++++++++++++++++-------- 1 files changed, 19 insertions(+), 8 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 2b004ee..0de345d 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -22,6 +22,7 @@ import hashlib import os import smtplib import sys +import textwrap import koji import pkgdb2client @@ -416,7 +417,7 @@ class DepChecker(object): bin_pkgs = self.by_src.get(name, []) rpm_pkg_names.extend([p.name for p in bin_pkgs]) - # dict for all dependent package for each to-be-removed package + # dict for all dependent packages for each to-be-removed package dep_map = OrderedDict() for name in packages: ignore = rpm_pkg_names @@ -566,18 +567,28 @@ def package_info(packages, release, orphans=None, failed=None): info += "Affected (co)maintainers\n" info += maintainer_info(affected_people) + + wrapper = textwrap.TextWrapper(break_long_words=False, + break_on_hyphens=False) + + def wrap_and_format(label, pkgs): + count = len(pkgs) + text = "{} ({}): {}".format(label, count, " ".join(pkgs)) + wrappedtext = "\n" + wrapper.fill(text) + "\n\n" + return wrappedtext + if orphans: - info += "\norphans: " + " ".join(orphans) - info += "\n\n" + info += wrap_and_format("Orphans", orphans) + orphans_breaking_deps = [o for o in orphans if o in dep_map and dep_map[o]] - info += "orphans (depended on): " + " ".join(orphans_breaking_deps) - info += "\n\n" + info += wrap_and_format("Orphans (dependend on)", orphans_breaking_deps) + orphans_not_breaking_deps = [o for o in orphans if o not in dep_map or not dep_map[o]] - info += "orphans (not depended on): " + " ".join( - orphans_not_breaking_deps) - info += "\n\n" + + info += wrap_and_format("Orphans (not depended on)", + orphans_not_breaking_deps) if failed: info += "\nFTBFS: " + " ".join(failed) info += "\n"

1 0

[releng] find_unblocked_orphans: Handle pkgdb2 errors
by Till Maas 07 Oct '14

07 Oct '14

commit 9bea7961aa127ca3c9d2d2250c7ab92d5c1129ab Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 20:46:22 2014 +0200 find_unblocked_orphans: Handle pkgdb2 errors Packages Fedora Branched without a branch for Fedora Branched cause an error when getting maintainer information. However, this should never happen: https://github.com/fedora-infra/pkgdb2/issues/95 scripts/find_unblocked_orphans.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index ceea03b..2b004ee 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -199,7 +199,9 @@ def get_people(package, branch=RAWHIDE_RELEASE["branch"]): sys.stderr.write( "Error getting maintainer of package {} on branch {}\n".format( package, branch)) + # FIXME: Write proper traceback sys.stderr.write(str(e)) + return [] pkginfo = pkginfo["packages"][0] people_ = [pkginfo["point_of_contact"]] people_.extend(associated(pkginfo, exclude=people_))

1 0

[releng] find_unblocked_orphans: Add newline
by Till Maas 07 Oct '14

07 Oct '14

commit 3c38698a21ee785689ef88a8b28b4b9c27abfa48 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 19:55:31 2014 +0200 find_unblocked_orphans: Add newline scripts/find_unblocked_orphans.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 43b8ad2..ceea03b 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -559,7 +559,7 @@ def package_info(packages, release, orphans=None, failed=None): table, affected_people = maintainer_table(packages, depchecker.people_dict) info += table - info += "\nThe following packages require above mentioned packages:\n" + info += "\n\nThe following packages require above mentioned packages:\n" info += dependency_info(dep_map, affected_people, depchecker.people_dict) info += "Affected (co)maintainers\n"

1 0

[releng] find_unblocked_orphans: Add debug info for exception
by Till Maas 07 Oct '14

07 Oct '14

commit 821dcda1bed274043cebef25b71986d0189fcbe4 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 19:55:09 2014 +0200 find_unblocked_orphans: Add debug info for exception scripts/find_unblocked_orphans.py | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 1ef656b..43b8ad2 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -193,7 +193,13 @@ def get_people(package, branch=RAWHIDE_RELEASE["branch"]): other_people.add(fas_name) return sorted(other_people) - pkginfo = pkgdb.get_package(package, branches=branch) + try: + pkginfo = pkgdb.get_package(package, branches=branch) + except Exception as e: + sys.stderr.write( + "Error getting maintainer of package {} on branch {}\n".format( + package, branch)) + sys.stderr.write(str(e)) pkginfo = pkginfo["packages"][0] people_ = [pkginfo["point_of_contact"]] people_.extend(associated(pkginfo, exclude=people_))

1 0

[releng] find_unblocked_orphans: Add email support
by Till Maas 07 Oct '14

07 Oct '14

commit ef1302571035960a6073b2f1d45b3a0705548a16 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 19:22:41 2014 +0200 find_unblocked_orphans: Add email support scripts/find_unblocked_orphans.py | 74 +++++++++++++++++++++++++++++++----- 1 files changed, 63 insertions(+), 11 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 195518c..1ef656b 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -17,8 +17,10 @@ from threading import Thread import argparse import cPickle as pickle import datetime +import email.mime.text import hashlib import os +import smtplib import sys import koji @@ -37,32 +39,42 @@ EPEL5_RELEASE = dict( source_repo='https://kojipkgs.fedoraproject.org/mash/updates/' 'el5-epel/SRPMS', tag='dist-5E-epel', - branch='el5') + branch='el5', + mailto='epel-devel(a)lists.fedoraproject.org', +) EPEL6_RELEASE = dict( repo='https://kojipkgs.fedoraproject.org/mash/updates/el6-epel/x86_64/', source_repo='https://kojipkgs.fedoraproject.org/mash/updates/' 'el6-epel/SRPMS', tag='dist-6E-epel', - branch='el6') + branch='el6', + mailto='epel-devel(a)lists.fedoraproject.org', +) EPEL7_RELEASE = dict( repo='https://kojipkgs.fedoraproject.org/mash/updates/epel7/x86_64/', source_repo='https://kojipkgs.fedoraproject.org/mash/updates/epel7/SRPMS', tag='epel7', - branch='epel7') + branch='epel7', + mailto='epel-devel(a)lists.fedoraproject.org', +) RAWHIDE_RELEASE = dict( repo='https://kojipkgs.fedoraproject.org/mash/rawhide/i386/os', source_repo='https://kojipkgs.fedoraproject.org/mash/rawhide/source/SRPMS', tag='f22', - branch='master') + branch='master', + mailto='devel(a)lists.fedoraproject.org', +) BRANCHED_RELEASE = dict( repo='https://kojipkgs.fedoraproject.org/mash/branched/i386/os', source_repo='https://kojipkgs.fedoraproject.org/mash/branched/source/SRPMS', tag='f21', - branch='f21') + branch='f21', + mailto='devel(a)lists.fedoraproject.org', +) RELEASES = { "rawhide": RAWHIDE_RELEASE, @@ -95,12 +107,30 @@ packages or a package that depends on one. Please orphan the affected package or retire your package to avoid broken dependencies. """ -FOOTER = """The script creating this output is run and developed by Fedora +FOOTER = """-- \nThe script creating this output is run and developed by Fedora Release Engineering. Please report issues at its trac instance: https://fedorahosted.org/rel-eng/ The sources of this script can be found at: https://git.fedorahosted.org/cgit/releng/tree/scripts/find_unblocked_orphan… """ + + +def send_mail(from_, to, subject, text, bcc=None): + if bcc is None: + bcc = [] + + msg = email.mime.text.MIMEText(text) + msg["Subject"] = subject + msg["From"] = from_ + msg["To"] = to + if isinstance(to, basestring): + to = [to] + smtp = smtplib.SMTP('127.0.0.1') + errors = smtp.sendmail(from_, to + bcc, msg.as_string()) + smtp.quit() + return errors + + pkgdb = pkgdb2client.PkgDB() @@ -555,7 +585,6 @@ def package_info(packages, release, orphans=None, failed=None): addresses = ["{0}(a)fedoraproject.org".format(p) for p in affected_people.keys() if p != ORPHAN_UID] - addresses = "Bcc: {0}\n".format(", ".join(addresses)) return info, addresses @@ -565,6 +594,13 @@ def main(): help="Do not look for orphans", default=False, action="store_true") parser.add_argument("--release", choices=RELEASES.keys(), default="rawhide") + parser.add_argument("--mailto", default=None, + help="Send mail to this address (for testing)") + parser.add_argument( + "--send", default=False, action="store_true", + help="Actually send mail including Bcc addresses to mailing list" + ) + parser.add_argument("--mailfrom", default="nobody(a)fedoraproject.org") parser.add_argument("failed", nargs="*", help="Additional packages, e.g. FTBFS packages") args = parser.parse_args() @@ -582,13 +618,29 @@ def main(): unblocked = unblocked_packages(sorted(list(set(list(orphans) + failed)))) sys.stderr.write('done\n') - print HEADER.format(RELEASES[args.release]["tag"].upper()) + text = HEADER.format(RELEASES[args.release]["tag"].upper()) info, addresses = package_info(unblocked, args.release, orphans=orphans, failed=failed) - print info - print FOOTER + text += "\n" + text += info + text += FOOTER + print text + + if args.mailto or args.send: + subject = "Orphan packages in " + args.release + if args.mailto: + mailto = args.mailto + else: + mailto = RELEASES[args.release]["mailto"] + if args.send: + bcc = addresses + else: + bcc = None + mail_errors = send_mail(args.mailfrom, mailto, subject, text, bcc) + if mail_errors: + sys.stderr.write("mail errors: " + repr(mail_errors) + "\n") - sys.stderr.write(addresses) + sys.stderr.write("Addresses: " + ", ".join(addresses) + "\n") if __name__ == "__main__": main()

1 0

[releng] find_unblocked_orphans: Make header generic
by Till Maas 07 Oct '14

07 Oct '14

commit 7798425326920a9cd01e2769986a36ab22cb055d Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 18:31:28 2014 +0200 find_unblocked_orphans: Make header generic scripts/find_unblocked_orphans.py | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index b4908f0..195518c 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -83,9 +83,16 @@ https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life According to https://fedoraproject.org/wiki/Schedule branching will occur not earlier than 2014-07-08. The packages will be retired shortly before. +""" + +HEADER = """The following packages are orphaned and might be retired eventually, +unless someone adopts them. If you know for sure that the package should be +retired, please do so now with a proper reason: +https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life Note: If you received this mail directly you (co)maintain one of the affected -packages or a package that depends on one. +packages or a package that depends on one. Please orphan the affected package or +retire your package to avoid broken dependencies. """ FOOTER = """The script creating this output is run and developed by Fedora @@ -575,7 +582,7 @@ def main(): unblocked = unblocked_packages(sorted(list(set(list(orphans) + failed)))) sys.stderr.write('done\n') - # print HEADER.format(RELEASES[args.release]["tag"].upper()) + print HEADER.format(RELEASES[args.release]["tag"].upper()) info, addresses = package_info(unblocked, args.release, orphans=orphans, failed=failed) print info

1 0

[releng] find_unblocked_orphans: Add newlines to output
by Till Maas 07 Oct '14

07 Oct '14

commit 8f31261d56cd39170069aa9abe58dd8eff032544 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 18:21:51 2014 +0200 find_unblocked_orphans: Add newlines to output scripts/find_unblocked_orphans.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index ee9cdc8..b4908f0 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -523,28 +523,28 @@ def package_info(packages, release, orphans=None, failed=None): info += maintainer_info(affected_people) if orphans: info += "\norphans: " + " ".join(orphans) - info += "\n" + info += "\n\n" orphans_breaking_deps = [o for o in orphans if o in dep_map and dep_map[o]] info += "orphans (depended on): " + " ".join(orphans_breaking_deps) - info += "\n" + info += "\n\n" orphans_not_breaking_deps = [o for o in orphans if o not in dep_map or not dep_map[o]] info += "orphans (not depended on): " + " ".join( orphans_not_breaking_deps) - info += "\n" + info += "\n\n" if failed: info += "\nFTBFS: " + " ".join(failed) info += "\n" ftbfs_breaking_deps = [o for o in failed if o in dep_map and dep_map[o]] info += "FTBFS (depended on): " + " ".join(ftbfs_breaking_deps) - info += "\n" + info += "\n\n" ftbfs_not_breaking_deps = [o for o in failed if o not in dep_map or not dep_map[o]] info += "FTBFS (not depended on): " + " ".join( ftbfs_not_breaking_deps) - info += "\n" + info += "\n\n" addresses = ["{0}(a)fedoraproject.org".format(p) for p in affected_people.keys() if p != ORPHAN_UID]

1 0

[releng] find_unblocked_orphans: Disable header
by Till Maas 07 Oct '14

07 Oct '14

commit 6768a25e46dec13fcfe1e0b4cf5fa36072a0f2bd Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 18:20:50 2014 +0200 find_unblocked_orphans: Disable header scripts/find_unblocked_orphans.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index f6dfa43..ee9cdc8 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -575,7 +575,7 @@ def main(): unblocked = unblocked_packages(sorted(list(set(list(orphans) + failed)))) sys.stderr.write('done\n') - print HEADER.format(RELEASES[args.release]["tag"].upper()) + # print HEADER.format(RELEASES[args.release]["tag"].upper()) info, addresses = package_info(unblocked, args.release, orphans=orphans, failed=failed) print info

1 0

[releng] find_unblocked_orphans: Sort imports
by Till Maas 07 Oct '14

07 Oct '14

commit 74a9109aaac54b8a77a30791ef0715a775de94b4 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 18:20:26 2014 +0200 find_unblocked_orphans: Sort imports scripts/find_unblocked_orphans.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 74d1f49..f6dfa43 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -11,15 +11,15 @@ # Jesse Keating <jkeating(a)redhat.com> # Till Maas <opensource(a)till.name> +from Queue import Queue from collections import OrderedDict +from threading import Thread +import argparse import cPickle as pickle import datetime import hashlib import os -import argparse -from Queue import Queue import sys -from threading import Thread import koji import pkgdb2client

1 0

[releng] find_unblocked_orphans: Support different releases
by Till Maas 07 Oct '14

07 Oct '14

commit a3194b4b0f7c58f998a51128a9a10b549a5ce981 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 18:19:24 2014 +0200 find_unblocked_orphans: Support different releases scripts/find_unblocked_orphans.py | 32 ++++++++++++++++++++++++++++---- 1 files changed, 28 insertions(+), 4 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index d8cf87c..74d1f49 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -32,6 +32,26 @@ except ImportError: with_texttable = False +EPEL5_RELEASE = dict( + repo='https://kojipkgs.fedoraproject.org/mash/updates/el5-epel/x86_64/', + source_repo='https://kojipkgs.fedoraproject.org/mash/updates/' + 'el5-epel/SRPMS', + tag='dist-5E-epel', + branch='el5') + +EPEL6_RELEASE = dict( + repo='https://kojipkgs.fedoraproject.org/mash/updates/el6-epel/x86_64/', + source_repo='https://kojipkgs.fedoraproject.org/mash/updates/' + 'el6-epel/SRPMS', + tag='dist-6E-epel', + branch='el6') + +EPEL7_RELEASE = dict( + repo='https://kojipkgs.fedoraproject.org/mash/updates/epel7/x86_64/', + source_repo='https://kojipkgs.fedoraproject.org/mash/updates/epel7/SRPMS', + tag='epel7', + branch='epel7') + RAWHIDE_RELEASE = dict( repo='https://kojipkgs.fedoraproject.org/mash/rawhide/i386/os', source_repo='https://kojipkgs.fedoraproject.org/mash/rawhide/source/SRPMS', @@ -47,6 +67,9 @@ BRANCHED_RELEASE = dict( RELEASES = { "rawhide": RAWHIDE_RELEASE, "branched": BRANCHED_RELEASE, + "epel7": EPEL7_RELEASE, + "epel6": EPEL6_RELEASE, + "epel5": EPEL5_RELEASE, } # pkgdb uid for orphan @@ -529,11 +552,12 @@ def package_info(packages, release, orphans=None, failed=None): return info, addresses -def main(release="rawhide"): +def main(): parser = argparse.ArgumentParser() parser.add_argument("--skip-orphans", dest="skip_orphans", help="Do not look for orphans", default=False, action="store_true") + parser.add_argument("--release", choices=RELEASES.keys(), default="rawhide") parser.add_argument("failed", nargs="*", help="Additional packages, e.g. FTBFS packages") args = parser.parse_args() @@ -544,15 +568,15 @@ def main(release="rawhide"): else: # list of orphans on the devel branch from pkgdb sys.stderr.write('Contacting pkgdb for list of orphans...') - orphans = orphan_packages(RELEASES[release]["branch"]) + orphans = sorted(orphan_packages(RELEASES[args.release]["branch"])) sys.stderr.write('done\n') sys.stderr.write('Getting builds from koji...') unblocked = unblocked_packages(sorted(list(set(list(orphans) + failed)))) sys.stderr.write('done\n') - print HEADER.format(RELEASES[release]["tag"].upper()) - info, addresses = package_info(unblocked, release, orphans=orphans, + print HEADER.format(RELEASES[args.release]["tag"].upper()) + info, addresses = package_info(unblocked, args.release, orphans=orphans, failed=failed) print info print FOOTER

1 0

[releng] find_unblocked_orphans: Use unique repo IDs
by Till Maas 07 Oct '14

07 Oct '14

commit 5d3230819707963039987059e0a186d20650e052 Author: Till Maas <opensource(a)till.name> Date: Tue Oct 7 18:18:42 2014 +0200 find_unblocked_orphans: Use unique repo IDs scripts/find_unblocked_orphans.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index aaaba8a..d8cf87c 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -14,6 +14,7 @@ from collections import OrderedDict import cPickle as pickle import datetime +import hashlib import os import argparse from Queue import Queue @@ -156,8 +157,10 @@ def setup_yum(repo=RAWHIDE_RELEASE["repo"], yb.conf.cache = 0 yb.repos.disableRepo('*') - yb.add_enable_repo('repo', [repo]) - yb.add_enable_repo('repo-source', [source_repo]) + # use digest to make repo id unique for each URL + yb.add_enable_repo('repo-' + hashlib.sha256(repo).hexdigest(), [repo]) + yb.add_enable_repo('repo-source-' + hashlib.sha256(repo).hexdigest(), + [source_repo]) yb.arch.archlist.append('src') return yb

1 0

[releng] find_unblocked_orphans: Make orphan_packages() generic
by Till Maas 06 Oct '14

06 Oct '14

commit d68aa61d4874666b9bda3bdec60205dfe233e257 Author: Till Maas <opensource(a)till.name> Date: Mon Oct 6 23:22:26 2014 +0200 find_unblocked_orphans: Make orphan_packages() generic scripts/find_unblocked_orphans.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index 8fb5193..aaaba8a 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -162,14 +162,15 @@ def setup_yum(repo=RAWHIDE_RELEASE["repo"], return yb -def orphan_packages(cache_filename='orphans.pickle'): +def orphan_packages(branch=RAWHIDE_RELEASE["branch"]): + cache_filename = 'orphans-{}.pickle'.format(branch) orphans = get_cache(cache_filename, default={}) if orphans: return orphans else: - pkgdbresponse = pkgdb.get_packages("", orphaned=True, - branches="master", page="all") + pkgdbresponse = pkgdb.get_packages( + "", orphaned=True, branches=branch, page="all") pkgs = pkgdbresponse["packages"] for p in pkgs: orphans[p["name"]] = p @@ -540,7 +541,7 @@ def main(release="rawhide"): else: # list of orphans on the devel branch from pkgdb sys.stderr.write('Contacting pkgdb for list of orphans...') - orphans = orphan_packages() + orphans = orphan_packages(RELEASES[release]["branch"]) sys.stderr.write('done\n') sys.stderr.write('Getting builds from koji...')

1 0

[releng] find_unblocked_orphans: Move people query to DepChecker()
by Till Maas 06 Oct '14

06 Oct '14

commit 369bfcec18f452604c640fb9f238cb4ea07402f9 Author: Till Maas <opensource(a)till.name> Date: Mon Oct 6 23:12:24 2014 +0200 find_unblocked_orphans: Move people query to DepChecker() scripts/find_unblocked_orphans.py | 74 +++++++++++++++++------------------- 1 files changed, 35 insertions(+), 39 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index cdbc02c..8fb5193 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -116,10 +116,6 @@ def write_cache(data, filename, cachedir='~/.cache'): pickle.dump(data, pickle_file) -people_queue = Queue() -people_dict = get_cache("orphans-people.pickle", default={}) - - def get_people(package, branch=RAWHIDE_RELEASE["branch"]): def associated(pkginfo, exclude=None): """ @@ -143,15 +139,6 @@ def get_people(package, branch=RAWHIDE_RELEASE["branch"]): return people_ -def people_worker(tag): - while True: - package = people_queue.get() - if package not in people_dict: - people_ = get_people(package, tag) - people_dict[package] = people_ - people_queue.task_done() - - def setup_yum(repo=RAWHIDE_RELEASE["repo"], source_repo=RAWHIDE_RELEASE["source_repo"]): """ Setup YumBase with two repos @@ -209,7 +196,6 @@ def unblocked_packages(packages, tagID=RAWHIDE_RELEASE["tag"]): [pkg] = result if not pkg[0]['blocked']: package_name = pkg[0]['package_name'] - people_queue.put(package_name) unblocked.append(package_name) else: print "ERROR: {pkgname}: {error}".format( @@ -218,10 +204,16 @@ def unblocked_packages(packages, tagID=RAWHIDE_RELEASE["tag"]): class DepChecker(object): - def __init__(self, yumbase): + def __init__(self, release): self._src_by_bin = None self._bin_by_src = None + self.release = release + yumbase = setup_yum(repo=RELEASES[release]["repo"], + source_repo=RELEASES[release]["source_repo"]) self.yumbase = yumbase + self.people_queue = Queue() + self.people_cache = "orphans-people-{}.pickle".format(release) + self.people_dict = get_cache(self.people_cache, default={}) def create_mapping(self): src_by_bin = {} # Dict of source pkg objects by binary package objects @@ -327,10 +319,27 @@ class DepChecker(object): prov) return OrderedDict(sorted(dependent_packages.items())) + def people_worker(self): + branch = RELEASES[self.release]["branch"] + while True: + package = self.people_queue.get() + if package not in self.people_dict: + people_ = get_people(package, branch) + self.people_dict[package] = people_ + self.people_queue.task_done() + def recursive_deps(self, packages, max_deps=20): + # Start threads to get information about (co)maintainers for packages + for i in range(0, 2): + people_thread = Thread(target=self.people_worker) + people_thread.daemon = True + people_thread.start() + # keep pylint silent + del i # get a list of all rpm_pkgs that are to be removed rpm_pkg_names = [] for name in packages: + self.people_queue.put(name) # Empty list if pkg is only for a different arch bin_pkgs = self.by_src.get(name, []) rpm_pkg_names.extend([p.name for p in bin_pkgs]) @@ -370,7 +379,7 @@ class DepChecker(object): ).setdefault(pkg, set()).add(dep) for srpm_name in new_srpm_names: - people_queue.put(srpm_name) + self.people_queue.put(srpm_name) ignore.extend(new_names) if allow_more: @@ -385,6 +394,11 @@ class DepChecker(object): sys.stderr.write("More than {0} broken deps for package" "'{1}', dependency check not" " completed\n".format(max_deps, name)) + + sys.stderr.write("Waiting for (co)maintainer information...") + self.people_queue.join() + sys.stderr.write("done\n") + write_cache(self.people_dict, self.people_cache) return dep_map # This function was stolen from pungi @@ -405,7 +419,7 @@ class DepChecker(object): sys.exit(1) -def maintainer_table(packages): +def maintainer_table(packages, people_dict): affected_people = {} if with_texttable: @@ -432,7 +446,7 @@ def maintainer_table(packages): return table, affected_people -def dependency_info(dep_map, affected_people): +def dependency_info(dep_map, affected_people, people_dict): info = "" for package_name, subdict in dep_map.items(): if subdict: @@ -463,12 +477,8 @@ def maintainer_info(affected_people): def package_info(packages, release, orphans=None, failed=None): - sys.stderr.write("Setting up yum...") - yumbase = setup_yum(repo=RELEASES[release]["repo"], - source_repo=RELEASES[release]["source_repo"]) - sys.stderr.write("done\n") sys.stderr.write("Setting up dependency checker...") - depchecker = DepChecker(yumbase) + depchecker = DepChecker(release) sys.stderr.write("done\n") info = "" sys.stderr.write('Calculating dependencies...') @@ -477,15 +487,10 @@ def package_info(packages, release, orphans=None, failed=None): dep_map = depchecker.recursive_deps(packages) sys.stderr.write('done\n') - sys.stderr.write("Waiting for (co)maintainer information...") - people_queue.join() - sys.stderr.write("done\n") - write_cache(people_dict, "orphans-people.pickle") - - table, affected_people = maintainer_table(packages) + table, affected_people = maintainer_table(packages, depchecker.people_dict) info += table info += "\nThe following packages require above mentioned packages:\n" - info += dependency_info(dep_map, affected_people) + info += dependency_info(dep_map, affected_people, depchecker.people_dict) info += "Affected (co)maintainers\n" info += maintainer_info(affected_people) @@ -538,15 +543,6 @@ def main(release="rawhide"): orphans = orphan_packages() sys.stderr.write('done\n') - # Start threads to get information about (co)maintainers for packages - for i in range(0, 2): - people_thread = Thread(target=people_worker, - args=(RELEASES[release]["tag"],)) - people_thread.daemon = True - people_thread.start() - # keep pylint silent - del i - sys.stderr.write('Getting builds from koji...') unblocked = unblocked_packages(sorted(list(set(list(orphans) + failed)))) sys.stderr.write('done\n')

1 0

[releng] find_unblocked_orphans: Use branch not tag
by Till Maas 06 Oct '14

06 Oct '14

commit bbe195e67e9a905deb83f9410d1466959e7c2bad Author: Till Maas <opensource(a)till.name> Date: Mon Oct 6 23:03:48 2014 +0200 find_unblocked_orphans: Use branch not tag scripts/find_unblocked_orphans.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index e22fc34..cdbc02c 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -120,7 +120,7 @@ people_queue = Queue() people_dict = get_cache("orphans-people.pickle", default={}) -def get_people(package, branch=RAWHIDE_RELEASE["tag"]): +def get_people(package, branch=RAWHIDE_RELEASE["branch"]): def associated(pkginfo, exclude=None): """

1 0

[releng] find_unblocked_orphans: Use not in
by Till Maas 06 Oct '14

06 Oct '14

commit 5a3c9f38eb7f301798a90144f7f5cf3bf37d6e6c Author: Till Maas <opensource(a)till.name> Date: Mon Oct 6 22:26:16 2014 +0200 find_unblocked_orphans: Use not in scripts/find_unblocked_orphans.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index f588cb4..e22fc34 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -497,7 +497,7 @@ def package_info(packages, release, orphans=None, failed=None): info += "orphans (depended on): " + " ".join(orphans_breaking_deps) info += "\n" orphans_not_breaking_deps = [o for o in orphans if - not o in dep_map or not dep_map[o]] + o not in dep_map or not dep_map[o]] info += "orphans (not depended on): " + " ".join( orphans_not_breaking_deps) info += "\n" @@ -509,7 +509,7 @@ def package_info(packages, release, orphans=None, failed=None): info += "FTBFS (depended on): " + " ".join(ftbfs_breaking_deps) info += "\n" ftbfs_not_breaking_deps = [o for o in failed if - not o in dep_map or not dep_map[o]] + o not in dep_map or not dep_map[o]] info += "FTBFS (not depended on): " + " ".join( ftbfs_not_breaking_deps) info += "\n"

1 0

[releng] find_unblocked_orphans: Introduce DepChecker()
by Till Maas 06 Oct '14

06 Oct '14

commit ce29f0dfdd4cdc4f80a80422c8c7cd7a729bf492 Author: Till Maas <opensource(a)till.name> Date: Mon Oct 6 22:23:39 2014 +0200 find_unblocked_orphans: Introduce DepChecker() Make it easier to check for other releases than Rawhide. scripts/find_unblocked_orphans.py | 375 +++++++++++++++++++------------------ 1 files changed, 191 insertions(+), 184 deletions(-) --- diff --git a/scripts/find_unblocked_orphans.py b/scripts/find_unblocked_orphans.py index b1ee20a..f588cb4 100755 --- a/scripts/find_unblocked_orphans.py +++ b/scripts/find_unblocked_orphans.py @@ -30,24 +30,29 @@ try: except ImportError: with_texttable = False -# Set some variables -# Some of these could arguably be passed in as args. -# If this is pre-branch, these repos should be rawhide; otherwise, -# they should be branched. -DEFAULT_REPO = 'http://kojipkgs.fedoraproject.org/mash/rawhide/i386/os' -DEFAULT_SOURCE_REPO = \ - 'http://kojipkgs.fedoraproject.org/mash/rawhide/source/SRPMS' -TAG = 'f21' # tag to check in koji - -# pre-branch, this should be master'. Post-branch, it will be something like -# fXY -RAWHIDE_BRANCHNAME = 'master' # pkgdb name for the devel branch + +RAWHIDE_RELEASE = dict( + repo='https://kojipkgs.fedoraproject.org/mash/rawhide/i386/os', + source_repo='https://kojipkgs.fedoraproject.org/mash/rawhide/source/SRPMS', + tag='f22', + branch='master') + +BRANCHED_RELEASE = dict( + repo='https://kojipkgs.fedoraproject.org/mash/branched/i386/os', + source_repo='https://kojipkgs.fedoraproject.org/mash/branched/source/SRPMS', + tag='f21', + branch='f21') + +RELEASES = { + "rawhide": RAWHIDE_RELEASE, + "branched": BRANCHED_RELEASE, +} # pkgdb uid for orphan ORPHAN_UID = 'orphan' HEADER = """The following packages are orphaned or did not build for two -releases and will be retired when Fedora ({0}) is branched, unless someone +releases and will be retired when Fedora ({}) is branched, unless someone adopts them. If you know for sure that the package should be retired, please do so now with a proper reason: https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life @@ -57,7 +62,7 @@ occur not earlier than 2014-07-08. The packages will be retired shortly before. Note: If you received this mail directly you (co)maintain one of the affected packages or a package that depends on one. -""".format(TAG.upper()) +""" FOOTER = """The script creating this output is run and developed by Fedora Release Engineering. Please report issues at its trac instance: @@ -115,7 +120,7 @@ people_queue = Queue() people_dict = get_cache("orphans-people.pickle", default={}) -def get_people(package, branch=RAWHIDE_BRANCHNAME): +def get_people(package, branch=RAWHIDE_RELEASE["tag"]): def associated(pkginfo, exclude=None): """ @@ -138,16 +143,17 @@ def get_people(package, branch=RAWHIDE_BRANCHNAME): return people_ -def people_worker(): +def people_worker(tag): while True: package = people_queue.get() if package not in people_dict: - people_ = get_people(package) + people_ = get_people(package, tag) people_dict[package] = people_ people_queue.task_done() -def setup_yum(repo=DEFAULT_REPO, source_repo=DEFAULT_SOURCE_REPO): +def setup_yum(repo=RAWHIDE_RELEASE["repo"], + source_repo=RAWHIDE_RELEASE["source_repo"]): """ Setup YumBase with two repos This code was mostly stolen from http://yum.baseurl.org/wiki/YumCodeSnippet/SetupArbitraryRepo @@ -168,28 +174,6 @@ def setup_yum(repo=DEFAULT_REPO, source_repo=DEFAULT_SOURCE_REPO): yb.arch.archlist.append('src') return yb -sys.stderr.write("Setting up yum...") -yb = setup_yum() -sys.stderr.write("done\n") - - -# This function was stolen from pungi -def SRPM(package): - """Given a package object, get a package object for the - corresponding source rpm. Requires yum still configured - and a valid package object.""" - srpm = package.sourcerpm.split('.src.rpm')[0] - (sname, sver, srel) = srpm.rsplit('-', 2) - try: - srpmpo = yb.pkgSack.searchNevra(name=sname, - ver=sver, - rel=srel, - arch='src')[0] - return srpmpo - except IndexError: - print >> sys.stderr, "Error: Cannot find a source rpm for %s" % srpm - sys.exit(1) - def orphan_packages(cache_filename='orphans.pickle'): orphans = get_cache(cache_filename, default={}) @@ -209,7 +193,7 @@ def orphan_packages(cache_filename='orphans.pickle'): return orphans -def unblocked_packages(packages, tagID=TAG): +def unblocked_packages(packages, tagID=RAWHIDE_RELEASE["tag"]): unblocked = [] kojisession = koji.ClientSession('https://koji.fedoraproject.org/kojihub') @@ -233,21 +217,22 @@ def unblocked_packages(packages, tagID=TAG): return unblocked -class BinSrcMapper(object): - def __init__(self): +class DepChecker(object): + def __init__(self, yumbase): self._src_by_bin = None self._bin_by_src = None + self.yumbase = yumbase def create_mapping(self): src_by_bin = {} # Dict of source pkg objects by binary package objects bin_by_src = {} # Dict of binary pkgobjects by srpm name - all_packages = yb.pkgSack.returnPackages() + all_packages = self.yumbase.pkgSack.returnPackages() # Populate the dicts for rpm_package in all_packages: if rpm_package.arch == 'src': continue - srpm = SRPM(rpm_package) + srpm = self.SRPM(rpm_package) src_by_bin[rpm_package] = srpm if srpm.name in bin_by_src: bin_by_src[srpm.name].append(rpm_package) @@ -269,142 +254,155 @@ class BinSrcMapper(object): self.create_mapping() return self._src_by_bin -sys.stderr.write("Setting up packager mapper...") -package_mapper = BinSrcMapper() -sys.stderr.write("done\n") - + def find_dependent_packages(self, srpmname, ignore): + """ Return packages depending on packages built from SRPM ``srpmname`` + that are built from different SRPMS not specified in ``ignore``. -def find_dependent_packages(srpmname, ignore): - """ Return packages depending on packages built from SRPM ``srpmname`` - that are built from different SRPMS not specified in ``ignore``. + :param ignore: list of SRPMs of packages that will not be returned + as dependent packages. + :type ignore: list() of str() - :param ignore: list of SRPMs of packages that will not be returned as - dependent packages. - :type ignore: list() of str() - - :returns: OrderedDict dependent_package: list of requires only provided - by package ``srpmname`` - {dep_pkg: [prov, ...]} - """ - # Some of this code was stolen from repoquery - dependent_packages = {} + :returns: OrderedDict dependent_package: list of requires only + provided by package ``srpmname`` {dep_pkg: [prov, ...]} + """ + # Some of this code was stolen from repoquery + dependent_packages = {} - # Handle packags not found in the repo - try: - rpms = package_mapper.by_src[srpmname] - except KeyError: - # If we don't have a package in the repo, there is nothing to do - sys.stderr.write("Package {0} not found in repo\n".format(srpmname)) - rpms = [] - - # provides of all packages built from ``srpmname`` - provides = [] - for pkg in rpms: - # add all the provides from the package as strings - string_provides = [yum.misc.prco_tuple_to_string(prov) - for prov in pkg.provides] - provides.extend(string_provides) - - # add all files as provides - # pkg.files is a dict with keys like "file" and "dir" - # values are a list of file/dir paths - for paths in pkg.files.itervalues(): - # sometimes paths start with "//" instead of "/" - # normalise "//" to "/": - # os.path.normpath("//") == "//", but - # os.path.normpath("///") == "/" - file_provides = [os.path.normpath('//%s' % fn) - for fn in paths] - provides.extend(file_provides) - - # Zip through the provides and find what's needed - for prov in provides: - # check only base provide, ignore specific versions - # "foo = 1.fc20" -> "foo" - base_provide = prov.split()[0] - - # Elide provide if also provided by another package - for pkg in yb.pkgSack.searchProvides(base_provide): - # FIXME: might miss broken dependencies in case the other provider - # depends on a to-be-removed package as well - if pkg.sourcerpm.rsplit('-', 2)[0] not in ignore: - break - else: - for dependent_pkg in yb.pkgSack.searchRequires(base_provide): - # skip if the dependent rpm package belongs to the - # to-be-removed Fedora package - if dependent_pkg in package_mapper.by_src[srpmname]: - continue - - # skip if the dependent rpm package is also a - # package that should be removed - if dependent_pkg.name in ignore: - continue - - # use setdefault to either create an entry for the - # dependent package or add the required prov - dependent_packages.setdefault(dependent_pkg, set()).add(prov) - return OrderedDict(sorted(dependent_packages.items())) - - -def recursive_deps(packages, max_deps=20): - # get a list of all rpm_pkgs that are to be removed - rpm_pkg_names = [] - for name in packages: - # Empty list if pkg is only for a different arch - bin_pkgs = package_mapper.by_src.get(name, []) - rpm_pkg_names.extend([p.name for p in bin_pkgs]) - - # dict for all dependent package for each to-be-removed package - dep_map = OrderedDict() - for name in packages: - ignore = rpm_pkg_names - dep_map[name] = OrderedDict() - to_check = [name] - allow_more = True - seen = [] - while True: - sys.stderr.write("to_check: {0}\n".format(repr(to_check))) - check_next = to_check.pop() - seen.append(check_next) - dependent_packages = find_dependent_packages(check_next, ignore) - if dependent_packages: - new_names = [] - new_srpm_names = set() - for pkg, dependencies in dependent_packages.items(): - if pkg.arch != "src": - srpm_name = package_mapper.by_bin[pkg].name - else: - srpm_name = pkg.name - if srpm_name not in to_check and \ - srpm_name not in new_names and \ - srpm_name not in seen: - new_names.append(srpm_name) - new_srpm_names.add(srpm_name) - - for dep in dependencies: - dep_map[name].setdefault( - srpm_name, - OrderedDict() - ).setdefault(pkg, set()).add(dep) - - for srpm_name in new_srpm_names: - people_queue.put(srpm_name) - - ignore.extend(new_names) - if allow_more: - to_check.extend(new_names) - dep_count = len(set(dep_map[name].keys() + to_check)) - if dep_count > max_deps: - allow_more = False - to_check = to_check[0:max_deps] - if not to_check: - break - if not allow_more: - sys.stderr.write("More than {0} broken deps for package" - "'{1}', dependency check not" - " completed\n".format(max_deps, name)) - return dep_map + # Handle packags not found in the repo + try: + rpms = self.by_src[srpmname] + except KeyError: + # If we don't have a package in the repo, there is nothing to do + sys.stderr.write("Package {0} not found in repo\n".format(srpmname)) + rpms = [] + + # provides of all packages built from ``srpmname`` + provides = [] + for pkg in rpms: + # add all the provides from the package as strings + string_provides = [yum.misc.prco_tuple_to_string(prov) + for prov in pkg.provides] + provides.extend(string_provides) + + # add all files as provides + # pkg.files is a dict with keys like "file" and "dir" + # values are a list of file/dir paths + for paths in pkg.files.itervalues(): + # sometimes paths start with "//" instead of "/" + # normalise "//" to "/": + # os.path.normpath("//") == "//", but + # os.path.normpath("///") == "/" + file_provides = [os.path.normpath('//%s' % fn) + for fn in paths] + provides.extend(file_provides) + + # Zip through the provides and find what's needed + for prov in provides: + # check only base provide, ignore specific versions + # "foo = 1.fc20" -> "foo" + base_provide = prov.split()[0] + + # Elide provide if also provided by another package + for pkg in self.yumbase.pkgSack.searchProvides(base_provide): + # FIXME: might miss broken dependencies in case the other + # provider depends on a to-be-removed package as well + if pkg.sourcerpm.rsplit('-', 2)[0] not in ignore: + break + else: + for dependent_pkg in self.yumbase.pkgSack.searchRequires( + base_provide): + # skip if the dependent rpm package belongs to the + # to-be-removed Fedora package + if dependent_pkg in self.by_src[srpmname]: + continue + + # skip if the dependent rpm package is also a + # package that should be removed + if dependent_pkg.name in ignore: + continue + + # use setdefault to either create an entry for the + # dependent package or add the required prov + dependent_packages.setdefault(dependent_pkg, set()).add( + prov) + return OrderedDict(sorted(dependent_packages.items())) + + def recursive_deps(self, packages, max_deps=20): + # get a list of all rpm_pkgs that are to be removed + rpm_pkg_names = [] + for name in packages: + # Empty list if pkg is only for a different arch + bin_pkgs = self.by_src.get(name, []) + rpm_pkg_names.extend([p.name for p in bin_pkgs]) + + # dict for all dependent package for each to-be-removed package + dep_map = OrderedDict() + for name in packages: + ignore = rpm_pkg_names + dep_map[name] = OrderedDict() + to_check = [name] + allow_more = True + seen = [] + while True: + sys.stderr.write("to_check: {0}\n".format(repr(to_check))) + check_next = to_check.pop() + seen.append(check_next) + dependent_packages = self.find_dependent_packages(check_next, + ignore) + if dependent_packages: + new_names = [] + new_srpm_names = set() + for pkg, dependencies in dependent_packages.items(): + if pkg.arch != "src": + srpm_name = self.by_bin[pkg].name + else: + srpm_name = pkg.name + if srpm_name not in to_check and \ + srpm_name not in new_names and \ + srpm_name not in seen: + new_names.append(srpm_name) + new_srpm_names.add(srpm_name) + + for dep in dependencies: + dep_map[name].setdefault( + srpm_name, + OrderedDict() + ).setdefault(pkg, set()).add(dep) + + for srpm_name in new_srpm_names: + people_queue.put(srpm_name) + + ignore.extend(new_names) + if allow_more: + to_check.extend(new_names) + dep_count = len(set(dep_map[name].keys() + to_check)) + if dep_count > max_deps: + allow_more = False + to_check = to_check[0:max_deps] + if not to_check: + break + if not allow_more: + sys.stderr.write("More than {0} broken deps for package" + "'{1}', dependency check not" + " completed\n".format(max_deps, name)) + return dep_map + + # This function was stolen from pungi + def SRPM(self, package): + """Given a package object, get a package object for the + corresponding source rpm. Requires yum still configured + and a valid package object.""" + srpm = package.sourcerpm.split('.src.rpm')[0] + (sname, sver, srel) = srpm.rsplit('-', 2) + try: + srpmpo = self.yumbase.pkgSack.searchNevra(name=sname, + ver=sver, + rel=srel, + arch='src')[0] + return srpmpo + except IndexError: + print >> sys.stderr, "Error: Cannot find a source rpm for %s" % srpm + sys.exit(1) def maintainer_table(packages): @@ -464,12 +462,19 @@ def maintainer_info(affected_people): return info -def package_info(packages, orphans=None, failed=None): +def package_info(packages, release, orphans=None, failed=None): + sys.stderr.write("Setting up yum...") + yumbase = setup_yum(repo=RELEASES[release]["repo"], + source_repo=RELEASES[release]["source_repo"]) + sys.stderr.write("done\n") + sys.stderr.write("Setting up dependency checker...") + depchecker = DepChecker(yumbase) + sys.stderr.write("done\n") info = "" sys.stderr.write('Calculating dependencies...') # Create yum object and depsolve out if requested. # TODO: add app args to either depsolve or not - dep_map = recursive_deps(packages) + dep_map = depchecker.recursive_deps(packages) sys.stderr.write('done\n') sys.stderr.write("Waiting for (co)maintainer information...") @@ -515,7 +520,7 @@ def package_info(packages, orphans=None, failed=None): return info, addresses -def main(): +def main(release="rawhide"): parser = argparse.ArgumentParser() parser.add_argument("--skip-orphans", dest="skip_orphans", help="Do not look for orphans", @@ -535,7 +540,8 @@ def main(): # Start threads to get information about (co)maintainers for packages for i in range(0, 2): - people_thread = Thread(target=people_worker) + people_thread = Thread(target=people_worker, + args=(RELEASES[release]["tag"],)) people_thread.daemon = True people_thread.start() # keep pylint silent @@ -545,8 +551,9 @@ def main(): unblocked = unblocked_packages(sorted(list(set(list(orphans) + failed)))) sys.stderr.write('done\n') - print HEADER - info, addresses = package_info(unblocked, orphans=orphans, failed=failed) + print HEADER.format(RELEASES[release]["tag"].upper()) + info, addresses = package_info(unblocked, release, orphans=orphans, + failed=failed) print info print FOOTER

1 0

Fedora Release Engineering meeting summary for 2014-10-06
by Dennis Gilmore 06 Oct '14

06 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 ====================================== #fedora-meeting-1: RELENG (2014-10-06) ====================================== Meeting started by dgilmore at 15:32:33 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting-1/2014-10-06/releng.2014-10… . Meeting summary - --------------- * init process (dgilmore, 15:32:44) * #6014 multiple builds in primary koji are lackign arm buildroot info (dgilmore, 15:34:01) * ACTION: all rebuild all effected packages (dgilmore, 15:36:47) * #5886 need method for distributing urgent fixes... urgently (dgilmore, 15:38:00) * LINK: https://fedorahosted.org/rel-eng/ticket/5886 (dgilmore, 15:38:03) * LINK: https://fedoraproject.org/wiki/Urgent_updates_policy (dgilmore, 15:40:42) * LINK: https://fedoraproject.org/wiki/Urgent_updates_policy (tyll, 15:41:42) * nothing new (tyll, 15:42:15) * #5775 switch to partitioned image for EC2; remove editing of menu.lst from image processing (dgilmore, 15:43:01) * LINK: https://fedorahosted.org/rel-eng/ticket/5775 (dgilmore, 15:43:07) * #5963 Orphaned vulnerable packages in EPEL (dgilmore, 15:46:56) * LINK: https://fedorahosted.org/rel-eng/ticket/5963 (dgilmore, 15:47:02) * ACTION: tyll to work on reporting dependncies to the list for the packages listed in this ticket and follow the fedora orphaned packages practices (dgilmore, 15:58:52) * Secondary Architectures updates (dgilmore, 16:00:02) * Secondary Architectures update - ppc (dgilmore, 16:00:04) * Secondary Architectures update - s390 (dgilmore, 16:05:52) * Secondary Architectures update - arm (dgilmore, 16:10:02) * Open Floor (dgilmore, 16:19:24) Meeting ended at 16:36:00 UTC. Action Items - ------------ * all rebuild all effected packages * tyll to work on reporting dependncies to the list for the packages listed in this ticket and follow the fedora orphaned packages practices Action Items, by person - ----------------------- * tyll * tyll to work on reporting dependncies to the list for the packages listed in this ticket and follow the fedora orphaned packages practices * **UNASSIGNED** * all rebuild all effected packages People Present (lines said) - --------------------------- * dgilmore (100) * nirik (43) * tyll (25) * pbrobinson (22) * sharkcz (14) * masta (9) * zodbot (4) * bochecha (2) * sgallagh (1) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJUMsU3AAoJEH7ltONmPFDR5qgP/2Jpyp1Qv36jcKB/eHyw7Whx IvmLdJyqS3KXs7rfWGIawaNY9bExbQdiCQh2m68vkeClvOpGEThSH6+1vhP2e+EA A8jjk1b5Pa0WJ3PYfC7JZ5rJsMkZYQghQB0KtBHw3+mBXUhHRKjMdR9SE+/ZssCx Az+Sn9bJ7EjOLJVSkZPMXvOyKCvU7RysJmmMdZ3RMmUyqBMZiSi5rjiPp/IQXUKO KnFG+xvta3mJYt3ntodWab7pS3DSf5GhqpEZ4iJitmnKhwkmUHotnz0SVNbt0yVJ gxYgqgBbb9q7CN4/4HOBChcHxF7iYWoqTeZj8Qf477l30NODF1eHNPPNo5kL2b6x MI2dZ0AAGhnoNkScS+ZSwG2t1LsfRIlHXC6+cSLdbhajjB+qc8Lq2nXwTL8u+Tir Vbwms6bd1VLkhpbxnB3idwl7hHeSg58mX7POUJkoBmJVx6FtrGpoAdUM68qmL7GD JFcPs7UjL2HNrOlqh4cmFRlUk99evsvmEKvF4d5yLyk944/mV0c4tlKRYnX6YVqz xMB6tLwPjF5pDuuiMKR7Lk/4Sa1GKjfQfjoF7GzjotPu4GpPDklw4No8yeCZ+DxL xwC6bPOf7lRWqhsqGOoRb7MLOrjfL/v1Due6wcmeJpT3/KhaQ3RgOvR3Ovzx2dvi BEelJIv5eF2XY9N7QcaQ =fd6C -----END PGP SIGNATURE-----

1 0

#6000: automate creation of excludelist for koji-shadow
by Fedora Release Engineering 06 Oct '14

06 Oct '14

#6000: automate creation of excludelist for koji-shadow -----------------------------+------------------------ Reporter: sharkcz | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ koj-shadow depends on excludelist to skip packages that contain ExcludeArch or ExclusiveArch tags making the package fail to build on secondary arch. The list is created by the srpm-exlcuded-arch.py script from releng git repo. It would be useful to automate the runs of this script and send the results to the releng list for example once in a week. This way the secondary arch maintainers could keep their copies up-to- date. -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6000> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 2

[Fedocal] Reminder meeting : Fedora Release Engineering
by Dennis Gilmore 05 Oct '14

05 Oct '14

Dear all, You are kindly invited to the meeting: Fedora Release Engineering on 2014-10-06 from 15:30:00 to 16:30:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Fedora Release Engineering weekly meeting Source: https://apps.fedoraproject.org/calendar//meeting/162/

1 0

#5914: Move fedmsg based blocking service to Fedora Infrastructure
by Fedora Release Engineering 04 Oct '14

04 Oct '14

#5914: Move fedmsg based blocking service to Fedora Infrastructure -----------------------------+------------------------ Reporter: till | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 21 Final | Component: koji Keywords: meeting | Blocked By: Blocking: | -----------------------------+------------------------ Move till's script to block retired packages in kojo to Fedora's infrastructure -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5914> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 4

fedora-release patch: Add system-release-product virtual Provides and Requires
by Stephen Gallagher 03 Oct '14

03 Oct '14

Patch is for F21 branch, but should work on master. We need to ensure that at least one of the fedora-release-[standard| $PRODUCT] packages is always installed.

2 1

[releng] buildepelbeta: Remove tabs
by Till Maas 01 Oct '14

01 Oct '14

commit 93852d2c8b815ebc6f57654cc402efedb6a3db3b Author: Till Maas <opensource(a)till.name> Date: Wed Oct 1 21:47:54 2014 +0200 buildepelbeta: Remove tabs scripts/buildepelbeta | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) --- diff --git a/scripts/buildepelbeta b/scripts/buildepelbeta index 3cd2cfa..8223a4d 100755 --- a/scripts/buildepelbeta +++ b/scripts/buildepelbeta @@ -1,7 +1,7 @@ #!/bin/bash # Copyright (C) 2013 Red Hat Inc. -# SPDX-License-Identifier: GPL-2.0+ +# SPDX-License-Identifier: GPL-2.0+ # runs currently on relepel01.phx2.fedoraproject.org @@ -12,8 +12,8 @@ MOCKCONFIG=rhel-compose-x86_64 [ -z "$DATE" ] && { - echo "usage: buildepelbeta <date>" - exit 1 + echo "usage: buildepelbeta <date>" + exit 1 } TMPDIR=`mktemp -d /tmp/epel.$DATE.XXXX` @@ -29,10 +29,10 @@ echo "start" | fedmsg-logger \ pushd $TMPDIR git clone git://git.fedorahosted.org/comps.git && { - pushd comps - make comps-epel7.xml - cp comps-epel7.xml /mnt/koji/mash/epel-$DATE/logs/ - popd + pushd comps + make comps-epel7.xml + cp comps-epel7.xml /mnt/koji/mash/epel-$DATE/logs/ + popd } popd

1 0

[releng] Updated tag rawhide-stable
by Till Maas 01 Oct '14

01 Oct '14

The lightweight tag 'rawhide-stable' was updated to point to: 2729264... buildbranched/rawhide: Fix mash calls It previously pointed to: 60bea54... buildbranched: Use $DIST for mash config edit NOTE: People pulling from the repository will not get the new tag. For more information, please see: http://live.gnome.org/Git/Help/TagUpdates

1 0

[releng] buildbranched/rawhide: Fix mash calls
by Till Maas 01 Oct '14

01 Oct '14

commit 272926406145042b3e3b62f785fec02b2482549e Author: Till Maas <opensource(a)till.name> Date: Wed Oct 1 21:22:36 2014 +0200 buildbranched/rawhide: Fix mash calls scripts/buildbranched | 3 +-- scripts/buildrawhide | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) --- diff --git a/scripts/buildbranched b/scripts/buildbranched index b1d9704..fd6ef49 100755 --- a/scripts/buildbranched +++ b/scripts/buildbranched @@ -119,8 +119,7 @@ send_fedmsg start mash.start log "starting mash" # Drop privs here so that we run as the masher UID -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/"${COMPSFILE}" $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 - "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 +mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 send_fedmsg done mash.complete diff --git a/scripts/buildrawhide b/scripts/buildrawhide index 0de0d6c..31c789f 100755 --- a/scripts/buildrawhide +++ b/scripts/buildrawhide @@ -110,8 +110,7 @@ send_fedmsg start mash.start log "starting mash" # Drop privs here so that we run as the masher UID -mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/"${COMPSFILE}" $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 - "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 +mock -r $MOCKCONFIG --uniqueext=$DATE --unpriv --chroot "mash $MASHOPTS -p $TREEPREFIX/development/$BRANCHED -o ${MASHDIR} --compsfile $logdir/${COMPSFILE} $BRANCHED$EXPANDARCH > $logdir/mash.log 2>&1" || exit 1 send_fedmsg done mash.complete

1 0

#6012: revive jbrout properly
by Fedora Release Engineering 01 Oct '14

01 Oct '14

#6012: revive jbrout properly -----------------------------+------------------------ Reporter: mcepl | Owner: rel-eng@… Type: task | Status: new Milestone: Fedora 20 Final | Component: other Keywords: | Blocked By: Blocking: | -----------------------------+------------------------ I have passed [https://bugzilla.redhat.com/show_bug.cgi?id=1139066 package review], tried to fix the package with git-request, but still, [https://admin.fedoraproject.org/pkgdb/package/jbrout/ there] is Orphan on the package, and I don't own devel branch. Could you please fix it for me, please? -- Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6012> Fedora Release Engineering <http://fedorahosted.org/rel-eng> Release Engineering for the Fedora Project

1 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

rel-eng October 2014