Hi all, Does anyone use gem2rpm to upgrade an existing rpm to new versions of upstream gems? I'm contemplating working on a pull request to make gem2rpm aware of an existing .spec file and only update specific sections such as: version, requires, buildrequires, and adding a changelog. As it is now, it overwrites the existing rpm spec, removing any changelog entries, etc. Is this a good idea? What do others do to regenerate the updated version, requires/buildrequires to avoid human error? Thanks,

On 06/23/2014 11:20 AM, Joe Rafaniello wrote: > Hi all, > > Does anyone use gem2rpm to upgrade an existing rpm to new versions of > upstream gems? > > I'm contemplating working on a pull request to make gem2rpm aware of an > existing .spec file and only update specific sections such as: version, > requires, buildrequires, and adding a changelog. As it is now, it > overwrites the existing rpm spec, removing any changelog entries, etc. > > Is this a good idea? What do others do to regenerate the updated version, > requires/buildrequires to avoid human error? > > Thanks, > You should be able to use polisher from where-ever to do this programmatically. Just updated the last outstanding PR to incorporate your latest feedback https://github.com/ManageIQ/polisher/pull/98 As always, glad to review any additional PR's w/ any new polisher features & enhancements. -Mo

On 06/23/2014 02:01 PM, Joe Rafaniello wrote: > ----- Original Message ----- >> On 06/23/2014 11:20 AM, Joe Rafaniello wrote: >>> Hi all, >>> >>> Does anyone use gem2rpm to upgrade an existing rpm to new versions of >>> upstream gems? >>> >>> I'm contemplating working on a pull request to make gem2rpm aware of an >>> existing .spec file and only update specific sections such as: version, >>> requires, buildrequires, and adding a changelog. As it is now, it >>> overwrites the existing rpm spec, removing any changelog entries, etc. >>> >>> Is this a good idea? What do others do to regenerate the updated >>> version, >>> requires/buildrequires to avoid human error? >>> >>> Thanks, >>> >> You should be able to use polisher from where-ever to do this >> programmatically. >> >> Just updated the last outstanding PR to incorporate your latest feedback >> >> https://github.com/ManageIQ/polisher/pull/98 >> >> As always, glad to review any additional PR's w/ any new polisher >> features & enhancements. >> >> -Mo >> > Yeah, Mo. > I'm wondering if the complexity should live in polisher or gem2rpm. > It seems strange to use gem2rpm for initial rpms and polisher for updates. > > If we can solve both initial and updates to a spec in gem2rpm, we can > eliminate similar logic in both. > There are some common aspects to the process and some differences. For creation gem2rpm uses erb-based templates to generate new spec files. For updating, polisher parses the spec & attempts to interpolate gem/rpm contents. There are tradeoffs either way, generating a new spec gives you a fresh start, but updating it can incorporate existing changes which still apply. As it stands polisher currently does alot of the parsing / updating legwork. Of course there are edge cases which can be addressed, but those should be able to be taken care of w/ small / targeted PR's (as issues are discovered). -Mo

On 06/23/2014 09:58 PM, Joe Rafaniello wrote: > > > If gem2rpm could handle both initial and updates to the spec, we can > cleanly separate the responsibilities and just delegate to gem2rpm for all > or some of that work. > I would be happy to have polisher be the wrapper around gem2rpm for larger > "workflows." > > I have only glanced at the gem2rpm code and don't know how much effort it > is to handle "updates", this is just a curiosity at this point. > > For example, I'm curious how are changes in packaging conventions currently > handled? > Through new gem2rpm templates? Yes. By default is uses this template[0] which was recently updated to reflect new ruby 2.1 changes. > Is there a migration path for updating an existing rpm spec from an old > template to a new template? > Is this all done manually currently? Are there tools to enforce/verify > packaging conventions? Vit, has written a set of scripts which update the specs to current guidelines[1], I don't know if some edge cases are missed though. You can follow a previous discussion[2]. And to chime in the discussion, changing only the sections you mentioned should not be that big a deal. - version is easy to be fetched from rubygems.org - Changelog is also easy - Requires are not needed for fc21 and above (could use a different template for other branches) - BR could be tricky, since current implementation [3] (correct me if I'm wring) checks development deps from gemspec and eg. tools needed for the test suites might not be in that list. So, you can't avoid manual intervention after all. As Josef wrote, ideally what we need is some sort of a web app that checks/builds every time a new version is out. [0] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide... [1] https://github.com/voxik/fermig [2] https://lists.fedoraproject.org/pipermail/ruby-sig/2014-April/001537.html [3] https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide... -- FAS : axilleas GPG : 0xABF99BE5 Blog: http://axilleas.me _______________________________________________ ruby-sig mailing list ruby-sig(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

Dne 24.6.2014 20:47, Joe Rafaniello napsal(a): > > ----- Original Message ----- >> On 06/23/2014 09:58 PM, Joe Rafaniello wrote: >>> >>> If gem2rpm could handle both initial and updates to the spec, we can >>> cleanly separate the responsibilities and just delegate to gem2rpm for >>> all >>> or some of that work. >>> I would be happy to have polisher be the wrapper around gem2rpm for >>> larger >>> "workflows." >>> >>> I have only glanced at the gem2rpm code and don't know how much effort it >>> is to handle "updates", this is just a curiosity at this point. >>> >>> For example, I'm curious how are changes in packaging conventions >>> currently >>> handled? >>> Through new gem2rpm templates? >> Yes. By default is uses this template[0] which was recently updated to >> reflect new ruby 2.1 changes. >> >>> Is there a migration path for updating an existing rpm spec from an old >>> template to a new template? >>> Is this all done manually currently? Are there tools to enforce/verify >>> packaging conventions? >> Vit, has written a set of scripts which update the specs to current >> guidelines[1], I don't know if some edge cases are missed though. You >> can follow a previous discussion[2]. >> >> And to chime in the discussion, changing only the sections you mentioned >> should not be that big a deal. >> >> - version is easy to be fetched from rubygems.org >> - Changelog is also easy >> - Requires are not needed for fc21 and above (could use a different >> template for other branches) >> - BR could be tricky, since current implementation [3] (correct me if >> I'm wring) checks development deps from gemspec and eg. tools needed for >> the test suites might not be in that list. So, you can't avoid manual >> intervention after all. >> >> As Josef wrote, ideally what we need is some sort of a web app that >> checks/builds every time a new version is out. >> >> >> [0] >> https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide... >> [1] https://github.com/voxik/fermig >> [2] >> https://lists.fedoraproject.org/pipermail/ruby-sig/2014-April/001537.html >> [3] >> https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide... >> >> -- >> FAS : axilleas >> GPG : 0xABF99BE5 >> Blog: http://axilleas.me >> _______________________________________________ >> ruby-sig mailing list >> ruby-sig(a)lists.fedoraproject.org >> https://admin.fedoraproject.org/mailman/listinfo/ruby-sig > Thanks. Those are all good references. > Does anyone have opinions as to where automation of spec updates belong? > I'm wondering why gem2rpm wasn't used for Vit's "fermig" scripts? Well, what is in the upstream fermig repository and what I am actually using during migration periods is not always the same. From my point of view, it is one shot library, which should be ideally used once a year (or how often the guidelines are updated) and that is it. Actually, I see where are you aiming, i.e. the upgrade of gem is good opportunity to update the .spec, etc. but from tooling POV, it is easier to do it in two steps, e.g. update everything in Fedora when guidelines are updated and upgrade to new version when new version is released. Vít

> It would be nice to have one tool that automated some of the rpm spec > changes for rubygems packages. > I hope to get some cycles to work on adding update to an existing spec in > gem2rpm in the next week or so, I'll report back anything I come up with. > > Thanks. > _______________________________________________ ruby-sig mailing list ruby-sig(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig

> Well, what is in the upstream fermig repository and what I am actually > using during migration periods is not always the same. From my point of > view, it is one shot library, which should be ideally used once a year > (or how often the guidelines are updated) and that is it. > > Actually, I see where are you aiming, i.e. the upgrade of gem is good > opportunity to update the .spec, etc. but from tooling POV, it is easier > to do it in two steps, e.g. update everything in Fedora when guidelines > are updated and upgrade to new version when new version is released. > > Vít > Corrected the line 7) below... Thanks, this is great information. I think it's easier to add "update" functionality to gem2rpm by ensuring the template and spec both get updated to latest conventions.

It's really complicated to try to apply changes from a spec generated with new conventions into a spec using old or no conventions.

----- Original Message ----- > On 06/25/2014 10:15 AM, Joe Rafaniello wrote: >>> Well, what is in the upstream fermig repository and what I am actually >>> using during migration periods is not always the same. From my point of >>> view, it is one shot library, which should be ideally used once a year >>> (or how often the guidelines are updated) and that is it. >>> >>> Actually, I see where are you aiming, i.e. the upgrade of gem is good >>> opportunity to update the .spec, etc. but from tooling POV, it is easier >>> to do it in two steps, e.g. update everything in Fedora when guidelines >>> are updated and upgrade to new version when new version is released. >>> >>> Vít >>> >> Corrected the line 7) below... >> >> Thanks, this is great information. >> I think it's easier to add "update" functionality to gem2rpm by ensuring >> the template and spec both get updated to latest conventions. > > Would have to agree w/ Vit on this one, it'd be easier to split things > into multiple separate use cases: > > - generate a new spec from scratch > > - updated a single package version to new specifications > > - updating a single package between versions > > The Unix philosophy is to build and use simple tools that do simple > things well. We should try and nail each of these down individually > before trying to abstract all three into a single stage / tool. > > >> It's really complicated to try to apply changes from a spec generated with >> new conventions into a spec using old or no conventions. > Have you attempted this? Perhaps some code would better illustrate what > you are going for? > > -Mo > > _______________________________________________ > ruby-sig mailing list > ruby-sig(a)lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/ruby-sig Thanks Mo, that's really my point. In order to update a gem version programmatically, there are a series of steps that should happen, regardless if it's the same or different tools:

If using the gem2rpm mechanism to generate a new spec file for the new gem version and merge the applicable changes, you have a few prerequisites: 1) update the template to latest conventions 2) update the existing spec to latest conventions 3) gem2pm using the updated template for new gem version to create the new version spec file

Then compare the new version spec to the old version spec and merge any desired changes. I don't think you can compare two specs unless they're using the same conventions, otherwise there would be lots of noise in the diff.

Dne 25.6.2014 16:15, Joe Rafaniello napsal(a): > > > > ----- Original Message ----- > > > ----- Original Message ----- >> Dne 24.6.2014 20:47, Joe Rafaniello napsal(a): >>> ----- Original Message ----- >>>> On 06/23/2014 09:58 PM, Joe Rafaniello wrote: >>>>> If gem2rpm could handle both initial and updates to the spec, we can >>>>> cleanly separate the responsibilities and just delegate to gem2rpm for >>>>> all >>>>> or some of that work. >>>>> I would be happy to have polisher be the wrapper around gem2rpm for >>>>> larger >>>>> "workflows." >>>>> >>>>> I have only glanced at the gem2rpm code and don't know how much effort >>>>> it >>>>> is to handle "updates", this is just a curiosity at this point. >>>>> >>>>> For example, I'm curious how are changes in packaging conventions >>>>> currently >>>>> handled? >>>>> Through new gem2rpm templates? >>>> Yes. By default is uses this template[0] which was recently updated to >>>> reflect new ruby 2.1 changes. >>>> >>>>> Is there a migration path for updating an existing rpm spec from an old >>>>> template to a new template? >>>>> Is this all done manually currently? Are there tools to enforce/verify >>>>> packaging conventions? >>>> Vit, has written a set of scripts which update the specs to current >>>> guidelines[1], I don't know if some edge cases are missed though. You >>>> can follow a previous discussion[2]. >>>> >>>> And to chime in the discussion, changing only the sections you mentioned >>>> should not be that big a deal. >>>> >>>> - version is easy to be fetched from rubygems.org >>>> - Changelog is also easy >>>> - Requires are not needed for fc21 and above (could use a different >>>> template for other branches) >>>> - BR could be tricky, since current implementation [3] (correct me if >>>> I'm wring) checks development deps from gemspec and eg. tools needed for >>>> the test suites might not be in that list. So, you can't avoid manual >>>> intervention after all. >>>> >>>> As Josef wrote, ideally what we need is some sort of a web app that >>>> checks/builds every time a new version is out. >>>> >>>> >>>> [0] >>>> https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide... >>>> [1] https://github.com/voxik/fermig >>>> [2] >>>> https://lists.fedoraproject.org/pipermail/ruby-sig/2014-April/001537.html >>>> [3] >>>> https://github.com/lutter/gem2rpm/blob/master/templates/fedora-21-rawhide... >>>> >>>> -- >>>> FAS : axilleas >>>> GPG : 0xABF99BE5 >>>> Blog: http://axilleas.me >>>> _______________________________________________ >>>> ruby-sig mailing list >>>> ruby-sig(a)lists.fedoraproject.org >>>> https://admin.fedoraproject.org/mailman/listinfo/ruby-sig >>> Thanks. Those are all good references. >>> Does anyone have opinions as to where automation of spec updates belong? >>> I'm wondering why gem2rpm wasn't used for Vit's "fermig" scripts? >> Well, what is in the upstream fermig repository and what I am actually >> using during migration periods is not always the same. From my point of >> view, it is one shot library, which should be ideally used once a year >> (or how often the guidelines are updated) and that is it. >> >> Actually, I see where are you aiming, i.e. the upgrade of gem is good >> opportunity to update the .spec, etc. but from tooling POV, it is easier >> to do it in two steps, e.g. update everything in Fedora when guidelines >> are updated and upgrade to new version when new version is released. >> >> Vít >> > Corrected the line 7) below... > > Thanks, this is great information. > I think it's easier to add "update" functionality to gem2rpm by ensuring > the template and spec both get updated to latest conventions. > It's really complicated to try to apply changes from a spec generated with > new conventions into a spec using old or no conventions. > > 1) Update or at least highlight changes needed to the template we're using > to abide by conventions > 2) Manually or script update the existing template to latest conventions in > 1) > 3) Commit the updated template > 4) Call the fermig scripts to update the existing .spec to latest > conventions > 5) Commit the .spec with latest conventions > 6) Use gem2rpm to generate a new .spec from the updated template in 2) for > the desired gem version > 7) Apply the applicable changes from 6) into 5) (requires, buildrequires, > version, etc.) > 8) Take a breathe and see if we broke things ;-) > > Do you have any links to distgit repos where the template and spec were > upgraded for conventions? > Vit, the scripts don't appear to be cumulative. So for example, I'd > imagine the f19.rb should be run before running update.rb which calls > f21.rb. > Is this correct? > I am afraid I don't follow you. But I'll try to make same remarks which hopefully answers some of your questions. * No, fermig does not do cumulative updates. It should be the tool, which should be run after guidelines update and it will update all packages in Rawhide at once and rebuilds them. Since all packages are up2date from that moment, you don't need to care about the history and it does not need to be cumulative. Unfortunately, this is just theory. * If you maintain several gems, you might create your own template for gem2rpm. The template would be unique for every gem you maintain. The template would be basically your latest .spec file, with a few macros remaining, Ideally, just the version macro. When new gem version is release, you regenerate the .spec using that template and you are done. * It should be possible to run fermig on gem2rpm templates and get it updated. This will get the template up2date with the latest guidelines and you can continue to use the template for updates. The success rate will depend on the template complexity. * Please note, that the rubygem- package updates are bit easier in F21+, since you don't need to care about Requires. The requires are autogenerated for you.

* If I should choose what will be my next target in a war for simplified gem packaging, then I'd love to focus to simplify %files sections. The first step might be something like %{gem_directories} macro, which would replace %{gem_instdir}, %{gem_spec} and %{gem_cache} set of macros, which are repeated in every gem. The next step of evolution might be something along the way Java (maven) packages do, e.g. you'd do just %file -f filelist and that was it. This would be one huge step forward to remove the need of any upgrade tool.

Vít _______________________________________________ ruby-sig mailing list ruby-sig(a)lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig