So, I've modified r2spec fairly heavily on my own. Pierre-Yves suggested I bring conversation straight here, so I'm doing that.
I'll put a SRPM of my r2spec up at the next point when it works. It worked earlier today, but of course I'm doing more surgery right now.
Here are the most visible changes I've made so far; Please hear them as suggestions, not demands. :)
+ Takes R package metadata from repositories directly through available.packages. Can load from many repos at once.
+ Cleanly handles multi-line fields in package metadata
+ Recursively chases Depends + including across repositories (i.e. CRAN depends on BioC) + Or Depends and Suggests, and Imports and Extends.
There's a lot more to this. Separate topic, separate post.
+ Uses a single template for specfile, instead of two long functions with literal strings.
+ Permits template to be specified on the command line
+ String manipulation by regexp instead of iterated 'replace' and such.
+ Maintains a dependency graph to permit building an automated rpmbuild order.
[ toy additions ]
outputs GraphViz (dot format) illustrations of dependency graphs.
I know this is lots and lots of change. I hope you don't see it as encroaching.
- Allen S. Rout
Hi Allen
Thanks for all the work done :-)
On Thu, 2009-09-03 at 16:42 -0400, Allen S. Rout wrote:
- Takes R package metadata from repositories directly through available.packages. Can load from many repos at once.
Is it still able to fetch information from the package itself (if you suddenly want to build the spec without internet connection) ?
Cleanly handles multi-line fields in package metadata
Uses a single template for specfile, instead of two long functions with literal strings.
- Permits template to be specified on the command line
String manipulation by regexp instead of iterated 'replace' and such.
That's for sure better than the crap I was using :)
- Recursively chases Depends
- including across repositories (i.e. CRAN depends on BioC)
- Or Depends and Suggests, and Imports and Extends.
There's a lot more to this. Separate topic, separate post.
I'm not sure here, why recursively chase the depends when you want to build a single RPM ?
- Maintains a dependency graph to permit building an automated rpmbuild order.
That's a cool feature :)
[ toy additions ]
outputs GraphViz (dot format) illustrations of dependency graphs.
I have a script that does it for the entire bioconductor repo.
I know this is lots and lots of change. I hope you don't see it as encroaching.
It's not GPL for nothing ;-)
IMHO R2spec is really a tool that should keep doing what it is good at, generating spec file. However I could see the need for another tool that allows to generate/build spec/rpm in a recursive manner. Basically allowing sysadmin to build an entire repo easily. Something for which you would do: tool --build http://link/to/some/package.tar.gz and which would generate all the spec file needed in order to be able to build this package.
But then again, it's just my opinion and it can be worth to just have everything in one tool instead of splitting :-)
Best regards,
Pierre
Pierre-Yves pingou-E11Oz7VxvVOXCRStZZN3OA@public.gmane.org writes:
IMHO R2spec is really a tool that should keep doing what it is good at, generating spec file. However I could see the need for another tool that allows to generate/build spec/rpm in a recursive manner. Basically allowing sysadmin to build an entire repo easily. Something for which you would do: tool --build http://link/to/some/package.tar.gz and which would generate all the spec file needed in order to be able to build this package.
I haven't seen any responses to my package-deps thinking. I'm hoping that's because everyone read it and said to themselves, "Wow, this Allen fellow is an utter genius: How profoundly correct he is! ".
Yeah, right.
OK, here's the SRPM of my current state.
http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.0-asr.src.rpm
When I run that against ggplot2, I get a "broad" dependency graph of 152 packages. That's the list of packages that "should" be installed to be able to R CMD CHECK all the packages, recursively.
That fact, the recursive Suggests graph, is why I think any tool that builds a single spec needs to do the math; because there's no way we can CHECK a package without doing it.
- Allen S. Rout
On Wednesday 09 September 2009 Allen S. Rout wrote:
I haven't seen any responses to my package-deps thinking. I'm hoping that's because everyone read it and said to themselves, "Wow, this Allen fellow is an utter genius: How profoundly correct he is! ".
Yeah, right.
The probable cause is more like "everyone would like to comment but REAL LIFE (tm) got in the way", at least it was for me.
OK, here's the SRPM of my current state.
http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.0-asr.src.rpm
When I run that against ggplot2, I get a "broad" dependency graph of 152 packages. That's the list of packages that "should" be installed to be able to R CMD CHECK all the packages, recursively.
Considering that cran has more than 1800 packages I would that this is small subset. ;-)
That fact, the recursive Suggests graph, is why I think any tool that builds a single spec needs to do the math; because there's no way we can CHECK a package without doing it.
What could be done, depending on the needs found, is to build a python library where both R2spec and your tools use the code from.
- Allen S. Rout
José Matos jamatos-BxEiAy0842I@public.gmane.org writes:
The probable cause is more like "everyone would like to comment but REAL LIFE (tm) got in the way", at least it was for me.
I understand entirely. :)
Considering that cran has more than 1800 packages I would that this is small subset. ;-)
yeah, but it's still honking huge compared to what I thought I'd be able to do. It was when I saw that depgraph that I decided I would just work on building sets instead of singles.
- Allen S. Rout
On Thursday 03 September 2009 Allen S. Rout wrote:
So, I've modified r2spec fairly heavily on my own. Pierre-Yves suggested I bring conversation straight here, so I'm doing that.
I'll put a SRPM of my r2spec up at the next point when it works. It worked earlier today, but of course I'm doing more surgery right now.
Just for curiosity, do you create the %test section on the first passage? I had to comment that section for bootstrap because I had cyclic dependencies where test section of package A would depend on package B, and the test of package B would depend on A.
- Allen S. Rout
José Matos jamatos-BxEiAy0842I@public.gmane.org writes:
Just for curiosity, do you create the %test section on the first passage? I had to comment that section for bootstrap because I had cyclic dependencies where test section of package A would depend on package B, and the test of package B would depend on A.
I'm hoping my previous message about dependencies has kind of answered this question. I've tried to be clear, but the topic is twisty and complex, and rife with politics.
My "branch" (If I may so dignify it) is now doing the following:
+ Generating the specfiles for a broad dependency tree of a package
+ Generating a rpm build script which will go through the specs, and - build a RPM - install the RPM
in an order which satisfies the narrow dependency tree.
I'm currently testing that one, it built the 162-package tree for ggplot2 with a relatively small number of RPM build errors, which I haven't yet solved.
I'm testing the r2spec package now, once I smack the bugs in my RPM of it I'll stick it up for kibitzing.
Once _that_ is complete, I intend to automate the running of the R CMD CHECKs.
- Allen S. Rout
José Matos jamatos-BxEiAy0842I@public.gmane.org writes:
So you running a R CMD CHECK for all the newly installed packages? Clever. :-)
Right. Since by definition we know we can't "CHECK packages before using the RPM", the best I could come up with was "CHECK packages before considering them suitable for distribution".
I would welcome some other theory; it totally squicks me to have the check facility available yet not use it before I get an RPM out of the process.
When I suggested doing a conservative CHECK during build, and then the full CHECK afterwards, the R folks shrugged and asked "why bother with the first one?". And I can't disagree.
- Allen S. Rout
On Thu, 3 Sep 2009, Allen S. Rout wrote:
So, I've modified r2spec fairly heavily on my own. Pierre-Yves suggested I bring conversation straight here, so I'm doing that.
I'll put a SRPM of my r2spec up at the next point when it works. It worked earlier today, but of course I'm doing more surgery right now.
I checked the list subject, and do not see that the URL for a new version has been mentioned. Pierre-Yves' version worked well for me with some mechanical edits to handle versioning valued, and to break some R CMD CHECK dependency loops to permit bootstrapping new package groups in.
This was on a packaging effort for a couple hundred CRAN financial and econometric R add on packages, earlier this year
-- Russ herrold
R P Herrold herrold-lD49m8UYmPxWk0Htik3J/w@public.gmane.org writes:
On Thu, 3 Sep 2009, Allen S. Rout wrote:
So, I've modified r2spec fairly heavily on my own. Pierre-Yves suggested I bring conversation straight here, so I'm doing that.
I'll put a SRPM of my r2spec up at the next point when it works. It worked earlier today, but of course I'm doing more surgery right now.
I checked the list subject, and do not see that the URL for a new version has been mentioned.
Wierd. I've thought I've mentioned several of them now. Have any of you seen any of those messages?
http://article.gmane.org/gmane.linux.redhat.fedora.r.devel/200
is the most recent, referring to my 2.6.2.
Pierre-Yves' version worked well for me with some mechanical edits to handle versioning valued, and to break some R CMD CHECK dependency loops to permit bootstrapping new package groups in.
This was on a packaging effort for a couple hundred CRAN financial and econometric R add on packages, earlier this year
I'm definitely focusing on building a whole dependency tree as a coordinated effort. I've attempted to formalize the (IMO poor) fit between the R ideas about package deps, and the RPM ideas.
http://article.gmane.org/gmane.linux.redhat.fedora.r.devel/177
- Allen S. Rout
On Mon, 28 Sep 2009, Allen S. Rout wrote:
http://article.gmane.org/gmane.linux.redhat.fedora.r.devel/200
is the most recent, referring to my 2.6.2.
http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.2-asr.src.rpm noted there is a dead link when I seek to pull it
-- Russ herrold
R P Herrold herrold-lD49m8UYmPxWk0Htik3J/w@public.gmane.org writes:
http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.2-asr.src.rpm noted there is a dead link when I seek to pull it
Well, crap. Should be there now.
It looks like the last time I copied over the 'binary' rpm, and not the src. They're both still there.
- Allen S. Rout
On Wed, 30 Sep 2009, Allen S. Rout wrote:
R P Herrold herrold-lD49m8UYmPxWk0Htik3J/w@public.gmane.org writes:
http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.2-asr.src.rpm noted there is a dead link when I seek to pull it
Well, crap. Should be there now.
thank you ... looking at it now
-- Russ herrold
r-devel@lists.fedoraproject.org
-
Allen S. Rout -
José Abílio Matos -
Pierre-Yves Chibon -
R P Herrold