In R2spec-3.0.3-1, if you set a local source (-s <source>) then the URL and source0 fields in the spec will remain empty. Such specs fail to build.
There are a variety of ways to address this, I'm not sure that what I've done is really a superior solution.
in RPackage.py, down the branch of "I've only got a source, not a source0", I added
self.source0 = source self.URL = source
(see below).
This means that URL is populated with a local file path, but that's really all that's available.
....
All I'm really trying to do right now is get ggplot to one of my customers. I wasn't gearing up for contributing much code, but I can, if that's what Pierre-Yves wants.
I have to say, this is being a distinctly frustrating process.
....
"below"...
def setSource(self, source0, source): ''' From the url retrieve the name of the source ''' if source != None and source0 != None: if source != source0.rsplit('/',1)[1] : sys.exit("Do not give 2 contradictory sources of information") elif source != None: try: self.source = source.rsplit('/',1)[1] self.path = source.rsplit('/',1)[0] self.source0 = source self.URL = source except IndexError, err: self.source = source self.path = '../' elif source0 != None: self.source = source0.rsplit('/',1)[1] self.path = '../' self.source0 = source0 else: sys.exit("The source are not correct ")
- Allen S. Rout
asr-hnhdhkBXzx8@public.gmane.org (Allen S. Rout) writes:
In R2spec-3.0.3-1, if you set a local source (-s <source>) then the URL and source0 fields in the spec will remain empty. Such specs fail to build.
There are a variety of ways to address this, I'm not sure that what I've done is really a superior solution.
in RPackage.py, down the branch of "I've only got a source, not a source0", I added
self.source0 = source self.URL = source(see below).
This means that URL is populated with a local file path, but that's really all that's available.
So, I see from your previous replies, Pierre-Yves, that this is more or less expected behavior. Understood.
I have the opinion that the current manner of failing doesn't communicate what's going on very well. Would you consider documenting this requirement in the usage message, or even failing with an informative message?
I also think that there are other sources of packages than the various repos. It seems incorrect to me to insist that we name one.
- Allen S. Rout
On Tue, 2011-04-12 at 16:57 -0400, Allen S. Rout wrote:
I have the opinion that the current manner of failing doesn't communicate what's going on very well. Would you consider documenting this requirement in the usage message, or even failing with an informative message?
As in print the error message in the console rather than in the build file ? Or do you have something else in mind ?
I also think that there are other sources of packages than the various repos. It seems incorrect to me to insist that we name one.
For this I had few ideas but I do not know how far I went with them: - One idea was to be able to add repositories on the ~/.R2spec. Now that I am thinking about it, I guess something like: [repository] bioc = http://bioconductor.org cran = http://cran.r-project.org mine = http://pingoured.fr where one could then call R2spec/R2rpm with --repo=mine or --repo=bioc or something like this.
- Another idea might be to just specify the repository via an argument --URL="http://pingoured.fr"
R2spec was designed to easily create spec file which would then be reviewed and corrected by human. It has evolved to include rpm building but I have the feeling that it might need a good redesign, maybe a version 3.0.0 ;-)
Best regards, Pierre
Pierre-Yves Chibon pingou-E11Oz7VxvVOXCRStZZN3OA@public.gmane.org writes:
On Tue, 2011-04-12 at 16:57 -0400, Allen S. Rout wrote:
I have the opinion that the current manner of failing doesn't communicate what's going on very well. Would you consider documenting this requirement in the usage message, or even failing with an informative message?
As in print the error message in the console rather than in the build file ? Or do you have something else in mind ?
Well, the error message is something along the lines of "Blank URL field". This doesn't lead me to assume "I should specify a repo, for this file that I have locally".
On the one hand, for the -source case, you could specify a file:/// URL (and source0) if the user doesn't give one.
If you don't like that, you could detect "Oh, they set a source but no repo", and give a diagnostic like "Must set a repo; please use one of --[... yadda yadda "
I also think that there are other sources of packages than the various repos. It seems incorrect to me to insist that we name one.
For this I had few ideas but I do not know how far I went with them:
- One idea was to be able to add repositories on the ~/.R2spec.
Now that I am thinking about it, I guess something like: [repository] bioc = http://bioconductor.org cran = http://cran.r-project.org mine = http://pingoured.fr where one could then call R2spec/R2rpm with --repo=mine or --repo=bioc or something like this.
- Another idea might be to just specify the repository via an argument
--URL="http://pingoured.fr"
I understand your repo-centric perspective, but I would encourage you to let people build from files without forcing them to invent a repo. But this is an aesthetic call; If you permit an escape valve like --URL, and an error message to point to its use, then the usability concerns are well answered.
R2spec was designed to easily create spec file which would then be reviewed and corrected by human. It has evolved to include rpm building but I have the feeling that it might need a good redesign, maybe a version 3.0.0 ;-)
The version I got from EPEL was 3.0.3.. ?
I would be delighted to help if you are interested in it. My previous wave of changes did not meet with your approval, and I inferred that this meant you look at the problem differently than I do. I am not offended by this, it's your project. But I did stop trying to contribute.
I definitely look at the problem from the perspective of "Automatic production of entire dependency chains". If you think that is a worthwhile direction to go, I would be happy to help get there.
- Allen S. Rout
On Tue, 2011-04-12 at 17:34 -0400, Allen S. Rout wrote:
R2spec was designed to easily create spec file which would then be reviewed and corrected by human. It has evolved to include rpm
building
but I have the feeling that it might need a good redesign, maybe a version 3.0.0 ;-)
The version I got from EPEL was 3.0.3.. ?
Sorry I meant 4.0.0
I would be delighted to help if you are interested in it. My previous wave of changes did not meet with your approval, and I inferred that this meant you look at the problem differently than I do. I am not offended by this, it's your project. But I did stop trying to contribute.
I definitely look at the problem from the perspective of "Automatic production of entire dependency chains". If you think that is a worthwhile direction to go, I would be happy to help get there.
R2spec remains a project designed to make the life of Fedora's R packager's easier, so it has to remain guideline compliant. That is a reason why putting the source0 as URL also is not *in my opinion* a good approach (while I actually do like the --URL option).
There was a time when I thought I would try to get an "automatic cran builder" working but atm I have other priorities.
Pierre
Pierre-Yves Chibon pingou-E11Oz7VxvVOXCRStZZN3OA@public.gmane.org writes:
R2spec remains a project designed to make the life of Fedora's R packager's easier, so it has to remain guideline compliant. That is a reason why putting the source0 as URL also is not *in my opinion* a good approach (while I actually do like the --URL option).
Well, like I said, from my perspective it's aesthetic, so I'd be delighted with it.
There was a time when I thought I would try to get an "automatic cran builder" working but atm I have other priorities.
Are you willing to record your preferences on this, especially with respect to how my previous work didn't fit your picture? If you did, then if I get around to it I could work in ways aligned with your plans.
- Allen S. Rout
On Wed, 2011-04-13 at 15:06 -0400, Allen S. Rout wrote:
R2spec remains a project designed to make the life of Fedora's R packager's easier, so it has to remain guideline compliant. That is a reason why putting the source0 as URL also is not *in my opinion* a good approach (while I actually do like the --URL option).
Well, like I said, from my perspective it's aesthetic, so I'd be delighted with it.
I'll try to push this on the git by the end of the week-end then :)
There was a time when I thought I would try to get an "automatic cran builder" working but atm I have other priorities.
Are you willing to record your preferences on this, especially with respect to how my previous work didn't fit your picture? If you did, then if I get around to it I could work in ways aligned with your plans.
Well at the moment I see more R2rpm/R2spec has a tool to generate rpm or spec individually as one would have to do to incorporate them into Fedora. It is a spec generator tool which push the work up to building the rpm (which it needs to fill the %file section anyway). Specs needs manual inspection and curation sometime, one needs to run rpmlint and so on.
My thoughts were that there was space for a tool to build the whole dependency tree and run R2spec/R2rpm in the right order. In my attempts to build CRAN I used your R script for that. I could this way create 7 files and then I was just running these 7 files on after the other. My problem was more or less: - Takes time to build (~1 week for the whole run)
- Lack of resources (although I improved on these but not enough)
- I did it the dirty way by doing : build all R packages in fileX in mock - Install all rpm - build all R packages in fileX+1 in mock - install all rpm.... - restart for another arch/release I actually wonder, if one wants to give a shot at such project in a long term, if looking into koji would not be a nice alternative: build all the R packages from fileX for all arch/release - populate the repo as they are build - build all the R packages from fileX+1 for all arch/release... And this way at the end you have a repo which you know works. I always had the wish to support at least i686 and x86_64 on Fedora n and n-1 RHEL 5 and 6, so 6 to 8 (if I added rawhide) build for each packages. For this I though koji would make sense, but I never got around to try to install it and configure vm builders.
- Not all the packages build, sometime because of the lack of outside dependencies. For this reason all the packages that fails have to be inspected by hand.
- I have a script to update existing spec file but it does not check for changes in the dependency tree and cran as well as bioconductor are so good at changing the dependency tree at every releases... So for every release of bioconductor, we have to update all the spec files for most of their elements but the changelog basically (sounds easy when I say it this way ^^).
Basically I would keep R2spec at a single package level and build on the top a cran2rpm or so.
All this makes me think that I still have few thousands spec files hanging somewhere and that I should actually make them accessible somewhere (with a warning that a significant part of it is probably outdated).
These were my thoughts when I was looking into it.
Hope this help,
Pierre
On Wednesday 13 April 2011 22:10:21 Pierre-Yves Chibon wrote:
All this makes me think that I still have few thousands spec files hanging somewhere and that I should actually make them accessible somewhere (with a warning that a significant part of it is probably outdated).
These were my thoughts when I was looking into it.
Hope this help,
Pierre
It would be nice to have a plan for this.
A boostrap stage where the dependencies are reduced just to build dependencies.
The identification of core packages required by other packages.
Debian has done it before, Dirk can provide feedback for specific issues we get.
José Matos jamatos-BxEiAy0842I@public.gmane.org writes:
It would be nice to have a plan for this.
A boostrap stage where the dependencies are reduced just to build dependencies.
So here
http://article.gmane.org/gmane.linux.redhat.fedora.r.devel/177
is the farthest I've gotten in formalizing that path. I've waved that around a few times; it doesn't seem to pick up a lot of traction. If the topic were simple, I'd tl;dr it, but I don't think it can really be done.
The identification of core packages required by other packages.
I think that's what I'd call the "Narrow dependency graph"; that is, "packages which are needed to install other packages", as opposed to "Packages which are needed to CMD CHECK other packages".
So; if we take R2rpm as a building block, then I suggest the following pseudocode, to build a repo:
build_a_repo()
Make a list from CRAN, consult some local metadata about busted packages.
Pick an unbuilt package at random, and build it, until done.
-----
build_a_package()
generate the narrow and wide dependency graphs for your package.
generate the narrow graph of your wide dependencies. (everything you need to install to run (everything you need to check (package p ))) I'll call that G'.
Remove from G' everything already installed.
Build everything in G' , disabling CHECK and ignoring Suggests, Enhances. [ Can we also avoid the build-it-once, check %files, build it again step at this stage? ]
Install everything in G'.
Build everything in G', enabling CHECK.
Packages that build and CHECK may be signed, and removed from the list of packages to be built.
-----
I split it up this way because it permits also the "Build this package and its depgraph", which is where I came in. :)
This process also suggests some additional caching, at least the package metadata from all the repos; It's going to be necessary to trace deps across repos, of course. If anyone has opinions about where that ought to go, fire away. I figure it'd go in /var/tmp/[something] , and not expire for a week or more. Since the build process is going to be multi-day, anthing shorter than that will just be insane.
- Allen S. Rout
On Wed, 2011-04-13 at 23:10 +0200, Pierre-Yves Chibon wrote:
On Wed, 2011-04-13 at 15:06 -0400, Allen S. Rout wrote:
R2spec remains a project designed to make the life of Fedora's R packager's easier, so it has to remain guideline compliant. That
is
a reason why putting the source0 as URL also is not *in my
opinion*
a good approach (while I actually do like the --URL option).
Well, like I said, from my perspective it's aesthetic, so I'd be delighted with it.
I'll try to push this on the git by the end of the week-end then :)
A bit late, but I just pushed the changes. Testers are welcome :)
Pierre
r-devel@lists.fedoraproject.org
-
Allen S. Rout -
José Abílio Matos -
Pierre-Yves Chibon