Thanks for the feedback!

On 08/16/2017 10:29 PM, Tom Callaway wrote:

• I do think that making -devel packages might be useful, but if you can

make the case as to why they aren't useful (as opposed to "harder to script"), you can probably get an exception for ROS packages.

My first thought was that in most use cases, people would use those ROS packages as base to build their own software on top of it, which would always require the devel package anyway. But on second thought, that's only for some of the packages, and people may want to just run ROS without a lot of custom software, so having separate devel packages is probably useful, I just haven't started working on the split.

• I don't think you want ROS libs in the ld cache, that seems like a

recipe for disaster. You also probably want to disable the autoprovides generation and instead do something like:

Provides: ros(library.so)

or

Provides: ros-kinetic(library.so)

That seems like a good idea, although I would have liked to use more of RPM's implicit dependency management instead of essentially re-adding all those Provides manually. Do you have an idea how we could add those Provides automatically, or at least in a scripted way without looking at the resulting packages first? So far I can generate all SPEC files without looking into the generated RPMs, (with the exceptions that have custom configs and/or templates) it would be nice if we could keep it that way.

• The unbundling is going to suck, except in the rare cases where the

bundled code is unmodified. Thankfully, you should be able to use the bundled code as long as you follow the Fedora Packaging Guidelines around that: https://fedoraproject.org/wiki/Bundled_Libraries

Yes, that's what I was aiming for. Unbundling would be ideal, but probably not feasible. But this is also a step that has to be done separately for each package, so far I've been trying to avoid this whenever possible.

Kind regards, Till

On 08/16/2017 10:29 PM, Tom Callaway wrote:

• I don't think you want ROS libs in the ld cache, that seems like a

recipe for disaster. You also probably want to disable the autoprovides generation and instead do something like:

Provides: ros(library.so)

or

Provides: ros-kinetic(library.so)

I've figured how to do that with a __ros_provides script (once I understood the RPM macro system, this was actually quite simple). But I'm not sure how to translate the Requires: accordingly. I have ROS libs that "Provides: ros(libfoo.so)", but I don't know how to create a "Requires: ros(libfoo.so)" in a package that uses libfoo.

So far I see two options: 1) Run /usr/lib/rpmdeps from my __ros_requires script and translate ROS libs accordingly. But this would mean I would have to check for each dependency whether it is part of a ROS package. 2) Don't use rpmdeps but instead run ldd manually and filter ldd's output. This makes the distinction between ROS libs and other libs easier, but I would have to recreate the behavior of rpmdeps, which sounds like a terrible idea.

So neither option seems really satisfactory. Any other ideas?

Kind regards, Till

On Thu, Aug 17, 2017 at 5:20 PM, Jose Luis Rivero jrivero@osrfoundation.org wrote:

Hello Till:

On 16/08/17 22:19, Till Hofmann wrote:

...

Hi all,

I'm currently working on packaging the whole ROS stack by automatically generating SPEC files with the help of the rosinstall_generator. I think the project is far enough to be shared on this list, but it is still work in progress.

I just want to mention that there was a previous effort to package ROS in Fedora in the past, mostly done by Rich Mattes:

• https://fedoraproject.org/wiki/SIGs/Robotics/ROS_Packaging

A similar work has been done in Debian recently:

• https://wiki.debian.org/DebianScience/Robotics/ROS/Motivation

You can check their approach which does not rely in ROS distribution (ROS debian packages[1] has no ROS distribution in their names) and also map one ROS package with one downstream distribution package.

Some patches in Debian and specially the work done in Fedora before could be useful to you.

Good luck!

[1] https://wiki.debian.org/DebianScience/Robotics/ROS/Packages

Hi Till,

This is great, thanks for taking this on! Expanding on what Jose mentioned: there have been a couple efforts to support ROS packaging in Fedora, none of which made it all the way to a usable product. A little bit of history:

As Jose said, I did try to package ROS in Fedora, around the fuerte timeframe in 2012. As a result of that effort, about half of the core ROS stack as outlined on the robotics sig wiki page is packaged for fedora, and the other half is sitting in review requests. When I did it, I took the approach that there would be one ros distro provided in the core Fedora repositories, and that they would be installed like any other system package (e.g. libs in /usr/lib<qual>, config in /etc, data in /usr/share.) I figured that if anyone needed support for a separate ros distro, they could continue to build manually and install to /opt. I was also creating the specfiles manually, so that I could create proper subpackages (-devel, -doc, etc.) in each individual case. I think that this effort closely maps to the debian science efforts, though they got a lot farther than I did. I did recently look at trying to bring the debian science patches to fedora to restart the packaging process, but I haven't had the time to dedicate to such an effort.

Around the 2014 timeframe, we started to investigate the possibility of having the ros buildfarm generate fedora RPMs in parallel to the Ubuntu debs, so that all of the ros distros would be available in a manner similar to Ubuntu. Scott Logan did the work there, and landed patches to the ROS 'bloom' tool enabling it to generate spec files for fedora. That infrastructure is in place, but we got hung up in working with the OSRF to enable RPM building in their buildfarm, and then that fizzled out as well.

As it stands today, a lot of ROS dependencies are packaged in Fedora, making it possible to build from source pretty easily. I think that to move forward safely, you need to decide for the long term whether you want to support: - ROS installation via OSRF resources - ROS installation via copr - ROS packages in the distribution proper

Supporting ROS as a copr gives you a lot of leeway to do things that don't conform to the packaging guidelines, such as installing parallel stacks to /opt, creating single rpms for each package, etc. To bring the packages into fedora proper, as Tom was getting at, you need to reconcile how to support multiple software stacks within the package guidelines (and possibly get some exemptions to the guidelines,) and how to auto-generate specs that meet the packaging guidelines (proper division of -devel packages, etc.)

In case that wasn't enough to worry about, there is also a "Modularity" effort going on in Fedora. I have not been able to figure out what they're doing, but [1] claims that it's modules can have different lifecycles than the base distribution. It may provide a model to maintain multiple ROS stacks in parallel, but I don't really have any more information than that.

I'm happy to add you to the existing ROS packages that have been reviewed, add you to the robotics-sig user group, and continue to discuss on the mailing list.

As a side note, I've recently been poking at ros2 packaging[2]. The ecosystem is a lot smaller at the moment, and it may be able to re-use some of the concepts and tooling that get worked out for ros 1.

[1] https://docs.pagure.org/modularity/ [2] https://copr.fedorainfracloud.org/coprs/g/robotics-sig/ros2/

Hi all,

thanks again to everyone for the valuable feedback!

On 08/20/2017 06:29 PM, Rich Mattes wrote:

On Thu, Aug 17, 2017 at 5:20 PM, Jose Luis Rivero jrivero@osrfoundation.org wrote:

...

Hello Till:

On 16/08/17 22:19, Till Hofmann wrote:

...

Hi all,

I'm currently working on packaging the whole ROS stack by automatically generating SPEC files with the help of the rosinstall_generator. I think the project is far enough to be shared on this list, but it is still work in progress.

I just want to mention that there was a previous effort to package ROS in Fedora in the past, mostly done by Rich Mattes:

• https://fedoraproject.org/wiki/SIGs/Robotics/ROS_Packaging

A similar work has been done in Debian recently:

• https://wiki.debian.org/DebianScience/Robotics/ROS/Motivation

You can check their approach which does not rely in ROS distribution (ROS debian packages[1] has no ROS distribution in their names) and also map one ROS package with one downstream distribution package.

Some patches in Debian and specially the work done in Fedora before could be useful to you.

Good luck!

[1] https://wiki.debian.org/DebianScience/Robotics/ROS/Packages

Hi Till,

This is great, thanks for taking this on! Expanding on what Jose mentioned: there have been a couple efforts to support ROS packaging in Fedora, none of which made it all the way to a usable product. A little bit of history:

As Jose said, I did try to package ROS in Fedora, around the fuerte timeframe in 2012. As a result of that effort, about half of the core ROS stack as outlined on the robotics sig wiki page is packaged for fedora, and the other half is sitting in review requests. When I did it, I took the approach that there would be one ros distro provided in the core Fedora repositories, and that they would be installed like any other system package (e.g. libs in /usr/lib<qual>, config in /etc, data in /usr/share.) I figured that if anyone needed support for a separate ros distro, they could continue to build manually and install to /opt. I was also creating the specfiles manually, so that I could create proper subpackages (-devel, -doc, etc.) in each individual case. I think that this effort closely maps to the debian science efforts, though they got a lot farther than I did. I did recently look at trying to bring the debian science patches to fedora to restart the packaging process, but I haven't had the time to dedicate to such an effort. > Around the 2014 timeframe, we started to investigate the possibility of having the ros buildfarm generate fedora RPMs in parallel to the Ubuntu debs, so that all of the ros distros would be available in a manner similar to Ubuntu. Scott Logan did the work there, and landed patches to the ROS 'bloom' tool enabling it to generate spec files for fedora. That infrastructure is in place, but we got hung up in working with the OSRF to enable RPM building in their buildfarm, and then that fizzled out as well.

Thanks for the summary, I've seen the Fedora wiki pages on ROS, although I didn't realize that many of those are already available in the repositories. I'm not seeing a list of the ROS packages packaged for Fedora, is that available somewhere? The build farm approach sounds like an interesting alternative, do you have a link where I can get further information about that?

I decided to start packaging from scratch rather than building on top of the existing packages for several reasons: 1. Since fuerte, there were quite some changes ROS in general and the build system in particular. I thought it would be easier to start from scratch, rather than building on top of the fuerte packages. 2. I want to avoid maintaining >200 SPEC files manually, and therefore came up with the SPEC file generation approach. Of course modifications may be necessary for some packages, but so far the number of package-specific changes is surprisingly small. 3. For me, the approach to put ROS packages in a separate directory (e.g., /usr/lib<qual>/ros) seemed like a better idea for now, because of missing SO versions and because it seems like this is still expected by ROS in many places. As an example it's not possible to provide separate install paths for the different target types such as binaries, libraries, devel files and so on. Additionally, in my understanding, packaging ROS in /usr/lib<qual>/ros is not prohibited by the packaging guidelines. Also, even if we package ROS in /usr/lib<qual>/ros, it provides several benefits over a local build in /opt: It is much easier to set up, especially if you have to manage multiple robots, it provides automatic updates, you can easily select and un-select packages that you want to have installed, you don't need to build locally, and there are probably many more reasons. In fact, I started packaging ROS because I was annoyed by managing 6 ROS installations on our robots, and only afterwards realized that the approach may be useful for distro-wide packages.

Anyway, I'd like to have a closer look at the existing packages, because I assume that they carry some patches that may still be useful, even when generating SPEC files.

As it stands today, a lot of ROS dependencies are packaged in Fedora, making it possible to build from source pretty easily. I think that to move forward safely, you need to decide for the long term whether you want to support:

• ROS installation via OSRF resources
• ROS installation via copr
• ROS packages in the distribution proper

Supporting ROS as a copr gives you a lot of leeway to do things that don't conform to the packaging guidelines, such as installing parallel stacks to /opt, creating single rpms for each package, etc. To bring the packages into fedora proper, as Tom was getting at, you need to reconcile how to support multiple software stacks within the package guidelines (and possibly get some exemptions to the guidelines,) and how to auto-generate specs that meet the packaging guidelines (proper division of -devel packages, etc.)

In the long run, my goal is to provide proper distribution packages for ROS, with COPR as a playing ground for development and testing. Splitting devel files into a sub-package should be fairly easy. Since the packaging guidelines regarding bundled libraries changed recently (last year?) and now allow bundled libraries as long as the package provides the respective bundled(library)s, unbundling all libraries is also not necessary anymore. Finding out all bundled libraries may result in a lot package-specific work, but at least we have an easy way to add these to the generated SPEC files. I think these are the main problems regarding the packaging guidelines.

In case that wasn't enough to worry about, there is also a "Modularity" effort going on in Fedora. I have not been able to figure out what they're doing, but [1] claims that it's modules can have different lifecycles than the base distribution. It may provide a model to maintain multiple ROS stacks in parallel, but I don't really have any more information than that.

I think modularity only helps us, and we could just decide to ignore it for now. In my understanding, it could provide us an easy way to support multiple ROS releases for the same Fedora release, e.g., having both kinetic and lunar modules for F26 without the hack of having the version in the name. That would mean that there is no ros-kinetic-desktop package, but instead two ros-desktop packages in separate modules.

I'm happy to add you to the existing ROS packages that have been reviewed, add you to the robotics-sig user group, and continue to discuss on the mailing list.

Yes, that would be nice as it would already give me more insight in the current status of ROS in Fedora. Please also add me to the user group.

I can grant access to the ROS pagure and my ROS COPR to anyone interested.

As a side note, I've recently been poking at ros2 packaging[2]. The ecosystem is a lot smaller at the moment, and it may be able to re-use some of the concepts and tooling that get worked out for ros 1.

I was totally unaware of ros2, thanks for the hint!

Kind regards, Till