secondary June 2016

secondary@lists.fedoraproject.org

3 participants
7 discussions

Re: multi-CPU optimization inside a distribution
by Dan Horák 30 Jun '16

30 Jun '16

On Thu, 30 Jun 2016 15:59:38 -0400 Solomon Peachy <pizza(a)shaftnet.org> wrote: > On Thu, Jun 30, 2016 at 08:04:57PM +0100, Richard W.M. Jones wrote: > > I will just snipe here that the situation would be better if POWER > > > 4 hardware was generally available to buy. > > The upcoming Raptor Talos POWER8 board [1] looks promising, but at > $3700 a pop just for the motherboard and an entry-level [1] CPU, it's > out of reach of many otherwise-interested developers. Assuming it > goes into production at all.. unfortunately it does not in foreseeable future (based on official update on their IRC channel). It was my hope to get newer workstation class HW and be able to lift the ppc64 (BE) support to Power7 and up (or so). With the OpenPOWER ecosystem there is still chance someone else will try it. Dan

1 0

Re: multi-CPU optimization inside a distribution
by Josh Boyer 30 Jun '16

30 Jun '16

On Thu, Jun 30, 2016 at 12:01 PM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote: > On Thu, 30 Jun 2016 16:13:59 +0200, Josh Boyer wrote: >> On Thu, Jun 30, 2016 at 9:53 AM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote: >> > But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer >> >>=Power8 hardware does use ppc64le which is already Power8 optimized. >> > So isn't this problem (somehow temporarily) solved for now? >> >> POWER8 can run in either endian mode. So no, the problem is not solved. > > End-users do not even know what is endianity. And all application developers > need to write software BE/LE-independently (such as using ntohl()/htonl() > etc.) so they also do not care whether the host is BE or LE. Only <1% of > developers for high performance core or very low-end libraries (like glibc or > glib) need to care about BE/LE. > > So why to run Power8 in BE? Power is switching to LE AFAIK which makes it > also better compatible with buggy software ignoring BE/LE difference which is > (as all the software) targeting primarily x86* (=LE). It isn't switching. It's adding. I'd maybe agree that LE will become more prevalent over time, particularly as it is used in cloud workloads. However, BE mode isn't going away anytime soon. >> Realistically, it isn't about either specifically. Each iteration of >> POWER tends to require tuning specifically for that generation if you >> want to get the most performance out of your software. > > Latest CPU features get utilized mostly only in inner loops of high > performance code which should use IFUNC. So building the whole distro for the > latest CPU brings only incompatibility with no real performance advantage. > > > At least this is my expectation, the OP misses any real benchmark numbers > whether the distro build for the latest CPU generation is worth it. > IIRC for x86* it was not worth it. You might not be wrong, but you're dismissing the market dynamics of POWER that I mentioned. Those same dynamics lead to the distribution components for ppc64 being built as 64-bit binaries, even though at the time 32-bit binaries for 99% of packages were actually more than sufficient. Comparing performance numbers within POWER is fine, but applying the "this is how we do it in x86" logic isn't really suitable. They are different markets. I believe the markets will perhaps merge over time, but they clearly haven't yet today. josh

1 0

Re: multi-CPU optimization inside a distribution
by Peter Robinson 30 Jun '16

30 Jun '16

On Thu, Jun 30, 2016 at 5:01 PM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote: > On Thu, 30 Jun 2016 16:13:59 +0200, Josh Boyer wrote: >> On Thu, Jun 30, 2016 at 9:53 AM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote: >> > But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer >> >>=Power8 hardware does use ppc64le which is already Power8 optimized. >> > So isn't this problem (somehow temporarily) solved for now? >> >> POWER8 can run in either endian mode. So no, the problem is not solved. > > End-users do not even know what is endianity. And all application developers > need to write software BE/LE-independently (such as using ntohl()/htonl() > etc.) so they also do not care whether the host is BE or LE. Only <1% of > developers for high performance core or very low-end libraries (like glibc or > glib) need to care about BE/LE. > > So why to run Power8 in BE? Power is switching to LE AFAIK which makes it > also better compatible with buggy software ignoring BE/LE difference which is > (as all the software) targeting primarily x86* (=LE). Data conversion is expensive, slow and problematic so once it's in one format it tends to stay that way. There's also legacy/proprietary tools that run as big endian on the Power64 platform. For a new deployment there's no reason not to do it as LE from the outset, and migration from other LE platforms makes sense but if you're in a BE world (AIX, Solaris, Z-series and friends) it makes sense to run in BE. There are some applications where the data comes in in BE format and it's costly to deal with that in real time, especially in high throughput scenarios. >> Realistically, it isn't about either specifically. Each iteration of >> POWER tends to require tuning specifically for that generation if you >> want to get the most performance out of your software. > > Latest CPU features get utilized mostly only in inner loops of high > performance code which should use IFUNC. So building the whole distro for the > latest CPU brings only incompatibility with no real performance advantage. We're looking at IFUNC as one option, Function multi-versioning in GCC 6 [1] is presumably also another option, although wouldn't be useful for already released enterprise releases, although obviously irrelevant for the Fedora conversation. > At least this is my expectation, the OP misses any real benchmark numbers > whether the distro build for the latest CPU generation is worth it. > IIRC for x86* it was not worth it. [1] https://lwn.net/Articles/691932/

1 0

Re: multi-CPU optimization inside a distribution
by Peter Robinson 30 Jun '16

30 Jun '16

On Thu, Jun 30, 2016 at 2:53 PM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote: > On Thu, 30 Jun 2016 12:12:37 +0200, Dan Horák wrote: >> For ppc64 (the big endian POWER) the base is set by the toolchain >> default which is Power4/ppc970. When Power7 came we were asked what are >> the options to take the advantage of these CPUs, 3 generations newer >> than the base. The solution was the introduction of ppc64p7 subarch > > But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer >>=Power8 hardware does use ppc64le which is already Power8 optimized. > So isn't this problem (somehow temporarily) solved for now? > > Or are really the Power7 (and not yet Power8) machines the real problem? No your confusing endian with CPU features. We support both Big Endian (ppc64) and Little Endian (ppc64le) options for the Power64 architecture. Both can run on Power8 hardware, in fact the builder VMs run like with with both BE and LE VMs running on the same physical Power8 host side by. There's shortly going to also be Power9 [1] with options that need optimisation to make the most of new architectural enhancements which will be primarily focused towards ppc64le (but will also work on ppc64) but we don't want to exclude Power8 from ppc64le. Peter [1] http://www.nextplatform.com/2016/04/18/power9-will-bring-competition-datace…

1 0

Re: multi-CPU optimization inside a distribution
by Josh Boyer 30 Jun '16

30 Jun '16

On Thu, Jun 30, 2016 at 9:53 AM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote: > On Thu, 30 Jun 2016 12:12:37 +0200, Dan Horák wrote: >> For ppc64 (the big endian POWER) the base is set by the toolchain >> default which is Power4/ppc970. When Power7 came we were asked what are >> the options to take the advantage of these CPUs, 3 generations newer >> than the base. The solution was the introduction of ppc64p7 subarch > > But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer >>=Power8 hardware does use ppc64le which is already Power8 optimized. > So isn't this problem (somehow temporarily) solved for now? POWER8 can run in either endian mode. So no, the problem is not solved. > Or are really the Power7 (and not yet Power8) machines the real problem? Realistically, it isn't about either specifically. Each iteration of POWER tends to require tuning specifically for that generation if you want to get the most performance out of your software. That tuning is incompatible with older generations of hardware (endian completely aside). So as Dan alludes to, this is fairly cumbersome. To make it widely useable, you default to the oldest generation you want to support. x86 theoretically has similar problems, but we tend to just not care as much about tuning at the distro level. The applications can tune however they'd like, but given that most datacenters are not homogeneous among x86 machines, it often means people cannot tune for one specific x86 generation because they have to run on a wider variety. Contrast that with POWER, where people using that tend to buy one or two very large systems to consolidate their workload on. They can and often do want the best performance they can eek out since they're consolidating, which means they often want the distro components tuned as well, particularly system libraries. Going forward, I'd seriously look at dropping anything older than POWER7 support. I know we all dearly love our Mac G5 machines (ppc970), but those are getting very very old at this point. POWER5 was a non-event for Fedora, POWER6 was a weird generation even for IBM. POWER7 and POWER8 are where most of the effort is, and that will only continue as next generation POWER chips come out. Given that Rackspace and others have actually started using the openPOWER stuff with POWER8, it probably makes sense to focus on where the users are headed and not where they were 10 years ago. josh

1 0

multi-CPU optimization inside a distribution
by Dan Horák 30 Jun '16

30 Jun '16

Hi all, we were talking about this item for some time, so let's start a thread for it to have the discussion and hopefully also a solution documented. This is meant as discussion initiation based on the situation in Fedora on POWER. I would like to ask the bigger experts than me to fill the missing details and options. Currently we set a minimal CPU level for an architecture (or use the toolchain default) on the distribution level (/usr/lib/rpm/redhat/rpmpc owned by redhat-rpm-config [1]). It allows the distro to run on such CPU and any newer evolution of it (omitting any kernel or hw issues), but it also means it doesn't generally take advantage of features and improvements in the newer CPUs. For ppc64 (the big endian POWER) the base is set by the toolchain default which is Power4/ppc970. When Power7 came we were asked what are the options to take the advantage of these CPUs, 3 generations newer than the base. The solution was the introduction of ppc64p7 subarch into the packaging and release engineering tools. But it showed more as a hack than a solution, touching rpm, yum, koji, .... The list of packages is manually maintained, requires manual updates to the buildsystem for new releases, seems new packaging tools (dnf) don't understand it correctly. Is there a way to make the subarch mechanism generic and integrated with the other tools? So we could have ppc64p8 and ppc64p9 inside Fedora for POWER ... Now I'm getting to an area where others are the experts :-) Glibc allows to build and install multiple per-cpu optimized runtimes that are selected based on the CPU. There is the IFUNC mechanism and function multi-versioning in GCC (user-friendly IFUNC) [2] to allow multiple implementations inside one library/binary. Some packages do the CPU detection during runtime and select the optimized functions themselves. There is also an option to introduce a "tertiary architecture" and rebuild everything for the new CPU, but keeping the rpm arch the same. But it has its own costs too. What do you think? Are there any recommendations for both the distro and package maintainers and upstream developers? I suppose even primary architectures are facing the same problem. [1] http://pkgs.fedoraproject.org/cgit/rpms/redhat-rpm-config.git/tree/rpmrc [2] https://lwn.net/Articles/691932/ Dan

1 0

Fwd: Fedora 24 is here!
by Peter Robinson 21 Jun '16

21 Jun '16

For those that didn't notice the Secondary arches are included in the Primary release announcement. Congrats and kudos to all architecture teams for all the hard work to get them out at the same time as primary! Peter ---------- Forwarded message ---------- From: Matthew Miller <mattdm(a)fedoraproject.org> Date: Tue, Jun 21, 2016 at 2:47 PM Subject: Fedora 24 is here! To: announce(a)lists.fedoraproject.org Today the Fedora Project is pleased to announce the general release of Fedora 24. Download it now from our Get Fedora site: Workstation: https://getfedora.org/workstation/ Server: https://getfedora.org/server/ Cloud: https://getfedora.org/cloud/ Spins: https://spins.fedoraproject.org/ Labs: https://labs.fedoraproject.org/ ARM: https://arm.fedoraproject.org/ Another Step in the Fedora Journey ---------------------------------- The Fedora Project has embarked on a great journey... redefining what an operating system should be for users and developers. Such innovation does not come overnight, and Fedora 24 is one big step on the road to the next generation of Linux distributions. But that does not mean that Fedora 24 is some "interim" release; there are great new features for Fedora users to deploy in their production environments right now! Workstation ----------- The Fedora 24 Workstation release features GNOME 3.20, with many usability improvements such as easier input device and printer settings, a better search interface, shortcut windows for keyboard commands, and more convenient music controls. Flatpak (formerly xdg-app) is another building-block feature, with Software able to track installed Flatpaks and adding more features in the future as the technology develops. The Software app has also grown features to provide a full system upgrade directly from the desktop from one Fedora release to the next, and the ability to provide labeling as well as reviews of available software. Fedora 24 continues our work on the X replacement, Wayland, a next-generation graphics stack. Although this release will not default to Wayland, it includes many improvements and is available as an option for users to try out, and potentially will be the default stack in Fedora 25. Server ------ Fedora 24 Server edition is more streamlined and introduces more modularity, which will become a major factor in future Fedora releases, even as unnecessary packages were removed and the installer has a smaller footprint. FreeIPA 4.3 is a major feature for Fedora 24 Server. FreeIPA is an integrated security information management solution. This new version of FreeIPA features simplified replica installation and improved replication technology management. Cloud ----- Fedora is on its way to being the best platform for containerized applications, from base Fedora container images to a full-featured platform as a service to run and manage them. As we continue on this part of the journey, we are packaging OpenShift Origin so it is easy to deploy. OpenShift Origin is a Platform as a Service system based around Kubernetes, a production-grade container orchestration project. OpenShift Origin is optimized for application development and deployment. Origin makes it easy for developers to get started building applications in containers and for operators to manage them. While not shipped in Fedora 24, per se, we have new infrastructure for developing container images with applications layered on top of the base Fedora Docker image. Fedora Developers will also see a layered image build service, which provides tools for Fedora contributors to start creating and shipping layered container images in Fedora 25 and beyond. Spins and Labs -------------- Fedora Spins and Labs are alternative Fedora versions that offer additional desktop environments, or other custom collections of software, alongside the three editions that are the primary focus for the project. Our Spins make it easy for people to use other desktop environments. Everyone has different needs and Spins are a great way for us to meet them. The Fedora 24 spins release showcases KDE Plasma, Xfce, LXDE, Mate-Compiz, and Cinnamon, all on the same Fedora base. Fedora Labs offer collections of software for specific purposes: Games, Robotics, and so on. They are pre-selected sets of software ideal for events or audiences with specialized interests that need corresponding software. Fedora 24 comes with a new lab focused on Astronomy, which contains a set of tools for astronomers and astrophysicists. ARM --- ARM images are available, as usual, for several use cases. Fedora 24 ships desktop images, such as Spins and Workstation, but also provides a Server image. A minimal Fedora image completes the wide set of install options for your ARM board. Alternate Fedora Architectures ------------------------------ For the first time, all of our secondary architectures, AArch64, Power64 (ppc64, ppc64le) and s390x, are released simultaneously, rather than trailing a little behind. This includes the Server edition for all architectures and the Cloud and Docker base images for AArch64 and Power64. Along with all the same enhancements as in the primary architectures in Fedora 24, AArch64 and Power64 now have support for golang, nodejs, mongodb; along with numerous other architecture-specific enhancements. The architecture release notes pages on the wiki have more specifics for each. Download these from: https://dl.fedoraproject.org/pub/fedora-secondary/releases/24/ Atomic Host ----------- Fedora Atomic Host releases on a two-week schedule, and each release is built on the latest released Fedora version. This schedule means the Atomic Host will now be currently built on Fedora 24. You can try one of the newer features with recent Fedora Atomic Host builds today. Since Fedora 23 was released, Atomic Host has added a developer mode that gives a better developer experience overall. When running in developer mode, the host will download and start Cockpit and a tmux session to make it easier to work at the console and obtain necessary information such as the root password or IP address. Make Fedora Even Better ----------------------- If you want to take an active hand in making Fedora even better, there are many ways you can contribute. There are many roles that you can participate. Visit http://whatcanidoforfedora.org/ for more information! While this is a general release, there is always a chance you may encounter bugs or missing features. To report issues encountered during testing, contact the Fedora QA team via the mailing list or in #fedora-qa on Freenode IRC. As testing progresses, common issues are tracked on the Common F24 Bugs page: https://fedoraproject.org/wiki/Common_F24_bugs For tips on reporting a bug effectively, read "how to file a bug report at https://fedoraproject.org/wiki/How_to_file_a_bug_report. Flock 2016: Krakow, Poland -------------------------- If you want to be a part of the journey that Fedora is taking, either as a current or interested Fedora contributor, one way to engage with our community is through Fedora premier events. The annual North American/European conference for Fedora contributors is Flock, which takes place August 2-5, 2016 in Krakow, Poland. Registration is now open at https://register.flocktofedora.org. For more information about our Latin American and Asia-Pacific Conferences, stay tuned for announcements on the Fedora Community Blog: https://communityblog.fedoraproject.org. -- Matthew Miller <mattdm(a)fedoraproject.org> Fedora Project Leader -- announce mailing list announce(a)lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/announce@lists.fedoraproject.org

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

secondary June 2016