10:24 a.m.
Hi all,
as a new Fedora Python maintainer, I have set myself a goal of moving Fedora to Python 3
as a default. This is going to be a multirelease effort that is going to affect lots of
Fedora parts. Since we will need to switch default package manager from Yum to DNF (which
is supposed to work with Python 3), we will need to wait for that. I've been told that
DNF should be default in F22, so that's my target, too. That should also give everyone
else plenty of time to work on other essential packages to make this happen.
Here is my analysis/proposal:
Before switching, we need to make sure that everything "important" (*) is Python
3 compatible. There are three steps I see in this transition:
1) Getting rid of Python 2 in mock minimal buildroot.
2) Porting Anaconda to Python 3.
3) Making all livecd packages depend on Python 3 by default (and eventually getting rid of
Python 2 from livecd) - this will also require switching from Yum to DNF as a default,
that is supposed to support Python 3.
( 4) Making as much of the remaining packages Python 3 compatible )
In past few days, I've been going through packages that are part of the above steps. I
have reported numerous bugs asking upstream and/or Fedora maintainers for help with
porting to Python 3. We have some spare cycles in our small Python packaging team, that we
will try to provide to whoever needs them most, but we're limited and we'll have
to rely on the upstreams to do most of the work. I'm attaching a document with list of
packages that need porting with some notes/links to opened bugs. Sometime soon, I'll
open a tracking bug for this, so that everyone can see where we are quickly.
(*) I call these "important" packages (in terms of being important for the
Python 3 switch)
From packaging point of view, this will probably require:
1)
Renaming python package to python2
2) Renaming python3 package to python
3) Switching the %{?with_python3} conditionals in specfiles to %{?with_python2} (we will
probably create a script to automate this, at least partially)
FAQ:
Q: Why do we need to switch to Python 3?
A: Because Python 2 is old, slower, less pythonic, doesn't get any more functionality
and it won't be that long before the official upstream support ends [1]
Q: How do I port to Python 3?
A: There are tons of tutorials and howtos about porting and the differencies in general.
E.g. [2] (general), [3] (c-extensions)
Q: What about Python 2?
A: We will maintain that at least as long as upstream supports it. After that, I'd
prefer dropping it, but since I know there will be people wanting to keep it around,
I'll gladly give the maintenance to someone else.
I'll be glad to answer all your questions and discuss the above points. Nothing is set
into stone and I'd love to hear your ideas and comments.
Thanks for reading this through!
Slavek.
--
Regards,
Bohuslav "Slavek" Kabrda.
[1] http://www.python.org/dev/peps/pep-0373/
[2] http://docs.python.org/dev/howto/pyporting.html
[3] http://docs.python.org/3/howto/cporting.html
11:33 a.m.
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
Hi all,
as a new Fedora Python maintainer, I have set myself a goal of
moving Fedora to Python 3 as a default.
That's a worthy goal!
From packaging point of view, this will probably require:
1) Renaming python package to python2
2) Renaming python3 package to python
3) Switching the %{?with_python3} conditionals in specfiles to %{?with_python2} (we will
probably create a script to automate this, at least partially)
But that seems like pointless busywork and churn. Sure, when python3
is ubiquitous, those checks can be dropped or reversed... But right
now it would be better to focus on actual python3 versions of all
packages — we're not there yet.
Zbyszek
--
they are not broken. they are refucktored
-- alxchk
12:10 p.m.
----- Original Message -----
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> Hi all,
> as a new Fedora Python maintainer, I have set myself a goal of
> moving Fedora to Python 3 as a default.
That's a worthy goal!
> From packaging point of view, this will probably require:
> 1) Renaming python package to python2
> 2) Renaming python3 package to python
> 3) Switching the %{?with_python3} conditionals in specfiles to
> %{?with_python2} (we will probably create a script to automate this, at
> least partially)
But that seems like pointless busywork and churn. Sure, when python3
is ubiquitous, those checks can be dropped or reversed... But right
now it would be better to focus on actual python3 versions of all
packages — we're not there yet.
Yes, current efforts should be directed to porting for Python 3, the packaging points I
mentioned will really be the very last step.
Thanks,
Slavek.
Zbyszek
--
they are not broken. they are refucktored
-- alxchk
--
Regards,
Bohuslav "Slavek" Kabrda.
11:53 a.m.
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
Hi all,
as a new Fedora Python maintainer, I have set myself a goal of moving
Fedora to Python 3 as a default.
I'm not sure we want to make python3 default depending on what your
definition of default is.
/usr/bin/python should refer to python2 --
http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
The python package itself should probably also remain python2 due to
dependencies and expectations from other distros and documentation --
I think I'd be -1 to changing this
The Fedora live images contain only python3, not python2 -- I'd be heavily
in favour of this. +1
This is going to be a multirelease effort
that is going to affect lots of Fedora parts. Since we will need to switch
default package manager from Yum to DNF (which is supposed to work with
Python 3), we will need to wait for that. I've been told that DNF should
be default in F22, so that's my target, too. That should also give
everyone else plenty of time to work on other essential packages to make
this happen.
Getting there at the same time as we get to DNF sounds like a good timeline.
(But see my note on anaconda below). +1
Here is my analysis/proposal:
Before switching, we need to make sure that everything "important" (*) is
Python 3 compatible. There are three steps I see in this transition:
1) Getting rid of Python 2 in mock minimal buildroot.
I'm not sure about this one as it will cause a lot of package churn. It
might be a necessary pain pointi or it might be a pain point we want to
defer until later in our porting efforts. Have to think about it more.
2) Porting Anaconda to Python 3.
+1 -- unfortunately, this probably depends on DNF.... So we may need to push
DNF in F22 and anaconda compatible with python3 in F23.
3) Making all livecd packages depend on Python 3 by default (and
eventually getting rid of Python 2 from livecd) - this will also require switching from
Yum to DNF as a default, that is supposed to support Python 3.
+1 -- this is what I see as the eventual goal (or perhaps, livecd python2
free followed by DVD python2 free followed by distro python2 free).
3.5) Switch tools that could target either python2 or python3 to target
python3. Currently the packaging guidelines say to target python2 to
control dep proliferation and because that's the most supported by the
larger python ecosystem. We should switch the recommendation when our
minimal environment must have python3 but does not need to have python2.
( 4) Making as much of the remaining packages Python 3 compatible )
We could talk quite a bit on this point -- How active do we want to be with
the things that aren't in one of the essential buckets from further up. We
could defer thinking about this until after we get the livecd python2-free,
though.
In past few days, I've been going through packages that are part
of the above steps. I have reported numerous bugs asking upstream and/or Fedora
maintainers for help with porting to Python 3. We have some spare cycles in our small
Python packaging team, that we will try to provide to whoever needs them most, but
we're limited and we'll have to rely on the upstreams to do most of the work.
I'm attaching a document with list of packages that need porting with some notes/links
to opened bugs. Sometime soon, I'll open a tracking bug for this, so that everyone can
see where we are quickly.
(*) I call these "important" packages (in terms of being important for the
Python 3 switch)
Cool. A list of packages that are on the livecd is good. One thing to
remember, though, is that the current Python Guidelines specify that we are
not to ship python3 versions of packages if upstream is not going to support
us in that effort:
https://fedoraproject.org/wiki/Packaging:Python#Subpackages
We could change that but I'm not 100% behind the idea of changing it. As
stated in the Guidelines:
"
[...]doing this on our own in Fedora is essentially creating a fork. That has
a large burden for maintaining the code, fixing bugs, porting when a new
version of upstream's code appears, managing a release schedule, and other
tasks normally handled by upstream. It's much better if we can cooperate
with upstream to share this work than doing it all on our own.
"
Luckily, in recent years I've only encountered a few upstreams that are
unwilling to look at python3 patches. Most upstreams are amenable to
taking patches that establish python3 compatibility. We just need to remain
clear that we have to work upstream to get these python3 versions into
fedora, not do it in our packages without upstream being on board.
From packaging point of view, this will probably require:
1) Renaming python package to python2
2) Renaming python3 package to python
-1: What are the benefits of this as the cost of this is very high in
several ways:
* updating our dependencies
* divergence from other distros (I believe that arch is the only distro that
has decided to ship python3 as "python". everyone else ships python3 as
python3)
* updating our documentation
* divergence from other upstream/googlable documentation
I could see us renaming the python package to python2, keeping a Virtual
Provide in the python2 package for python (and similar for all of the
subpackages and python-doc package), and leaving python3 as it is. This
might be a stepping stone to when the internet's memory hasstarted
associating "python" with python3 instead of python2.
3) Switching the %{?with_python3} conditionals in specfiles to
%{?with_python2} (we will probably create a script to automate this, at least partially)
-1: This one doesn't make any sense to do. The third-party python library
ecosystem is highly weighted for python2. There are only a handful of
libraries that support python3 and not python2. There are a boatload of
libraries that support python2 and not python3. We're starting from a base
of existing python2 packages that may add support for python3. The
conditionals are there to enable packaging of that situation.
FAQ:
Q: Why do we need to switch to Python 3?
A: Because Python 2 is old, slower, less pythonic, doesn't get any more functionality
and it won't be that long before the official upstream support ends [1]
Although I agree with the need to switch to python3, I don't think the first
three reasons are very compelling arguments (they're only half-truths) -- we
should concentrate on the last reason and also on features that python3
has that pyhton2 doesn't. Chained exceptions are a pretty nice thing, for
instance.
Q: How do I port to Python 3?
A: There are tons of tutorials and howtos about porting and the differencies in general.
E.g. [2] (general), [3] (c-extensions)
The best two tutorials for python3 porting are likely:
https://wiki.ubuntu.com/Python/3 (Will be moving to the python.org wiki in
the near future)
http://python3porting.com/
Q: What about Python 2?
A: We will maintain that at least as long as upstream supports it. After that, I'd
prefer dropping it, but since I know there will be people wanting to keep it around,
I'll gladly give the maintenance to someone else.
<nod> 2015 is right around the corner... I think someone else will get
stuck maintaining the package :-/
I'll be glad to answer all your questions and discuss the above points. Nothing is
set into stone and I'd love to hear your ideas and comments.
I sent out a message earlier that we should have a python sig/python
guidelines discussion at flock. I think that nick and I are the only two
that can definitely attend that in person.
Can anyone else make this timeslot on IRC?
http://flock2013.sched.org/event/281138262885f34d97408cfe65cdf21b?iframe=...
Planning for python3 and any needed updates to the Guidelines surrounding
this are one of the things I wanted us to discuss.
[..]
One thing it might be nice to see in the below list is what things we have
some upstream control over already. I believe the gdb work is being driven
by dmalcolm. anaconda and yum/dnf are things we are upstream for. etc.
Knowing about this responsibility will help us to understand where we
control our own destiny and where we're dependent on other upstreams.
In some cases where upstream isn't going to port (for instance, dead
upstream), we may need to either port to a different upstream (potentially
large one-time cost) or fork upstream (ongoing maintainance burden).
One specific note:
- python-pycurl - TODO -
https://github.com/p/pycurl/pull/28 (is this the official upstream?)
You probably need to find someone to take over upstream maintainance of
python-pycurl. Over the past two or three years, various people have
stepped up to take over upstream and never gotten more than a release out
the door.
-Toshio
7:41 p.m.
On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
/usr/bin/python should refer to python2 --
http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
But
when python2 is no longer installed by default, surely you want to
get a python prompt when you type 'python'?
10:44 p.m.
On Jul 18, 2013 5:42 PM, "Michael Catanzaro" <mike.catanzaro(a)gmail.com>
wrote:
On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
> /usr/bin/python should refer to python2 --
> http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
But when python2 is no longer installed by default, surely you want to
get a python prompt when you type 'python'?
Yes, and for a long time, I'm going to want to get a python2 prompt. Which
means installing the package for python2, not having python3 start on that
case.
1:20 a.m.
On 07/19/2013 05:44 AM, Toshio Kuratomi wrote:
On Jul 18, 2013 5:42 PM, "Michael Catanzaro" <mike.catanzaro(a)gmail.com
<mailto:mike.catanzaro@gmail.com>> wrote:
>
> On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
> > /usr/bin/python should refer to python2 --
> > http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
> But when python2 is no longer installed by default, surely you want to
> get a python prompt when you type 'python'?
Yes, and for a long time, I'm going to want to get a python2 prompt.
Which means installing the package for python2, not having python3 start
on that case.
Why do you want to use old version as default? That's very conservative
approach for Fedora ;-)
Upstream plans to support it until 2015 (maybe little longer). Fedora
needs to be prepared for such step, so it's the right time to start
working on it.
Marcela
4:17 a.m.
On Fri, Jul 19, 2013 at 08:20:02AM +0200, Marcela Mašláňová wrote:
On 07/19/2013 05:44 AM, Toshio Kuratomi wrote:
>
>On Jul 18, 2013 5:42 PM, "Michael Catanzaro" <mike.catanzaro(a)gmail.com
><mailto:mike.catanzaro@gmail.com>> wrote:
> >
> > On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
> > > /usr/bin/python should refer to python2 --
> > > http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
> > But when python2 is no longer installed by default, surely you want to
> > get a python prompt when you type 'python'?
>
>Yes, and for a long time, I'm going to want to get a python2 prompt.
>Which means installing the package for python2, not having python3 start
>on that case.
>
>
>
Why do you want to use old version as default? That's very
conservative approach for Fedora ;-)
This is not just about porting packages that Fedora is shipping. There are
a quadrillion and one random scripts out in the wild with #!/usr/bin/python
in them, that only work with python2. If Fedora makes /usr/bin/python be a
python3 interpretor, we'll break pretty much all of them. I don't think we
want to be doing that for a very long time yet, if ever.
Far better to encourage people to explicitly use /usr/bin/python2 and
/usr/bin/python3 explicitly and discourage any use of plain /usr/bin/python,
but definitely not change the semantics of the latter.
Upstream plans to support it until 2015 (maybe little longer).
Fedora needs to be prepared for such step, so it's the right time to
start working on it.
Given the amount of python2 code out there, I wouldn't be at all surprised
if someone steps up to maintain python2 beyond the date at which its
current maintainer ceases work. Of course such maintainence work would
likely be important bug fixes & security updates only, not feature work.
So while I encourage a Fedora effort to get onto Python3 by default,
well before 2015, I don't think we should assume that Python2 support
is definitely going to stop in 2015.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
7:32 a.m.
From: berrange(a)redhat.com
Far better to encourage people to explicitly use /usr/bin/python2 and
/usr/bin/python3 explicitly and discourage any use of plain
/usr/bin/python,
but definitely not change the semantics of the latter.
I think this is by far the best thing to do. After all "import this"
says:
Explicit is better than implicit.
--
John Florian
10:04 a.m.
On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
On Fri, Jul 19, 2013 at 08:20:02AM +0200, Marcela Mašláňová wrote:
> On 07/19/2013 05:44 AM, Toshio Kuratomi wrote:
> >
> >On Jul 18, 2013 5:42 PM, "Michael Catanzaro"
<mike.catanzaro(a)gmail.com
> ><mailto:mike.catanzaro@gmail.com>> wrote:
> > >
> > > On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
> > > > /usr/bin/python should refer to python2 --
> > > > http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing
this
> > > But when python2 is no longer installed by default, surely you want to
> > > get a python prompt when you type 'python'?
> >
> >Yes, and for a long time, I'm going to want to get a python2 prompt.
> >Which means installing the package for python2, not having python3 start
> >on that case.
> >
> >
> >
> Why do you want to use old version as default? That's very
> conservative approach for Fedora ;-)
This is not just about porting packages that Fedora is shipping. There are
a quadrillion and one random scripts out in the wild with #!/usr/bin/python
in them, that only work with python2. If Fedora makes /usr/bin/python be a
python3 interpretor, we'll break pretty much all of them. I don't think we
want to be doing that for a very long time yet, if ever.
Far better to encourage people to explicitly use /usr/bin/python2 and
/usr/bin/python3 explicitly and discourage any use of plain /usr/bin/python,
but definitely not change the semantics of the latter.
> Upstream plans to support it until 2015 (maybe little longer).
> Fedora needs to be prepared for such step, so it's the right time to
> start working on it.
Given the amount of python2 code out there, I wouldn't be at all surprised
if someone steps up to maintain python2 beyond the date at which its
current maintainer ceases work. Of course such maintainence work would
likely be important bug fixes & security updates only, not feature work.
So while I encourage a Fedora effort to get onto Python3 by default,
well before 2015, I don't think we should assume that Python2 support
is definitely going to stop in 2015.
+1 Breaking completely needlessly backwards compatibility this way is
really not a good idea.
--
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
Turkish proverb
10:16 a.m.
Le Ven 19 juillet 2013 17:04, Tomas Mraz a écrit :
On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
> So while I encourage a Fedora effort to get onto Python3 by
default,
> well before 2015, I don't think we should assume that Python2 support
> is definitely going to stop in 2015.
+1 Breaking completely needlessly backwards compatibility this way is
really not a good idea.
OTOH, it's better to give a heads-up this way (with the option to change
shebangs as short-term workaround) than drop users cold at python2 removal
time (which *will* eventually come). Some people really need some breakage
to notice they're missing a migration.
--
Nicolas Mailhot
10:24 a.m.
From: nicolas.mailhot(a)laposte.net
Le Ven 19 juillet 2013 17:04, Tomas Mraz a écrit :
> On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
>> So while I encourage a Fedora effort to get onto Python3 by default,
>> well before 2015, I don't think we should assume that Python2 support
>> is definitely going to stop in 2015.
>
> +1 Breaking completely needlessly backwards compatibility this way is
> really not a good idea.
OTOH, it's better to give a heads-up this way (with the option to change
shebangs as short-term workaround) than drop users cold at python2
removal
time (which *will* eventually come). Some people really need some
breakage
to notice they're missing a migration.
Excellent point! If I were caught unaware, I'd much rather have to fix up
a bunch shebang lines and learn that I need to get going on my migration
pronto than the alternative of finding one day that python2 is just gone
where one is left with either a hurried port or downloading python2 from
external sources.
--
John Florian
10:35 a.m.
On Fri, 2013-07-19 at 11:24 -0400, John.Florian(a)dart.biz wrote:
> From: nicolas.mailhot(a)laposte.net
>
>
> Le Ven 19 juillet 2013 17:04, Tomas Mraz a écrit :
> > On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
>
> >> So while I encourage a Fedora effort to get onto Python3 by default,
> >> well before 2015, I don't think we should assume that Python2 support
> >> is definitely going to stop in 2015.
> >
> > +1 Breaking completely needlessly backwards compatibility this way is
> > really not a good idea.
>
> OTOH, it's better to give a heads-up this way (with the option to change
> shebangs as short-term workaround) than drop users cold at python2
removal
> time (which *will* eventually come). Some people really need some
breakage
> to notice they're missing a migration.
Excellent point! If I were caught unaware, I'd much rather have to fix up
a bunch shebang lines and learn that I need to get going on my migration
pronto than the alternative of finding one day that python2 is just gone
where one is left with either a hurried port or downloading python2 from
external sources.
The heads-up can be done before the python2 dropping by
removing /usr/bin/python from python2 package.
--
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
Turkish proverb
10:58 a.m.
From: tmraz(a)redhat.com
On Fri, 2013-07-19 at 11:24 -0400, John.Florian(a)dart.biz wrote:
> > From: nicolas.mailhot(a)laposte.net
> >
> >
> > Le Ven 19 juillet 2013 17:04, Tomas Mraz a écrit :
> > > On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
> >
> > >> So while I encourage a Fedora effort to get onto Python3 by
default,
> > >> well before 2015, I don't think we should
assume that Python2
support
> > >> is definitely going to stop in 2015.
> > >
> > > +1 Breaking completely needlessly backwards compatibility this way
is
> > > really not a good idea.
> >
> > OTOH, it's better to give a heads-up this way (with the option to
change
> > shebangs as short-term workaround) than drop users cold at
python2
> removal
> > time (which *will* eventually come). Some people really need some
> breakage
> > to notice they're missing a migration.
>
>
> Excellent point! If I were caught unaware, I'd much rather have to
fix
up
> a bunch shebang lines and learn that I need to get going on my
migration
> pronto than the alternative of finding one day that python2 is
just
gone
> where one is left with either a hurried port or downloading
python2
from
> external sources.
The heads-up can be done before the python2 dropping by
removing /usr/bin/python from python2 package.
Ah, true indeed. However, I still like the idea of being explicit. If
you want py2, say so in the shebang line. Likewise with py3. How can we
get the very blunt "head's up" and be explicit too? I have no idea when
py2 will truly go away, but I'm convinced it will eventually and I'd like
to create my works in the best possible fashion. Thus far I've relied on
the implicit py->py2 and explicit py3 invocations as that's the way I've
seen Fedora set examples, but I think this should really be more detailed
in the packaging guidelines.
--
John Florian
11:41 a.m.
On Fri, Jul 19, 2013 at 11:58:35AM -0400, John.Florian(a)dart.biz wrote:
> From: tmraz(a)redhat.com
>
> On Fri, 2013-07-19 at 11:24 -0400, John.Florian(a)dart.biz wrote:
> >
> >
> > Excellent point! If I were caught unaware, I'd much rather have to fix up
> > a bunch shebang lines and learn that I need to get going on my migration
> > pronto than the alternative of finding one day that python2 is just gone
> > where one is left with either a hurried port or downloading python2 from
> > external sources.
>
> The heads-up can be done before the python2 dropping by
> removing /usr/bin/python from python2 package.
Ah, true indeed. However, I still like the idea of being explicit. If you
want py2, say so in the shebang line. Likewise with py3. How can we get the
very blunt "head's up" and be explicit too? I have no idea when py2 will
truly
go away, but I'm convinced it will eventually and I'd like to create my works
in the best possible fashion. Thus far I've relied on the implicit py->py2 and
explicit py3 invocations as that's the way I've seen Fedora set examples, but I
think this should really be more detailed in the packaging guidelines.
I'm not sure if this is a good idea (or in what time frame it may be a good
idea. It's definitely too early now and likely too early for F22) but -- if
you want something explicit, we could replace /usr/bin/python with something
that prints out an error message to switch shebang lines to use
/usr/bin/python2 and exits.
(Perhaps a warning message to switch shebang lines could be done now,
though).
-Toshio
11:50 a.m.
On Fri, Jul 19, 2013 at 10:11 PM, Toshio Kuratomi <a.badger(a)gmail.com> wrote:
I'm not sure if this is a good idea (or in what time frame it may
be a good
idea. It's definitely too early now and likely too early for F22) but -- if
you want something explicit, we could replace /usr/bin/python with something
that prints out an error message to switch shebang lines to use
/usr/bin/python2 and exits.
(Perhaps a warning message to switch shebang lines could be done now,
though).
I personally think it is too early for even that. If one asks me to
make necessary changes only to the projects I am upstream, I will have
to clone myself many times to do that in any near future.
Kushal
10:30 a.m.
On Fri, Jul 19, 2013 at 05:16:03PM +0200, Nicolas Mailhot wrote:
Le Ven 19 juillet 2013 17:04, Tomas Mraz a écrit :
> On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
>> So while I encourage a Fedora effort to get onto Python3 by default,
>> well before 2015, I don't think we should assume that Python2 support
>> is definitely going to stop in 2015.
>
> +1 Breaking completely needlessly backwards compatibility this way is
> really not a good idea.
OTOH, it's better to give a heads-up this way (with the option to change
shebangs as short-term workaround) than drop users cold at python2 removal
time (which *will* eventually come). Some people really need some breakage
to notice they're missing a migration.
I don't think we're anywhere near the point in time where such an
approach is worth the pain it will inflict on people. As above,
I'm pretty sceptical that maintainence of Python2 is actually going
to stop in 2015, as I think there are too many people and/or
organizations who will find they have reasons for wanting to stick
on python2 for a long time. Maybe I'll be proved wrong on this in
time, but I doubt it, as there's far too many prior examples of
people/companies sticking with ancient software versions because
the cost of maintaining them is less than the cost of porting
them.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
10:53 a.m.
Le Ven 19 juillet 2013 17:30, Daniel P. Berrange a écrit :
On Fri, Jul 19, 2013 at 05:16:03PM +0200, Nicolas Mailhot wrote:
>
> Le Ven 19 juillet 2013 17:04, Tomas Mraz a écrit :
> > On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
>
> >> So while I encourage a Fedora effort to get onto Python3 by default,
> >> well before 2015, I don't think we should assume that Python2 support
> >> is definitely going to stop in 2015.
> >
> > +1 Breaking completely needlessly backwards compatibility this way is
> > really not a good idea.
>
> OTOH, it's better to give a heads-up this way (with the option to change
> shebangs as short-term workaround) than drop users cold at python2
> removal
> time (which *will* eventually come). Some people really need some
> breakage
> to notice they're missing a migration.
I don't think we're anywhere near the point in time where such an
approach is worth the pain it will inflict on people. As above,
I'm pretty sceptical that maintainence of Python2 is actually going
to stop in 2015, as I think there are too many people and/or
organizations who will find they have reasons for wanting to stick
on python2 for a long time. Maybe I'll be proved wrong on this in
time, but I doubt it, as there's far too many prior examples of
people/companies sticking with ancient software versions because
the cost of maintaining them is less than the cost of porting
them.
Well, in practical terms if you want to do it graciously you need
time of python renaming in fedora = fedora python2 removal time - one rhel
cycle
because some people won't migrate before their stuff is broken in an
enterprise distro, so for the heads up to be effective it needs to be
advertised a full rhel cycle
--
Nicolas Mailhot
2:21 p.m.
Daniel P. Berrange (berrange(a)redhat.com) said:
I don't think we're anywhere near the point in time where
such an
approach is worth the pain it will inflict on people. As above,
I'm pretty sceptical that maintainence of Python2 is actually going
to stop in 2015, as I think there are too many people and/or
organizations who will find they have reasons for wanting to stick
on python2 for a long time. Maybe I'll be proved wrong on this in
time, but I doubt it, as there's far too many prior examples of
people/companies sticking with ancient software versions because
the cost of maintaining them is less than the cost of porting
them.
Especially since I do believe there are large software infrastructures even
more tied to python2 than Fedora is with yum/anaconda/etc. (Hellooooooo,
OpenStack!)
Bill
3:59 a.m.
On Fri, Jul 19, 2013 at 03:21:12PM -0400, Bill Nottingham wrote:
Daniel P. Berrange (berrange(a)redhat.com) said:
> I don't think we're anywhere near the point in time where such an
> approach is worth the pain it will inflict on people. As above,
> I'm pretty sceptical that maintainence of Python2 is actually going
> to stop in 2015, as I think there are too many people and/or
> organizations who will find they have reasons for wanting to stick
> on python2 for a long time. Maybe I'll be proved wrong on this in
> time, but I doubt it, as there's far too many prior examples of
> people/companies sticking with ancient software versions because
> the cost of maintaining them is less than the cost of porting
> them.
Especially since I do believe there are large software infrastructures even
more tied to python2 than Fedora is with yum/anaconda/etc. (Hellooooooo,
OpenStack!)
There is an effort in openstack to try to make the code python 3 friendly,
but they wish to retain python 2.x compatibility for a while yet since
that's what most of the OS platforms they're targetting have most widely
available. So they're seeing if it is possible to get code that works on
say python 2.7 and 3.x concurrently. AFAIK, there's not official timeline
yet for declaring that openstack formally supports python 3.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
11:36 a.m.
On Fri, Jul 19, 2013 at 10:17:51AM +0100, Daniel P. Berrange wrote:
Far better to encourage people to explicitly use /usr/bin/python2 and
/usr/bin/python3 explicitly and discourage any use of plain /usr/bin/python,
but definitely not change the semantics of the latter.
+1*10^6
> Upstream plans to support it until 2015 (maybe little longer).
> Fedora needs to be prepared for such step, so it's the right time to
> start working on it.
Given the amount of python2 code out there, I wouldn't be at all surprised
if someone steps up to maintain python2 beyond the date at which its
current maintainer ceases work. Of course such maintainence work would
likely be important bug fixes & security updates only, not feature work.
So while I encourage a Fedora effort to get onto Python3 by default,
well before 2015, I don't think we should assume that Python2 support
is definitely going to stop in 2015.
<nod> At this time, I'm pretty sure that someone will step up to
maintain
python2 packages in all major distros after the 2015 date that upstream
stops supporting a CPython2. There's several different ways that could
shake out --
* Distros independently maintain python2 packages
* Distros get together and maintain python2 as a unified project. Possibly
even using the CPython upstream revision control system and issue tracker.
* Distros switch to alternate implementations of python2 (such as pypy) for
their python2 compatible code.
It's too early for me to tell which of these is likely to happen. It
depends mostly on how important python2 still is when upstream CPython
finally decides to stop supporting python2.
-Toshio
11:53 a.m.
On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
Far better to encourage people to explicitly use /usr/bin/python2
and
/usr/bin/python3 explicitly and discourage any use of plain /usr/bin/python
Note the GNOME discussion here:
https://mail.gnome.org/archives/desktop-devel-list/2012-November/msg00005...
One ugly part is that Debian Wheezy does not have "python2". That
combined with the Arch Linux people who made /usr/bin/python = python3
is a terrible mess for people who just want random portable scripts.
For GNOME we decided to just try to ensure python2 always exists:
https://mail.gnome.org/archives/desktop-devel-list/2012-November/msg00017...
12:18 p.m.
On Fri, Jul 19, 2013 at 12:53:20PM -0400, Colin Walters wrote:
On Fri, 2013-07-19 at 10:17 +0100, Daniel P. Berrange wrote:
> Far better to encourage people to explicitly use /usr/bin/python2 and
> /usr/bin/python3 explicitly and discourage any use of plain /usr/bin/python
Note the GNOME discussion here:
https://mail.gnome.org/archives/desktop-devel-list/2012-November/msg00005...
One ugly part is that Debian Wheezy does not have "python2". That
combined with the Arch Linux people who made /usr/bin/python = python3
is a terrible mess for people who just want random portable scripts.
For GNOME we decided to just try to ensure python2 always exists:
https://mail.gnome.org/archives/desktop-devel-list/2012-November/msg00017...
<nod> These two concerns were reasons that PEP394 was written.
http://www.python.org/dev/peps/pep-0394/#rationale
and CPython2's install was changed to create the python2 symlink:
http://www.python.org/dev/peps/pep-0394/#application-to-the-cpython-refer...
which should make it into Debian at some point.
-Toshio
8:38 a.m.
On Fri, Jul 19, 2013 at 11:17 AM, Daniel P. Berrange
<berrange(a)redhat.com> wrote:
Far better to encourage people to explicitly use /usr/bin/python2
and
/usr/bin/python3 explicitly and discourage any use of plain /usr/bin/python,
but definitely not change the semantics of the latter.
Right, treating the existence of /usr/bin/python as a
mistake/historical accident is reasonable.
(As a philosophical aside, isn't it rather sad that we don't have any
established conventions for handling such transition/compatibility
issues? We are treating project and every every case as unique, when
they really aren't; just about any API provider faces the same
versioning/compatibility issues sooner or later. If only there were
some kind of consensus for "how things are done" that any project
could adopt - and were expected to adopt - already in the early stages
of the project when nobody thinks a migration will ever be needed.)
Mirek
11:24 a.m.
On Fri, Jul 19, 2013 at 08:20:02AM +0200, Marcela Mašláňová wrote:
On 07/19/2013 05:44 AM, Toshio Kuratomi wrote:
>
>On Jul 18, 2013 5:42 PM, "Michael Catanzaro" <mike.catanzaro(a)gmail.com
><mailto:mike.catanzaro@gmail.com>> wrote:
> >
> > On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
> > > /usr/bin/python should refer to python2 --
> > > http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
> > But when python2 is no longer installed by default, surely you want to
> > get a python prompt when you type 'python'?
>
>Yes, and for a long time, I'm going to want to get a python2 prompt.
>Which means installing the package for python2, not having python3 start
>on that case.
>
>
>
Why do you want to use old version as default? That's very
conservative approach for Fedora ;-)
This is the wrong mental model of the relationship between python2 and
python3. As a comparison this argument is akin to saying, "C is the old
version of C++. We should be porting all our code to use the C++ compiler
in Fedora" C is much more compatible to C++ than python2 is to python3 so
this would be a less drastic change.
python2 and python3 are separate languages. There is a lot of similarity
between the two and with recent enough versions of python2 (2.7) and python3
(python3.4) and some external libraries (python-six) and by sticking to some
specific coding styles ( http://python3porting.com/noconv.html ) and by
sometimes resorting to having separate files for some python2-specific
routines vs python3-specific routines you can write code that is valid and
runs under either language. That does not mean that they are the same
language.
Upstream plans to support it until 2015 (maybe little longer).
Fedora
needs to be prepared for such step, so it's the right time to start
working on it.
I am wholeheartedly in favor of "working on it". But things like
switching
the default /usr/bin/python are not "working on it". Those are backwards
incompatible, end user visible, changes of expectations that we do not need
to implement now in order to be a leader. When people invoke python, they
expect python2. When they want python3, they expect to type python3 to get
that. At the moment, causing /usr/bin/python to invoke python3 is just
going to cause confusion and end user confusion (arch is the only linux
distro that sets /usr/bin/python to point to python3).
As an example here, let's suppose that we were to stop shipping firefox by
default and started shipping chromium instead. Would we make
/usr/bin/firefox invoke /usr/bin/chromium? Would we have a firefox icon
that started chromium when clicked? When applied here, it does seem to make
more sense for end user's to notice that there isn't a firefox icon and
either consciously open up a different web browser or else install firefox
to get their preferred browsing experience, right? /usr/bin/python is the
same way. People have tons of scripts on their systems with
#!/usr/bin/python shebang lines. If we change /usr/bin/python to
point to /usr/bin/python3 then we'll force them to deal with either porting,
going through all their scripts and replacing them with /usr/bin/python2, or
switching to another distribution which doesn't change the meanings of well
known interpreter paths for no gain.
All this is not to say that there might not be a time in the future when it
makes sense to have python3 assume the /usr/bin/python name. However, that
time is *after* the collective user consciousness has stopped using
/usr/bin/python, not before. When we can go to pycon and talks are assumed
to be targeting python3 rather than python2. After people wanting to do
a simple calculation on the CLI stop doing /usr/bin/python -c 'print 5 + 6'.
This is probably after alternate python implementations like pypy and jython
either die off or finally make the switch to implementing python3. My guess
is this will be after there's a RHEL release that includes /usr/bin/python3
in the default install since RHEL and CentOS are major parts of our
ecosystem and will have to be able to use the python3 syntax on both their
RHEL servers and Fedora machines. To some extent, this also depends on what
other large distros set as their expectations: Debian and Ubuntu currently
point /usr/bin/python to python2. AFAIK, only Arch is pointing to python3.
My intuition is this will be at least two years after we stop shipping
/usr/bin/python (pointing to python2) by default but this could be wrong --
it's about the expectations of the Fedora community and the Linux and python
development communities as a whole.
-Toshio
1:40 a.m.
----- Original Message -----
On Fri, Jul 19, 2013 at 08:20:02AM +0200, Marcela Mašláňová wrote:
> On 07/19/2013 05:44 AM, Toshio Kuratomi wrote:
> >
> >On Jul 18, 2013 5:42 PM, "Michael Catanzaro"
<mike.catanzaro(a)gmail.com
> ><mailto:mike.catanzaro@gmail.com>> wrote:
> > >
> > > On Thu, 2013-07-18 at 09:53 -0700, Toshio Kuratomi wrote:
> > > > /usr/bin/python should refer to python2 --
> > > > http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing
this
> > > But when python2 is no longer installed by default, surely you want to
> > > get a python prompt when you type 'python'?
> >
> >Yes, and for a long time, I'm going to want to get a python2 prompt.
> >Which means installing the package for python2, not having python3 start
> >on that case.
> >
> >
> >
> Why do you want to use old version as default? That's very
> conservative approach for Fedora ;-)
This is the wrong mental model of the relationship between python2 and
python3. As a comparison this argument is akin to saying, "C is the old
version of C++. We should be porting all our code to use the C++ compiler
in Fedora" C is much more compatible to C++ than python2 is to python3 so
this would be a less drastic change.
python2 and python3 are separate languages. There is a lot of similarity
between the two and with recent enough versions of python2 (2.7) and python3
(python3.4) and some external libraries (python-six) and by sticking to some
specific coding styles ( http://python3porting.com/noconv.html ) and by
sometimes resorting to having separate files for some python2-specific
routines vs python3-specific routines you can write code that is valid and
runs under either language. That does not mean that they are the same
language.
The problem is that you're basically saying "my mental model is the right
one", which is not necessarily true for everyone (and not necessarily true
generally). Taking your arguments a bit further, Python 2.6 and 2.7 are different
languages too, since there are some backward incompatible additions to Python 2.7.
> Upstream plans to support it until 2015 (maybe little longer).
Fedora
> needs to be prepared for such step, so it's the right time to start
> working on it.
>
I am wholeheartedly in favor of "working on it". But things like switching
the default /usr/bin/python are not "working on it". Those are backwards
incompatible, end user visible, changes of expectations that we do not need
to implement now in order to be a leader. When people invoke python, they
expect python2. When they want python3, they expect to type python3 to get
that. At the moment, causing /usr/bin/python to invoke python3 is just
going to cause confusion and end user confusion (arch is the only linux
distro that sets /usr/bin/python to point to python3).
As an example here, let's suppose that we were to stop shipping firefox by
default and started shipping chromium instead. Would we make
/usr/bin/firefox invoke /usr/bin/chromium? Would we have a firefox icon
that started chromium when clicked? When applied here, it does seem to make
more sense for end user's to notice that there isn't a firefox icon and
either consciously open up a different web browser or else install firefox
to get their preferred browsing experience, right? /usr/bin/python is the
same way. People have tons of scripts on their systems with
#!/usr/bin/python shebang lines. If we change /usr/bin/python to
point to /usr/bin/python3 then we'll force them to deal with either porting,
going through all their scripts and replacing them with /usr/bin/python2, or
switching to another distribution which doesn't change the meanings of well
known interpreter paths for no gain.
Again, this is all based on your "mental model" and on your assumption that it
is the correct one. I just don't agree.
All this is not to say that there might not be a time in the future
when it
makes sense to have python3 assume the /usr/bin/python name. However, that
time is *after* the collective user consciousness has stopped using
/usr/bin/python, not before. When we can go to pycon and talks are assumed
to be targeting python3 rather than python2. After people wanting to do
a simple calculation on the CLI stop doing /usr/bin/python -c 'print 5 + 6'.
This is probably after alternate python implementations like pypy and jython
either die off or finally make the switch to implementing python3. My guess
is this will be after there's a RHEL release that includes /usr/bin/python3
in the default install since RHEL and CentOS are major parts of our
ecosystem and will have to be able to use the python3 syntax on both their
RHEL servers and Fedora machines. To some extent, this also depends on what
other large distros set as their expectations: Debian and Ubuntu currently
point /usr/bin/python to python2. AFAIK, only Arch is pointing to python3.
My intuition is this will be at least two years after we stop shipping
/usr/bin/python (pointing to python2) by default but this could be wrong --
it's about the expectations of the Fedora community and the Linux and python
development communities as a whole.
-Toshio
--
Regards,
Bohuslav "Slavek" Kabrda.
2:04 a.m.
On Tue, 2013-07-23 at 02:40 -0400, Bohuslav Kabrda wrote:
> python2 and python3 are separate languages. There is a lot of
similarity
> between the two and with recent enough versions of python2 (2.7) and
python3
> (python3.4) and some external libraries (python-six) and by sticking
to some
> specific coding styles ( http://python3porting.com/noconv.html ) and
by
> sometimes resorting to having separate files for some
python2-specific
> routines vs python3-specific routines you can write code that is
valid and
> runs under either language. That does not mean that they are the
same
> language.
>
The problem is that you're basically saying "my mental model is the
right one", which is not necessarily true for everyone (and not
necessarily true generally). Taking your arguments a bit further,
Python 2.6 and 2.7 are different languages too, since there are some
backward incompatible additions to Python 2.7.
Python 2.7 might not be backward compatible with 2.6 or 2.5 but 2.5 and
2.6 is forward compatible with 2.7, which is not true for python 2.* vs
3.*.
That makes them more of different languages than 2.6 vs 2.7.
Pierre
2:26 a.m.
----- Original Message -----
On Tue, 2013-07-23 at 02:40 -0400, Bohuslav Kabrda wrote:
> > python2 and python3 are separate languages. There is a lot of
> similarity
> > between the two and with recent enough versions of python2 (2.7) and
> python3
> > (python3.4) and some external libraries (python-six) and by sticking
> to some
> > specific coding styles ( http://python3porting.com/noconv.html ) and
> by
> > sometimes resorting to having separate files for some
> python2-specific
> > routines vs python3-specific routines you can write code that is
> valid and
> > runs under either language. That does not mean that they are the
> same
> > language.
> >
>
> The problem is that you're basically saying "my mental model is the
> right one", which is not necessarily true for everyone (and not
> necessarily true generally). Taking your arguments a bit further,
> Python 2.6 and 2.7 are different languages too, since there are some
> backward incompatible additions to Python 2.7.
Python 2.7 might not be backward compatible with 2.6 or 2.5 but 2.5 and
2.6 is forward compatible with 2.7, which is not true for python 2.* vs
3.*.
Good point, but not entirely true, IMHO.
If you look at Python 2.7 release notes [1], you'll find out that e.g. float-int
conversions round differently. So if you targeted your code for 2.6 working under the
assumption that the rounding will have a certain result, your script will likely break
(e.g. what you wrote for 2.6 may fail in 2.7).
That makes them more of different languages than 2.6 vs 2.7.
Pierre
[1] http://docs.python.org/dev/whatsnew/2.7.html
--
Regards,
Bohuslav "Slavek" Kabrda.
11:32 p.m.
On Tue, Jul 23, 2013 at 02:40:56AM -0400, Bohuslav Kabrda wrote:
The problem is that you're basically saying "my mental model is the right
one", which is not necessarily true for everyone (and not necessarily true
generally). Taking your arguments a bit further, Python 2.6 and 2.7 are
different languages too, since there are some backward incompatible
additions to Python 2.7.
Even more directly, I am saying that your mental model is wrong. Now then,
I assumed that you would not find that palatable therefore I backed my
mental model up with links to the python-dev mailing list. (The three
threads from the PEP and the earlier one that I linked to explicitly in my
email). What are the links to upstream that you are basing your mental
model on?
-Toshio
4:37 a.m.
Dne 24.7.2013 06:32, Toshio Kuratomi napsal(a):
On Tue, Jul 23, 2013 at 02:40:56AM -0400, Bohuslav Kabrda wrote:
> The problem is that you're basically saying "my mental model is the right
> one", which is not necessarily true for everyone (and not necessarily true
> generally). Taking your arguments a bit further, Python 2.6 and 2.7 are
> different languages too, since there are some backward incompatible
> additions to Python 2.7.
>
Even more directly, I am saying that your mental model is wrong. Now then,
I assumed that you would not find that palatable therefore I backed my
mental model up with links to the python-dev mailing list. (The three
threads from the PEP and the earlier one that I linked to explicitly in my
email). What are the links to upstream that you are basing your mental
model on?
-Toshio
So what is the story with Arch and Python 3? Is that considered total
failure after almost 3 years they did that? Just asking ...
Vít
7:19 a.m.
On Wed, Jul 24, 2013 at 6:32 AM, Toshio Kuratomi <a.badger(a)gmail.com> wrote:
On Tue, Jul 23, 2013 at 02:40:56AM -0400, Bohuslav Kabrda wrote:
>
> The problem is that you're basically saying "my mental model is the right
> one", which is not necessarily true for everyone (and not necessarily true
> generally). Taking your arguments a bit further, Python 2.6 and 2.7 are
> different languages too, since there are some backward incompatible
> additions to Python 2.7.
>
Even more directly, I am saying that your mental model is wrong. Now then,
I assumed that you would not find that palatable therefore I backed my
mental model up with links to the python-dev mailing list. (The three
threads from the PEP and the earlier one that I linked to explicitly in my
email). What are the links to upstream that you are basing your mental
model on?
Ignoring the naming/versioning/PR aspects, Python 3 is developed by
the approximately the same group of developers, who have been quite
public about planning to stop maintenance of the Python 2 branch.
That means Fedora will need to migrate eventually[1], so thinking
about the two versions as just continuing the same language makes more
_practical sense for Fedora_. Treating them as two separate languages
that can just peacefully coexist obfuscates the need to migrate, and
the need for every Python package maintainer to plan for migration.
Mirek
[1] Ignoring the possibilities that upstream will change their mind
about Python 2, or that somebody will pick it up and continue
maintaining it. For all I know that might well happen - but from what
I've seen I don't feel comfortable betting the ability to continue to
run anaconda, yum, koji and other critical infrastructure on such an
outcome.
7:49 a.m.
----- Original Message -----
On Tue, Jul 23, 2013 at 02:40:56AM -0400, Bohuslav Kabrda wrote:
>
> The problem is that you're basically saying "my mental model is the right
> one", which is not necessarily true for everyone (and not necessarily true
> generally). Taking your arguments a bit further, Python 2.6 and 2.7 are
> different languages too, since there are some backward incompatible
> additions to Python 2.7.
>
Even more directly, I am saying that your mental model is wrong. Now then,
I assumed that you would not find that palatable therefore I backed my
mental model up with links to the python-dev mailing list. (The three
threads from the PEP and the earlier one that I linked to explicitly in my
email). What are the links to upstream that you are basing your mental
model on?
Since Python upstream really cares about these things, I started a discussion about this
on their mailing list [1]. So far, it seems that people prefer my mental model, but this
is judging from just 4 answers, 2 of which mentioned this. So let's wait and see.
BTW, if Python 2 and Python 3 were different languages, then IMO it wouldn't make
sense to point /usr/bin/python to Python 3, while upstream plans to actually give the
recommendation to do so sometime in the future.
-Toshio
--
Regards,
Bohuslav "Slavek" Kabrda.
[1] http://mail.python.org/pipermail/python-dev/2013-July/127516.html
2:13 a.m.
----- Original Message -----
Since Python upstream really cares about these things, I started a
discussion
about this on their mailing list [1]. So far, it seems that people prefer my
mental model, but this is judging from just 4 answers, 2 of which mentioned
this. So let's wait and see.
BTW, if Python 2 and Python 3 were different languages, then IMO it wouldn't
make sense to point /usr/bin/python to Python 3, while upstream plans to
actually give the recommendation to do so sometime in the future.
So, judging from what upstream has recommended (and how they will modify pep 394), we
should:
- Mandate usage of /usr/bin/python2 and prohibit usage of /usr/bin/python (I think this
can be done automatically in %__os_install_post, or somewhere similar) - we should
probably do that ASAP
- Rename python to python2 and add "Provides: python" for the time being (the
similar should probably be done for all python packages to keep things consistent) - we
can do this for F21
- Somewhere in the future switch "Provides: python" to python3 stuff and
possibly /usr/bin/python to point to python3, according to upstream recommendation (2015?)
to keep consistent across multiple platforms as much as possible.
Just BTW:
$ repoquery --whatrequires /usr/bin/python | wc -l
1262
If this seems ok, then I'll go on and propose the first step to FPC.
Thanks,
Slavek.
5:26 a.m.
On Jul 25, 2013 12:13 AM, "Bohuslav Kabrda" <bkabrda(a)redhat.com> wrote:
Rather than 2015 I'd tie it to when upstream switches the recommendation in
pep 394.
-Toshio
----- Original Message -----
> Since Python upstream really cares about these things, I started a
discussion
> about this on their mailing list [1]. So far, it seems that
people
prefer my
> mental model, but this is judging from just 4 answers, 2 of
which
mentioned
> this. So let's wait and see.
> BTW, if Python 2 and Python 3 were different languages, then IMO it
wouldn't
> make sense to point /usr/bin/python to Python 3, while upstream
plans to
> actually give the recommendation to do so sometime in the future.
So, judging from what upstream has recommended (and how they will modify
pep 394),
we should:
- Mandate usage of /usr/bin/python2 and prohibit usage of
/usr/bin/python
(I think this can be done automatically in %__os_install_post, or
somewhere
similar) - we should probably do that ASAP
<nod>. Which means patching in a bunch of packages. For os_install_post
modification you're talking about a check that fails the build if
/usr/bin/python is used? I note that upstream envisions some cases where
/usr/bin/python is correct but I don't recall any package where we'd have
invoked that clause (the case is when the code will run on either python2
or on python3)
/me goes to the upstream thread to ask Nick what distutils/setuptools/etc
do when they rewrite shebang lines.
- Rename python to python2 and add "Provides: python" for
the time being
(the similar should probably be done for all python packages to keep
things
consistent) - we can do this for F21
Renaming other packages gets problematic. See the previous discussion on
python-devel(a)lists.fp.o that tomspur championed. (This was one of the
items I was hoping could be discussed at flock) You wouldn't expect to
report bugs for the python3-foo library against the python2-foo package for
instance. One possibility that we talked about was to stop shipping
python3 packages as subpackages of the python2 modules.
- Somewhere in the future switch "Provides: python" to
python3 stuff and
possibly /usr/bin/python to point to python3, according to
upstream
recommendation (2015?) to keep consistent across multiple platforms as much
as possible.
5:49 a.m.
----- Original Message -----
On Jul 25, 2013 12:13 AM, "Bohuslav Kabrda" <
bkabrda(a)redhat.com > wrote:
>
> ----- Original Message -----
> > Since Python upstream really cares about these things, I started a
> > discussion
> > about this on their mailing list [1]. So far, it seems that people prefer
> > my
> > mental model, but this is judging from just 4 answers, 2 of which
> > mentioned
> > this. So let's wait and see.
> > BTW, if Python 2 and Python 3 were different languages, then IMO it
> > wouldn't
> > make sense to point /usr/bin/python to Python 3, while upstream plans to
> > actually give the recommendation to do so sometime in the future.
> >
>
> So, judging from what upstream has recommended (and how they will modify
> pep 394), we should:
> - Mandate usage of /usr/bin/python2 and prohibit usage of /usr/bin/python
> (I think this can be done automatically in %__os_install_post, or
> somewhere similar) - we should probably do that ASAP
<nod>. Which means patching in a bunch of packages. For
os_install_post
modification you're talking about a check that fails the build if
/usr/bin/python is used? I note that upstream envisions some cases where
/usr/bin/python is correct but I don't recall any package where we'd have
invoked that clause (the case is when the code will run on either python2 or
on python3)
I meant something that would actually rewrite the lines, even though
that may be very expensive to run for every build... Failing the build is another option,
yes. I'll try to have a look at how these would be implemented.
/me goes to the upstream thread to ask Nick what
distutils/setuptools/etc do
when they rewrite shebang lines.
Hmm, patching setuptools to do this for us sounds
interesting, although not everything that has /usr/bin/python uses setuptools for build,
unfortunately.
> - Rename python to python2 and add "Provides: python"
for the time being
> (the similar should probably be done for all python packages to keep
> things consistent) - we can do this for F21
Renaming other packages gets problematic. See the previous discussion
on
python-devel(a)lists.fp.o that tomspur championed. (This was one of the items
I was hoping could be discussed at flock) You wouldn't expect to report bugs
for the python3-foo library against the python2-foo package for instance.
One possibility that we talked about was to stop shipping python3 packages
as subpackages of the python2 modules.
Yep, I remember. I'm thinking about this
(hope I'm not repeating anyone's idea from before):
What if we use one python-foo.srpm to produce two python2-foo.rpm and python3-foo.rpm, but
not produce python-foo.rpm? Assuming that python2-foo has Provides: python-foo, I think
that pretty much achieves what we want - unless I'm missing something. The only glitch
I can see is that we would need to tell users that they need to report all bugs for
python-foo component. But they can do the mapping from python3-foo to python-foo now, so
it shouldn't hurt so much, IMO.
> - Somewhere in the future switch "Provides: python" to
python3 stuff and
> possibly /usr/bin/python to point to python3, according to upstream
> recommendation (2015?) to keep consistent across multiple platforms as
> much as possible.
>
Rather than 2015 I'd tie it to when upstream switches the
recommendation in
pep 394.
That is what I meant :) 2015 was just a wild guess of when that might
happen.
-Toshio
--
Regards,
Bohuslav "Slavek" Kabrda.
12:57 p.m.
On Thu, Jul 25, 2013 at 06:49:41AM -0400, Bohuslav Kabrda wrote:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
<nod>. Which means patching in a bunch of packages. For os_install_post
modification you're talking about a check that fails the build if /usr/bin/
python is used? I note that upstream envisions some cases where /usr/bin/
python is correct but I don't recall any package where we'd have invoked
that clause (the case is when the code will run on either python2 or on
python3)
I meant something that would actually rewrite the lines, even though that may
be very expensive to run for every build... Failing the build is another
option, yes. I'll try to have a look at how these would be implemented.
I would be hesitant to do rewriting of the shebang lines. I suppose that we do
re-byte-compile the .pyc and .pyo now but it seems different to be changing
something that is also source code and human readable automatically.
Some other places to make this same change: we also have to have people
change anywhere that they use "python" in scripts (for instance, some python
applications ship a shell script that invokes python /path/to/python_file.py
) and our rpm macros should probably also change to use /usr/bin/python2
instead of /usr/bin/python
/me goes to the upstream thread to ask Nick what distutils/setuptools/etc
do when they rewrite shebang lines.
Hmm, patching setuptools to do this for us sounds interesting, although not
everything that has /usr/bin/python uses setuptools for build, unfortunately.
<nod> And not so unfortunately... setuptools has some pretty iffy code :-/
> - Rename python to python2 and add "Provides: python" for the time
being
(the similar should probably be done for all python packages to keep things
consistent) - we can do this for F21
Renaming other packages gets problematic. See the previous discussion on
python-devel(a)lists.fp.o that tomspur championed. (This was one of the
items I was hoping could be discussed at flock) You wouldn't expect to
report bugs for the python3-foo library against the python2-foo package for
instance. One possibility that we talked about was to stop shipping
python3 packages as subpackages of the python2 modules.
Yep, I remember. I'm thinking about this (hope I'm not repeating anyone's
idea
from before):
What if we use one python-foo.srpm to produce two python2-foo.rpm and
python3-foo.rpm, but not produce python-foo.rpm? Assuming that python2-foo has
Provides: python-foo, I think that pretty much achieves what we want - unless
I'm missing something. The only glitch I can see is that we would need to tell
users that they need to report all bugs for python-foo component. But they can
do the mapping from python3-foo to python-foo now, so it shouldn't hurt so
much, IMO.
<nod> a main package that doesn't get built triggers an intense dislike
for
me but that may be the best thing to do. I do still think that having two
separate SRPMs is advantageous to packages (I've worked on several packages:
docutils, nose) where the package doesn't build or run correctly on one
version of the interpreter and so it holds back package updates. dgilmore
pointed out to me yesterday that because python3 is not currently building
for arm, a lot of python packages won't build for arm despite their python2
versions (which is what actual software in the distribution depends on)
building fine.
> - Somewhere in the future switch "Provides:
python" to python3 stuff and
possibly /usr/bin/python to point to python3, according to upstream
recommendation (2015?) to keep consistent across multiple platforms as much
as possible.
>
Rather than 2015 I'd tie it to when upstream switches the recommendation in
pep 394.
That is what I meant :) 2015 was just a wild guess of when that might happen.
<nod> True enough :-)
-Toshio
6:24 p.m.
On Thu, Jul 25, 2013 at 06:49:41AM -0400, Bohuslav Kabrda wrote:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
/me goes to the upstream thread to ask Nick what distutils/setuptools/etc
do when they rewrite shebang lines.
Hmm, patching setuptools to do this for us sounds interesting, although not
everything that has /usr/bin/python uses setuptools for build, unfortunately.
Haven't gotten much response yet but:
http://mail.python.org/pipermail/distutils-sig/2013-July/022002.html
I think the use of sys.executable means if we use /usr/bin/python2 setup.py
instead of /usr/bin/python setup.py then we'll start to get shebang lines
with /usr/bin/python2 instead of embedded /usr/bin/python.
That will get us through many of the applications that are installed by
packages.
-Toshio
1:41 a.m.
----- Original Message -----
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> Hi all,
> as a new Fedora Python maintainer, I have set myself a goal of moving
> Fedora to Python 3 as a default.
I'm not sure we want to make python3 default depending on what your
definition of default is.
/usr/bin/python should refer to python2 --
http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
So, my definition of default is "all system tools use Python 3, it is the only Python
that gets to minimal buildroot/minimal Fedora installation" - that means:
- livecd can still ship Python 2
- /usr/bin/python points to Python 3
- Please note, that the pep you're referring to also states that "python should
refer to the same target as python2 but may refer to python3 on some bleeding edge
distributions", so this wouldn't really be going against the pep.
The python package itself should probably also remain python2 due to
dependencies and expectations from other distros and documentation --
I think I'd be -1 to changing this
The Fedora live images contain only python3, not python2 -- I'd be heavily
in favour of this. +1
> This is going to be a multirelease effort
> that is going to affect lots of Fedora parts. Since we will need to switch
> default package manager from Yum to DNF (which is supposed to work with
> Python 3), we will need to wait for that. I've been told that DNF should
> be default in F22, so that's my target, too. That should also give
> everyone else plenty of time to work on other essential packages to make
> this happen.
Getting there at the same time as we get to DNF sounds like a good timeline.
(But see my note on anaconda below). +1
> Here is my analysis/proposal:
> Before switching, we need to make sure that everything "important" (*) is
> Python 3 compatible. There are three steps I see in this transition:
> 1) Getting rid of Python 2 in mock minimal buildroot.
I'm not sure about this one as it will cause a lot of package churn. It
might be a necessary pain pointi or it might be a pain point we want to
defer until later in our porting efforts. Have to think about it more.
If you look at the minimal mock buildroot for rawhide now, the only thing that is drawing
in Python is gdb because of it's Python bindings (if I'm not mistaken). So
compiling GDB against Python 3, which should work with newest gdb, will accomplish this
AFAICS.
> 2) Porting Anaconda to Python 3.
+1 -- unfortunately, this probably depends on DNF.... So we may need to push
DNF in F22 and anaconda compatible with python3 in F23.
DNF is a continuous effort. I believe that DNF will provide it's Python 3 bindings
sooner than in F22, so Anaconda devels can simultaneously do porting to Python 3 as well
as to DNF. IMO this is good thing, since they will just do one big rewrite instead of two
smaller.
> 3) Making all livecd packages depend on Python 3 by default (and
eventually
> getting rid of Python 2 from livecd) - this will also require switching
> from Yum to DNF as a default, that is supposed to support Python 3.
+1 -- this is what I see as the eventual goal (or perhaps, livecd python2
free followed by DVD python2 free followed by distro python2 free).
3.5) Switch tools that could target either python2 or python3 to target
python3. Currently the packaging guidelines say to target python2 to
control dep proliferation and because that's the most supported by the
larger python ecosystem. We should switch the recommendation when our
minimal environment must have python3 but does not need to have python2.
IMO we should switch this for F21, since livecd ships Python 3 anyway, so the switch
doesn't have to happen in one point, but can be continuous.
> ( 4) Making as much of the remaining packages Python 3
compatible )
>
We could talk quite a bit on this point -- How active do we want to be with
the things that aren't in one of the essential buckets from further up. We
could defer thinking about this until after we get the livecd python2-free,
though.
This is really the last step, that is somehow tied what you mentioned as a reaction to 3)
- going through the rest of packages on DVD and then whole distro. This will take few more
releases I guess, but it is not that important as sorting out livecd.
> In past few days, I've been going through packages that are
part of the
> above steps. I have reported numerous bugs asking upstream and/or Fedora
> maintainers for help with porting to Python 3. We have some spare cycles
> in our small Python packaging team, that we will try to provide to whoever
> needs them most, but we're limited and we'll have to rely on the upstreams
> to do most of the work. I'm attaching a document with list of packages
> that need porting with some notes/links to opened bugs. Sometime soon,
> I'll open a tracking bug for this, so that everyone can see where we are
> quickly.
>
> (*) I call these "important" packages (in terms of being important for
the
> Python 3 switch)
>
Cool. A list of packages that are on the livecd is good. One thing to
remember, though, is that the current Python Guidelines specify that we are
not to ship python3 versions of packages if upstream is not going to support
us in that effort:
https://fedoraproject.org/wiki/Packaging:Python#Subpackages
We could change that but I'm not 100% behind the idea of changing it. As
stated in the Guidelines:
"
[...]doing this on our own in Fedora is essentially creating a fork. That has
a large burden for maintaining the code, fixing bugs, porting when a new
version of upstream's code appears, managing a release schedule, and other
tasks normally handled by upstream. It's much better if we can cooperate
with upstream to share this work than doing it all on our own.
"
Luckily, in recent years I've only encountered a few upstreams that are
unwilling to look at python3 patches. Most upstreams are amenable to
taking patches that establish python3 compatibility. We just need to remain
clear that we have to work upstream to get these python3 versions into
fedora, not do it in our packages without upstream being on board.
Yep. When I was opening the bugs, I tried to open them in the sense "please work with
upstream to port to Python 3". Only when I found out that upstream supports Python 3,
I asked the maintainer to add python3- subpackage.
> From packaging point of view, this will probably require:
> 1) Renaming python package to python2
> 2) Renaming python3 package to python
-1: What are the benefits of this as the cost of this is very high in
several ways:
* updating our dependencies
* divergence from other distros (I believe that arch is the only distro that
has decided to ship python3 as "python". everyone else ships python3 as
python3)
* updating our documentation
* divergence from other upstream/googlable documentation
I could see us renaming the python package to python2, keeping a Virtual
Provide in the python2 package for python (and similar for all of the
subpackages and python-doc package), and leaving python3 as it is. This
might be a stepping stone to when the internet's memory hasstarted
associating "python" with python3 instead of python2.
> 3) Switching the %{?with_python3} conditionals in specfiles to
> %{?with_python2} (we will probably create a script to automate this, at
> least partially)
>
-1: This one doesn't make any sense to do. The third-party python library
ecosystem is highly weighted for python2. There are only a handful of
libraries that support python3 and not python2. There are a boatload of
libraries that support python2 and not python3. We're starting from a base
of existing python2 packages that may add support for python3. The
conditionals are there to enable packaging of that situation.
And this situation will be changing in the future. Right now, there are not so many Python
packages in Fedora that only support Python 2 (I didn't count, but you don't see
them too often these days). IMO Fedora should lead the way of making Python 3 "the
Python" and Python 2 "the old compat version". This also makes sense in the
traditional linux-distro "one version of package" that we should be trying to
pursue.
>
> FAQ:
> Q: Why do we need to switch to Python 3?
> A: Because Python 2 is old, slower, less pythonic, doesn't get any more
> functionality and it won't be that long before the official upstream
> support ends [1]
>
Although I agree with the need to switch to python3, I don't think the first
three reasons are very compelling arguments (they're only half-truths) -- we
should concentrate on the last reason and also on features that python3
has that pyhton2 doesn't. Chained exceptions are a pretty nice thing, for
instance.
So first three reason:
- Python 2 is old - how is that a half-truth?
- Slower - yes, in the beginning, Python 3 was significantly slower because of nonoptimal
code after the rewrite from Python 3. But with Python 3.3 for instance, you get tons of
speed improvements - decimal module for instance got a significant boost. Brett Cannon had
a nice presentation about speed benchmarking [1]. Yes, Python 3 is slower in some areas,
but mostly it's faster.
- Less Pythonic - where do I start with this? Python 3 got rid of tons of unnecessary
syntactic constructs as well as builtin object methods. E.g. "print" vs.
"print()"; exception raising syntax; dict.iteritems() removed and dict.items()
only left, more consistent unicode handling etc. So in the sense of having only one way to
do things, Python 3 is more pythonic than Python 2. If you read through zen of python, you
can find more arguments for this (e.g. making int and long one type - "Special cases
aren't special enough to break the rules."; simplification/rewrite of parts of
stdlib - "Simple is better than complex.", etc.)
> Q: How do I port to Python 3?
> A: There are tons of tutorials and howtos about porting and the
> differencies in general. E.g. [2] (general), [3] (c-extensions)
>
The best two tutorials for python3 porting are likely:
https://wiki.ubuntu.com/Python/3 (Will be moving to the python.org wiki in
the near future)
http://python3porting.com/
Thanks for the links.
> Q: What about Python 2?
> A: We will maintain that at least as long as upstream supports it. After
> that, I'd prefer dropping it, but since I know there will be people
> wanting to keep it around, I'll gladly give the maintenance to someone
> else.
>
<nod> 2015 is right around the corner... I think someone else will get
stuck maintaining the package :-/
>
> I'll be glad to answer all your questions and discuss the above points.
> Nothing is set into stone and I'd love to hear your ideas and comments.
>
I sent out a message earlier that we should have a python sig/python
guidelines discussion at flock. I think that nick and I are the only two
that can definitely attend that in person.
Can anyone else make this timeslot on IRC?
http://flock2013.sched.org/event/281138262885f34d97408cfe65cdf21b?iframe=...
I'll try.
Planning for python3 and any needed updates to the Guidelines
surrounding
this are one of the things I wanted us to discuss.
[..]
One thing it might be nice to see in the below list is what things we have
some upstream control over already. I believe the gdb work is being driven
by dmalcolm. anaconda and yum/dnf are things we are upstream for. etc.
Knowing about this responsibility will help us to understand where we
control our own destiny and where we're dependent on other upstreams.
In some cases where upstream isn't going to port (for instance, dead
upstream), we may need to either port to a different upstream (potentially
large one-time cost) or fork upstream (ongoing maintainance burden).
One specific note:
> - python-pycurl - TODO -
> https://github.com/p/pycurl/pull/28 (is this the
> official upstream?)
You probably need to find someone to take over upstream maintainance of
python-pycurl. Over the past two or three years, various people have
stepped up to take over upstream and never gotten more than a release out
the door.
Yeah, I'll try to speak to Fedora's maintainer of pycurl to see if he has any
updates on this and we'll see.
-Toshio
Thanks for your thoughts.
Slavek.
--
Regards,
Bohuslav "Slavek" Kabrda.
[1]
https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-th...
1:59 a.m.
On 19. 7. 2013 at 02:41:23, Bohuslav Kabrda wrote:
... snip ...
> +1 -- unfortunately, this probably depends on DNF.... So we may
need to
> push DNF in F22 and anaconda compatible with python3 in F23.
DNF is a continuous effort. I believe that DNF will provide it's Python 3
bindings sooner than in F22, so Anaconda devels can simultaneously do
porting to Python 3 as well as to DNF. IMO this is good thing, since they
will just do one big rewrite instead of two smaller.
That's right, we are already working on the API stabilization and testing dnf
with Anaconda is one part of the entire effort. However there is only so much
we can do ourselves. Stronger participation of the Anaconda team is required,
as we need to know what parts are they missing and what parts need improving.
... snip ...
Thanks
Jan
3:11 p.m.
On Fri, Jul 19, 2013 at 02:41:23AM -0400, Bohuslav Kabrda wrote:
----- Original Message -----
> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> > Hi all,
> > as a new Fedora Python maintainer, I have set myself a goal of moving
> > Fedora to Python 3 as a default.
>
> I'm not sure we want to make python3 default depending on what your
> definition of default is.
>
> /usr/bin/python should refer to python2 --
> http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
>
So, my definition of default is "all system tools use Python 3, it is the only
Python that gets to minimal buildroot/minimal Fedora installation" - that means:
I'm okay with this portion of the definition. One note is I would be hesitant
about the timing of python3 being the only python that is installed into the
minimal buildroot. This should probably happen in rawhide right after
a branching.
- livecd can still ship Python 2
I would consider this to be the goal that we should shoot for, though. We
are constantly fighting for space on the install images and we know that
people who install Fedora would like to have the ability to slim down what
is installed. Shooting for no-python2 on the livecds and after that, no
python2 on the install dvds, and still later, no need for python2 in the
packages in the repository seem like milestones that have actual real value
for end users.
- /usr/bin/python points to Python 3
I am firmly against this. more depth was in my reply to mmaslano although
I'll reply to one thing here:
- Please note, that the pep you're referring to also states
that "python
should refer to the same target as python2 but may refer to python3 on
some bleeding edge distributions", so this wouldn't really be going
against the pep.
This is a misinterpretion of the PEP. (This section is confusing, though:
"python should refer to the same target as python2" is a recommendation to
distributions. "but may refer to python3 on some bleeding edge
distributions" is a statement of fact for end users to watch out for) See
the recommendation section:
"For the time being, it is recommended that python should refer to python2
(however, some distributions have already chosen otherwise; see the
Rationale and Migration Notes below)."
and Future Changes Section:
http://www.python.org/dev/peps/pep-0394/#future-changes-to-this-recommend...
"
It is anticipated that there will eventually come a time where the third
party ecosystem surrounding Python 3 is sufficiently mature for this
recommendation to be updated to suggest that the python symlink refer to
python3 rather than python2.
This recommendation will be periodically reviewed over the next few years,
and updated when the core development team judges it appropriate.
"
The "may refer to python3" phrase is an acknowledgment that arch has moved
to /usr/bin/python == python3 and isn't going to revert even though upstream
thinks it's a... premature time to make that switch. (To be fair to arch,
the discussion and PEP happened as a result of arch making that switch so
they'd already committed to that before the consensus was formed that this
would be a bad thing to do atthis time. We don't have that excuse ;-)
If you'd like to read the discussions for yourself, there are three threads
linked from the PEP. An even earlier one is at:
http://mail.python.org/pipermail/python-dev/2010-November/105252.html
> The python package itself should probably also remain python2 due to
> dependencies and expectations from other distros and documentation --
> I think I'd be -1 to changing this
>
> The Fedora live images contain only python3, not python2 -- I'd be heavily
> in favour of this. +1
>
> > This is going to be a multirelease effort
> > that is going to affect lots of Fedora parts. Since we will need to switch
> > default package manager from Yum to DNF (which is supposed to work with
> > Python 3), we will need to wait for that. I've been told that DNF should
> > be default in F22, so that's my target, too. That should also give
> > everyone else plenty of time to work on other essential packages to make
> > this happen.
>
> Getting there at the same time as we get to DNF sounds like a good timeline.
> (But see my note on anaconda below). +1
>
> > Here is my analysis/proposal:
> > Before switching, we need to make sure that everything "important"
(*) is
> > Python 3 compatible. There are three steps I see in this transition:
> > 1) Getting rid of Python 2 in mock minimal buildroot.
>
> I'm not sure about this one as it will cause a lot of package churn. It
> might be a necessary pain pointi or it might be a pain point we want to
> defer until later in our porting efforts. Have to think about it more.
>
If you look at the minimal mock buildroot for rawhide now, the only thing that is drawing
in Python is gdb because of it's Python bindings (if I'm not mistaken). So
compiling GDB against Python 3, which should work with newest gdb, will accomplish this
AFAICS.
<nod> I thought that python might be one of the packages that showed up
in the dep chain here: http://fedoraproject.org/wiki/Packaging:Guidelines#Exceptions_2
which would mean that packages might be leaving it out of BuildRequires
right now. I wasn't able to find a dep chain leading back to python from
one of those so I think that any package which isn't explicitly
BuildRequiring python just has a bug. No objection here although as noted
earlier, we should probably do this right after a rawhide branch so that any
of these bugs are found and fixed in plenty of time.
> > 2) Porting Anaconda to Python 3.
>
> +1 -- unfortunately, this probably depends on DNF.... So we may need to push
> DNF in F22 and anaconda compatible with python3 in F23.
>
DNF is a continuous effort. I believe that DNF will provide it's Python 3 bindings
sooner than in F22, so Anaconda devels can simultaneously do porting to Python 3 as well
as to DNF. IMO this is good thing, since they will just do one big rewrite instead of two
smaller.
Well.... 1) If DNF lands their python3 bindings sooner, that's fine for the
timeline. But if they don't, anaconda can't be finished until after. So
this is something to note as a key piece of the switch. 2) Fedora
anaconda experience (and general open source development experience as
well... you've read the Joel on Software article about netscape, right?)
would tend to show that big rewrites are worse than several smaller ones.
Sure, smaller ones mean that you touch the same code a few times before
you're satisfied with it. But smaller ones mean that you can stop partway
through (say, for instance, because we needed a few extra weeks to port to
DNF and that doesn't leave us enough time to complete the python3 port
[even assuming that that doesn't take longer than anticipated] in time for
Fedora 22's release date) which is just anticipating that Murphy's Law is
inevitably going to throw a wrench into your timeline for completion.
> > 3) Making all livecd packages depend on Python 3 by default
(and eventually
> > getting rid of Python 2 from livecd) - this will also require switching
> > from Yum to DNF as a default, that is supposed to support Python 3.
>
> +1 -- this is what I see as the eventual goal (or perhaps, livecd python2
> free followed by DVD python2 free followed by distro python2 free).
>
>
> 3.5) Switch tools that could target either python2 or python3 to target
> python3. Currently the packaging guidelines say to target python2 to
> control dep proliferation and because that's the most supported by the
> larger python ecosystem. We should switch the recommendation when our
> minimal environment must have python3 but does not need to have python2.
>
IMO we should switch this for F21, since livecd ships Python 3 anyway, so the switch
doesn't have to happen in one point, but can be continuous.
Ehhh... I don't think the livecd having to ship python3 is a good measure
for this. I think something considerably more minimal than the livecd would
be better. talked to mattdm (since he's been working on minimal
environments) and he suggested @core or @standard groups might be
appropriate.) The idea is to avoid doubling the needed python stacks on
minimal environments until necessary. Switching tools that have the option
of running on either one or the other of the stacks to python3 prematurely
means that we start doubling the the python stacks needed before it's
necessary.
> > ( 4) Making as much of the remaining packages Python 3
compatible )
> >
>
> We could talk quite a bit on this point -- How active do we want to be with
> the things that aren't in one of the essential buckets from further up. We
> could defer thinking about this until after we get the livecd python2-free,
> though.
>
This is really the last step, that is somehow tied what you mentioned as
a reaction to 3) - going through the rest of packages on DVD and then
whole distro. This will take few more releases I guess, but it is not that
important as sorting out livecd.
yeah, this strikes me as extending far into the future. I will note,
though, that ideas about changing /usr/bin/python to point to python3
probably come in the latter stages of this step rather than before.
>
> > 3) Switching the %{?with_python3} conditionals in specfiles to
> > %{?with_python2} (we will probably create a script to automate this, at
> > least partially)
> >
> -1: This one doesn't make any sense to do. The third-party python library
> ecosystem is highly weighted for python2. There are only a handful of
> libraries that support python3 and not python2. There are a boatload of
> libraries that support python2 and not python3. We're starting from a base
> of existing python2 packages that may add support for python3. The
> conditionals are there to enable packaging of that situation.
>
And this situation will be changing in the future. Right now, there are not so many
Python packages in Fedora that only support Python 2 (I didn't count, but you
don't see them too often these days).
Uh.... What's your methodology? This is a very, very, very bad estimate but
I think it'll show that we need more than an anecdote to prove that
statement:
$ repoquery -q 'python3-*' |wc -l
259
$ repoquery -q 'python-*' |wc -l
1099
Also, just to be clear, you do understand why switching the conditionals
doesn't work for our existing packages, correct?
IMO Fedora should lead the way of making Python 3 "the
Python" and Python 2 "the old compat version". This also makes sense in the
traditional linux-distro "one version of package" that we should be trying to
pursue.
-1. I'm serious, you've got the wrong conceptual model of the relationship
between python2 and python3 stuck in your head and it's coloring what you're
trying to achieve in bad ways. Python3 is not an upgrade to Python2.
Python3 is a new language. It is compatible in many ways. If you can
target recent enough versions (at least python-2.6 but python2.7 is better
and python-3.3) you can set out to purposefully code things that work on
both languages. But if you're writing general, working python2 code using
idioms and thought processes that you've mastered over the last 10 years,
chances are extremely high that not even your simple scripts are going to
run without modification.
> >
> > FAQ:
> > Q: Why do we need to switch to Python 3?
> > A: Because Python 2 is old, slower, less pythonic, doesn't get any more
> > functionality and it won't be that long before the official upstream
> > support ends [1]
> >
> Although I agree with the need to switch to python3, I don't think the first
> three reasons are very compelling arguments (they're only half-truths) -- we
> should concentrate on the last reason and also on features that python3
> has that pyhton2 doesn't. Chained exceptions are a pretty nice thing, for
> instance.
>
So first three reason:
- Python 2 is old - how is that a half-truth?
C is older. Let's get rid of that first. Old is not a reason to switch
or get rid of it.
- Slower - yes, in the beginning, Python 3 was significantly slower
because of nonoptimal code after the rewrite from Python 3. But with Python 3.3 for
instance, you get tons of speed improvements - decimal module for instance got a
significant boost. Brett Cannon had a nice presentation about speed benchmarking [1]. Yes,
Python 3 is slower in some areas, but mostly it's faster.
* pypy is faster and mostly python2 compatible. Many shops that want speed
are switching to that rather than python3. (Not saying this is a good
idea for a distro to do. I'm just saying that making the speed argument
is not compelling).
* people don't use python for raw speed of processing. They really just
care if it's fast enough. People who write python code would be happy to
take speed increases if they were free. But python2 to python3 requires
porting code so it comes with a significant cost. Speed is a side effect
of switching, not a reason to switch.
- Less Pythonic - where do I start with this? Python 3 got rid of
tons of unnecessary syntactic constructs as well as builtin object methods. E.g.
"print" vs. "print()"; exception raising syntax; dict.iteritems()
removed and dict.items() only left, more consistent unicode handling etc. So in the sense
of having only one way to do things, Python 3 is more pythonic than Python 2. If you read
through zen of python, you can find more arguments for this (e.g. making int and long one
type - "Special cases aren't special enough to break the rules.";
simplification/rewrite of parts of stdlib - "Simple is better than complex.",
etc.)
pythonic is a very vague statement and I wouldn't consider most of your list
to be examples of those. Yes, python3 may be a *better* language (and
I would include most of your list as "features of python3 that python2 does
not have) but a more pythonic language... that's not something that you can
readily measure. For instance, I can make the case that python3's unicode
handling is less pythonic than python2 as it violates three rules of the zen
of python:
Explicit is better than implicit.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
(To be fair, python2 violated some of these rules in its unicode handling as
well, although errors should never pass silently would probably take some
work to convince most people :-)
Anyhow, I stick to my assertion that we should be talking mostly about
upstream support ending as the reason to switch and also the features that
python3 provides that python2 does not [and as noted, I'd lump your
"pythonic" list into this category.] Stating a compelling argument of why
people should change isn't just about identifying all the things that
the change will do. It's also about identifying the things that the change
will do that are important to the person and resonate with their needs.
-Toshio
10:04 a.m.
Le Ven 19 juillet 2013 22:11, Toshio Kuratomi a écrit :
Python3 is not an upgrade to Python2.
Python3 is a new language. It is compatible in many ways. If you can
target recent enough versions (at least python-2.6 but python2.7 is better
and python-3.3) you can set out to purposefully code things that work on
both languages. But if you're writing general, working python2 code using
idioms and thought processes that you've mastered over the last 10 years,
chances are extremely high that not even your simple scripts are going to
run without modification.
How exactly is it different from when gcc grew standard C++ behaviour and
most C++ apps broke right and left?
http://gcc.gnu.org/gcc-2.96.html
--
Nicolas Mailhot
11:08 a.m.
On Saturday 20 July 2013 17:04:47 Nicolas Mailhot wrote:
How exactly is it different from when gcc grew standard C++ behaviour
and
most C++ apps broke right and left?
http://gcc.gnu.org/gcc-2.96.html
The same can be said about C++/11 that clearly looks (at least to me) as a new and better
language than C++/03 (or older).
I am aware that this goes against Toshio's statement, a statement that I can
understand and agree but really python3 is a better python than python2 was (is). Surely
that sometimes age make us wiser.
On the other hand according to python's zen:
$ python -m 'this' | grep practical
Although practicality beats purity.
That is why I agree with Toshio POV.
FWIW, and before you ask, most of the time before expressing my thoughts I don't need
to fire a python session. ;-)
Regards,
--
José Abílio
2:13 a.m.
----- Original Message -----
I'm okay with this portion of the definition. One note is I
would be
hesitant
about the timing of python3 being the only python that is installed into the
minimal buildroot. This should probably happen in rawhide right after
a branching.
To be more specific, I meant "installed into the minimal buildroot by default",
which only means building GDB with Python 3, as I've written elsewhere. IMO it
shouldn't cause any problems, because all the packages that use python 2 during build
should BR: python2-devel anyway, right? (maybe some packages with build scripts written in
python 2 may have minor issues, but these should be solvable by simply adding BR: python
or rewriting to python 3)
> - livecd can still ship Python 2
I would consider this to be the goal that we should shoot for, though. We
are constantly fighting for space on the install images and we know that
people who install Fedora would like to have the ability to slim down what
is installed. Shooting for no-python2 on the livecds and after that, no
python2 on the install dvds, and still later, no need for python2 in the
packages in the repository seem like milestones that have actual real value
for end users.
Ok, +1
> - /usr/bin/python points to Python 3
I am firmly against this. more depth was in my reply to mmaslano although
I'll reply to one thing here:
> - Please note, that the pep you're referring to also states that "python
> should refer to the same target as python2 but may refer to python3 on
> some bleeding edge distributions", so this wouldn't really be going
> against the pep.
>
This is a misinterpretion of the PEP. (This section is confusing, though:
"python should refer to the same target as python2" is a recommendation to
distributions. "but may refer to python3 on some bleeding edge
distributions" is a statement of fact for end users to watch out for) See
the recommendation section:
"For the time being, it is recommended that python should refer to python2
(however, some distributions have already chosen otherwise; see the
Rationale and Migration Notes below)."
and Future Changes Section:
http://www.python.org/dev/peps/pep-0394/#future-changes-to-this-recommend...
"
It is anticipated that there will eventually come a time where the third
party ecosystem surrounding Python 3 is sufficiently mature for this
recommendation to be updated to suggest that the python symlink refer to
python3 rather than python2.
This recommendation will be periodically reviewed over the next few years,
and updated when the core development team judges it appropriate.
"
The "may refer to python3" phrase is an acknowledgment that arch has moved
to /usr/bin/python == python3 and isn't going to revert even though upstream
thinks it's a... premature time to make that switch. (To be fair to arch,
the discussion and PEP happened as a result of arch making that switch so
they'd already committed to that before the consensus was formed that this
would be a bad thing to do atthis time. We don't have that excuse ;-)
If you'd like to read the discussions for yourself, there are three threads
linked from the PEP. An even earlier one is at:
http://mail.python.org/pipermail/python-dev/2010-November/105252.html
One way or the other, the PEP states that people should be careful about what
/usr/bin/python points to, since it may be python 2 as well as python 3. This gives us the
freedom to do the switch when we want and not break the upstream expectations (yes, I know
that huge number of people expect /usr/bin/python to point to python 3, but this pep
basically says that it's a bad idea to assume that).
>
> > The python package itself should probably also remain python2 due to
> > dependencies and expectations from other distros and documentation --
> > I think I'd be -1 to changing this
> >
> > The Fedora live images contain only python3, not python2 -- I'd be
> > heavily
> > in favour of this. +1
> >
> > > This is going to be a multirelease effort
> > > that is going to affect lots of Fedora parts. Since we will need to
> > > switch
> > > default package manager from Yum to DNF (which is supposed to work with
> > > Python 3), we will need to wait for that. I've been told that DNF
> > > should
> > > be default in F22, so that's my target, too. That should also give
> > > everyone else plenty of time to work on other essential packages to
> > > make
> > > this happen.
> >
> > Getting there at the same time as we get to DNF sounds like a good
> > timeline.
> > (But see my note on anaconda below). +1
> >
> > > Here is my analysis/proposal:
> > > Before switching, we need to make sure that everything
"important" (*)
> > > is
> > > Python 3 compatible. There are three steps I see in this transition:
> > > 1) Getting rid of Python 2 in mock minimal buildroot.
> >
> > I'm not sure about this one as it will cause a lot of package churn. It
> > might be a necessary pain pointi or it might be a pain point we want to
> > defer until later in our porting efforts. Have to think about it more.
> >
>
> If you look at the minimal mock buildroot for rawhide now, the only thing
> that is drawing in Python is gdb because of it's Python bindings (if I'm
> not mistaken). So compiling GDB against Python 3, which should work with
> newest gdb, will accomplish this AFAICS.
>
<nod> I thought that python might be one of the packages that showed up
in the dep chain here:
http://fedoraproject.org/wiki/Packaging:Guidelines#Exceptions_2
which would mean that packages might be leaving it out of BuildRequires
right now. I wasn't able to find a dep chain leading back to python from
one of those so I think that any package which isn't explicitly
BuildRequiring python just has a bug. No objection here although as noted
earlier, we should probably do this right after a rawhide branch so that any
of these bugs are found and fixed in plenty of time.
> > > 2) Porting Anaconda to Python 3.
> >
> > +1 -- unfortunately, this probably depends on DNF.... So we may need to
> > push
> > DNF in F22 and anaconda compatible with python3 in F23.
> >
>
> DNF is a continuous effort. I believe that DNF will provide it's Python 3
> bindings sooner than in F22, so Anaconda devels can simultaneously do
> porting to Python 3 as well as to DNF. IMO this is good thing, since they
> will just do one big rewrite instead of two smaller.
>
Well.... 1) If DNF lands their python3 bindings sooner, that's fine for the
timeline. But if they don't, anaconda can't be finished until after. So
this is something to note as a key piece of the switch. 2) Fedora
anaconda experience (and general open source development experience as
well... you've read the Joel on Software article about netscape, right?)
would tend to show that big rewrites are worse than several smaller ones.
Sure, smaller ones mean that you touch the same code a few times before
you're satisfied with it. But smaller ones mean that you can stop partway
through (say, for instance, because we needed a few extra weeks to port to
DNF and that doesn't leave us enough time to complete the python3 port
[even assuming that that doesn't take longer than anticipated] in time for
Fedora 22's release date) which is just anticipating that Murphy's Law is
inevitably going to throw a wrench into your timeline for completion.
And still, they can write Python 3 code that is compatible with both Python 2 and Python
3, so if DNF fails to provide Python 3 bindings in time, they can run on Python 2.
> > > 3) Making all livecd packages depend on Python 3 by
default (and
> > > eventually
> > > getting rid of Python 2 from livecd) - this will also require switching
> > > from Yum to DNF as a default, that is supposed to support Python 3.
> >
> > +1 -- this is what I see as the eventual goal (or perhaps, livecd python2
> > free followed by DVD python2 free followed by distro python2 free).
> >
> >
> > 3.5) Switch tools that could target either python2 or python3 to target
> > python3. Currently the packaging guidelines say to target python2 to
> > control dep proliferation and because that's the most supported by the
> > larger python ecosystem. We should switch the recommendation when our
> > minimal environment must have python3 but does not need to have python2.
> >
>
> IMO we should switch this for F21, since livecd ships Python 3 anyway, so
> the switch doesn't have to happen in one point, but can be continuous.
>
Ehhh... I don't think the livecd having to ship python3 is a good measure
for this. I think something considerably more minimal than the livecd would
be better. talked to mattdm (since he's been working on minimal
environments) and he suggested @core or @standard groups might be
appropriate.) The idea is to avoid doubling the needed python stacks on
minimal environments until necessary. Switching tools that have the option
of running on either one or the other of the stacks to python3 prematurely
means that we start doubling the the python stacks needed before it's
necessary.
As I replied to mattdm, I'm not against switching this at a single time (although
doing more little steps is better than doing a single huge step ;)).
> > > ( 4) Making as much of the remaining packages Python 3
compatible )
> > >
> >
> > We could talk quite a bit on this point -- How active do we want to be
> > with
> > the things that aren't in one of the essential buckets from further up.
> > We
> > could defer thinking about this until after we get the livecd
> > python2-free,
> > though.
> >
>
> This is really the last step, that is somehow tied what you mentioned as
> a reaction to 3) - going through the rest of packages on DVD and then
> whole distro. This will take few more releases I guess, but it is not that
> important as sorting out livecd.
>
yeah, this strikes me as extending far into the future. I will note,
though, that ideas about changing /usr/bin/python to point to python3
probably come in the latter stages of this step rather than before.
> >
> > > 3) Switching the %{?with_python3} conditionals in specfiles to
> > > %{?with_python2} (we will probably create a script to automate this, at
> > > least partially)
> > >
> > -1: This one doesn't make any sense to do. The third-party python
> > library
> > ecosystem is highly weighted for python2. There are only a handful of
> > libraries that support python3 and not python2. There are a boatload of
> > libraries that support python2 and not python3. We're starting from a
> > base
> > of existing python2 packages that may add support for python3. The
> > conditionals are there to enable packaging of that situation.
> >
>
> And this situation will be changing in the future. Right now, there are not
> so many Python packages in Fedora that only support Python 2 (I didn't
> count, but you don't see them too often these days).
>
Uh.... What's your methodology? This is a very, very, very bad estimate but
I think it'll show that we need more than an anecdote to prove that
statement:
$ repoquery -q 'python3-*' |wc -l
259
$ repoquery -q 'python-*' |wc -l
1099
Heh, true :) I based this on packages that I review (out of which likely 90 % or so have
python3- subpackage) and I didn't really count. Well, it is high time we start working
on these 700 :)
Also, just to be clear, you do understand why switching the
conditionals
doesn't work for our existing packages, correct?
Please be more specific. What do you mean "doesn't work"?
> IMO Fedora should lead the way of making Python 3 "the
Python" and Python 2
> "the old compat version". This also makes sense in the traditional
> linux-distro "one version of package" that we should be trying to pursue.
>
-1. I'm serious, you've got the wrong conceptual model of the relationship
between python2 and python3 stuck in your head and it's coloring what you're
trying to achieve in bad ways. Python3 is not an upgrade to Python2.
Python3 is a new language. It is compatible in many ways. If you can
target recent enough versions (at least python-2.6 but python2.7 is better
and python-3.3) you can set out to purposefully code things that work on
both languages. But if you're writing general, working python2 code using
idioms and thought processes that you've mastered over the last 10 years,
chances are extremely high that not even your simple scripts are going to
run without modification.
And again, I'm saying that your conceptual model is not necessarily the correct one.
>
> > >
> > > FAQ:
> > > Q: Why do we need to switch to Python 3?
> > > A: Because Python 2 is old, slower, less pythonic, doesn't get any
more
> > > functionality and it won't be that long before the official upstream
> > > support ends [1]
> > >
> > Although I agree with the need to switch to python3, I don't think the
> > first
> > three reasons are very compelling arguments (they're only half-truths) --
> > we
> > should concentrate on the last reason and also on features that python3
> > has that pyhton2 doesn't. Chained exceptions are a pretty nice thing,
> > for
> > instance.
> >
>
> So first three reason:
> - Python 2 is old - how is that a half-truth?
>
C is older. Let's get rid of that first. Old is not a reason to switch
or get rid of it.
Python 2 has Python 3 as a successor, C does not (yes, I know you'll say C++, but this
is really not the case).
> - Slower - yes, in the beginning, Python 3 was significantly slower because
> of nonoptimal code after the rewrite from Python 3. But with Python 3.3
> for instance, you get tons of speed improvements - decimal module for
> instance got a significant boost. Brett Cannon had a nice presentation
> about speed benchmarking [1]. Yes, Python 3 is slower in some areas, but
> mostly it's faster.
* pypy is faster and mostly python2 compatible. Many shops that want speed
are switching to that rather than python3. (Not saying this is a good
idea for a distro to do. I'm just saying that making the speed argument
is not compelling).
* people don't use python for raw speed of processing. They really just
care if it's fast enough. People who write python code would be happy to
take speed increases if they were free. But python2 to python3 requires
porting code so it comes with a significant cost. Speed is a side effect
of switching, not a reason to switch.
It is one of the reasons to switch for me.
> - Less Pythonic - where do I start with this? Python 3 got rid
of tons of
> unnecessary syntactic constructs as well as builtin object methods. E.g.
> "print" vs. "print()"; exception raising syntax;
dict.iteritems() removed
> and dict.items() only left, more consistent unicode handling etc. So in
> the sense of having only one way to do things, Python 3 is more pythonic
> than Python 2. If you read through zen of python, you can find more
> arguments for this (e.g. making int and long one type - "Special cases
> aren't special enough to break the rules."; simplification/rewrite of
> parts of stdlib - "Simple is better than complex.", etc.)
>
pythonic is a very vague statement and I wouldn't consider most of your list
to be examples of those. Yes, python3 may be a *better* language (and
I would include most of your list as "features of python3 that python2 does
not have) but a more pythonic language... that's not something that you can
readily measure. For instance, I can make the case that python3's unicode
handling is less pythonic than python2 as it violates three rules of the zen
of python:
Explicit is better than implicit.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
Eh, I really don't see where Python 3 unicode handling violates these. Could you be
more specific?
(To be fair, python2 violated some of these rules in its unicode
handling as
well, although errors should never pass silently would probably take some
work to convince most people :-)
Anyhow, I stick to my assertion that we should be talking mostly about
upstream support ending as the reason to switch and also the features that
python3 provides that python2 does not [and as noted, I'd lump your
"pythonic" list into this category.] Stating a compelling argument of why
people should change isn't just about identifying all the things that
the change will do. It's also about identifying the things that the change
will do that are important to the person and resonate with their needs.
-Toshio
--
Regards,
Bohuslav "Slavek" Kabrda.
7:37 p.m.
On 19/07/13 02:41 -0400, Bohuslav Kabrda wrote:
----- Original Message -----
> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
>> FAQ:
>> Q: Why do we need to switch to Python 3?
>> A: Because Python 2 is old, slower, less pythonic, doesn't get any more
>> functionality and it won't be that long before the official upstream
>> support ends [1]
>>
> Although I agree with the need to switch to python3, I don't think the first
> three reasons are very compelling arguments (they're only half-truths) -- we
> should concentrate on the last reason and also on features that python3
> has that pyhton2 doesn't. Chained exceptions are a pretty nice thing, for
> instance.
>
So first three reason:
- Python 2 is old - how is that a half-truth?
- Slower - yes, in the beginning, Python 3 was significantly slower
because of nonoptimal code after the rewrite from Python 3. But with
Python 3.3 for instance, you get tons of speed improvements -
decimal module for instance got a significant boost. Brett Cannon
had a nice presentation about speed benchmarking [1]. Yes, Python 3
is slower in some areas, but mostly it's faster.
Mind to share some grounds for this claim? My negligible experience
told me the contrary, but perhaps timeit module is a bad indicator.
Otherwise yes, let's get gently beyond 3000.
--
Jan
2:16 a.m.
----- Original Message -----
On 19/07/13 02:41 -0400, Bohuslav Kabrda wrote:
> ----- Original Message -----
>> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
>>> FAQ:
>>> Q: Why do we need to switch to Python 3?
>>> A: Because Python 2 is old, slower, less pythonic, doesn't get any more
>>> functionality and it won't be that long before the official upstream
>>> support ends [1]
>>>
>> Although I agree with the need to switch to python3, I don't think the
>> first
>> three reasons are very compelling arguments (they're only half-truths) --
>> we
>> should concentrate on the last reason and also on features that python3
>> has that pyhton2 doesn't. Chained exceptions are a pretty nice thing, for
>> instance.
>>
>
> So first three reason:
> - Python 2 is old - how is that a half-truth?
> - Slower - yes, in the beginning, Python 3 was significantly slower
> because of nonoptimal code after the rewrite from Python 3. But with
> Python 3.3 for instance, you get tons of speed improvements -
> decimal module for instance got a significant boost. Brett Cannon
> had a nice presentation about speed benchmarking [1]. Yes, Python 3
> is slower in some areas, but mostly it's faster.
Mind to share some grounds for this claim? My negligible experience
told me the contrary, but perhaps timeit module is a bad indicator.
There is the presentation that I referenced in the email that you reacted to (referenced
by [1]). You can also have a look at pystone benchmark results (a quick googling around
gave me e.g. http://www.levigross.com/post/2340736877/pystone-benchmark-on-2-6-2-7-3-2)
Otherwise yes, let's get gently beyond 3000.
--
Jan
--
devel mailing list
devel(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
--
Regards,
Bohuslav "Slavek" Kabrda.
8:15 a.m.
On 23/07/13 03:16 -0400, Bohuslav Kabrda wrote:
----- Original Message -----
> On 19/07/13 02:41 -0400, Bohuslav Kabrda wrote:
>> ----- Original Message -----
>>> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
>>>> FAQ:
>>>> Q: Why do we need to switch to Python 3?
>>>> A: Because Python 2 is old, slower, less pythonic, doesn't get any
more
>>>> functionality and it won't be that long before the official upstream
>>>> support ends [1]
>>>>
>>> Although I agree with the need to switch to python3, I don't think the
>>> first
>>> three reasons are very compelling arguments (they're only half-truths)
--
>>> we
>>> should concentrate on the last reason and also on features that python3
>>> has that pyhton2 doesn't. Chained exceptions are a pretty nice thing,
for
>>> instance.
>>>
>>
>> So first three reason:
>> - Python 2 is old - how is that a half-truth?
>> - Slower - yes, in the beginning, Python 3 was significantly slower
>> because of nonoptimal code after the rewrite from Python 3. But with
>> Python 3.3 for instance, you get tons of speed improvements -
>> decimal module for instance got a significant boost. Brett Cannon
>> had a nice presentation about speed benchmarking [1]. Yes, Python 3
>> is slower in some areas, but mostly it's faster.
>
> Mind to share some grounds for this claim? My negligible experience
> told me the contrary, but perhaps timeit module is a bad indicator.
>
There is the presentation that I referenced in the email that you
reacted to (referenced by [1]).
Saw the presentation before actually replying -- it doesn't show
any evidence Python 3.3 would be noticably faster. And my micro
tests unfortunately fell into the "slower" category as nicely
demonstrated in the graphs there.
You can also have a look at pystone benchmark results (a quick
googling
around gave me e.g.
http://www.levigross.com/post/2340736877/pystone-benchmark-on-2-6-2-7-3-2)
This is more than two years old. The score reliability of this benchmark
is also questionable (not speaking about order of magnitude difference as
with PyPy).
> Otherwise yes, let's get gently beyond 3000.
--
Jan
1:12 a.m.
----- Original Message -----
On 23/07/13 03:16 -0400, Bohuslav Kabrda wrote:
> ----- Original Message -----
>> On 19/07/13 02:41 -0400, Bohuslav Kabrda wrote:
>>> ----- Original Message -----
>>>> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
>>>>> FAQ:
>>>>> Q: Why do we need to switch to Python 3?
>>>>> A: Because Python 2 is old, slower, less pythonic, doesn't get
any more
>>>>> functionality and it won't be that long before the official
upstream
>>>>> support ends [1]
>>>>>
>>>> Although I agree with the need to switch to python3, I don't think
the
>>>> first
>>>> three reasons are very compelling arguments (they're only
half-truths)
>>>> --
>>>> we
>>>> should concentrate on the last reason and also on features that python3
>>>> has that pyhton2 doesn't. Chained exceptions are a pretty nice
thing,
>>>> for
>>>> instance.
>>>>
>>>
>>> So first three reason:
>>> - Python 2 is old - how is that a half-truth?
>>> - Slower - yes, in the beginning, Python 3 was significantly slower
>>> because of nonoptimal code after the rewrite from Python 3. But with
>>> Python 3.3 for instance, you get tons of speed improvements -
>>> decimal module for instance got a significant boost. Brett Cannon
>>> had a nice presentation about speed benchmarking [1]. Yes, Python 3
>>> is slower in some areas, but mostly it's faster.
>>
>> Mind to share some grounds for this claim? My negligible experience
>> told me the contrary, but perhaps timeit module is a bad indicator.
>>
>
> There is the presentation that I referenced in the email that you
> reacted to (referenced by [1]).
Saw the presentation before actually replying -- it doesn't show
any evidence Python 3.3 would be noticably faster. And my micro
tests unfortunately fell into the "slower" category as nicely
demonstrated in the graphs there.
Yes, it is not _noticably_ faster, I agree. The thing is that micro tests do tend to be
slower on Python 3.3 because of different integer and string representations. Things start
to change when you start to use stdlib.
> You can also have a look at pystone benchmark results (a quick
googling
> around gave me e.g.
> http://www.levigross.com/post/2340736877/pystone-benchmark-on-2-6-2-7-3-2)
This is more than two years old. The score reliability of this benchmark
is also questionable (not speaking about order of magnitude difference as
with PyPy).
Yeah, PyPy is known to be a lot faster than any cPython. The problem with PyPy is that C
extension support is still in beta [1] and requires some patching. So unfortunately PyPy
is not suitable for default Python implementation for distribution such as Fedora, IMO.
>> Otherwise yes, let's get gently beyond 3000.
--
Jan
--
Regards,
Bohuslav "Slavek" Kabrda.
[1] http://doc.pypy.org/en/latest/faq.html#do-cpython-extension-modules-work-...
4:05 a.m.
On 07/18/2013 06:53 PM, Toshio Kuratomi wrote:
/usr/bin/python should refer to python2 --
http://www.python.org/dev/peps/pep-0394/ I'd be -1 to changing this
Quoting from this PEP:
python should refer to the same target as python2 but may refer to
python3 on some bleeding edge distributions
I always thought that Fedora (and especially rawhide) is bleading edge
distribution :)
--
Miroslav Suchy
Red Hat, Software Engineer
10:56 p.m.
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
From packaging point of view, this will probably require:
1) Renaming python package to python2
2) Renaming python3 package to python
3) Switching the %{?with_python3} conditionals in specfiles to %{?with_python2} (we will
probably create a script to automate this, at least partially)
Renaming the python package to python2 kind of makes sense, but renaming
the python3 package to python seems needlessly confusing. Wouldn't it
make sense to just keep python2 and python3 side by side without
ambiguity until some long future date when python2 disappears?
--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
4:27 a.m.
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
Hi all,
as a new Fedora Python maintainer, I have set myself a goal of moving Fedora to Python 3
as a default. This is going to be a multirelease effort that is going to affect lots of
Fedora parts. Since we will need to switch default package manager from Yum to DNF (which
is supposed to work with Python 3), we will need to wait for that. I've been told that
DNF should be default in F22, so that's my target, too. That should also give everyone
else plenty of time to work on other essential packages to make this happen.
Here is my analysis/proposal:
Before switching, we need to make sure that everything "important" (*) is
Python 3 compatible. There are three steps I see in this transition:
1) Getting rid of Python 2 in mock minimal buildroot.
2) Porting Anaconda to Python 3.
3) Making all livecd packages depend on Python 3 by default (and eventually getting rid
of Python 2 from livecd) - this will also require switching from Yum to DNF as a default,
that is supposed to support Python 3.
( 4) Making as much of the remaining packages Python 3 compatible )
If we do any work on python3 conversions, it must be done in the context
of respective upstream projects, and not a Fedora custom addon. It will
also quite likely require non-negligable dev resources to make a big dent
in it. It is unlikely to be as simple as just telling upstream to run 2to3,
because a great many upstreams are going to want to provide ongoing support
for both Python2 and Python3 to satisfy non cutting edge distros.
From packaging point of view, this will probably require:
1) Renaming python package to python2
Renaming this is fine, particularly if we also add a Provides: python
2) Renaming python3 package to python
This is a bad idea IMHO. Python3 is not upgrade compatible with functionality
previously provided in the current python package. So such a rename is going
to needlessly cause pain & suffering.
3) Switching the %{?with_python3} conditionals in specfiles to
%{?with_python2}
(we will probably create a script to automate this, at least partially)
I don't think we need do any automated conversion of that sort. It is perfectly
reasonable for packages to in fact have both %{with_python3} and %{with_python2}
conditionals present. By all means add %{with_python2} to allow builds without
the legacy python2 stuff, but no need to blanket remove %{with_python3} - leave
it upto the maintainer to decide if they wish to have conditionals for this, or
unconditionally build both.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
8:31 a.m.
On Fri, Jul 19, 2013 at 11:27 AM, Daniel P. Berrange
<berrange(a)redhat.com> wrote:
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> Hi all,
> as a new Fedora Python maintainer, I have set myself a goal of moving Fedora to
Python 3 as a default. This is going to be a multirelease effort that is going to affect
lots of Fedora parts. Since we will need to switch default package manager from Yum to DNF
(which is supposed to work with Python 3), we will need to wait for that. I've been
told that DNF should be default in F22, so that's my target, too. That should also
give everyone else plenty of time to work on other essential packages to make this
happen.
>
> Here is my analysis/proposal:
> Before switching, we need to make sure that everything "important" (*) is
Python 3 compatible. There are three steps I see in this transition:
> 1) Getting rid of Python 2 in mock minimal buildroot.
> 2) Porting Anaconda to Python 3.
> 3) Making all livecd packages depend on Python 3 by default (and eventually getting
rid of Python 2 from livecd) - this will also require switching from Yum to DNF as a
default, that is supposed to support Python 3.
> ( 4) Making as much of the remaining packages Python 3 compatible )
If we do any work on python3 conversions, it must be done in the context
of respective upstream projects, and not a Fedora custom addon.
IMHO this is precisely the kind of integration where a distribution is
perfectly justified to carry local patches, even against explicit
wishes of upstream if necessary (though cooperating with upstream and
finding a way to integrate the patch upstream is much better). We
shouldn't block the transition just because a one or two upstreams
refuse to port (or, more likely, a dozen or two upstreams do not exist
any more).
Mirek
2:29 p.m.
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
3) Making all livecd packages depend on Python 3 by default (and
eventually getting rid of Python 2 from livecd) - this will also require
switching from Yum to DNF as a default, that is supposed to support Python
I have a concern about bloating @core and by extension the cloud image.
Right now, python is about 5% of the total on-disk usage. I'd hate to see
that go to 10%. Therefore, I'd like to see a goal of making the transition
for usage in the base cloud image go entirely from python2 to python3 in
one release cycle.
(Roughly, that's @core + cloud-init, which isn't currently on your list.)
--
Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ <mattdm(a)fedoraproject.org>
3:45 p.m.
On Fri, Jul 19, 2013 at 03:29:57PM -0400, Matthew Miller wrote:
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> 3) Making all livecd packages depend on Python 3 by default (and
> eventually getting rid of Python 2 from livecd) - this will also require
> switching from Yum to DNF as a default, that is supposed to support Python
I have a concern about bloating @core and by extension the cloud image.
Right now, python is about 5% of the total on-disk usage. I'd hate to see
that go to 10%. Therefore, I'd like to see a goal of making the transition
for usage in the base cloud image go entirely from python2 to python3 in
one release cycle.
(Roughly, that's @core + cloud-init, which isn't currently on your list.)
This seems like a very very worthy goal. I took a brief look at @core
(I didn't recurse down the whole dep chain so hopefully I didn't miss an
iceberg) and that looks doable. The switch from yum to DNF is the big
thing.
cloud-init is python so it might need to be ported in and of itself. Some
of its dependencies also don't have python3 ports:
* configobj -- no upstream python3 port and the third-party port never made
it to a release (the code may work but it's unmaintained). If someone
is ready to become the upstream for this, indications are that it would be
welcomed by both the port's original author and the original ConfigObj
maintainer.
* policycoreutils-python -- not sure if this interfaces with the C libraries
or is about implementing higher level functionality. Nearly anytime you
interface with C you have to make choices about bytes vs text and problems
with encoding. That could slow up this work.
* python-boto -- Looks like there's a badly out-of-date branch in the
upstream SCM repository to add python3 support. That may be based on an
API incompatible update, though (I see that there's work on boto3...)
boto is going to need to talk over the wire to AWS so there's going to be
a lot of places where bytes vs text decisions have to be made. All in
all, this package feels like it might be the hardest piece of the
cloud-init porting.
* python-cheetah -- Development slowed way down after they made their last
release in 2010 and announced that the next release cheetah-3.0 would
include python3 support. Probably need to contact upstream about this and
may need to prepare the patch to do the port.
-Toshio
4:30 p.m.
On Fri, Jul 19, 2013 at 01:45:41PM -0700, Toshio Kuratomi wrote:
* python-cheetah -- Development slowed way down after they made their
last
release in 2010 and announced that the next release cheetah-3.0 would
include python3 support. Probably need to contact upstream about this and
may need to prepare the patch to do the port.
What I'd _really_ like to do is get cheetah factored out of cloud-init.
(https://bugzilla.redhat.com/show_bug.cgi?id=974327). It brings in a whole
dependency chain of which python2 vs. python3 is the least of the troubles.
--
Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ <mattdm(a)fedoraproject.org>
4:44 p.m.
On Fri, Jul 19, 2013 at 05:30:36PM -0400, Matthew Miller wrote:
On Fri, Jul 19, 2013 at 01:45:41PM -0700, Toshio Kuratomi wrote:
> * python-cheetah -- Development slowed way down after they made their last
> release in 2010 and announced that the next release cheetah-3.0 would
> include python3 support. Probably need to contact upstream about this and
> may need to prepare the patch to do the port.
What I'd _really_ like to do is get cheetah factored out of cloud-init.
(https://bugzilla.redhat.com/show_bug.cgi?id=974327). It brings in a whole
dependency chain of which python2 vs. python3 is the least of the troubles.
If your needs are very minimal, python3-tempita might be a good choice.
If you actually do need more features than that, python-mako and
python-jinja2 are popular. Note that both of those have a few deps (but
hopefully not as bad as cheetah). (Also -- the python3 version of mako has
less deps than the python2 version... I think that it just because those
deps haven't been ported to python3 yet and the package can operate with
reduced functionality without them. the deps fo the python3 version might
expand i nthe future).
-Toshio
1:45 a.m.
----- Original Message -----
On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> 3) Making all livecd packages depend on Python 3 by default (and
> eventually getting rid of Python 2 from livecd) - this will also require
> switching from Yum to DNF as a default, that is supposed to support Python
I have a concern about bloating @core and by extension the cloud image.
Right now, python is about 5% of the total on-disk usage. I'd hate to see
that go to 10%. Therefore, I'd like to see a goal of making the transition
for usage in the base cloud image go entirely from python2 to python3 in
one release cycle.
(Roughly, that's @core + cloud-init, which isn't currently on your list.)
Doing everything in one shot sounds reasonable from this POV, I'll try to put this
into my plan (I'll go through cloud-init the same way I went through livecd and post
my findings here).
--
Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ <mattdm(a)fedoraproject.org>
--
Regards,
Bohuslav "Slavek" Kabrda.
2:02 p.m.
On 18/07/13 08:24 AM, Bohuslav Kabrda wrote:
Hi all,
as a new Fedora Python maintainer, I have set myself a goal of moving Fedora to Python 3
as a default. This is going to be a multirelease effort that is going to affect lots of
Fedora parts. Since we will need to switch default package manager from Yum to DNF (which
is supposed to work with Python 3), we will need to wait for that. I've been told that
DNF should be default in F22, so that's my target, too. That should also give everyone
else plenty of time to work on other essential packages to make this happen.
Here is my analysis/proposal:
Before switching, we need to make sure that everything "important" (*) is
Python 3 compatible. There are three steps I see in this transition:
1) Getting rid of Python 2 in mock minimal buildroot.
2) Porting Anaconda to Python 3.
3) Making all livecd packages depend on Python 3 by default (and eventually getting rid
of Python 2 from livecd) - this will also require switching from Yum to DNF as a default,
that is supposed to support Python 3.
( 4) Making as much of the remaining packages Python 3 compatible )
I am very surprised infrastructure has not planned to gradually move to
python 3 since its release. I understand that due to RHEL, python2 will
remain used, in Fedora it is disappointing very little initiative were
proposed in the past for a better transition.
i applaud the effort which is really needed. Fedora is supposed to be a
distribution with latest stable technologies. Python 2.x support is
coming to the end.
--
Luya Tshimbalanga
Graphic & Web Designer
E: luya(a)fedoraproject.org
W: http://www.coolest-storm.net
2:23 p.m.
On 23 July 2013 13:02, Luya Tshimbalanga <luya(a)fedoraproject.org> wrote:
On 18/07/13 08:24 AM, Bohuslav Kabrda wrote:
> Hi all,
> as a new Fedora Python maintainer, I have set myself a goal of moving
> Fedora to Python 3 as a default. This is going to be a multirelease effort
> that is going to affect lots of Fedora parts. Since we will need to switch
> default package manager from Yum to DNF (which is supposed to work with
> Python 3), we will need to wait for that. I've been told that DNF should be
> default in F22, so that's my target, too. That should also give everyone
> else plenty of time to work on other essential packages to make this happen.
>
> Here is my analysis/proposal:
> Before switching, we need to make sure that everything "important" (*) is
> Python 3 compatible. There are three steps I see in this transition:
> 1) Getting rid of Python 2 in mock minimal buildroot.
> 2) Porting Anaconda to Python 3.
> 3) Making all livecd packages depend on Python 3 by default (and
> eventually getting rid of Python 2 from livecd) - this will also require
> switching from Yum to DNF as a default, that is supposed to support Python
> 3.
> ( 4) Making as much of the remaining packages Python 3 compatible )
>
>
> I am very surprised infrastructure has not planned to gradually move to
python 3 since its release. I understand that due to RHEL, python2 will
remain used, in Fedora it is disappointing very little initiative were
proposed in the past for a better transition.
Don't be disappointed. Go help. The above has been the plan since Python 3
came out. We can plan all we want, but if no one has time to work on it
then it doesn't mean much. We can't shut down the old infrastructure and we
can't stop implementing things that people require now versus 6-9 months
from now when a python3 rewrite would be done with the people currently
here.
--
Stephen J Smoogen.
2:40 a.m.
On 23/07/13 12:23 PM, Stephen John Smoogen wrote:
Don't be disappointed. Go help. The above has been the plan since
Python 3 came out. We can plan all we want, but if no one has time to
work on it then it doesn't mean much. We can't shut down the old
infrastructure and we can't stop implementing things that people
require now versus 6-9 months from now when a python3 rewrite would be
done with the people currently here.
--
Stephen J Smoogen.
I'll do as I can to help executing the plan. Thanks for the encouragement.
Luya
11:28 p.m.
On Tue, Jul 23, 2013 at 12:02:04PM -0700, Luya Tshimbalanga wrote:
I am very surprised infrastructure has not planned to gradually move
to python 3 since its release. I understand that due to RHEL, python2
will remain used, in Fedora it is disappointing very little
initiative were proposed in the past for a better transition.
Note -- if you want infrastructure to move to python3, the first step would
be to get python3 into EPEL. There are quite a few things that could go
into an EPEL python3 stack once we have python-3.3 there. (Although EPEL
will probably have to decide whether they want to create a bunch of forward
compat packages [since they have older non-python3-capable versions of some
packages there] or instead create separate python3 SRPM-level packages for
the new code. And if we do move to separate packages in EPEL, whether we
want to revise our packaging in Fedora to have separate packages for python2
and python3 in Fedora.
Infrastructure really doesn't have any chance to migrate to python3 at all
until there's a python3 runtime available there. (Note -- there's other
things holding it back as well -- for instance, the lack of a python3 port
for the web framework that underlies a few critical services, and the fact
that python2 will have to be run on the servers for basic system things like
yum and ansible until we get a RHEL version with DNF and a python3 version
of ansible. But without a python3 runtime, not even those things that can
move independent of those constraints can migrate.)
-Toshio
2:51 a.m.
On 23/07/13 09:28 PM, Toshio Kuratomi wrote:
Note -- if you want infrastructure to move to python3, the first step would
be to get python3 into EPEL. There are quite a few things that could go
into an EPEL python3 stack once we have python-3.3 there. (Although EPEL
will probably have to decide whether they want to create a bunch of forward
compat packages [since they have older non-python3-capable versions of some
packages there] or instead create separate python3 SRPM-level packages for
the new code. And if we do move to separate packages in EPEL, whether we
want to revise our packaging in Fedora to have separate packages for python2
and python3 in Fedora.
Infrastructure really doesn't have any chance to migrate to python3 at all
until there's a python3 runtime available there. (Note -- there's other
things holding it back as well -- for instance, the lack of a python3 port
for the web framework that underlies a few critical services, and the fact
that python2 will have to be run on the servers for basic system things like
yum and ansible until we get a RHEL version with DNF and a python3 version
of ansible. But without a python3 runtime, not even those things that can
move independent of those constraints can migrate.)
-Toshio
EPEL did not have python3 ? By looking at EL6 repository, I realized
Blender is still at 2.49b instead of the current version 2.68.
Will it be possible to bring on EL6 or future EL7? Now that the topic is
raised, it is a good opportunity to gradually start the migration.
Although my focus is currently on Design team, I may provide some
assistance as I can.
Luya
3939
days inactive
3946
days old
60 comments
23 participants
participants (23)
-
Andrew McNabb -
Bill Nottingham -
Bohuslav Kabrda -
Colin Walters -
Daniel P. Berrange -
Jan Pokorný -
Jan Zelený -
John.Florian＠dart.biz -
José Abílio Matos -
Kushal Das -
Luya Tshimbalanga -
Marcela Mašláňová -
Matthew Miller -
Michael Catanzaro -
Miloslav Trmač -
Miroslav Suchý -
Nicolas Mailhot -
Pierre-Yves Chibon -
Stephen John Smoogen -
Tomas Mraz -
Toshio Kuratomi -
Vít Ondruch -
Zbigniew Jędrzejewski-Szmek