11:48 a.m.
Hi all,
looks like PyXML package is deprecated since python itself provides xml
mechanisms.
When you look deeper,
python's xml provides:
"dom", "parsers", "sax", "etree"
and PyXML provides:
'dom', 'marshal', 'parsers', 'sax', 'schema',
'utils', 'xpath', 'xslt'
So, PyXML duplicates dom, parsers and sax (and looks like python's is in
better shape). Is any package using marshall, schema or any other not in
python itself?
Deprecate PyXML or just remove duplicated parts?
RR
4:11 a.m.
On 21.2.2012 18:48, Roman Rakus wrote:
So, PyXML duplicates dom, parsers and sax (and looks like
python's is in
better shape). Is any package using marshall, schema or any other not in
python itself?
Deprecate PyXML or just remove duplicated parts?
What packages require PyXML? Could they be rebuilt just with xml tools
in stock Python (I think so)? Did you try?
Matěj
4:21 a.m.
On 02/22/2012 11:11 AM, Matej Cepl wrote:
On 21.2.2012 18:48, Roman Rakus wrote:
> So, PyXML duplicates dom, parsers and sax (and looks like python's is in
> better shape). Is any package using marshall, schema or any other not in
> python itself?
>
> Deprecate PyXML or just remove duplicated parts?
What packages require PyXML? Could they be rebuilt just with xml tools
in stock Python (I think so)? Did you try?
Matěj
I guess rebuilding isn't enough. Only running affected scripts can show
errors. Packages requiring PyXML in F16 follows:
$ repoquery --whatrequires PyXML
SOAPpy-0:0.11.6-12.fc16.noarch
bkchem-0:0.14.0-3.pre2.fc15.noarch
comoonics-cdsl-py-0:0.2-18.noarch
comoonics-cluster-py-0:0.1-24.noarch
fedora-business-cards-0:0.2.4.3-2.fc15.noarch
grc-0:0.70-7.fc15.noarch
heartbeat-0:3.0.4-1.fc15.1.x86_64
inksmoto-0:0.7.0-5.fc15.noarch
libopensync-plugin-google-calendar-1:0.22-5.fc15.x86_64
openxcap-0:1.1.2-3.fc15.noarch
pida-0:0.5.1-13.fc15.x86_64
pypar2-0:1.4-7.fc15.noarch
python-MythTV-0:0.24.1-4.fc16.x86_64
python-MythTV-0:0.24.2-1.fc16.x86_64
python-ZSI-0:2.0-9.fc15.noarch
python-nova-0:2011.3-4.fc16.noarch
python-nova-0:2011.3.1-2.fc16.noarch
python-webdav-library-0:0.3.0-1.fc16.noarch
salt-0:0.9.6-2.fc16.noarch
spacewalk-backend-tools-0:1.4.39-1.fc16.noarch
subscription-manager-0:0.99.4-1.fc16.x86_64
synce-sync-engine-0:0.15.1-1.fc16.x86_64
xen-0:4.1.1-8.fc16.x86_64
xen-0:4.1.2-6.fc16.x86_64
zeroinstall-injector-0:1.2-1.fc16.noarch
RR
9:54 a.m.
On Tue, Feb 21, 2012 at 06:48:11PM +0100, Roman Rakus wrote:
Hi all,
looks like PyXML package is deprecated since python itself provides
xml mechanisms.
When you look deeper,
python's xml provides:
"dom", "parsers", "sax", "etree"
and PyXML provides:
'dom', 'marshal', 'parsers', 'sax', 'schema',
'utils', 'xpath', 'xslt'
So, PyXML duplicates dom, parsers and sax (and looks like python's is
in better shape). Is any package using marshall, schema or any other
not in python itself?
Deprecate PyXML or just remove duplicated parts?
It's not as simple as saying that a library provides something that has the
same names as modules in the stdlib, you also have to figure out
compatibility and whether removing it will cause any problems for software
that Fedora ships.
Looking at the sourceforge page, the authors of PyXML are heavily involved
in python core development so it's likely that they worked to merge the
useful bits of PyXML into the stdlib before they abandoned it:
http://sourceforge.net/projects/pyxml/
However, it also looks like PyXML is a collection of works that the
sourceforge authors didn't necessarily originate. With that in mind, they
may not have been able to get permission of the various original authors to
merge a particular module into the python stdlib. However, there may exist
independent upstream versions of those modules that would be better to ship
than shipping PyXML in those cases.
The best way to proceed is likely similar to how I looked at whether it
would be okay to retire python-sqlite2: Take all the packages that depend on
PyXML and grep through their sources to find where they use the PyXML
modules (rpm -ql PyXML shows that everything in PyXML is in an "_xmlplus"
python package so you'll see things like "import _xmlplus.dom" and
"from
_xmlplus import dom". grepping for _xmlplus will probably work). In some
cass, you'll likely find this was an old dep and the source no longer uses
it. Others may be conditionalized:
try:
from xml import sax
except ImportError:
from _xmlplus import sax
Testing these packages to know that they behave properly is good but at
least you can be confident that the upstream code is intended to work with
the stdlib modules instead of the PyXML modules.
If you find any code that's using _xmlplus unconditionally, you'll have to
write patches to use the stdlib or separate modules. Then test your changes
and send the patches upstream. Given the upstream note that PyXML is
deprecated and the authors do not intend for people to use it, this category
would hopefully be very small. But you won't know until you look.
-Toshio
10:19 a.m.
On 02/23/2012 04:54 PM, Toshio Kuratomi wrote:
On Tue, Feb 21, 2012 at 06:48:11PM +0100, Roman Rakus wrote:
> Hi all,
> looks like PyXML package is deprecated since python itself provides
> xml mechanisms.
> When you look deeper,
> python's xml provides:
> "dom", "parsers", "sax", "etree"
> and PyXML provides:
> 'dom', 'marshal', 'parsers', 'sax', 'schema',
'utils', 'xpath', 'xslt'
>
> So, PyXML duplicates dom, parsers and sax (and looks like python's is
> in better shape). Is any package using marshall, schema or any other
> not in python itself?
>
> Deprecate PyXML or just remove duplicated parts?
>
It's not as simple as saying that a library provides something that has the
same names as modules in the stdlib, you also have to figure out
compatibility and whether removing it will cause any problems for software
that Fedora ships.
Looking at the sourceforge page, the authors of PyXML are heavily involved
in python core development so it's likely that they worked to merge the
useful bits of PyXML into the stdlib before they abandoned it:
http://sourceforge.net/projects/pyxml/
However, it also looks like PyXML is a collection of works that the
sourceforge authors didn't necessarily originate. With that in mind, they
may not have been able to get permission of the various original authors to
merge a particular module into the python stdlib. However, there may exist
independent upstream versions of those modules that would be better to ship
than shipping PyXML in those cases.
The best way to proceed is likely similar to how I looked at whether it
would be okay to retire python-sqlite2: Take all the packages that depend on
PyXML and grep through their sources to find where they use the PyXML
modules (rpm -ql PyXML shows that everything in PyXML is in an "_xmlplus"
python package so you'll see things like "import _xmlplus.dom" and
"from
_xmlplus import dom". grepping for _xmlplus will probably work). In some
cass, you'll likely find this was an old dep and the source no longer uses
it. Others may be conditionalized:
try:
from xml import sax
except ImportError:
from _xmlplus import sax
Look at stdlib xml - it tries to import _xmlplus. And
it will replace
stdlib with nonstd. It's kind of "what?". I can try to report bug on it.
Testing these packages to know that they behave properly is good but at
least you can be confident that the upstream code is intended to work with
the stdlib modules instead of the PyXML modules.
If you find any code that's using _xmlplus unconditionally, you'll have to
write patches to use the stdlib or separate modules. Then test your changes
and send the patches upstream. Given the upstream note that PyXML is
deprecated and the authors do not intend for people to use it, this category
would hopefully be very small. But you won't know until you look.
-Toshio
Currently I'm going through packages and using pylint on *.py files on
%preped sources. And using --deprecated-modules option of pylint. I will
post results, report bugs and so on...
Anyway, you're right that better is to ask upstream how different are
stdlib and _xmlplus.
RR
11 a.m.
On 23.2.2012 16:54, Toshio Kuratomi wrote:
It's not as simple as saying that a library provides something
that has the
same names as modules in the stdlib, you also have to figure out
compatibility and whether removing it will cause any problems for software
that Fedora ships.
Completely agree with what you were writing, just to note that IIRC
PyXML was the first Python XML library, so I would believe plenty of
projects use it just because of conservatism/laziness/fear of change.
Matěj
11:09 a.m.
On Thu, Feb 23, 2012 at 05:19:01PM +0100, Roman Rakus wrote:
Look at stdlib xml - it tries to import _xmlplus. And it will replace stdlib
with nonstd. It's kind of "what?". I can try to report bug on it.
Ugh. Yeah -- so it looks like the code there assumes that python-2.7 xml
libraries don't have any changes that aren't in the 0.8.4 release of PyXML.
This is untrue, but many of the changes won't affect what the code does:
"if value in dict" vs "if dict.has_key(value)" type stuff and work
arounds
for older python releases. There are a few changes which might affect the
output of the code but someone would need to analyze it more to know for
sure.
The method used also makes it harder for people to simply grep the sources
and tell if code really depends on PyXML or not as an import of the stdlib's
xml module may be getting _xmlplus instead of the stdlib code.
Currently I'm going through packages and using pylint on *.py files on %preped
sources. And using --deprecated-modules option of pylint. I will post results,
report bugs and so on...
It's great that you're doing this. Hopefully PyXML has been deprecated
long
enough that you'll find most code only needs the stdlib to function.
-Toshio
6:32 a.m.
Quoting Roman Rakus (2012-02-22 11:21:38)
On 02/22/2012 11:11 AM, Matej Cepl wrote:
> On 21.2.2012 18:48, Roman Rakus wrote:
>> So, PyXML duplicates dom, parsers and sax (and looks like python's is in
>> better shape). Is any package using marshall, schema or any other not in
>> python itself?
>>
>> Deprecate PyXML or just remove duplicated parts?
>
> What packages require PyXML? Could they be rebuilt just with xml tools
> in stock Python (I think so)? Did you try?
>
> Matěj
>
I guess rebuilding isn't enough. Only running affected scripts can show
errors. Packages requiring PyXML in F16 follows:
$ repoquery --whatrequires PyXML
...
python-webdav-library-0:0.3.0-1.fc16.noarch
Fixed in rawhide/F17. Thanks for pointing it out
--
Stanislav Ochotnicky <sochotnicky(a)redhat.com>
Software Engineer - Base Operating Systems Brno
PGP: 7B087241
Red Hat Inc. http://cz.redhat.com
6:03 p.m.
On 22.2.2012 11:21, Roman Rakus wrote:
fedora-business-cards-0:0.2.4.3-2.fc15.noarch
I wonder why this is on the list: if I am not mistaken, it doesn't use
anything else than xml.dom.minidom (in generate.py), which was already
present in python 2.4 (the oldest Python currently living in Fedora/EPEL
universe). And yes PyXML is hard-coded in Requires: (maybe this is just
one more example why hard coded Requires are evil).
Putting the maintainer on Cc: to ask for the reasons why this package
wants PyXML at all.
Best,
Matěj
11:33 p.m.
On Mon, Feb 27, 2012 at 01:03:32AM +0100, Matej Cepl wrote:
On 22.2.2012 11:21, Roman Rakus wrote:
>fedora-business-cards-0:0.2.4.3-2.fc15.noarch
I wonder why this is on the list: if I am not mistaken, it doesn't
use anything else than xml.dom.minidom (in generate.py), which was
already present in python 2.4 (the oldest Python currently living in
Fedora/EPEL universe). And yes PyXML is hard-coded in Requires:
(maybe this is just one more example why hard coded Requires are
evil).
Putting the maintainer on Cc: to ask for the reasons why this
package wants PyXML at all.
I'll remove the dependency sometime this week.
--
Ian Weller <ian(a)ianweller.org>
4:45 a.m.
On Tue, Feb 21, 2012 at 06:48:11PM +0100, Roman Rakus wrote:
Hi all,
looks like PyXML package is deprecated since python itself provides
xml mechanisms.
When you look deeper,
python's xml provides:
"dom", "parsers", "sax", "etree"
and PyXML provides:
'dom', 'marshal', 'parsers', 'sax', 'schema',
'utils', 'xpath', 'xslt'
So, PyXML duplicates dom, parsers and sax (and looks like python's
is in better shape). Is any package using marshall, schema or any
other not in python itself?
Deprecate PyXML or just remove duplicated parts?
PyXML is not maintained by upstream for many years it should not be used
hence. Distribution specific PyXML-0.8.4-python2.6.patch included in
srpm is warning of ongoing issues.
Anyway it provides many features which are missing in python stdlib as
far as I know (for example xpath or magnificent HTML-to-DOM reader).
There are alternatives outside of stdlib described here:
http://wiki.python.org/moin/PythonXml
(lxml is my personal favorite, however it is not compatible with PyXML and
it isn't pure python).
I believe PyXML should be kept unchanged (my personal code rely on that
as well) but deprecated and its users should be strongly encouraged to
switch to some alternative if stdlib doesn't satisfy their requirements.
--
Regards,
Marian
6:28 p.m.
On Thu, Feb 23, 2012 at 9:09 AM, Toshio Kuratomi <a.badger(a)gmail.com> wrote:
On Thu, Feb 23, 2012 at 05:19:01PM +0100, Roman Rakus wrote:
>
> Currently I'm going through packages and using pylint on *.py files on %preped
> sources. And using --deprecated-modules option of pylint. I will post results,
> report bugs and so on...
>
It's great that you're doing this. Hopefully PyXML has been deprecated long
enough that you'll find most code only needs the stdlib to function.
To revive a very old thread --
Did you get anywhere with this? I've just found that the latest
version of docutils doesn't run if pyxml is installed (due to the
stdlib replacing its own implementations with pyxml's implementation
if pyxml is installed.) For now, in rawhide, I've added a Conflicts
on PyXml but that's not going to work into the future (not the least
because you can't install packages that (perhaps bogusly) require
pyxml at the same time as docutils.)
I can see several ways forward --
* Deprecate pyxml as you were thinking of doing
* Locally patch the python stdlib to not replace its implementation of
xml with pyxml's. I don't know if upstream python will take that as
python-2.7 is in maintainance mode but they may as it's somewhat
agreed that this importing of pyxml is not kosher in upstream (and has
been removed in python-3.x)
* I can file a bug and we can figure out how to fix PyXML so that
docutils works when it's installed. (We may have to do this anyway as
there's a chance I may need to push a docutils update to older Fedora
for some bugfixes.)
Thanks,
-Toshio
1:05 p.m.
On Fri, Jul 20, 2012 at 4:28 PM, Toshio Kuratomi <a.badger(a)gmail.com> wrote:
I can see several ways forward --
* Deprecate pyxml as you were thinking of doing
* Locally patch the python stdlib to not replace its implementation of
xml with pyxml's. I don't know if upstream python will take that as
python-2.7 is in maintainance mode but they may as it's somewhat
agreed that this importing of pyxml is not kosher in upstream (and has
been removed in python-3.x)
Some followup. I've made a page for removing pyxml:
https://fedoraproject.org/wiki/User:Toshio/Remove_PyXML
dmalcolm added the dep tree. I remember that fedora-business-card is
a false dep from when this came up in February. I'll remove the dep
shortly. Since SOAPpy affects so much stuff I had a look. It seems
like the dep is false there as well. The README says that PyXML is
needed but inspection of the code and the ChangeLog, ReleaseNotes, and
the upstream scm point to that being a documentation bug; PyXML
requirement was supposedly removed in 2003.
-Toshio
5:16 p.m.
Just finished analyzing all of the packages that claim a PyXML dependency:
https://fedoraproject.org/wiki/User:Toshio/Remove_PyXML#Dep_analysis
There seem to be a hefty number of packages where we can remove the
dependency (the Easy fixes section). I've opened bugs for those in
case the maintainers know something about the PyXML dep that I'm
unaware of.
Due to the fact that PyXML overwrites the python stdlib xml module the
possibility does exist that I've missed some feature that PyXML adds
to some portion of the stdlib xml module. As an example of what level
of replacement that can occur: Both modules have sax.handler. The sax
handlers can both be configured by setting sax.handler.feature_*
attributes. However, PyXML has feature_namespace_prefixes while the
stdlib module does not. If you run across one of these in course of
trying to remove the PyXML dependency, please let me know, update the
wiki page, add to the bug report what's blocking things, etc. I can
try to check for these things across all of the PyXML-using packages
if I'm made aware that they exist.
The Require Coding section is more problematic. I'll list them all here:
* bkchem: uses xpath. If this is all, it's probably doable to write a
patch that uses lxml's xpath instead. Since it's a plugin, it's also
possible to stop shipping that plugin.
* libopensync-plugin-google-calendar: uses xpath. Writing a patch
shoul dbe as doable as bkchem
* python-ZSI: I think we could fix with a package update. Upstream
has a 2.1alpha1 release that Debian and Gentoo both ship that would
fix this for us.
* subscription-manager looks like it just needs a port to a different
library (the code isn't even xml related.. it's date parsing)
* spacewalk-backend: both uses of PyXML-only code were in test cases.
One looks like it could be ported to some other library (it's for
converting between character encodings). The other one is for writing
xml using a SAX api. I haven't looked at that one too hard yet.
* openxcap makes use of a non-stdlib feature of the sax reader. I
haven't looked into this one too hard either but I suspect there's not
just a drop-in replacement for this. Some sort of custom code will
have to be written to handle what they're trying to do.
* comoonics* -- these packages make heavy use of the PyXML-only API.
I think someone will need to spend a while getting to know the code
and porting it to use a different library. We could also drop the
comoonics packages until upstream ports.
rrakus, dmalcolm, others -- how do you want to proceed? I can open
bugs for the remaining packages and we can plow forward on this but
some of the packages needing code changes may not make it for F18 and
have to be blocked. We can also pursue one of the alternatives (for
instance, making the python2 stdlib not replace itself with PyXML and
patching those package swhich can't be ported in time to import
_xmlplus instead of import xml.
-Toshio
4294
days inactive
4448
days old
14 comments
7 participants
participants (7)
-
Ian Weller -
Marian Ganisin -
Matej Cepl -
Otto Rey -
Roman Rakus -
Stanislav Ochotnicky -
Toshio Kuratomi