9:05 a.m.
I had this build failure:
Package: sagemath-6.5-7.fc22
Status: failed
Built by: pcpa
ID: 672175
Started: Sat, 25 Jul 2015 22:52:10 UTC
Finished: Sun, 26 Jul 2015 07:57:28 UTC
Closed tasks:
-------------
Task 10480570 on arm04-builder10.arm.fedoraproject.org
Task Type: build (noarch)
Link: https://koji.fedoraproject.org/koji/taskinfo?taskID=10480570
mismatch when analyzing sagemath-doc-en-6.5-7.fc22.noarch.rpm, rpmdiff
output was:
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages database in /var/lib/rpm
added /usr/share/doc/sagemath/output/html/en/reference/hecke
[...]
The interesting part, is that the above build generated
packages, but ended up with failed state.
After that, I checked contents of the armv7hl and x86_64
trees, and noticed that they are, indeed not identical.
The way documentation is generated with sphinx, should
be part of the cause (when it pickle/unpicke python states, etc),
and sometimes it even adds the location of a file to the docs,
e.g. telling where source is located, causes diffs, from
/usr/lib/python2.7/... vs /usr/lib64/python2.7/...
Should I make the doc packages arch specific?
Thanks,
Paulo
10:20 a.m.
On Dom, 2015-07-26 at 11:05 -0300, Paulo César Pereira de Andrade wrote:
I had this build failure:
Package: sagemath-6.5-7.fc22
Status: failed
Built by: pcpa
ID: 672175
Started: Sat, 25 Jul 2015 22:52:10 UTC
Finished: Sun, 26 Jul 2015 07:57:28 UTC
Closed tasks:
-------------
Task 10480570 on arm04-builder10.arm.fedoraproject.org
Task Type: build (noarch)
Link: https://koji.fedoraproject.org/koji/taskinfo?taskID=10480570
mismatch when analyzing sagemath-doc-en-6.5-7.fc22.noarch.rpm, rpmdiff
output was:
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages database in /var/lib/rpm
added /usr/share/doc/sagemath/output/html/en/reference/hecke
[...]
The interesting part, is that the above build generated
packages, but ended up with failed state.
After that, I checked contents of the armv7hl and x86_64
trees, and noticed that they are, indeed not identical.
The way documentation is generated with sphinx, should
be part of the cause (when it pickle/unpicke python states, etc),
and sometimes it even adds the location of a file to the docs,
e.g. telling where source is located, causes diffs, from
/usr/lib/python2.7/... vs /usr/lib64/python2.7/...
I suspect package is using %{_libdir} and libdir can't be used in noarch
packages. %{_libdir} expands to /usr/lib on arm and /usr/lib64/ on
x86_64. This is a general problem when we try translate Debian packages
to Fedora, I don't have time now to explain better but we had some
topics about related subjects in Packaging mailing list ...
Python also have different scriptlets for arch and noarch packages. On
x86_64 systems, we can find some noarch packages in /usr/lib/python2.7/
and python arched in /usr/lib64/python2.7/ .
Should I make the doc packages arch specific?
Thanks,
Paulo
--
Sérgio M. B.
10:50 a.m.
2015-07-26 12:20 GMT-03:00 Sérgio Basto <sergio(a)serjux.com>:
On Dom, 2015-07-26 at 11:05 -0300, Paulo César Pereira de Andrade wrote:
> I had this build failure:
>
> Package: sagemath-6.5-7.fc22
> Status: failed
> Built by: pcpa
> ID: 672175
> Started: Sat, 25 Jul 2015 22:52:10 UTC
> Finished: Sun, 26 Jul 2015 07:57:28 UTC
>
> Closed tasks:
> -------------
> Task 10480570 on arm04-builder10.arm.fedoraproject.org
> Task Type: build (noarch)
> Link: https://koji.fedoraproject.org/koji/taskinfo?taskID=10480570
>
> mismatch when analyzing sagemath-doc-en-6.5-7.fc22.noarch.rpm, rpmdiff
> output was:
> error: cannot open Packages index using db5 - Permission denied (13)
> error: cannot open Packages database in /var/lib/rpm
> error: cannot open Packages database in /var/lib/rpm
> added /usr/share/doc/sagemath/output/html/en/reference/hecke
> [...]
>
> The interesting part, is that the above build generated
> packages, but ended up with failed state.
> After that, I checked contents of the armv7hl and x86_64
> trees, and noticed that they are, indeed not identical.
> The way documentation is generated with sphinx, should
> be part of the cause (when it pickle/unpicke python states, etc),
> and sometimes it even adds the location of a file to the docs,
> e.g. telling where source is located, causes diffs, from
> /usr/lib/python2.7/... vs /usr/lib64/python2.7/...
I suspect package is using %{_libdir} and libdir can't be used in noarch
packages. %{_libdir} expands to /usr/lib on arm and /usr/lib64/ on
x86_64. This is a general problem when we try translate Debian packages
to Fedora, I don't have time now to explain better but we had some
topics about related subjects in Packaging mailing list ...
Python also have different scriptlets for arch and noarch packages. On
x86_64 systems, we can find some noarch packages in /usr/lib/python2.7/
and python arched in /usr/lib64/python2.7/ .
A good share of the diffs is documentation is printing addresses of
objects, at the time the documentation was generated, for example:
---8<---
diff -rNu
armv7hl/usr/share/doc/sagemath/output/html/en/reference/calculus/sage/gsl/dft.html
x86_64/usr/share/doc/sagemath/output/html/en/reference/calculus/sage/gsl/dft.html
--- armv7hl/usr/share/doc/sagemath/output/html/en/reference/calculus/sage/gsl/dft.html
2015-07-26 12:25:01.039419016 -0300
+++ x86_64/usr/share/doc/sagemath/output/html/en/reference/calculus/sage/gsl/dft.html
2015-07-26 12:25:06.410419222 -0300
@@ -226,7 +226,7 @@
<dl class="method">
<dt id="sage.gsl.dft.IndexedSequence.dft">
-<tt
class="descname">dft</tt><big>(</big><em>chi=<function
<lambda> at 0xac336d70></em><big>)</big><a
class="headerlink"
href="#sage.gsl.dft.IndexedSequence.dft" title="Permalink to this
definition">¶</a></dt>
+<tt
class="descname">dft</tt><big>(</big><em>chi=<function
<lambda> at 0x7f27a03f4140></em><big>)</big><a
class="headerlink" href="#sage.gsl.dft.IndexedSequence.dft"
title="Permalink to this definition">¶</a></dt>
<dd><p>A discrete Fourier transform “over <img
class="math"
src="../../_images/math/7bab5d1f8fefbcece570be83647d253afa56e0df.png"
alt="\QQ"/>” using exact
<img class="math"
src="../../_images/math/9373b89a1028693ea48d96bef05ec06f42c4ba41.png"
alt="N"/>-th roots of unity.</p>
<p>EXAMPLES:</p>
---8<---
Example of the above, "live":
http://doc.sagemath.org/html/en/reference/calculus/sage/gsl/dft.html#sage...
About %{_libdir}, there are some in this pattern:
---8<---
diff -rNu armv7hl/usr/share/doc/sagemath/output/html/en/reference/todolist.html
x86_64/usr/share/doc/sagemath/output/html/en/reference/todolist.html
--- armv7hl/usr/share/doc/sagemath/output/html/en/reference/todolist.html
2015-07-26 12:25:03.271419101 -0300
+++ x86_64/usr/share/doc/sagemath/output/html/en/reference/todolist.html
2015-07-26 12:25:08.671419308 -0300
@@ -83,36 +83,36 @@
<p class="last">Deprecate this in favor of a method called <img
class="math"
src="_images/math/107b545f3b4d727d47ab3a406ddd5b3a92cd2fda.png"
alt="center()"/> once
subalgebras are properly implemented in Sage.</p>
</div>
-<p class="todo-source">(The <a class="reference internal"
href="algebras/sage/algebras/clifford_algebra.html#index-0"><em>original
entry</em></a> is located in
/usr/lib/python2.7/site-packages/sage/algebras/clifford_algebra.py:docstring
of sage.algebras.clifford_algebra.CliffordAlgebra.center_basis, line
12.)</p>
+<p class="todo-source">(The <a class="reference internal"
href="algebras/sage/algebras/clifford_algebra.html#index-0"><em>original
entry</em></a> is located in
/usr/lib64/python2.7/site-packages/sage/algebras/clifford_algebra.py:docstring
of sage.algebras.clifford_algebra.CliffordAlgebra.center_basis, line
12.)</p>
---8<---
Others are in some generated js, after searching the first
difference, it looks like this,
diff at char 255:
armv7hl:
Search.setIndex({envversion:42,terms:{represent:[1,2],all:[5,1,3,6,8,12,13],x3y:13,partial:0,sch:3,joyner:14,illustr:[0,2,4,6,7,8,15],skip:9,lfsr_sequenc:9,hamburg:14,roe:7,month:3,'_k1':0,plot_list:15,short_weierstrass_model:10,'_k2':0,ellipt:13,legal:3,edu:3,follow:
x86_64:
Search.setIndex({envversion:42,terms:{represent:[1,2],all:[5,1,3,6,8,12,13],x3y:13,partial:0,sch:3,joyner:14,illustr:[0,2,4,6,7,8,15],skip:9,lfsr_sequenc:9,hamburg:14,roe:7,month:3,'_k1':0,plot_list:15,short_weierstrass_model:10,'_k2':0,ellipt:13,legal:3,classic:12,edu:3,follow
> Should I make the doc packages arch specific?
>
> Thanks,
> Paulo
--
Sérgio M. B.
Thanks,
Paulo
8:34 p.m.
Excerpts from paulo.cesar.pereira.de.andrade's message of 2015-07-27 00:05 +10:00:
Should I make the doc packages arch specific?
Rather than trying to make Sphinx spit out bitwise-identical output on
every arch (which sounds like fighting a losing battle), could you just
build the doc subpackage on only one arch?
%ifarch x86_64
%package doc
BuildArch: noarch
...
%endif
I think Koji still counts this a regular noarch subpackage and it should
therefore be included in the Fedora trees for all arches.
--
Dan Callaghan <dcallagh(a)redhat.com>
Senior Software Engineer, Products & Technologies Operations
Red Hat, Inc.
10:37 p.m.
2015-07-27 22:34 GMT-03:00 Dan Callaghan <dcallagh(a)redhat.com>:
Excerpts from paulo.cesar.pereira.de.andrade's message of
2015-07-27 00:05 +10:00:
> Should I make the doc packages arch specific?
Rather than trying to make Sphinx spit out bitwise-identical output on
every arch (which sounds like fighting a losing battle), could you just
build the doc subpackage on only one arch?
%ifarch x86_64
%package doc
BuildArch: noarch
...
%endif
This looks like a very wise way of handling it. Actually, while debugging
it, I found that the translated documentation was not being properly
generated, and after fixing it, it would take like 3 to 4 times longer
to generate docs, and doc generation was already almost 80% of
the package build time...
I think Koji still counts this a regular noarch subpackage and it
should
therefore be included in the Fedora trees for all arches.
In the worst case, it would generate -doc packages only for x86_64,
where most users interested on reading it would be using.
--
Dan Callaghan <dcallagh(a)redhat.com>
Senior Software Engineer, Products & Technologies Operations
Red Hat, Inc.
Thanks! I will try this way :)
Paulo
3:03 a.m.
On Tue, Jul 28, 2015 at 4:37 AM, Paulo César Pereira de Andrade
<paulo.cesar.pereira.de.andrade(a)gmail.com> wrote:
2015-07-27 22:34 GMT-03:00 Dan Callaghan
<dcallagh(a)redhat.com>:
> Excerpts from paulo.cesar.pereira.de.andrade's message of 2015-07-27 00:05
+10:00:
>> Should I make the doc packages arch specific?
>
> Rather than trying to make Sphinx spit out bitwise-identical output on
> every arch (which sounds like fighting a losing battle), could you just
> build the doc subpackage on only one arch?
>
> %ifarch x86_64
> %package doc
> BuildArch: noarch
> ...
> %endif
This looks like a very wise way of handling it. Actually, while debugging
It's not, it breaks all secondary architectures.
it, I found that the translated documentation was not being properly
generated, and after fixing it, it would take like 3 to 4 times longer
to generate docs, and doc generation was already almost 80% of
the package build time...
That tells me the process of generating docs is broken, or they're
very good docs and worth the wait!
> I think Koji still counts this a regular noarch subpackage and it
should
> therefore be included in the Fedora trees for all arches.
In the worst case, it would generate -doc packages only for x86_64,
where most users interested on reading it would be using.
And won't generate docs for any of the secondary arches which don't
have any x86_64 build capacity, please don't do this.
9:57 a.m.
2015-07-28 5:03 GMT-03:00 Peter Robinson <pbrobinson(a)gmail.com>:
>> %ifarch x86_64
>> %package doc
>> BuildArch: noarch
>> ...
>> %endif
>
> This looks like a very wise way of handling it. Actually, while debugging
It's not, it breaks all secondary architectures.
> it, I found that the translated documentation was not being properly
> generated, and after fixing it, it would take like 3 to 4 times longer
> to generate docs, and doc generation was already almost 80% of
> the package build time...
That tells me the process of generating docs is broken, or they're
very good docs and worth the wait!
It is the later. It has its problems of course, but the documentation
is really very good and complete, documenting every single interface.
It is "live", once running the sagemath "notebook", one can modify
the examples, run with different input, etc.
Thanks,
Paulo
11:51 p.m.
On 07/28/2015 03:34 AM, Dan Callaghan wrote:
Excerpts from paulo.cesar.pereira.de.andrade's message of
2015-07-27 00:05 +10:00:
> Should I make the doc packages arch specific?
Rather than trying to make Sphinx spit out bitwise-identical output on
every arch (which sounds like fighting a losing battle), could you just
build the doc subpackage on only one arch?
This is a blatant hack.
Packages builds must be reproducable on any supported architecture.
Ralf
3:01 a.m.
On Tue, Jul 28, 2015 at 2:34 AM, Dan Callaghan <dcallagh(a)redhat.com> wrote:
Excerpts from paulo.cesar.pereira.de.andrade's message of
2015-07-27 00:05 +10:00:
> Should I make the doc packages arch specific?
Rather than trying to make Sphinx spit out bitwise-identical output on
every arch (which sounds like fighting a losing battle), could you just
build the doc subpackage on only one arch?
%ifarch x86_64
%package doc
BuildArch: noarch
...
%endif
I think Koji still counts this a regular noarch subpackage and it should
therefore be included in the Fedora trees for all arches.
Dan,
This is completely NOT appropriate, it breaks on secondary arches
where they then end up with no documentation due to the lack of any
x86_64. Please DO NOT do this and please revert the change on any
packages you might have made this change.
5:35 p.m.
Excerpts from Peter Robinson's message of 2015-07-28 18:01 +10:00:
This is completely NOT appropriate, it breaks on secondary arches
where they then end up with no documentation due to the lack of any
x86_64. Please DO NOT do this and please revert the change on any
packages you might have made this change.
I haven't used that hack on any of my packages but I learnt about it
from the ipxe package which uses it to produce noarch subpackages
containing boot images. I guess ipxe is broken on secondary arches as
well, but it's acceptable for that package because the only alternative
would be to make it x86-only?
--
Dan Callaghan <dcallagh(a)redhat.com>
Senior Software Engineer, Products & Technologies Operations
Red Hat, Inc.
3:58 a.m.
On 07/26/2015 04:05 PM, Paulo César Pereira de Andrade wrote:
Should I make the doc packages arch specific?
No, this is not a reason to make them arch-specific. A lot of packages
give different results when built twice in a row, on the *same*
architecture.
There is an effort under way to change that, called “reproducible
builds”. The hard part is any reproducibility at all, identical noarch
builds across architectures are likely just some additional work on top
of it.
--
Florian Weimer / Red Hat Product Security
9:52 a.m.
2015-07-28 5:58 GMT-03:00 Florian Weimer <fweimer(a)redhat.com>:
On 07/26/2015 04:05 PM, Paulo César Pereira de Andrade wrote:
> Should I make the doc packages arch specific?
No, this is not a reason to make them arch-specific. A lot of packages
give different results when built twice in a row, on the *same*
architecture.
There is an effort under way to change that, called “reproducible
builds”. The hard part is any reproducibility at all, identical noarch
builds across architectures are likely just some additional work on top
of it.
I believe that if there is a check for bit by bit identical noarch
packages, it would also be mandatory some way to tell that
any minor difference is ok and expected, and use the noarch
built on that arch...
--
Florian Weimer / Red Hat Product Security
Thanks,
Paulo
6:59 a.m.
On Tue, Jul 28, 2015 at 10:52 AM, Paulo César Pereira de Andrade
<paulo.cesar.pereira.de.andrade(a)gmail.com> wrote:
2015-07-28 5:58 GMT-03:00 Florian Weimer <fweimer(a)redhat.com>:
> On 07/26/2015 04:05 PM, Paulo César Pereira de Andrade wrote:
>
>> Should I make the doc packages arch specific?
>
> No, this is not a reason to make them arch-specific. A lot of packages
> give different results when built twice in a row, on the *same*
> architecture.
>
> There is an effort under way to change that, called “reproducible
> builds”. The hard part is any reproducibility at all, identical noarch
> builds across architectures are likely just some additional work on top
> of it.
I believe that if there is a check for bit by bit identical noarch
packages, it would also be mandatory some way to tell that
any minor difference is ok and expected, and use the noarch
built on that arch...
> --
> Florian Weimer / Red Hat Product Security
Thanks,
Paulo
Please note that *changing* a package from arch to noarch requires
extra attention. The switch of architecture is not normally regarded
as an upgrade path, by default, so you may have to insert an
"Obsoletes" statement to ensure that older versions of the alternative
architecture get upgraded.
11:59 a.m.
On 07/28/2015 10:58 AM, Florian Weimer wrote:
On 07/26/2015 04:05 PM, Paulo César Pereira de Andrade wrote:
> Should I make the doc packages arch specific?
No, this is not a reason to make them arch-specific. A lot of packages
give different results when built twice in a row, on the *same*
architecture.
There is an effort under way to change that, called “reproducible
builds”.
Actually, reproducable builds wrt. docs have been subject to Fedora
Packaging since Fedora day #1 and repeatedly have been subject to
discussions of details (e.g. doxygen repeatedly had introduced docs
breakages)
Packages which do not comply to this rule are broken.
Ralf
4:05 p.m.
On 07/28/2015 06:59 PM, Ralf Corsepius wrote:
Actually, reproducable builds wrt. docs have been subject to Fedora
Packaging since Fedora day #1 and repeatedly have been subject to
discussions of details (e.g. doxygen repeatedly had introduced docs
breakages)
Packages which do not comply to this rule are broken.
Can you provide a citation in the guidelines? As far as I can tell,
javadoc hasn't been patched not to put the build date into the resulting
HTML documentation, so a lot of Java packages are not compliant with the
above.
--
Florian Weimer / Red Hat Product Security
4:09 p.m.
Am 01.08.2015 um 23:05 schrieb Florian Weimer:
On 07/28/2015 06:59 PM, Ralf Corsepius wrote:
> Actually, reproducable builds wrt. docs have been subject to Fedora
> Packaging since Fedora day #1 and repeatedly have been subject to
> discussions of details (e.g. doxygen repeatedly had introduced docs
> breakages)
>
> Packages which do not comply to this rule are broken.
Can you provide a citation in the guidelines? As far as I can tell,
javadoc hasn't been patched not to put the build date into the resulting
HTML documentation, so a lot of Java packages are not compliant with the
above
it's simply not true - there are no reproducable builds in Fedora and
frankly currently very few distributions have them at all
3184
days inactive
3190
days old
15 comments
8 participants
participants (8)
-
Dan Callaghan -
Florian Weimer -
Nico Kadel-Garcia -
Paulo César Pereira de Andrade -
Peter Robinson -
Ralf Corsepius -
Reindl Harald -
Sérgio Basto