2:29 p.m.
Greetings.
image composes using oz take a long time on our s390x image builder,
(due to them being in another datacenter). They take so long, they timeout.
See: https://pagure.io/koji/pull-request/837 from sinny and
https://pagure.io/koji/pull-request/841 which makes this value
configurable in kojid.conf
I would like to:
1. Update koji on buildvm-s390x-02 to the version I just built in
rawhide/f28 that has a backport of the patch that makes this value
configurable.
2. Apply the following ansible patch to set the timeout only on the
buildvm-s390x-02 builder:
diff --git a/roles/koji_builder/templates/kojid.conf
b/roles/koji_builder/templates/kojid.conf
index 832d80c..9f10d9c 100644
--- a/roles/koji_builder/templates/kojid.conf
+++ b/roles/koji_builder/templates/kojid.conf
@@ -20,7 +20,12 @@ maxjobs=25
keepalive=False
rpmbuild_timeout=172800
-
+{% if ansible_hostname.startswith('buildvm-s390x-02') %}
+; Set oz timeout higher on x390x image builder to allow it to finish.
+; Install timeout(seconds) for image build
+; if it's unset, use the number in /etc/oz/oz.cfg, supported since
oz-0.16.0
+oz_install_timeout=14400
+{% endif %}
use_createrepo_c=True
{% if host in groups['buildvm-s390x'] %}
3. Run the playbook on it to update the config and restart kojid.
+1s?
kevin
2:42 p.m.
+1. Do we know 14400 will be enough?
-re
On Fri, Mar 16, 2018 at 3:29 PM, Kevin Fenzi <kevin(a)scrye.com> wrote:
> Greetings.
>
> image composes using oz take a long time on our s390x image builder,
> (due to them being in another datacenter). They take so long, they timeout.
>
> See: https://pagure.io/koji/pull-request/837 from sinny and
> https://pagure.io/koji/pull-request/841 which makes this value
> configurable in kojid.conf
>
> I would like to:
>
> 1. Update koji on buildvm-s390x-02 to the version I just built in
> rawhide/f28 that has a backport of the patch that makes this value
> configurable.
>
> 2. Apply the following ansible patch to set the timeout only on the
> buildvm-s390x-02 builder:
>
> diff --git a/roles/koji_builder/templates/kojid.conf
> b/roles/koji_builder/templates/kojid.conf
> index 832d80c..9f10d9c 100644
> --- a/roles/koji_builder/templates/kojid.conf
> +++ b/roles/koji_builder/templates/kojid.conf
> @@ -20,7 +20,12 @@ maxjobs=25
> keepalive=False
>
> rpmbuild_timeout=172800
> -
> +{% if ansible_hostname.startswith('buildvm-s390x-02') %}
> +; Set oz timeout higher on x390x image builder to allow it to finish.
> +; Install timeout(seconds) for image build
> +; if it's unset, use the number in /etc/oz/oz.cfg, supported since
> oz-0.16.0
> +oz_install_timeout=14400
> +{% endif %}
> use_createrepo_c=True
>
> {% if host in groups['buildvm-s390x'] %}
>
> 3. Run the playbook on it to update the config and restart kojid.
>
> +1s?
>
> kevin
>
>
> _______________________________________________
> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>
2:51 p.m.
New subject: Freeze Break Request: update koji and oz_timeout on
buildvm-s390x-02
On 03/16/2018 12:42 PM, Ricky Elrod wrote:
+1. Do we know 14400 will be enough?
We don't, but I figured we could start there and adjust it if it wasn't
enough still.
kevin
--
-re
On Fri, Mar 16, 2018 at 3:29 PM, Kevin Fenzi <kevin(a)scrye.com> wrote:
> Greetings.
>
> image composes using oz take a long time on our s390x image builder,
> (due to them being in another datacenter). They take so long, they timeout.
>
> See: https://pagure.io/koji/pull-request/837 from sinny and
> https://pagure.io/koji/pull-request/841 which makes this value
> configurable in kojid.conf
>
> I would like to:
>
> 1. Update koji on buildvm-s390x-02 to the version I just built in
> rawhide/f28 that has a backport of the patch that makes this value
> configurable.
>
> 2. Apply the following ansible patch to set the timeout only on the
> buildvm-s390x-02 builder:
>
> diff --git a/roles/koji_builder/templates/kojid.conf
> b/roles/koji_builder/templates/kojid.conf
> index 832d80c..9f10d9c 100644
> --- a/roles/koji_builder/templates/kojid.conf
> +++ b/roles/koji_builder/templates/kojid.conf
> @@ -20,7 +20,12 @@ maxjobs=25
> keepalive=False
>
> rpmbuild_timeout=172800
> -
> +{% if ansible_hostname.startswith('buildvm-s390x-02') %}
> +; Set oz timeout higher on x390x image builder to allow it to finish.
> +; Install timeout(seconds) for image build
> +; if it's unset, use the number in /etc/oz/oz.cfg, supported since
> oz-0.16.0
> +oz_install_timeout=14400
> +{% endif %}
> use_createrepo_c=True
>
> {% if host in groups['buildvm-s390x'] %}
>
> 3. Run the playbook on it to update the config and restart kojid.
>
> +1s?
>
> kevin
>
>
> _______________________________________________
> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>
_______________________________________________
infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
3:25 p.m.
+1 to try
On Fri, Mar 16, 2018 at 8:51 PM, Kevin Fenzi <kevin(a)scrye.com> wrote:
> On 03/16/2018 12:42 PM, Ricky Elrod wrote:
>> +1. Do we know 14400 will be enough?
>
> We don't, but I figured we could start there and adjust it if it wasn't
> enough still.
>
> kevin
> --
>>
>> -re
>>
>> On Fri, Mar 16, 2018 at 3:29 PM, Kevin Fenzi <kevin(a)scrye.com> wrote:
>>> Greetings.
>>>
>>> image composes using oz take a long time on our s390x image builder,
>>> (due to them being in another datacenter). They take so long, they timeout.
>>>
>>> See: https://pagure.io/koji/pull-request/837 from sinny and
>>> https://pagure.io/koji/pull-request/841 which makes this value
>>> configurable in kojid.conf
>>>
>>> I would like to:
>>>
>>> 1. Update koji on buildvm-s390x-02 to the version I just built in
>>> rawhide/f28 that has a backport of the patch that makes this value
>>> configurable.
>>>
>>> 2. Apply the following ansible patch to set the timeout only on the
>>> buildvm-s390x-02 builder:
>>>
>>> diff --git a/roles/koji_builder/templates/kojid.conf
>>> b/roles/koji_builder/templates/kojid.conf
>>> index 832d80c..9f10d9c 100644
>>> --- a/roles/koji_builder/templates/kojid.conf
>>> +++ b/roles/koji_builder/templates/kojid.conf
>>> @@ -20,7 +20,12 @@ maxjobs=25
>>> keepalive=False
>>>
>>> rpmbuild_timeout=172800
>>> -
>>> +{% if ansible_hostname.startswith('buildvm-s390x-02') %}
>>> +; Set oz timeout higher on x390x image builder to allow it to finish.
>>> +; Install timeout(seconds) for image build
>>> +; if it's unset, use the number in /etc/oz/oz.cfg, supported since
>>> oz-0.16.0
>>> +oz_install_timeout=14400
>>> +{% endif %}
>>> use_createrepo_c=True
>>>
>>> {% if host in groups['buildvm-s390x'] %}
>>>
>>> 3. Run the playbook on it to update the config and restart kojid.
>>>
>>> +1s?
>>>
>>> kevin
>>>
>>>
>>> _______________________________________________
>>> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
>>> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>>>
>> _______________________________________________
>> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
>> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>>
>
>
>
> _______________________________________________
> infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
>
10:18 a.m.
New subject: Freeze Break Request: update koji and oz_timeout on
buildvm-s390x-02
On 03/16/2018 03:29 PM, Kevin Fenzi wrote:
Greetings.
image composes using oz take a long time on our s390x image builder,
(due to them being in another datacenter). They take so long, they timeout.
See: https://pagure.io/koji/pull-request/837 from sinny and
https://pagure.io/koji/pull-request/841 which makes this value
configurable in kojid.conf
I would like to:
1. Update koji on buildvm-s390x-02 to the version I just built in
rawhide/f28 that has a backport of the patch that makes this value
configurable.
2. Apply the following ansible patch to set the timeout only on the
buildvm-s390x-02 builder:
Thanks Kevin. We are now hitting another issue on s390x during anaconda.
https://koji.fedoraproject.org/koji/taskinfo?taskID=25761596
```
10:58:45,692 ERR anaconda:anaconda: startup_utils: Waiting for modules to be started timed
out.
10:58:45,804 ERR anaconda:anaconda: stdout: Anaconda DBus modules failed to start on
time.
```
Sinny, can you investigate this new issue?
Dusty
10:32 p.m.
On Sat, Mar 17, 2018 at 8:48 PM, Dusty Mabe <dusty(a)dustymabe.com> wrote:
On 03/16/2018 03:29 PM, Kevin Fenzi wrote:
> Greetings.
>
> image composes using oz take a long time on our s390x image builder,
> (due to them being in another datacenter). They take so long, they timeout.
>
> See: https://pagure.io/koji/pull-request/837 from sinny and
> https://pagure.io/koji/pull-request/841 which makes this value
> configurable in kojid.conf
>
> I would like to:
>
> 1. Update koji on buildvm-s390x-02 to the version I just built in
> rawhide/f28 that has a backport of the patch that makes this value
> configurable.
>
> 2. Apply the following ansible patch to set the timeout only on the
> buildvm-s390x-02 builder:
Thanks Kevin. We are now hitting another issue on s390x during anaconda.
https://koji.fedoraproject.org/koji/taskinfo?taskID=25761596
```
10:58:45,692 ERR anaconda:anaconda: startup_utils: Waiting for modules to be started
timed out.
10:58:45,804 ERR anaconda:anaconda: stdout: Anaconda DBus modules failed to start on
time.
```
Sinny, can you investigate this new issue?
Last two F28 S390x cloud-base compose from 20180317 [1] and
20180318[2] looks good.
It might be a temporary issue, will look into it if error occurs again.
[1]
https://kojipkgs.fedoraproject.org/compose/branched/Fedora-28-20180317.n....
[2]
https://kojipkgs.fedoraproject.org/compose/branched/Fedora-28-20180318.n....
Dusty
_______________________________________________
infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
--
http://sinny.io/
10:33 p.m.
On Sat, Mar 17, 2018 at 12:59 AM, Kevin Fenzi <kevin(a)scrye.com> wrote:
Greetings.
image composes using oz take a long time on our s390x image builder,
(due to them being in another datacenter). They take so long, they timeout.
See: https://pagure.io/koji/pull-request/837 from sinny and
https://pagure.io/koji/pull-request/841 which makes this value
configurable in kojid.conf
Thanks a lot Kevin for this fix!
I would like to:
1. Update koji on buildvm-s390x-02 to the version I just built in
rawhide/f28 that has a backport of the patch that makes this value
configurable.
2. Apply the following ansible patch to set the timeout only on the
buildvm-s390x-02 builder:
diff --git a/roles/koji_builder/templates/kojid.conf
b/roles/koji_builder/templates/kojid.conf
index 832d80c..9f10d9c 100644
--- a/roles/koji_builder/templates/kojid.conf
+++ b/roles/koji_builder/templates/kojid.conf
@@ -20,7 +20,12 @@ maxjobs=25
keepalive=False
rpmbuild_timeout=172800
-
+{% if ansible_hostname.startswith('buildvm-s390x-02') %}
+; Set oz timeout higher on x390x image builder to allow it to finish.
+; Install timeout(seconds) for image build
+; if it's unset, use the number in /etc/oz/oz.cfg, supported since
oz-0.16.0
+oz_install_timeout=14400
+{% endif %}
use_createrepo_c=True
{% if host in groups['buildvm-s390x'] %}
3. Run the playbook on it to update the config and restart kojid.
+1s?
kevin
_______________________________________________
infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org
--
http://sinny.io/
11:07 a.m.
New subject: Freeze Break Request: update koji and oz_timeout on
buildvm-s390x-02
On 03/18/2018 08:33 PM, Sinny Kumari wrote:
On Sat, Mar 17, 2018 at 12:59 AM, Kevin Fenzi <kevin(a)scrye.com>
wrote:
> Greetings.
>
> image composes using oz take a long time on our s390x image builder,
> (due to them being in another datacenter). They take so long, they timeout.
>
> See: https://pagure.io/koji/pull-request/837 from sinny and
> https://pagure.io/koji/pull-request/841 which makes this value
> configurable in kojid.conf
Thanks a lot Kevin for this fix!
No problem. :)
kevin
2229
days inactive
2232
days old
infrastructure@lists.fedoraproject.org
7 comments
5 participants
participants (5)
-
Dusty Mabe -
Kevin Fenzi -
Patrick Uiterwijk -
Ricky Elrod -
Sinny Kumari