Hi, In the koji/__init__.py:fastUpload method there is a blocksize parameter that is used to determine how many bytes are uploaded to the koji hub per request. By default it is set to 1MB and as far as I can tell that is the value actually used. Is there a way to change this value via koji configuration? A larger blocksize value could significantly improve the upload throughput.

On Tue, Mar 1, 2016 at 4:48 PM, Mike McLean <mikem(a)redhat.com&gt; wrote: > On 03/01/2016 11:09 AM, Dennis Gregorovic wrote: >> Hi, >> >> In the koji/__init__.py:fastUpload method there is a blocksize >> parameter that is used to determine how many bytes are uploaded to >> the koji hub per request. By default it is set to 1MB and as far as >> I can tell that is the value actually used. Is there a way to change >> this value via koji configuration? A larger blocksize value could >> significantly improve the upload throughput. For people on the local LAN sure, but the further away you are the more likely you'll have issues, I suspect people in India, Australia and other places much more will cause issues. > There is no way to change it via configuration. It used to be smaller > and we raised it for performance. At the time at least, 1M seemed to be > roughly the point of diminishing returns. It really depends on the > connection overhead. Is it hard to make it a config option? > Worth noting that there is also a hard coded block size on the other end. Why? > Does anyone have time to experiment with different permutations? Ah, time, I wish.

On Tuesday, March 01, 2016 01:44:12 PM Dennis Gregorovic wrote: > On 03/01/2016 12:05 PM, Peter Robinson wrote: >> On Tue, Mar 1, 2016 at 4:48 PM, Mike McLean <mikem(a)redhat.com&gt; wrote: >>> On 03/01/2016 11:09 AM, Dennis Gregorovic wrote: >>>> Hi, >>>> >>>> In the koji/__init__.py:fastUpload method there is a blocksize >>>> parameter that is used to determine how many bytes are uploaded to >>>> the koji hub per request. By default it is set to 1MB and as far as >>>> I can tell that is the value actually used. Is there a way to change >>>> this value via koji configuration? A larger blocksize value could >>>> significantly improve the upload throughput. >> >> For people on the local LAN sure, but the further away you are the >> more likely you'll have issues, I suspect people in India, Australia >> and other places much more will cause issues. >> >>> There is no way to change it via configuration. It used to be smaller >>> and we raised it for performance. At the time at least, 1M seemed to be >>> roughly the point of diminishing returns. It really depends on the >>> connection overhead. >> >> Is it hard to make it a config option? >> >>> Worth noting that there is also a hard coded block size on the other end. >> >> Why? >> >>> Does anyone have time to experiment with different permutations? >> >> Ah, time, I wish. > > I haven't tested with brew, but we did some brief testing with uploads to > Pulp, which uses a similar upload model. Over a wan, we saw an 8x increase > in upload throughput when using a 10MB block size vs 1MB. > > I don't know if upload speed is generally an issue or not for koji > deployments, but I wanted to throw this out there. I've created an RFE for > the backlog. > > https://pagure.io/koji/issue/30 > > Cheers > -- Dennis Thanks Dennis, Uploads of big items like live iso's, cloud and arm images are extremely slow in fedora. It takes somewhere around 10 to 20 Minutes to upload 1Gb Dennis