8:30 a.m.
Hi
When we announced the FC6 statistics milestone of a million users, it
was described as the amount of downloads in many places. This widespread
confusion even internally in Red Hat is probably many software projects
like say Firefox usually advertise their download rates. One way to
avoid this is provide both download and user estimates except that we
dont really have download numbers.
We could ask each of the mirrors to send us the data but that is a bit
tedious and mirrors might not participate. A better way might be a
mirror list designed to allow us to keep tab of the downloads. sf.net
for example has a web page that is not a static list of mirrors but a
web page with a round robin system. We could do something like that or
geo ip based page which suggests the mirrors near to them. I dont think
we would be able to know partial downloads with this but it allows us a
more accurate estimate of much of these downloads actually translate
into users. Comments?
Rahul
8:41 a.m.
On Mon, 2007-02-19 at 20:00 +0530, Rahul Sundaram wrote:
Hi
When we announced the FC6 statistics milestone of a million users, it
was described as the amount of downloads in many places. This widespread
confusion even internally in Red Hat is probably many software projects
like say Firefox usually advertise their download rates. One way to
avoid this is provide both download and user estimates except that we
dont really have download numbers.
We could ask each of the mirrors to send us the data but that is a bit
tedious and mirrors might not participate. A better way might be a
mirror list designed to allow us to keep tab of the downloads. sf.net
for example has a web page that is not a static list of mirrors but a
web page with a round robin system. We could do something like that or
geo ip based page which suggests the mirrors near to them. I dont think
we would be able to know partial downloads with this but it allows us a
more accurate estimate of much of these downloads actually translate
into users. Comments?
those web page things work not at all with automatic downloading tools.
For example: yum.
how about we continue with what Matt Domsch has been working on and we
evolve the set of data we have.
-sv
8:45 a.m.
seth vidal wrote:
those web page things work not at all with automatic downloading tools.
For example: yum.
They dont need to. Yum would continue to use whatever mirror management
it does. I am just talking about a web interface for ISO downloads.
how about we continue with what Matt Domsch has been working on and we
evolve the set of data we have.
Does it help collect more data than we already can?
Rahul
8:47 a.m.
On Mon, 2007-02-19 at 20:15 +0530, Rahul Sundaram wrote:
seth vidal wrote:
>
> those web page things work not at all with automatic downloading tools.
> For example: yum.
They dont need to. Yum would continue to use whatever mirror management
it does. I am just talking about a web interface for ISO downloads.
So we'll have two different systems for when someone connects to:
http://download.fedora.redhat.com?
that seems, umm, complex.
>
> how about we continue with what Matt Domsch has been working on and we
> evolve the set of data we have.
Does it help collect more data than we already can?
It helps get us a better handle on mirrors. Which can only help.
-sv
9:13 a.m.
seth vidal wrote:
On Mon, 2007-02-19 at 20:15 +0530, Rahul Sundaram wrote:
> seth vidal wrote:
>
>> those web page things work not at all with automatic downloading tools.
>> For example: yum.
> They dont need to. Yum would continue to use whatever mirror management
> it does. I am just talking about a web interface for ISO downloads.
>
So we'll have two different systems for when someone connects to:
http://download.fedora.redhat.com?
that seems, umm, complex.
Yum could connect to say mirrors.fedoraproject.org and the web interface
could be download.fedoraproject.org. Wouldnt that work? Again, I am only
looking to collect the download rates for the iso images. Not individual
packages at this point.
Rahul
8:48 a.m.
On Mon, Feb 19, 2007 at 08:15:44PM +0530, Rahul Sundaram wrote:
seth vidal wrote:
>
>those web page things work not at all with automatic downloading tools.
>For example: yum.
They dont need to. Yum would continue to use whatever mirror management
it does. I am just talking about a web interface for ISO downloads.
>
>how about we continue with what Matt Domsch has been working on and we
>evolve the set of data we have.
Does it help collect more data than we already can?
That's the goal. One thing I'm adding is for the mirror admins to be
able to upload their rsync/ftp/http logs back to our central database
to be analyzed. I haven't done anything more than put in a
placeholder yet, but that's the goal. It would of course be opt-in
from the perspective of the mirror admin, and they are free to
sanitize the logs before sending them to us (e.g. send only data
related to the Fedora directories on their servers). This way we get
the actual traffic data as seen by the mirrors as opposed to a landing
page which isn't.
--
Matt Domsch
Software Architect
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com
10:19 a.m.
On Mon, Feb 19, 2007 at 08:48:34AM -0600, Matt Domsch wrote:
On Mon, Feb 19, 2007 at 08:15:44PM +0530, Rahul Sundaram wrote:
> seth vidal wrote:
>
> >
> >those web page things work not at all with automatic downloading tools.
> >For example: yum.
>
> They dont need to. Yum would continue to use whatever mirror management
> it does. I am just talking about a web interface for ISO downloads.
>
> >
> >how about we continue with what Matt Domsch has been working on and we
> >evolve the set of data we have.
>
> Does it help collect more data than we already can?
That's the goal. One thing I'm adding is for the mirror admins to be
able to upload their rsync/ftp/http logs back to our central database
to be analyzed.
I don't think that will be easy. The larger mirrors which by
definition have more weight in statistics carry more than just
Fedora. And having them filter their logs for feedback is a cumbersome
step most mirror admins won't have time to do. So you'll lose the
bigger players.
I remember Bill asking on redhat-mirror-list-d for some sites to send
back statistics and the feedback was really close to nothing.
The method of redirects is IMHO the best metric gathering tool. It
just needs to stay out of the way, e.g. a broken redirector killing
all the mirrors simulaneously is A Bad Thing.
--
Axel.Thimm at ATrpms.net
9:13 a.m.
Rahul Sundaram wrote:
seth vidal wrote:
> those web page things work not at all with automatic downloading tools.
> For example: yum.
They dont need to. Yum would continue to use whatever mirror management
it does. I am just talking about a web interface for ISO downloads.
But don't most people get the ISO from Bittorrent? Would you attempt to
count them as well?
Matt
9:17 a.m.
Matthew Gillen wrote:
>>those web page things work not at all with automatic
downloading tools.
>>For example: yum.
>They dont need to. Yum would continue to use whatever mirror management
>it does. I am just talking about a web interface for ISO downloads.
But don't most people get the ISO from Bittorrent? Would you attempt to
count them as well?
I am quite doubtfull. I do not know the speed of others mirrors but I do
not cap download speed on my FTP server and it is usually faster to
download it directly from the server than to get it from Bittorent.
François
9:24 a.m.
Francois Petillon wrote:
I am quite doubtfull. I do not know the speed of others mirrors but I
do not cap download speed on my FTP server and it is usually faster to
download it directly from the server than to get it from Bittorent.
François
200,000+ downloads for zod related media, not insignificant.
-Mike
9:21 a.m.
Matthew Gillen wrote:
But don't most people get the ISO from Bittorrent?
AFAIK, no. Most people dont use bittorrent. They are not aware of the
technology, it hasnt worked out to be faster for some of them and we
dont particularly make it easier since we dont provide a bittorrent
client by default in Fedora yet. Moreover there are things like firewall
restrictions, policy against using bittorrent etc in many places.
Would you attempt to
count them as well?
Thats already tracked in http://fedoraproject.org/wiki/Statistics. See
the section called downloads.
Rahul
6274
days inactive
6274
days old
infrastructure@lists.fedoraproject.org
10 comments
7 participants
participants (7)
-
Axel Thimm -
Francois Petillon -
Matt Domsch -
Matthew Gillen -
Mike McGrath -
Rahul Sundaram -
seth vidal