I have created a script that will backup some files from my web server to my local server via the internet.. I will be using Rsync over SSH.. I am unsure of the best way to impliment compression since bothe Rsync and SSH have the ability to compress data..

On Wed, 2004-04-21 at 02:38, Tom 'Needs A Hat' Mitchell wrote: >On a local link I turn compression off. The CPU effort and latency to >compress then uncompress does not justify the time saved in transfer >time. > >Also most digital content (images, rpms) do not compress enough to >justify the cycles. Some increase in size.... > >Since this depends on your content and your local system capabilities >I can only advise you to list all the possible environment knobs in >your script and then benchmark by turning them on and off. > >Of interest if compression proves to be an advantage for backups then >you should check into compression on the httpd server side. It is >possible to present compressed content to an aware client that is then >expanded locally by the browser. > >If you think about it a distant proxy server could do this and make >the link look faster. Some services are apparently doing this and >charging extra for it. As a content provider you should do this to >save both you and your customers bandwidth. It might be interesting >to make sure that precompressed content is not expanded to make a link >look slow. I seriously doubt that any service would like to be caught >doing this ... but ... ya never know. > >The backup impact of this is that the pages on the web site are >compressed already and will not compress any more. So why bother. > > Something else to consider is if you are keeping multiple copies of the backup. Tar is much faster at doing this than rsync _if_ the files being backed up do not already exist on the other end. Rsync gets its speed by moving only what has changed (it compares both ends of the connection before deciding). Therefore, if you are keeping only a single copy of the backup, you can use rsync, but if you are keeping multiple copies, for example a complete copy of the data for every day of the past month, then I think tar may be the better choice. (there are reasons for doing it either way, but I won't get into that here, just providing an option)

On Wed, Apr 21, 2004 at 06:59:37AM +0100, WipeOut wrote: >I have created a script that will backup some files from my web server >to my local server via the internet.. I will be using Rsync over SSH.. > >I am unsure of the best way to impliment compression since bothe Rsync >and SSH have the ability to compress data.. > > On a local link I turn compression off. The CPU effort and latency to compress then uncompress does not justify the time saved in transfer time.

Also most digital content (images, rpms) do not compress enough to justify the cycles. Some increase in size.... Since this depends on your content and your local system capabilities I can only advise you to list all the possible environment knobs in your script and then benchmark by turning them on and off.

Of interest if compression proves to be an advantage for backups then you should check into compression on the httpd server side. It is possible to present compressed content to an aware client that is then expanded locally by the browser.