Hi!
I am going to make local repository for fedora updates and extras and I am wondering what is the best tool for it: rsync or wget.
Regrads, Sasa
On Fri, 2005-04-22 at 11:48 +0200, Sasa Stupar wrote:
Hi!
I am going to make local repository for fedora updates and extras and I am wondering what is the best tool for it: rsync or wget.
It depends on the remote server you are trying to mirror.
If it provides rsync-access, then rsync probably is the preferable choice.
If it doesn't provide rsync-access, then other tools like wget, lftp, curl or other tools (there are plenty of them, all have pros and cons) are possible escapes.
Ralf
--On 22. april 2005 12:16 +0200 Ralf Corsepius rc040203@freenet.de wrote:
On Fri, 2005-04-22 at 11:48 +0200, Sasa Stupar wrote:
Hi!
I am going to make local repository for fedora updates and extras and I am wondering what is the best tool for it: rsync or wget.
It depends on the remote server you are trying to mirror.
If it provides rsync-access, then rsync probably is the preferable choice.
If it doesn't provide rsync-access, then other tools like wget, lftp, curl or other tools (there are plenty of them, all have pros and cons) are possible escapes.
Ralf
What I need is to also remove old versions automatically while transfering the new ones. I was playing with wget mirror function but it takes everything from the root dir on the remote server but I only need for fc3. Looks like lftp could do this job or am I mistaken?
Sasa
Sasa Stupar wrote:
--On 22. april 2005 12:16 +0200 Ralf Corsepius rc040203@freenet.de wrote:
On Fri, 2005-04-22 at 11:48 +0200, Sasa Stupar wrote:
Hi!
I am going to make local repository for fedora updates and extras and I am wondering what is the best tool for it: rsync or wget.
It depends on the remote server you are trying to mirror.
If it provides rsync-access, then rsync probably is the preferable choice.
If it doesn't provide rsync-access, then other tools like wget, lftp, curl or other tools (there are plenty of them, all have pros and cons) are possible escapes.
Ralf
What I need is to also remove old versions automatically while transfering the new ones. I was playing with wget mirror function but it takes everything from the root dir on the remote server but I only need for fc3. Looks like lftp could do this job or am I mistaken?
Sasa
Wget will only download whatever rpm packages that you still don't have; I am using wget to download all the fc3 rpm package updates - wget first checks those updates that I have previously downloaded in a specific directory, and compares them against the repository rpm update packages, and will only download those new ones (which my rpm update directory still doesn't have yet), I am at work now and my fc3 box that takes care all the rpm update packages is at home, basically the command is like ; wget -nd --mirror ftp.redhat.fedora\path\to\the\rpm\updates*.rpm -nd This will download rpm packages only and will no duplicate any directory in the source repository. --mirror This will compare the rpm update packages existing in your system against the source depository rpm update packages, if newer version of the rpm update is available in the source repository, then wget will just download that one, and will ignor those you have already had.
I used this way to keep one of my systems to download all the rpm update packages, and my other machines can use them to do the rpm update instead of downloading same thing from Internet again.
On Fri, 2005-04-22 at 14:05 +0200, Sasa Stupar wrote:
--On 22. april 2005 12:16 +0200 Ralf Corsepius rc040203@freenet.de wrote:
On Fri, 2005-04-22 at 11:48 +0200, Sasa Stupar wrote:
Hi!
I am going to make local repository for fedora updates and extras and I am wondering what is the best tool for it: rsync or wget.
It depends on the remote server you are trying to mirror.
If it provides rsync-access, then rsync probably is the preferable choice.
If it doesn't provide rsync-access, then other tools like wget, lftp, curl or other tools (there are plenty of them, all have pros and cons) are possible escapes.
Ralf
What I need is to also remove old versions automatically while transfering the new ones. I was playing with wget mirror function but it takes everything from the root dir on the remote server but I only need for fc3. Looks like lftp could do this job or am I mistaken?
I use the following script to mirror CentOS updates for my servers at work ... it should be possible to modify it to work with fedora with some tweaking. One thing I like about wget is that you can specify a maximum bandwidth so as to not swamp your internet connection.
The one thing you will have to check is the "--cut-dirs=8" this will vary depending on your source repository layout.
#!/bin/sh # # mirror-centos.sh - Mirror CentOS Linux Updates #
# Variables MIRROR="mirror.cs.wisc.edu" WBDIR="pub/mirrors/linux/caosity.org/centos" VERSIONS="4" ARCHS="SRPMS i386 x86_64" WBUPDATES="/home/ftp/pub/centos" PROXY="http://firewall.company.com:8080/"
# Limit Transfer Rate to 10K bytes per second MAXRATE="10k"
# Do the mirror for d in $VERSIONS; do for y in $ARCHS; do cd $WBUPDATES ## env http_proxy=$PROXY \ wget --non-verbose --no-host-directories --cut-dirs=8 --mirror \ --passive-ftp --limit-rate=$MAXRATE \ --directory-prefix=$WBUPDATES/$d/updates/$y \ ftp://$MIRROR/$WBDIR/$d/updates/$y ##--output-file=/tmp/mirror-centos.log
# the RPMS should be checked for valid sigs, not ready for that yet ##rpm --checksig $WBUPDATES/$d/updates/$y/*.rpm | grep 'NOT OK' done done
# Done exit
lftp is my favoriate way to do it. I have a cron job that runs like this:
lftp -f /path/to/config
and the config file looks like this:
open mirrors.kernel.org mirror -e /fedora/core/updates/3/i386/ /path/to/local/repo
This keeps my local repo synced with my favorite mirror.
Hope this helps.
Thomas
--On 22. april 2005 13:23 -0500 Thomas Cameron thomas.cameron@camerontech.com wrote:
lftp is my favoriate way to do it. I have a cron job that runs like this:
lftp -f /path/to/config
and the config file looks like this:
open mirrors.kernel.org mirror -e /fedora/core/updates/3/i386/ /path/to/local/repo
This keeps my local repo synced with my favorite mirror.
Hope this helps.
Thomas
Thanx. That's exactly what I was looking for.
Sasa
-
Barry Yu -
Paul -
Ralf Corsepius -
Sasa Stupar -
thomas cameron