[mirror-admin] quick-fedora-mirror

[Jason L Tibbitts III]

>>>>> "CA" == Chuck Anderson <cra at wpi.edu> writes:

CA> Since deletes are being done before the rsync run, the repo will be
CA> in an inconsistent state during the time period between these
CA> deletes and the full rsync run.

Actually if you look at the current code you'll see that the deletes are
done after the rsync run.  It's changing rapidly (at least while I'm
awake) but it has settled down somewhat.

CA> Am I correct in assuming that if the deletes are moved to after the
CA> big rsync run, that will cause directory mtimes to be wrong, which
CA> is why they were moved to before the big rsync run in the first
CA> place?

Well, it will cause them to be different.  I am not entirely sure that
it is critical for directory mtimes to be the same as on the server and
I'm pretty sure there will always be the possibility the timestamps
changing in a way that isn't corrected by anything but a complete rsync
run.  And in any case, if that's something I have to give up in order to
be able to start transferring files in six seconds instead of eleven
hours, I'll take it.

However, I haven't completely figured out the best way to do things.
Having all of the timestamps match the master repository as closely as
possible is definitely a reasonable goal in any case.

CA> - do the deletes after the full big rsync
CA> - manually fixup the mtimes after the deletions, using the alldirs
CA>   lists

If you mean the file list that comes from the server, it does not
contain mtimes.  (Many, if not most, of the timestamps in there are
ctimes.)  It's not really useful for this purpose.

Some random ideas:

When doing deletions, it should be possible to extract the mtime of the
enclosing directories and touch -m them back, but even then you have to
be careful about the order in which you do things.  I wonder how rsync
does it, since it does plenty of manipulation after it's transferred
things.

It is possible that rsync has an option to delete files from the
--files-from list which do not exist on the server, in which case we
could just let it do the deletions.  I thought I tried a few things but
if there is such an option I didn't find it.

Maybe some smart person here can cook up a "delete and restore timestamp
on containing directory" function while I get some sleep.

 - J<

--