[mirror-admin] redundant transfers testing->updates

[Chris Schanzle]

On 01/19/2012 08:58 PM, Dennis Gilmore wrote:
> El Thu, 19 Jan 2012 12:57:02 -0500
> Chris Schanzle<schanzle at nist.gov>  escribió:
>> On 07/13/2011 09:24 AM, Chris Schanzle wrote:
>>> A friendly suggestion...
>>>
>>> For mirrors that carry testing+updates, as packages move from
>>> testing to updates, they are transferred again as new files in
>>> updates.  It would be a significant reduction in traffic if
>>> packages could be be hardlinked from testing to updates, then
>>> removed from testing after a reasonable delay (24hrs?).
>>
>>
>> The recent hardlinking for directory hashing refreshed my memory from
>> the above suggestion, which had no further discussion.  Stupid idea
>> or decided not worth implementing due to infrastructure design issues?
>>
>> --
>
> this was supposed to b happening. maybe i need to add a manual hardlink
> or --link-dest on the scripts that push bits to the mirrors.
>
> Thanks for the reminder.
>
> Dennis

Thanks!  Don't forget to watch out for hardlink failures due to just permissions being different (I think that got nipped a while back).

Here's a sample package that came through the process which shows the landing in updates/testing, later promoted to updates, but was not hardlinked first and was removed from updates/testing same day it landed in updates/.

$ grep pvm-gui-3.4.6-1.fc16 fedora-rsync-{201112,201201}.log
/local/tmp/fedora-rsync-201112.log:2011/12/30 02:55:47 [9567] >f+++++++++ updates/testing/16/i386/pvm-gui-3.4.6-1.fc16.i686.rpm
/local/tmp/fedora-rsync-201112.log:2011/12/30 03:01:15 [9567] >f+++++++++ updates/testing/16/x86_64/pvm-gui-3.4.6-1.fc16.x86_64.rpm
/local/tmp/fedora-rsync-201201.log:2012/01/12 03:17:34 [15367] >f+++++++++ updates/16/i386/pvm-gui-3.4.6-1.fc16.i686.rpm
/local/tmp/fedora-rsync-201201.log:2012/01/12 03:24:04 [15367] >f+++++++++ updates/16/x86_64/pvm-gui-3.4.6-1.fc16.x86_64.rpm
/local/tmp/fedora-rsync-201201.log:2012/01/12 03:28:17 [15361] *deleting   updates/testing/16/i386/pvm-gui-3.4.6-1.fc16.i686.rpm
/local/tmp/fedora-rsync-201201.log:2012/01/12 03:29:08 [15361] *deleting   updates/testing/16/x86_64/pvm-gui-3.4.6-1.fc16.x86_64.rpm

Again, rsync may end up deleting things first depending on sort order and --delete-{before,after} options, so it's important to have an overlap of time where when a package is promoted from testing to updates, it remains in testing for a while (perhaps a day).

Best regards,
Chris


--