[mirror-admin] Deduplication

[Dennis Gilmore]

On Wednesday, November 25, 2015 07:46:42 AM Carsten Otto wrote:
> On Tue, Nov 24, 2015 at 01:25:51PM -0600, Dennis Gilmore wrote:
> > I would be curious to know where you achieved the savings.  everything
> > that is duplicated should be hardlinked already, as long as you mirror
> > the hardlinks you should not see this.  I would appreciate help in
> > making sure we provide the content correctly.
> 
> http://ftp.halifax.rwth-aachen.de/~cotto/duperemove-fedora.log.gz
> 
> I don't fully understand the output, but it seems a fair share of
> packages contain duplicated data.
> 
> [0x5ab1400] Dedupe 2 extents (id: 966d56b7) with target: (0.0, 407.3M), 
> "/pub/fedora//linux/updates/testing/23/armhfp/s/supertuxkart-data-0.9.1-2.f
> c23.noarch.rpm" [0x7098850] Dedupe 2 extents (id: 16e2747b) with target:
> (0.0, 407.4M), 
> "/pub/fedora//linux/updates/testing/22/armhfp/s/supertuxkart-data-0.9.1-1.f
> c22.noarch.rpm"

they likely contain mostly the same content but they are signed by different 
gpg keys and have some changed data, so they can not be hardlinked. I am not 
sure how we can do better here. 

Dennis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.ale.org/pipermail/mirror-admin/attachments/20151125/30c9f7ce/attachment.sig>
-------------- next part --------------
--