[ale] ISCSI array on virtual machine

Thu Apr 28 16:56:22 EDT 2016

What filesystem and kernel versions were you using during that time?  If
all those transitions were made using the same filesystem, I'm impressed by
the quality of the filesystem code.

In the past I have see transitions like that tickle latent bugs in the
filesystem code (or device mapper code or md code or block layer or virtual
memory subsystem).  I usually create a fresh filesystem and rsync
contents.  Partly it's to get the free defragmentation, but also it's for
fear of bugs.

On Thu, Apr 28, 2016 at 4:31 PM, Jeff Jansen <bamakojeff at gmail.com> wrote:

> Using LVM costs you almost nothing and it offers tremendous advantages.
> Even if you don't need those advantages now, having them available for free
> (or nearly) is a great asset as a sysadmin.
>
> Over five years we had a backup system that began as a single hard drive
> in a single machine, which became a RAID array in a single machine, which
> became two RAID arrays in two machines connected by DRBD and a crossover
> cable, which became multiple RAID arrays in multiple HA machines across a
> WAN.
>
> Having LVM as part of the underlying architecture made all those changes,
> while not "easy," much easier than it would have been without it.  If you
> use LVM but then never change partition on the 8 TB drive, you'll never
> know it's there, and it will never cause you any trouble.  But if you do
> ever decide to make changes, you will be immensely grateful for the
> possibilities it opens up for you.
>
> HTH
>
> Jeff
>
> On Thu, Apr 28, 2016 at 10:34 AM, Todor Fassl <fassl.tod at gmail.com> wrote:
>
>> But, Jim, what I'm asking is why bother carving it up at all? What
>> benefit is there in that?
>>
>> I get that if you use LVM and ext4 file systems, you can resize the
>> partitions. But if I made the whole 8T one big partition, I'd never have
>> any reason to resize it.
>>
>> PS: One thing I forgot to mention that may be of critical importance is
>> that quotas are enforced on this drive.
>>
>>
>>
>>
>>
>> On 04/28/2016 06:16 AM, Jim Kinney wrote:
>>
>>> I have a large drive array for my department. I use LVM to carve it up. I
>>> leave a huge chunk unallocated so I can extend logical partitions as
>>> required. That dodges the need to shrink existing partitions and allows
>>> XFS
>>> as filesystem.
>>> On Apr 28, 2016 3:27 AM, "Todor Fassl" <fassl.tod at gmail.com> wrote:
>>>
>>> With respect to your question about using LVM ... I guess that was sort
>>>> of
>>>> my original question. If I just allocate the whole 8T to one big
>>>> partition,
>>>> I'd have no reason to use LVM. But I can see the need to use LVM if I
>>>> continue with the scheme where I split the drive into partitions for
>>>> faculty, grads, and staff.
>>>>
>>>> On 04/27/2016 02:27 PM, Jim Kinney wrote:
>>>>
>>>> If you need de-dup, ZFS is the only choice and be ready to throw a lot
>>>>> of RAM into the server so it can do it's job. I was looking at dedupe
>>>>> on 80TB and the RAM hit was 250GB.
>>>>> XFS vs EXT4.
>>>>> XFS is the better choice.
>>>>> XFS does everything EXT4 does except shrink. It was designed for (then
>>>>> very) large files (video) and works quite well with smaller files. It's
>>>>> as fast as EXT4 but will handle larger files and many, many more of
>>>>> them. I want to say exabytes but not certain. Petabytes are OK
>>>>> filesystem sizes with XFS right now. I have no experience with a
>>>>> filesystem of that size but I expect there to be some level of metadata
>>>>> performance hit.
>>>>> If there's the slightest chance of a need to shrink a partition (You
>>>>> _are_ using LVM, right?) then XFS will bite you and require relocation,
>>>>> tear down, rebuild, relocation. Not a fun process.
>>>>> A while back, an install onto a 24 TB RAID6 array refused to budge
>>>>> using EXT4. While EXT4 is supposed to address that kind of size, it had
>>>>> bugs and unimplemented plans for expansion features that were blockers.
>>>>> I used XFS instead and never looked back. XFS has a very complete
>>>>> toolset for maintenance/repair needs.
>>>>> On Wed, 2016-04-27 at 13:54 -0500, Todor Fassl wrote:
>>>>>
>>>>> I need to setup a new file server on a virtual machine with an
>>>>>> attached
>>>>>> ISCSI array. Two things I am obsessing over -- 1. Which file system
>>>>>> to
>>>>>> use and 2. Partitioning scheme.
>>>>>>
>>>>>> The ISCSI array is attached to a ubuntu 16.04 virtual machine. To
>>>>>> tell
>>>>>> you the truth, I don't even know how that is done. I do not manage
>>>>>> the
>>>>>> VMware cluster.  In fact, I think the Dell technitian actually ddid
>>>>>> that
>>>>>> for us. It looks like a normal 8T hard drive on /dev/sdb to the
>>>>>> virtual
>>>>>> machine. The ISCSI array is configured for RAID6 so from what I
>>>>>> understand, all I have to do is choose a file system appropriate for
>>>>>> my
>>>>>> end user's needs. Even though the array looks like a single hard
>>>>>> drive,
>>>>>> I don't have to worry about software RAID or anyhthing like that.
>>>>>>
>>>>>> Googling shows me no clear advantage to ext4, xfs, or zfs. I haven't
>>>>>> been able to find a page that says any one of those is an obvious
>>>>>> choice
>>>>>> in my situation. I have about 150 end-users with nfs mounted home
>>>>>> directories. We also have a handful of people using Windows so the
>>>>>> file
>>>>>> server will have samba installed. It's a pretty good mix of large
>>>>>> files
>>>>>> and small files since different users are doing drastically
>>>>>> different
>>>>>> things. There are users who never do anything but read email and
>>>>>> browse
>>>>>> the web and others doing fluid dynamic simulations on small
>>>>>> supercomputers.
>>>>>>
>>>>>> Secondthing I've been going back and forth on in my own mind is
>>>>>> whether
>>>>>> to do away with seperate partitions for faculty, staff, and grad
>>>>>> students. My co-worker says that's probably an artifact of the days
>>>>>> when
>>>>>> partition sizes were limited. That was before my time here. The last
>>>>>> 2
>>>>>> times we rebuilt our file server, we just maintained the
>>>>>> partitioning
>>>>>> scheme and just made the sizes  times larger. But sometimes the
>>>>>> faculty
>>>>>> partition got filled up while there was still plenty of space left
>>>>>> on
>>>>>> the grad partition. Or it might be the other way around. If we
>>>>>> munged
>>>>>> them all together, that wouldn't happen. The only downside I see to
>>>>>> doing that is that if the faculty partition gets hosed, the grad
>>>>>> partition wouldn't be effected. But that seems like a pretty
>>>>>> arbitrary
>>>>>> choice. We could just assign users randomly to one partition or
>>>>>> another.
>>>>>> When you're setting up a NAS for use by a lot of users, is it
>>>>>> considered
>>>>>> best practice to split it up to limit the damage from a messed up
>>>>>> file
>>>>>> system? I mean, hopefully, that never happens anyway, right?
>>>>>>
>>>>>> Right now, I've got it configured as one gigantic 8T ext4 partition.
>>>>>> But
>>>>>> we won't be going live with it until the end of May so I have plenty
>>>>>> of
>>>>>> time to completely rebuild it.
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ale mailing list
>>>>>> Ale at ale.org
>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>
>>>>>>
>>>>> --
>>>> Todd
>>>> _______________________________________________
>>>> Ale mailing list
>>>> Ale at ale.org
>>>> http://mail.ale.org/mailman/listinfo/ale
>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>> http://mail.ale.org/mailman/listinfo
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>>>
>>>
>> --
>> Todd
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>

-- 
  Ed Cashin <ecashin at noserose.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20160428/ec7d3220/attachment.html>