[ale] Can't Boot! Need Super Block Wizard

Andrew Wade andrewiwade at gmail.com
Wed Jun 13 15:04:54 EDT 2012


Sorry about that, the picture didn't work properly so I'll remove it.

I've run into superblock issues and have done the following the fix it (I
wrote up a doc on my wiki):

System Hangs at Boot with Size in Superblock error - Fix

It appears that for some of our servers, someone has attemped to extend a
filesystem and forgot to run the resize2fs command.

This makes it so that the filesystem is extend for X amount, but the actual
logical volume isn't resized to utilize this extra space.   What I've found
so far is some of our x nodes for the mount point:

/usr/WebSphere70    were extended, but not resized.   So if the server
crashes and the OS tries to FSCK the filesystem because it saw an
incomplete journal entry (normal), it will not be able to fix the logical
volume.  So what you need to do is the following:



On Boot:

Console Error Picture Removed


Then enter the root password and press enter.

Then find the name of the logical volume that has the superblock error:  In
this case, /usr/WebSphere70.   You'll want to find the logical volume that
this corresponds to:   cat /etc/fstab | more    and you'll see that mount
point is linked to   /dev/vg02/lv
 cat /etc/fstab | more
....
and you'll see that mount point is linked to:
/dev/vg02/lv_uwas70



So next, you'll want to fix the superblock and FSCK the disk   (if you just
fsck the disk, it won't work):
 mke2fs -S /dev/vg02/lv_uwas70 && fsck /dev/vg02/lv_uwas70

Then answer yes to repair each item.  Once it's finished, you can reboot
and see if it comes up cleanly.



OS Boots up, but claims the filesystem does not have a journal and
therefore won't mount:
 #Make a journal
tune2fs -j /dev/vg02/lv_uwas70

#mount the filesystem
mount /usr/WebSphere70

Then it mounts correctly!


On Wed, Jun 13, 2012 at 3:03 PM, Andrew Wade <andrewiwade at gmail.com> wrote:

> Marvin,
>
> I've run into superblock issues and have done the following the fix it (I
> wrote up a doc on my wiki):
>
> System Hangs at Boot with Size in Superblock error - Fix
>
> It appears that for some of our servers, someone has attemped to extend a
> filesystem and forgot to run the resize2fs command.
>
> This makes it so that the filesystem is extend for X amount, but the
> actual logical volume isn't resized to utilize this extra space.   What
> I've found so far is some of our x nodes for the mount point:
>
> /usr/WebSphere70    were extended, but not resized.   So if the server
> crashes and the OS tries to FSCK the filesystem because it saw an
> incomplete journal entry (normal), it will not be able to fix the logical
> volume.  So what you need to do is the following:
>
>
>
> On Boot:
>
> Console Error:
>
>
> Then enter the root password and press enter.
>
> Then find the name of the logical volume that has the superblock error:
> In this case, /usr/WebSphere70.   You'll want to find the logical volume
> that this corresponds to:   cat /etc/fstab | more    and you'll see that
> mount point is linked to   /dev/vg02/lv
>  cat /etc/fstab | more
> ....
> and you'll see that mount point is linked to:
> /dev/vg02/lv_uwas70
>
>
>
> So next, you'll want to fix the superblock and FSCK the disk   (if you
> just fsck the disk, it won't work):
>  mke2fs -S /dev/vg02/lv_uwas70 && fsck /dev/vg02/lv_uwas70
>
> Then answer yes to repair each item.  Once it's finished, you can reboot
> and see if it comes up cleanly.
>
>
>
> OS Boots up, but claims the filesystem does not have a journal and
> therefore won't mount:
>  #Make a journal
> tune2fs -j /dev/vg02/lv_uwas70
>
> #mount the filesystem
> mount /usr/WebSphere70
>
> Then it mounts correctly!
>
>
> On Wed, Jun 13, 2012 at 2:32 PM, Michael H. Warfield <mhw at wittsend.com>wrote:
>
>> On Wed, 2012-06-13 at 13:00 -0400, mdkliman at aol.com wrote:
>>
>> > I'm "The Marvin" who's calling for help.
>>
>> > Once upon-a-time I was technical - during the minicomputer era - but
>> > stopped short of Linux immersion.  I understand what I'm reading and
>> > am grateful for the helpful comments I've received.
>>
>> > I'm sure the drives were inaccessible to the system when we tried to
>> > reboot (red lamps were lighted).  They appear to be online now (green
>> > lamps are lighted).  However, I don't know where to go from here.  I
>> > am not conversant with Linux or its utilities.
>>
>> Red lamps and green lamps???  What kind of server is this we are talking
>> about and what kind of drives?  Are they external drives?  In another
>> message, I think you made a remark about "rdrives" here:
>>
>> On Wed, 2012-06-13 at 12:41 -0400, mdkliman at aol.com wrote:
>> > Robert,
>> >
>> > I know the root password, but have no idea what utility to run. The
>> > system is where it failed, except that the rdrives that couldn't be
>> > accessed are now back online.
>>
>> What are these "rdrives" and how many drives are we talking about???
>> Are you referring to raid drives?  Is this a raid array cabinet?
>>
>> Have you rebooted the server since the drives came on-line?  I have seen
>> this in the past where drives were unavailable at boot up due to
>> (generally brief) power loss or fluctuations.  If you haven't, I
>> seriously doubt you have anything to loose at this point and would go
>> ahead and reboot the system first, without doing a power down.
>>
>> If that works, great.  If not, trying powering it all down and bringing
>> it back up after a few minutes.
>>
>> If either of those work, your first order of business is to get your
>> data OFF that system.  There is no way you can trust it going forward.
>>
>> There are some recovery applications available such as dd-rescue which
>> can be used to recover data from a moderately damaged drive and there
>> are other techniques for use if you've blown a controller board or an
>> interface.  But we would need a lot more information first.
>>
>> If someone gets out there (I'm totally unavailable for at least a couple
>> of weeks) and makes it work, it doesn't mean they "made it all better".
>> You need to upgrade that system or history will repeat itself and it
>> will be costly.
>>
>> Regards,
>> Mike
>>
>> > My programmer took a quick look and insists that I get Admin help to
>> > get the system back in order.  I am seeking someone knowledgeable who
>> > will come on-site and attempt to "clean up" the file system or
>> > determine that it's trashed.
>> >
>> > Glug, glug, glug ...
>> > Marvin in Norcross, GA
>> > (404) 433-0900
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jim Lynch <ale_nospam at fayettedigital.com>
>> > To: Atlanta Linux Enthusiasts <ale at ale.org>
>> > Sent: Wed, Jun 13, 2012 6:55 am
>> > Subject: Re: [ale] Can't Boot! Need Super Block Wizard
>> >
>> >
>> > On 06/13/2012 05:32 AM, Matthew wrote:
>> >  I am more curious, Does the Marvin know how to do everything that we
>> >  are suggesting? I rather just go down there and fix it. I just don't
>> >  have time to drive down 10 hours, fix it, and drive back.
>> > e did say he had a programmer that was familiar with Linux but probably
>> > asn't a true sys admin.  I'm hoping Marvin can answer your question
>> > hen he comes online.
>> > Jim.
>> > ______________________________________________
>> > le mailing list
>> > le at ale.org
>> > ttp://mail.ale.org/mailman/listinfo/ale
>> > ee JOBS, ANNOUNCE and SCHOOLS lists at
>> > ttp://mail.ale.org/mailman/listinfo
>> >
>> >
>> > _______________________________________________
>> > Ale mailing list
>> > Ale at ale.org
>> > http://mail.ale.org/mailman/listinfo/ale
>> > See JOBS, ANNOUNCE and SCHOOLS lists at
>> > http://mail.ale.org/mailman/listinfo
>>
>> --
>> Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
>>   /\/\|=mhw=|\/\/          | (678) 463-0932 |
>> http://www.wittsend.com/mhw/
>>   NIC whois: MHW9          | An optimist believes we live in the best of
>> all
>>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20120613/65d6ccf2/attachment-0001.html 


More information about the Ale mailing list