Jim,<br><br>You really need to fsck that volume to correct the problem.<br><br>Since it is in read only mode, you need to umount it or umount -l it if it doesn't respond to umount.<br><br>Then fsck the logical volume. Then once that completes you need to remount it. Even if the application is happily running, if it tries to retrieve a piece of data on a errored filesystem you risk feeding it bad data which it could send downstream (to a DB or another post processing application) and run into even more problems.<br>
<br>I'd suggest telling the customer that you need to fix the filesystem and get some outage time from the app owners, then umount and fsk it. If that fails, you'll have to go to single user mode and fsck it (just make sure you unmount it even in single user mode before the fsck).<br>
<br>Andrew<br><br><div class="gmail_quote">On Tue, Jan 11, 2011 at 12:43 PM, Jim Kinney <span dir="ltr"><<a href="mailto:jim.kinney@gmail.com">jim.kinney@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<p>Sadly it's bullet biting time. <br>
</p><div><div></div><div class="h5">
<div class="gmail_quote">On Jan 11, 2011 12:16 PM, "Brian Pitts" <<a href="mailto:brian@polibyte.com" target="_blank">brian@polibyte.com</a>> wrote:<br type="attribution">> Hi,<br>> <br>> I have a server whose link to an iSCSI volume was interrupted for so<br>
> long that writes were lost and the volume was remounted read-only.<br>> <br>> kernel: Buffer I/O error on device sdf1, logical block 1<br>> 0174<br>> kernel: lost page write due to I/O error on sdf1<br>> kernel: Aborting journal on device sdf1.<br>
> <br>> A long-running series of processes are currently using the volume<br>> without problems, but of course they will blow up if they try to write<br>> anything to it. I thought I might be able to remount it read-write<br>
> without unmounting it first, but this isn't the case.<br>> <br>> If I try "mount -o remount,rw /mnt/foo", I'm told<br>> "mount: block device /dev/disk/by-label/foo is write-protected, mounting<br>
> read-only" and "EXT3-fs error (device sdf1): ext3_remount: Abort forced<br>> by user" is logged.<br>> <br>> Does anyone have any other ideas, or do I need to bite the bullet and<br>> interrupt the running job to unmount and remount the volume?<br>
> <br>> -- <br>> All the best,<br>> Brian Pitts<br>> _______________________________________________<br>> Ale mailing list<br>> <a href="mailto:Ale@ale.org" target="_blank">Ale@ale.org</a><br>> <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> See JOBS, ANNOUNCE and SCHOOLS lists at<br>> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br></div>
</div></div><br>_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br></blockquote></div><br>