Exchange Server Forums

antivirus removed and corrupted log file -URGENT HELP PLEASE

Users viewing this topic: none

Logged in as: Guest

Printable Version

All Forums >> [Microsoft Exchange 2003] >> Information Stores >> antivirus removed and corrupted log file -URGENT HELP PLEASE

Page: [1]

Message

<< Older Topic Newer Topic >>

antivirus removed and corrupted log file -URGENT HELP P... - 26.Jun.2004 4:53:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Hi

At 10:30 last night the Mcaffee labled a log file for 2 mailbox stores as a virus and then tried removing it from the log file folder. In my panic I copied the quarantined file over the E03.log file. Stupid, I know. The databases are have now been labled as inconsistent. NO BACKUPS! We were in the process of changing our backup system so the last valid backup is for the 12 of June.

The other mailbox stores have been remounted and are operational.

As mail was starting to queue and we are a 24/7 company I spent 5 hours recreating user accounts.
They only way to do this was to delete the mailbox and recreat a new one in a new location. Moving the mailbox to a new store kept failing. All users from the inconsitent databases are operational with empty mailboxes!

I have tried running eseutil /p but has only given me a working blank database.

I then dismounted that and copied a copy of the edb and stm files to that edb.

I then tried running exmerge and it tells me there something like "no user mailbox information was found in the database specified".

Can anyone please give me advice as to recover the information in those databases and keep my job?

Regards
Viv "[Eek!]"

[ June 26, 2004, 04:56 PM: Message edited by: Vivienne ]

Post #: 1

Featured Links*

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 11:55:00 AM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

So you still have the .EDB/.STM files containing the mailbox data?

Have you tried to mount the database through the Recovery Storage Group feature? If not this is worth a try, as you can then export the mailbox data from the .edb files using Exmerge.

The quicker but more expensive method would be to buy a copy of OnTrack's PowerControls, which can recover mailbox data from an unmounted mailbox store.

(in reply to Vivienne)

Post #: 2

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 4:06:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Hi

I tried with one of the databases that way, but which did mount but had no info in it? It showed no mailboxes.. is it supposed to be blank?

I made a copy of the edb and stm files so I have deleted the originals and are now working from the copies. I still have all the log files right up to the time the corruption occured, it was just the main one E03.log that was quarantined. I was wondering if I could use those and manually replay them somehow. I deleted the original edb and stm files, as I had run eseutil /r (which failed) and eseutil /p which gave me a blank database..... [Mad]

I will try the recovery storage group again with the copies of the edb, stm.

Thanks
Viv

(in reply to Vivienne)

Post #: 3

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 5:57:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Created the Recovery Storage group and added the db to be recovered. Copied the edb and stm files to that location, tried mounting the db and got the following error:

"Atleast one of this stores database files is missing. Mounting this store will force the creation of an empty database." I clicked on no. The edb file is still 12GB. I take it that the only option now is to restore the backup of the 12th of june or use "powercontrol" software?

I have downloaded the trail version of ontracks powercontrol so I can see if there are any mailboxes that I can retrieve......

Thanks for the advice!
Viv

[ June 27, 2004, 06:05 PM: Message edited by: Vivienne ]

(in reply to Vivienne)

Post #: 4

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 7:45:00 PM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

Hi again Vivienne,

Unfortunately you can't replay the log files if you already ran Eseutil /P on the database, you should really have run Eseutil /R first [Roll Eyes]

As the log files are now useless, I would defintely try to give PowerControls a shot.

(in reply to Vivienne)

Post #: 5

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 8:38:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

I didnt think of that. I ran the eseutil /p and when it came up empty, I deleted the empty database and replaced them with the file backup of the edb file. I didnt think it would affect the log files [Frown]

I did an integrity check on the copy of the edb file and got the following output:
C:\Program Files\Exchsrvr\bin>eseutil /k g:\backup\operations\opsmailboxstore.e
db

Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating CHECKSUM mode...
Database: g:\backup\operations\opsmailboxstore.edb
Streaming File: g:\backup\operations\opsmailboxstore.STM
Temp. Database: TEMPCHKSUM3608.EDB

File: g:\backup\operations\opsmailboxstore.edb

Checksum Status (% complete)

0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|----|----|----|----|----|
...................................................

2985090 pages seen
0 bad checksums
1216 uninitialized pages
0 wrong page numbers

186569 reads performed
11660 MB read
190 seconds taken
61 MB/second
194161869 milliseconds used
1040 milliseconds per read
1750 milliseconds for the slowest read
31 milliseconds for the fastest read

File: g:\backup\operations\opsmailboxstore.STM
ERROR: database was not shutdown cleanly (dirty shutdown)

Operation terminated with error -550 (JET_errDatabaseDirtyShutdown, Database was
not shutdown cleanly. Recovery must first be run to properly complete database
operations for the previous shutdown.) after 194.266 seconds.
===========================================

I did the same check on the first 10 log file and they all passed the intergrity check (but that was after I ran eseutil /p). I take it now that just means they are useless.....

Regards
Viv

(in reply to Vivienne)

Post #: 6

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 9:03:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

I went back through the eventlogs and found the events relating to this mess:

First Event log message:
Information Store (2804) Operations: An attempt to move the file "L:\Logs\Operations\E03.log"
to "L:\Logs\Operations\E030B3E8.log" failed with system error 2 (0x00000002):
"The system cannot find the file specified. ". The move file operation will fail
with error -1811 (0xfffff8ed).

Second:
Information Store (2804) Operations: Unable to create a new logfile because the database
cannot write to the log drive. The drive may be read-only, out of disk space,
misconfigured, or corrupted. Error -1811.

Third:
Information Store (2804) Operations: The logfile sequence in "L:\Logs\Operations\"
has been halted due to a fatal error. No further updates are possible for the databases
that use this logfile sequence. Please correct the problem and restart or restore
from backup.

Fourth:
Information Store (2804) Operations: Unable to rollback operation #59148660 on database
H:\mdbdata\Operations\Ops Mailbox Store.edb. Error: -510.
All future database updates will be rejected.

5th:
An error occurred while writing to the database log file of storage group "Operations".
Attempting to unmount all databases in this storage group.

6th:
Database error 0xfffffbbe occurred in function JTAB_BASE::EcPrepareUpdate while accessing
the database "Operations\Ops Mailbox Store".

7th:
Database error 0xfffffbbe occurred in function JetRollbackTransaction while accessing
the database "Operations\Ops Mailbox Store".

8th:
Error 0xfffffbbe returned from closing database table, called
from function JTAB_BASE::EcCloseTable on table Folders.

The first event message mentions "E030B3E8.log" the last log in the log folder is E030B3E6.log?

[ June 27, 2004, 09:05 PM: Message edited by: Vivienne ]

(in reply to Vivienne)

Post #: 7

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 9:36:00 PM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

You could also try to give PSS a call, you may get a Support Engineer with a few good ideas...

(in reply to Vivienne)

Post #: 8

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 10:29:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Thanks for all your help. I will give MS a call in the morning.

Just to add to an earlier comment, I ran eseutil /r before I ran eseutil /p but that failed. I used eseutil /p as a last resort.

Thanks again
Viv

(in reply to Vivienne)

Post #: 9

RE: antivirus removed and corrupted log file -URGENT HE... - 27.Jun.2004 10:44:00 PM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

You're welcome...

Best of luck!

(in reply to Vivienne)

Post #: 10

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 8:22:00 AM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Thanks Henrik,

The trail version of PowerControl shows all the mailboxes. So if Microsoft can't help, then I will purchase PowerControl. It looks like quite a good product. Why doesn't microsoft provide software like this....????

Thanks
Viv

(in reply to Vivienne)

Post #: 11

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 7:19:00 PM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

Yes I only had positive experiences with Powercontrols too. Well Microsoft are slowly moving in this direction, first they offered RSG and who know maybe one day they will implement a feature similar to Powercontrols, making it possible to access mailbox data in a dismounted mailbox store.

(in reply to Vivienne)

Post #: 12

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 8:18:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Called MS and they told me that they are not a data recovery company. They gave me some options, which I ran on the databases and it repaired the database, I was able to remount the db's. With all looking good (db's the same size as before), I deleted the file copies of the databases.

I then convinced my company to purchase PowerControl as Exmerge doesn't work if any changes had been made in AD. I then ran defrag on the offline database as suggested by MS. The databases are now much smaller and after restoring some users mailboxes with Powercontrols they complained about missing info. Atleast 2 mailboxes are missing.....

Can you recommend any software that will retreive the 18gb and the 12gb databases after being deleted, that are safe for production servers?

Thanks
Viv

(in reply to Vivienne)

Post #: 13

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 9:01:00 PM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

So the databases are up and running and all mailbox data are intact after the call to PSS right?

I don't quite understand the last part of what you're writing, you tried to restore two users mailboxes with Powercontrols?

Why do you want to restore the two 18 and 12GB databases?

(in reply to Vivienne)

Post #: 14

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 9:33:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

Yes, they are up and running, but missing alot of info. Even though the db's are now operational they are missing alot of users mail and also complete mailboxes are missing.

Being a 24/7 company, so when this happened over the weekend, the mail was starting to queue (1500+ mails). The only option I had to stop mails bouncing was to get the users mailboxes working to continue the flow of mail. As I couldnt access the corrupted store to move them the only choice was to deleted all mailboxes and recreated new ones in a different storage group. This gave me time to try and sort out the damaged database.

After running eseutil /p again on the damaged db's (MS's suggestion), deleted the chk file and all logs, I was able to mount the databases. They were still the correct size ie 18gb and 12gb. Thinking all was okay, I deleted the file copy of the damaged db's. Then dismounted the repaired db's and ran /d which reduced the size of the db's to 9gb and 7gb. Then reading on an MS kb that exmerge will only work if AD is unaltered, I purchased PowerControls. I then started copying the users mailbox info from the restored db to their new mailboxes. A few users who's mailboxes that I had restored phoned to say they are missing alot of info. After that I checked the restored db and noticed that 2 mailboxes in the restored db's were missing, there might be more. The /p or /d had removed corrupt info and their mailboxes must have been in that corruption.

For this reason, I would like to "undelete" the file copies of the db's I deleted this morning. I am hoping that now that I have PowerControl, I will be able to retreive all the missing info and mailboxes from the original damaged db's.

Regards
Viv

[ June 28, 2004, 09:44 PM: Message edited by: Vivienne ]

(in reply to Vivienne)

Post #: 15

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 9:51:00 PM

Henrik Walther

Posts: 6928
Joined: 21.Nov.2002
From: Copenhagen, Denmark
Status: offline

Okay I understand now.

Undelete from Executive Software are one of the best, though I haven't tried to Exchange database files with it. But it may be worth a try.

Undelete - Reliable File Recovery:
http://www.executive.com/file-recovery/file-recovery.asp

(in reply to Vivienne)

Post #: 16

RE: antivirus removed and corrupted log file -URGENT HE... - 28.Jun.2004 9:59:00 PM

Vivienne

Posts: 57
Joined: 8.Jul.2003
From: Cape Town
Status: offline

I was reading up on that tonight. I will download the trial and see if it finds the deleted db's.

Lets hope this is the end of a disasterous few days! What else can go wrong.... [Wink]