Exchange Server Forums
Forums |
Register |
Login |
My Profile |
Inbox |
RSS 
|
My Subscription |
My Forums |
Address Book |
Member List |
Search |
FAQ |
Ticket List |
Log Out
Urgent - Index shot
|
Users viewing this topic:
none
|
Logged in as: Guest
|
Login  | |
|
 Limited time MSExchange.org offer! -- 1.Sep.2008 1:00:00 PM
|
|
TechGenix and SolarWinds have partnered to provide free copies of SolarWinds Exchange Monitor to all visitors who join the MSExchange.org Forums. SolarWinds Exchange Monitor is a handy desktop dashboard that continuously monitors Microsoft Exchange to deliver real-time insight into Exchange services, mail queue sizes, and host server health. Learn more about Exchange Monitor and the free offer!
|
Urgent - Index shot - 3.Nov.2008 4:21:07 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
Hi all,
just stumbled in to get some help in an issue resulting from a bad PC400 - DIMM in the box affected. ( We' re runing E2K3SP2SE on a W2K3 box. ) The faulty DIMM has been replaced and the streaming - errors related to it went away, but I'm still pi$$ed with the effects it left me with. Most important, the store is functional and mounts fine.
I'm having a recurring 467 error from source ESE in the event viewer telling me that the FidDeleteTimeIndex in Table IndexAge is damaged.
NB: I have 222 errors in the log saying that backup was unable to read everything ( as a result from the ArcServe backup agent connecting to the store. In ArcServe itself I have Read Errors in NamedPipes )
What I've tried aleady reads as follows:
- I ' ve drawn a manual backup of the affected priv1.edb file and ran ESEUTIL /mh against the store resulting in clean logs ( no umont issue, all logs available and OK. ).
- Running a ISINTEG against it resulted in some hundred fixes on 3 or 4 tables, the last message was none ( meaning there was no "done" output or such at the end, if any ).
After remounting the store, which it does fine, the error persists. I fear that it WILL grow and kill my repository if it remains unfixed. So, I dismounted it again.
- Next, I ran an integrity check using ESEUTIL against it.Took some hours before terminating with a JET error. ( Can't remember the errorcode as it was around 2 am and I forgot to make a note of it )
I guess I should run a hard repair now ?
Some othe option would be Exmerge, but I want to avoid having to merge LOTS of boxes into PST - Files and pumping these back into a new store, restoring the ACLs and stuff.
There is no other exchange box available to move the store elsewhere and, as this is a standard E2K3, I cannot build another storage group.
Can someone point me in the appropriate direction ?
D.
PS. There is NO backup ( except the faulty EDB ) as the LTO Drive had quit at the same time  and the new one refuses to read the old tapes. 
< Message edited by Old_Fart -- 3.Nov.2008 4:23:33 AM >
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 4:37:35 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
Well, i must say that you did a good troubleshooting step - however not having a backup isn't something an Exchange guy should be proud of. I guess you would now realize that. We can talk about that at a different time, at the moment let's focus on the issue.
Did you get this sort of DB defrag errors ?
results:
Operation terminated with error -338 (JET_errBadParentPageLink,
Database corrupted) after 1308.32 seconds.
Is it possible for you to paste event id description completely. Also, are you getting any other event along with this event - for example 9296 or 476
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 5:12:03 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
Hi,
and thanks a lot for replying quickly. Not having a backup is NOT an option, I know. I wonder who already had this kind of $hit ( Exchange box malfunctioning on Friday night and the LTO drive exiting with a bad clutch at the damn same time and the new one refusing to readl the old tapes which I have to initialize again. ) Murphy rules, I'm afraid.
I did not have the error you mentioned, but a JET read [ ....] error, I might run a ISINTAG / ESEUTIL defrag / check against the backup files to reproduce the error. The actual errorcode returned was 612 or 613, but I'm pretty sure it was a 613. According to Frank Carius' msexchangefaq.de this might point us to a 1018 growing somewhere under the hood.
The event description/s is / are as follows; EN = English translation of the original german text in the eventviewer.
1. )
Ereignistyp: Fehler
Ereignisquelle: ESE
Ereigniskategorie: Datenbankbeschädigung / EN: Database Damage
Ereigniskennung: 467
Datum: 02.11.2008
Zeit: 20:54:31
Benutzer: Nicht zutreffend
Computer: MTA0
Beschreibung:
Information Store (4504) Erste Speichergruppe: Datenbank C:\Programme\Exchsrvr\mdbdata\priv1.edb: Index FidDeleteTimeIndex von Tabelle IndexAge ist beschädigt (0).
EN: First Storage Group: Database [ ... ] Index [ ... ] from Table [ ... ] is damaged (0 ).
2. )
Backup failure / agent exiting whilst reading:
Ereignistyp: Fehler
Ereignisquelle: ESE
Ereigniskategorie: Protokollierung/Wiederherstellung
Ereigniskennung: 215
Datum: 31.10.2008
Zeit: 23:52:42
Benutzer: Nicht zutreffend
Computer: MTA0
Beschreibung:
Information Store (5792) Erste Speichergruppe: Die Sicherung wurde beendet, weil sie vom Client abgebrochen wurde oder weil die Verbindung mit dem Client fehlgeschlagen ist.
EN: First Storage Group: Backup was terminated because the client aborted it or because the connection to Client the was terminated, too.
3. ) The following error went away after replacing the faulty DIMM.
Ereignistyp: Fehler
Ereignisquelle: ESE
Ereigniskategorie: Protokollierung/Wiederherstellung
Ereigniskennung: 478
Datum: 31.10.2008
Zeit: 23:52:37
Benutzer: Nicht zutreffend
Computer: MTA0
Beschreibung:
Information Store (5792) Fehler bei Überprüfung der aus Datei "C:\Programme\Exchsrvr\mdbdata\priv1.stm" bei Offset 15290912768 (0x000000038f68d000) für 4096 (0x00001000) Bytes gelesenen Streamingseite durch eine Inkonsistenz der Seitenprüfsumme. Die erwartete Prüfsumme war 4227052534 (0x00000000fbf3b3f6), die tatsächliche Prüfsumme 4227052022 (0x00000000fbf3b1f6). Fehler -613 (0xfffffd9b) bei Leseoperation. Wenn dieser Zustand andauert, stellen Sie die Datenbank aus einer vorherigen Sicherung wieder her.
EN Error -613 [HEX] whilst performing read operation [ ....consistency checker ...unexpected value...] . If this condition persists, please restore the DB from a previous backup.
Edit: The Online Defrag runs fine every night.
< Message edited by Old_Fart -- 3.Nov.2008 5:14:08 AM >
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 5:23:35 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
Would it be possible to run the following commands and sending me the reports or uploading them at some location for further analysis.
I am trying to avoid running a repair at the moment.
eseutil /m [ database_name ] /p3733132 > Pages.txt
eseutil /ms priv1.edb >space.txt
isinteg -s <servername> -dump -l c:\dump1.txt
These commands should not take more than 5-10 mins each.
Email (dot) tushar (at) gmail (dot) com
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 5:29:58 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
Sure, I' ll do that.
Do you want me to run that against a copy of the live db ( which I have touched in the above documented way ) ? If so, I must do that tonight ( it's 11:30 am now ) as drawing the backup takes some time. I cannot take the store offline now.
If you want me to run it against the faulty / untouched db ( of which I do have a copy ) I can do that right now. We might allow some time for the commands to execute I guess.
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 7:15:25 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
I have sent you a PM, please have a look
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 7:36:40 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
Thanks again,
you should have the logs from the live DB right now.
please let me know if this is suffcient.
D,
|
|
|
|
|
RE: Urgent - Index shot - 3.Nov.2008 12:54:55 PM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
My advice is to get the database repaired - its the only and last solution in this.
you may want to run it on the copied database and follow that with defrag.
bring it back on the server - run the isinteg - keep fingers crossed and hope things would return soon to normal state. (which it should)
|
|
|
|
|
RE: Urgent - Index shot - 4.Nov.2008 2:44:30 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
Hi,
that's what I thought, too - somehow. You mean to run ESEUTIL /R oder hard repair using the /P switch ?
Can you drop a few lines on what's damaged ?
|
|
|
|
|
RE: Urgent - Index shot - 4.Nov.2008 3:06:05 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
Eseutil /r is simply a soft recovery - this will never clean any sort of corruption - it is only used to bring store to consistent state.
Eseutil /p is the command used for removing complete corruption from any database. This may be required to be followed by isinteg (to remove logical corruption - just as removing those pointer links left behind by hard repair).
Using the reports it is for sure that a particular mailbox / message isn't corrupted. It seems more of a corruption at tree / page level - which I wasn't much able to detect (meant which tree/page to be precise)
|
|
|
|
|
RE: Urgent - Index shot - 4.Nov.2008 3:23:02 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
Can you think of a worst - case scenario whilst throwing the /P switch ?
|
|
|
|
|
RE: Urgent - Index shot - 4.Nov.2008 3:30:03 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
For avoiding those worst case scenario's its always recommended to take a backup copy of the database before you do perform hard repair.
Plus there could be hundreds of those messages that may terminate hard repair, for example -1605 (the worst) or -1216 or -613, why not try crossing your fingers - or have a cup of coffee to ease the tension.
After, all you always have a backup of database taken.
|
|
|
|
|
RE: Urgent - Index shot - 4.Nov.2008 4:47:55 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
As i said take a copy of the database and you may play around with maintenance when you want.
Ideally perform this over post-production hours.
|
|
|
|
|
RE: Urgent - Index shot - 10.Nov.2008 3:21:19 AM
|
|
|
Old_Fart
Posts: 9
Joined: 3.Nov.2008
Status: offline
|
All's well.
The store was repaired successfully, eseutil /p deleted some "unicode fixup table".
After defragmentation and integrity check using isinteg, it mounted fine and the errors are gone. No data losses ( as of now ). total = 14 hours.
Will check if the backup agent runs fine tonight, if so - we're done.
Thanks again for your help !
I guess I may delete the old store's logfiles now ?
|
|
|
|
|
RE: Urgent - Index shot - 10.Nov.2008 4:04:09 AM
|
|
|
Exchange_Geek
Posts: 1026
Joined: 31.Dec.2006
Status: offline
|
Wonderful. That's good news.
I would request you to keep those old log files - until you have taken a new backup successfully. Once you are through with new backup - you can delete your old log files.
Keep us posted about updates.
All the best.
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
Printable Version
