Exchange Server Forums
Forums |
Register |
Login |
My Profile |
Inbox |
RSS 
|
My Subscription |
My Forums |
Address Book |
Member List |
Search |
FAQ |
Ticket List |
Log Out
Private information store too big even after defrag utility
|
Users viewing this topic:
none
|
Logged in as: Guest
|
Login  | |
|
 Limited time MSExchange.org offer! -- 1.Sep.2008 1:00:00 PM
|
|
TechGenix and SolarWinds have partnered to provide free copies of SolarWinds Exchange Monitor to all visitors who join the MSExchange.org Forums. SolarWinds Exchange Monitor is a handy desktop dashboard that continuously monitors Microsoft Exchange to deliver real-time insight into Exchange services, mail queue sizes, and host server health. Learn more about Exchange Monitor and the free offer!
|
Private information store too big even after defrag uti... - 16.Jan.2004 5:18:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
I'm running Exchange Server 5.5 SP4 on an NT4 SP6a box. Even after running the eseutil to defrag the private information store, it is still 13.4 GB. But when I look at the mailbox resources in Exchange Admin util, they all only add up to ~1.5GB. What am I missing?
When our private IS reached the 16GB limit, it crashed Exchange. I deleted all I could before defragging but I'm still not sure why it's so big when it seems it should be much smaller.
We are also using McAfee VirusScan on the mail server.
Any help is appreciated.
- Rim
|
|
|
|
|
RE: Private information store too big even after defrag... - 17.Jan.2004 5:16:00 PM
|
|
|
zbnet
Posts: 815
Joined: 25.Sep.2003
From: Manchester, UK
Status: offline
|
At one time there was a bug in the Admin program that, when a mailbox got bigger than a certain size (I think 2GB), it showed the size as zero. I'm not sure whether that big was fixed, or if it was in what version, but you might be suffering from that.
A good candidate to check would be the IMS administrators mailbox that is set in the IMS - if you have Notifications turned on, you might have got a lot of system messages to that account, which could over time result in the mailbox being too big to see in Admin.
|
|
|
|
|
RE: Private information store too big even after defrag... - 19.Jan.2004 4:57:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
Thanks for the response.
We did experience the bug you posted about. The admin mailbox had over 2 million messages but showed up as 0KB. Attempts to clean the mailbox, including via Clean Mailbox and Outlook client, failed so we deleted the account and recreated it. We ran the defrag utility and did get back 1GB worth, but we're still at 13.4GB.
|
|
|
|
|
RE: Private information store too big even after defrag... - 19.Jan.2004 5:42:00 PM
|
|
|
ismajor
Posts: 27
Joined: 2.Jul.2003
From: Irving, TX
Status: offline
|
This was an offline defrag you ran with eseutil? I'm just asking since online defrags don't actually recover database whitepace.
|
|
|
|
|
RE: Private information store too big even after defrag... - 19.Jan.2004 8:31:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
Yes, offline - stopped the information store service every time before running defrag.
To give you more details, our Exchange Server crapped out and we discovered it was due to the private IS reaching the 16GB limit. We defragged offline and it got down to about 15.5GB - enough to get Exchange working again. We saw the admin mailbox had over 2 million messages but was showing 0KB mailbox resource. We deleted the account and mailbox and recreated it. I also deleted some non-current employee accounts but it wasn't much - maybe 200MB according to Mailbox Resources list. We did offline defrag again and got it down to 13.4GB. Adding up all the mailbox resources, it looks like it should only be around 1.5GB.
This was late last week and our private IS has actually grown to 13.9GB since then. I'm not sure, but from what I've read, if the actual size needed for the IS is smaller than the allocated size (what it takes up on the hard drive), the file won't get larger unless needed. So, let's say our priv IS really only does need 1.5GB but the file is 13.4GB. When new messages come in, the file shouldn't get bigger until the IS actually becomes larger than 13.4GB. Is this correct? If so, then our priv.edb file shouldn't get larger, but it has. It does seem there's that much info on our IS, but get this, we only have 20 users! And like I said, I've added up all the listed Mailbox Resources and they just add up to 1.5GB.
The only other thing I was thinking of was McAfee GroupShield for Exchange and the quarantine function. I deleted a lot of quarantined messages (with attachments) but I haven't run the defrag since then but priv.edb has grown even after that. From what I've read, the quarantined items are stored in a separate file, not in priv.edb, but I'm not absolutely positive.
I'm scratching my head on this one.
- Rim
[ January 19, 2004, 08:35 PM: Message edited by: rim ]
|
|
|
|
|
RE: Private information store too big even after defrag... - 20.Jan.2004 9:30:00 AM
|
|
|
zbnet
Posts: 815
Joined: 25.Sep.2003
From: Manchester, UK
Status: offline
|
Okay, a number of things to say about this.
If there is white space (unused space) in the priv, then yes, the store will use this space first before increasing the size of the file when writing new mail. If your file is growing in size (and you can only really be sure how big it is when you have the IS service stopped), then there is no white space inside the file - you really have that many messages.
Test this by looking for the 1221 event that the online defrag writes each night (database maintenance window). There will be separate events for the priv and the pub.
If you really do have this much mail and it's growing, the big question is why? Do you have a mail loop? Maybe a rule on an unattended mailbox that's bouncing messages out to another rule externally that's bouncing them back in again? Set IMS logging on and look at the traffic being sent. Ask the users to review their rules.
Whilst you're checking users, get them to confirm that the flag is ticked in Outlook to Delete the Delete Items folder when logging out - if they take this flag off, their mailboxes donÆt ever throw away the contents of the deleted items folder.
Also, what's your Deleted Item Retention set to? If this is a really big number, this will effectively stop anything being deleted. Most people set the DIR to somewhere between 7 and 28 days (if they have it set at all, but it's generally a good idea).
|
|
|
|
|
RE: Private information store too big even after defrag... - 20.Jan.2004 7:29:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
I had also created a support request with Microsoft (the $99 kind) and I've included the response in case it may help others.
The diagnostic dump command is:
ESEUTIL.EXE /ms priv.edb > priv.txt
quote:
I carefully checked the eseutil /ms dump and found some abnormal values:
1-24 Tbl 1775866 808341 3790
<Long Values> LV 1775870 264770 54
Msg Tbl 1020 2756981 15442
<Long Values> LV 1021 939346 4923
There are two abnormal <Long Values>. LetÆs see the objectÆs size:
1775870*4096=7.2GB and 939346*4096=3.8GB. They are very huge.
These objects may orphaned or corrupted long values.
As you know, Long-values (LVs) are created when a column is too large to store with the rest of the record. Internally, the Exchange Server database engine breaks large columns into separate parts; these are the long-values. Long-values are stored in a separate binary tree (B-Tree) and each LV is given a table-wide unique identifier (the long-value ID [LID]).
How the orphaned or corrupted long values occurs:
Orphaned Long-Values
To save space, the Exchange Server database engine provides the ability for multiple records to share the same LV (similar to, but not exactly related to the information store concept of single-instance storage for messages). To do this, a reference count is attached to each LV. When the reference count reaches zero, the LV is deleted. If the Exchange Server database engine is shut down (by a crash, power cut, or blue screen error messages) after dereferencing an LV, but before expunging it from the database, the LV will be ôorphaned,ö that is, its reference count is set to zero, but it is never removed. Orphaned LVs are invisible to anyone using the database because they are logically deleted, but still take up space because they have not been physically removed.
Corrupted Long-Values
As mentioned above, LVs are stored as chunks of data in the long-value tree of a table. Each LV is prefixed by a header (ôLVROOTö) that contains the length and reference count of the LV. In rare cases (such as the B-Tree split problem), the LVROOT of an LV can be overwritten. This corrupts the LV, but doesnÆt actually lose any data. An Exchange Server information store (Store.exe) may stop responding or error out trying to access this LV.
To deal with such a problem, renew database is the best way. We can export userÆs mailbox via exmerge tool and renew Exchange database. After that, we can import userÆs mailbox back.
Some questions and answers about the Exmerge Utility for your reference:
http://support.microsoft.com/default.aspx?scid=kb;en-us;265441
So basically there are two objects in priv.edb and they both add up to 11GB. This looks like where the problem lies. I cannot account for what these objects are so they are probably corrupt. I'm looking at EXMERGE now to see if I can just export those two objects somehow. This would give us the free space and then there would be no rush to run the defrag utility, since we are not running out of hard disk space.
[added]
Oh, and our Deleted Item Retention period was already set to 1 day.
[ January 20, 2004, 07:31 PM: Message edited by: rim ]
|
|
|
|
|
RE: Private information store too big even after defrag... - 21.Jan.2004 9:33:00 AM
|
|
|
zbnet
Posts: 815
Joined: 25.Sep.2003
From: Manchester, UK
Status: offline
|
A very interesting response from MS.
I think you misunderstand their reference to ExMerge. They recommend using ExMerge to remove all the user mailbox deta from the store (to archive it), then to delete and recreate the store, then to put the mailbox data back again (although this will result inloss of historical single instance storage, it will still be better than having 11GB of rubbish in the store!).
You will not be able to remove these orphaned objects with ExMerge, because they are not messages as such - they are internal database constructs that are not exposed via the MAPI interface (which is what ExMerge uses).
|
|
|
|
|
RE: Private information store too big even after defrag... - 21.Jan.2004 5:33:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
Well, I understood what MS support said. I just never ran EXMERGE before so I didn't know what options were available on that utility. But yes, it cannot do what I hoped it could.
Anyway, running it on all our mailboxes yielded .pst files that totalled 1.91GB - a much more realistic number. Looking at exmerge's logs, there was one mailbox that had 4,000 error messages about being unable to copy a message. Fortunately, we did not need it any longer so I deleted it. If I'm lucky, the two objects were in that mailbox and now they're gone? I'm monitoring our priv.edb file to see if it still grows in size.
If that mailbox was not the problem, then I will have to re-create the private IS.
|
|
|
|
|
RE: Private information store too big even after defrag... - 22.Jan.2004 9:13:00 AM
|
|
|
zbnet
Posts: 815
Joined: 25.Sep.2003
From: Manchester, UK
Status: offline
|
Check your overnight event log for 1221 (white space report), this will show whether the 2 objects are gone or not.
|
|
|
|
|
RE: Private information store too big even after defrag... - 22.Jan.2004 8:57:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
I tried looking for that event ID before and have not found any. We have all of our logging set to the default of none, which is basically just critical stuff. What do I need to turn up logging on and to what level?
|
|
|
|
|
RE: Private information store too big even after defrag... - 23.Jan.2004 2:14:00 PM
|
|
|
zbnet
Posts: 815
Joined: 25.Sep.2003
From: Manchester, UK
Status: offline
|
I have my logging set to None by default as well - but this doesn't stop the 1221 events being reported.
Are you sure you have Database Maintanence windows defined? It's on the Server properties in Admin.
|
|
|
|
|
RE: Private information store too big even after defrag... - 28.Jan.2004 12:46:00 PM
|
|
|
Guest
|
why don't you upgrade Exchange 5.5 to exchange 5.5 enterprise edition which has an unlimited IS size? it takes about 5 minutes.
you can buy a 2000 enterprise license to do this proceedure then all you need is the disk which microsoft should be able to dig up (i have one) then just run srvmax.exe and there u go.
|
|
|
|
|
RE: Private information store too big even after defrag... - 2.Feb.2004 10:00:00 PM
|
|
|
rim
Posts: 21
Joined: 15.Jan.2004
From: Alexandria, VA
Status: offline
|
I've just been through hell. ![[Mad]](/image/smiles/mad.gif)
Actually, I think the export and import went fine. The priv.edb got down to under 3 GB. I then went ahead and tried to run our backup software but it kept giving me a Dr. Watson (never did it before). Tried other things and thought the mail server may need a reboot - it never booted up to the OS (blank screen with blinking cursor on upper left - after POST).
Long story short, I ended up building another server as the Exchange Server and restored from tape. Unfortunately, we lost a day's worth of data. Yes, I should've backed up just before any of the export business. ![[Roll Eyes]](/image/smiles/rolleyes.gif) I worked 37 hours straight on it (including the original export/import).
What's worse is, our priv.edb is back to 14.1 GB in size and our Exchange Server is standard edition again. Even though I installed the enterprise edition when I rebuilt the new server, since what I was restoring from was standard edition, it basically made the server standard edition (confirmed by MS support).
MS Support told me all I need to do is re-run Exchange enterprise setup and choose the re-install option and it would upgrade to enterprise. At this point, though, we've just let our users use it as is for now and maybe looking at other options (outsourcing mail being one of them). Fun, fun, fun.
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
Printable Version
![[Confused]](/image/smiles/confused.gif)
![[Smile]](/image/smiles/smile.gif)
Anyway, we are planning on upgrading to Exch2000 but that is still in the planning stage so this is just interim.
![[Eek!]](/image/smiles/eek.gif)
Probably got nothing but complaints from your users? ![[Frown]](/image/smiles/frown.gif)
