Exchange Server Forums
Forums |
Register |
Login |
My Profile |
Inbox |
RSS 
|
My Subscription |
My Forums |
Address Book |
Member List |
Search |
FAQ |
Ticket List |
Log Out
Outlook 2007 / Exchange 2003 slowness ... due to GC access ?
|
Users viewing this topic:
none
|
Logged in as: Guest
|
Login  | |
|
Outlook 2007 / Exchange 2003 slowness ... due to GC acc... - 2.Dec.2008 9:26:39 AM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Hey all...
At my company (~700 employees), we have Outlook 2007 users, accessing a single Exch2003/sp2 server (no front end) ... and frequently, a lot of LAN-based users complain of serious lag/delay when switching between items within Outlook (server-based mailbox, calendar, psts, etc.). Enabling cached mode helps a little, but the problem persists. Users utilize Outlook Anywhere quite a bit, so exact levels of http vs IP access to Exchange is unknown but probably about equal, if not skewed to the http side. Additionally, the Exchange server has its logs and dbs on seperate SAN-based disks, which are 'optimized'. (Raid type, seperate spindles, fiber throughput, etc)
We have 1 domain, with 3 local DCs, 2 of which are GCs. The Exch server auto identifies the 2 local GCs in DSAccess... however, depending on which one is actively serving the GAL, our monitoring software shows the cpu of the GC getting hit very hard. (OS Processor queue length exceeding high thresholds). One word of note: the GC's have 2GB of memory, but the /3GB switch is currently NOT enabled on them.
So my question....
What is causing the extreme Outlook client slowness? Is it due to the (active lookup) GC not being able to handle the load? Should I create another DC/GC, and enable the /3GB switch? Should I manually configure the Exch server's DSAccess to just that GC to test? Should I network-isolate that additional GC so that only the Exchange server can do lookups on it? Or, will having 4 DCs/3 GCs share the load enough such that the clients will no longer see the delays in mail access? Would adding a front-end server help? Should I enable /3GB on all of the GCs?
Just trying to determine what the "problem" is, and what steps will make the biggest speed improvements for the clients....?
Thank you for any help.....
mia
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 2.Dec.2008 12:52:04 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
Exchange is hard on global catalogs, which is why you need them as close to Exchange as possible. However it will only use one, you can have 100s of the the things, but Exchange will use just the one.
The main performance hit with Exchange is usually storage.
However I would start with the Exchange Troubleshooting tool and see if that flags anything. You can download the tool from Microsoft.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 2.Dec.2008 12:59:55 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Thanks Sembee... we have run the Exch Troubleshooting tool, Best Practices Analyzer, etc... and made most of the changes they recommended. Performance still suffers. FYI... network/LAN appears not to be contributing to the issue.
So if Exchange is hard on the GC, and the 2 systems are on the same subnet, would activating /3GB on the GC dramatically help? Would isolating a DC/GC to the Exchange server (so nothing else hits it) help?
Thanks again for any advice here...
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 2.Dec.2008 1:23:09 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
The 3gb switch isn't supported on domain controllers, so doing that isn't going to help.
You have to ask yourself why the clients are hammering away at the domain controllers so hard. I have designed and consulted on larger sites than that and the DCs have been the least of my worries.
Have changes been made to the Exchange server, for example reducing the permissions cache time? That can increase the load on the server significantly.
Are the clients using the other domain controllers at all? If you hold down CTRL and then right click on the Outlook icon you can select Connection Status. That will show you what DC is being used (Directory Access).
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 2.Dec.2008 2:20:13 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Hmm... the document on MSExchange.org--10 Tips to Optimize Exchange 2003 by Rui Silva--specifically states that the /3GB should be enabled on all GC's.... ?
Can you elaborate on the "permissions cache time" ? How this is set, and how it could have been reduced as you are stating? (hotfix, service pack, BES-server-related, etc)
Outlook clients use whichever (DSAccess--GC) server the Exchange server is using. Similar to holding CTRL/right clicking the Outlook icon--within Outlook, you can select the Address Book, right-click on the "Global Address List" pull-down menu and select properties. The server listed there is the one (here at my company) that is generating the "OS Processor Queue Length" thresholds being exceeded.... ie, the DC/GC that our "clients are hammering away at"......
Appreciate the ideas......
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 2.Dec.2008 7:14:04 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
The 3gb switch is a classic arguing point. I don't even think Microsoft are 100% sure. I have seen KB articles state no you can't and others say yes you can. Its a mess.
Of course the easy way round it is to switch the DCs to 64 bit version of Windows - then the problem goes away.
The setting that I have indicated is not changed by any installation that I am aware of. Even third party, because of its performance hit. My Google fu seems to be off at this time of night, as I can only find the Exchange 2007 version of the article, not the Exchange 2003 version.
I have seen Outlook use other domain controllers in the past, so if everything is going to one DC then that is certainly why there is a performance kit. Is one of the domain controllers quicker than the other? Most clients (including Exchange) will use the first DC to respond.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 7:42:33 AM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Simon, yes, I too couldn't locate anything on my searches for that document either.
Doesnt Exchange determine the directory lookup GC to be used based on the DSAccess list, and then pass that to the Outlook clients?
Is there no way to load balance (or round robin) the clients accross the local GCs?
Even at 3:30am this morning, clicking between folders in my Inbox took ~13 seconds to switch between them. I logged onto the DC/GC my Outlook was using for directory lookups, and sure enough, the resources on that GC were getting hit very hard.... yet nothing was running on the server (no backups, scans, tests, replication, etc.)
So, is the slowness simply due to the concentration of Outlook clients hitting that single GC? Even in the middle of the night when no-one is using email...?
thanks
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 1:46:34 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
If you are seeing performance issues in the middle of the night when there are no users on the network, then that would tend to point the problem being elsewhere - not Outlook clients.
Have you tried changing domain controllers?
Does this DC hold any other FSMO roles?
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 2:25:19 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
I hear you, Simon... it just appears that the DC/GC which is 'actively' handling directory lookups --when its resources get taxed heavily-- Ourlook client performance suffers at roughly the same time.... so, just making the guess here as to the connection.... but at 3am, I guess that wouldnt make sense...
We currently have local 3 DCs, 2 of which are GCs.
The DC/GC which is the current Outlook directory/GAL server, also holds these FSMO roles--domain RID, Schema Master and Domain Naming Master.
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 2:40:11 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
I would start looking at whether the DC is the cause of the problem, by moving the FSMO roles and GC to another machine and then either restarting the Exchange services to get it to find another GV or forcing to use another GC.
If it does prove to the be DC then DCPROMO it out, drop in to a workgroup and rebuild it.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 2:50:42 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
We have manually forced Exchange to use both DC2 and then DC3, with the same performance results. Whichever of these 2 GCs we point it at, we still see Outlook performance degredation.
DC1 is primary DNS/WINS/PDC/Infrastructure Master
DC2 is secondary GC/DNS/RID/Schema/Domain Naming Master
DC3 is a GC
I am bringing DC4 online as an additional GC... and I assume we should then spread the FSMO roles across more of the DCs?
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 3:01:15 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
I believe that Microsoft state that one of the FSMO roles should not be on the same domain controller as the global catalog, unless all of the servers are global catalogs. Personally I don't see the point in having more than two GCs unless the FSMO roles are going to be spread out on to non-DCs, which would mean in theory seven DCs would be required.
I have just verified that the Exchange 2007 version of the instructions for the mailbox permissions cache time is valid for Exchange 2000/2003 as well.
http://technet.microsoft.com/en-us/library/bb684892.aspx
Therefore I would be checking that registry key to see if has been changed to 20 seconds or something like that.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 3:08:20 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Thanks... I will check, but my co-sys-admins said this entry was never manually changed, per their knowledge...
20 seconds or 20 minutes, per this document?
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 3:10:47 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
20 minutes. 20 seconds would be an error - an error I have seen done before which brought Exchange to a halt.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 3:16:57 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Ok, so those reg values are not on the Exch server... ? So, not the issue ?
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 3:32:44 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
If those values are not there, then it is the default, which is two hours. So as you say, that rules out that as being the cause.
This may be something that has to go to Microsoft. The behaviour you are seeing is not normal.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 3:45:25 PM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Simon, thanks so much for your assistance/troubleshooting....
Anyone else have any suggestions before I have to pay MS?
thx
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 3.Dec.2008 4:29:38 PM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
I have put this problem to another group (which I can't say where due to NDA).
One of the first responses has been to check for restrictions.
The example used was:
"If I have 700 mailboxes, and a group that contains 500 of them, and then I put that group on the restrictions of all but a couple mailboxes, then every time anyone sends an email to anyone that has the restriction the categorizer expands the group."
That could mean that every email sent is generating a high number of LDAP lookups.
Another thought was to use netmon to look at the traffic, but as it looks like LDAP, that would mean disabling LDAP signing, which may not be something that you can entertain.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 4.Dec.2008 7:59:54 AM
|
|
|
mia450r
Posts: 87
Joined: 28.Jun.2006
Status: offline
|
Thanks Simon, I will look into this.
Another issue we discovered recently, which may have had an impact on performance is we had recently consolidated 2 domains into one, and a large number of the groups were Universal groups, which I believe (in one domain) can cause lookup problems/delays.
I will look into your suggestion on restrictions.....
thanks!
mia
|
|
|
|
|
RE: Outlook 2007 / Exchange 2003 slowness ... due to GC... - 4.Dec.2008 11:46:11 AM
|
|
|
Sembee
Posts: 4093
Joined: 17.Jan.2008
From: Somewhere near London, UK
Status: offline
|
The only time that large groups are an issue is when they are expanded. That shouldn't be happening at 3am, unless the groups are used as restrictions on all of the objects and someone sends a single message.
I think you need to use netmon, see what the traffic is. Once you know that, then you can look for the cause.
Simon.
_____________________________
Simon Butler,
Exchange MVP
Blog: http://blog.sembee.co.uk/
Web: http://www.amset.info/
In the UK? Hire me: http://www.sembee.co.uk/
Exchange Resources: http://exbpa.com/
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
Printable Version
