• RSS
  • Twitter
  • FaceBook

Exchange Server Forums

Forums | Register | Login | My Profile | Inbox | RSS RSS icon | My Subscription | My Forums | Address Book | Member List | Search | FAQ | Ticket List | Log Out

Recovery installation problem

Users viewing this topic: none

Logged in as: Guest
  Printable Version
All Forums >> [Microsoft Exchange 2010] >> Installation >> Recovery installation problem Page: [1]
Login
Message << Older Topic   Newer Topic >>
Recovery installation problem - 25.Oct.2010 4:34:34 AM   
tnx

 

Posts: 5
Joined: 24.Oct.2010
Status: offline
Hello.
I got stuck in prerequisites for recovery installation. I hope someone more experienced could give me a hint what to do.

Let me explain some background please.
I have my own training server where I learn about Exchange 2010. To increase my own motivation I have placed my wife’s mailbox there. So even though it is a training server and I am doing nasty things to it, I am highly motivated to make it work again every time. Last thing I want is to listen to my wife criticizing my IT skills. ;-)

I am using VMware ESXi box with four servers: DC01, DC02, EX01 and EX02. All are running Windows 2008 R2. EX01 and EX02 have Enterprise edition of Windows server and Standard edition of Exchange 2010. I have installed EX01 first with CAS, HUB, MBX roles. I have installed EX02 later also with CAS, HUB, MBX roles. I have formed DAG a created a copy of mailbox database. Everything was working fine. Then I have powered off the EX01 server to simulate server crash. Then I started to correct the damage. Everything works fine again on the EX02. Now I would like to do a recovery install on the EX01. I have installed a clean OS, installed prerequisites. Now I turned back to EX02 and started the prerequisites to recover DAG member.
I have run:
[PS] Remove-MailboxDatabaseCopy mdb01\ex01
Confirm
Are you sure you want to perform this action?
Removing database copy for database "mdb01" on server "ex01".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): y
WARNING: Couldn't communicate with the Microsoft Exchange Replication service on server "EX01.tnx.cz" to pick up new configuration changes for database "mdb01". Make sure that the service is running and that the server has network connectivity. Error: A server-side administrative operation has failed. The Microsoft Exchange Replication service may not be running on server EX01.tnx.cz. Specific RPC error message: Error 0x6d9 (There are no more endpoints available from the endpoint mapper) from cli_RpcsNotifyChangedReplayConfiguration
WARNING: Couldn't delete the replication state for database mdb01 on server ex01. Error: Requested registry access is not allowed.
WARNING: The copy of mailbox database "mdb01" on server "ex01" has been removed. If necessary, manually delete the database copy's files located at "d:\exchange\md01\logs" and "d:\exchange\md01\db\mdb01.edb" on that server.
WARNING: The search catalog could not be dismounted for the database 'mdb01' on server 'ex01'. The files may be locked while the Microsoft Exchange Search service is still running.
WARNING: The requested data move replication constraint, SecondCopy, isn't valid for mailbox database mdb01 because it either has only one copy or has copies in only one datacenter. The constraint has been set to None instead.
 
I think that this was expected.
 
But when running second command…:
[PS] Remove-DatabaseAvailabilityGroupServer -Identity dag01 -MailboxServer ex01
Confirm
Are you sure you want to perform this action?
Removing Mailbox server "ex01" from database availability group "dag01".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): y
WARNING: The operation wasn't successful because an error was encountered. You may find more details in log file
"C:\ExchangeSetupLogs\DagTasks\dagtask_2010-10-25_08-00-06.151_remove-databaseavailabiltygroupserver.log".
There was a problem changing the quorum model for database availability group dag01. Error: An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"SetClusterQuorumResource() failed with 0x1725. Error: A quorum of cluster nodes was not present to form a cluster"' failed.
    + CategoryInfo          : InvalidArgument: (:) [Remove-DatabaseAvailabilityGroupServer], DagTaskProblemChangingQuorumException
    + FullyQualifiedErrorId : 722AAE81,Microsoft.Exchange.Management.SystemConfigurationTasks.RemoveDatabaseAvailabilityGroupServer
 
I am not sure whether to send the whole log. I hope it is not a problem:
 
remove-databaseavailabiltygroupserver started on machine EX02.
[2010-10-25T08:00:06] remove-dagserver started
[2010-10-25T08:00:06] commandline:         $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2010-10-25T08:00:06] Option 'Identity' = 'dag01'.
[2010-10-25T08:00:06] Option 'MailboxServer' = 'ex01'.
[2010-10-25T08:00:06] Option 'ConfigurationOnly' = ''.
[2010-10-25T08:00:06] Option 'WhatIf' = ''.
[2010-10-25T08:00:06] Process: w3wp w3wp.exe:7032.
[2010-10-25T08:00:06] User context = 'NT AUTHORITY\SYSTEM'.
[2010-10-25T08:00:06]   Member of group 'Everyone'.
[2010-10-25T08:00:06]   Member of group 'BUILTIN\Users'.
[2010-10-25T08:00:06]   Member of group 'NT AUTHORITY\SERVICE'.
[2010-10-25T08:00:06]   Member of group 'CONSOLE LOGON'.
[2010-10-25T08:00:06]   Member of group 'NT AUTHORITY\Authenticated Users'.
[2010-10-25T08:00:06]   Member of group 'NT AUTHORITY\This Organization'.
[2010-10-25T08:00:06]   Member of group 'BUILTIN\IIS_IUSRS'.
[2010-10-25T08:00:06]   Member of group 'LOCAL'.
[2010-10-25T08:00:06]   Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2010-10-25T08:00:06]   Member of group 'BUILTIN\Administrators'.
[2010-10-25T08:00:06] Updated Progress 'Validating the parameters.' 2%.
[2010-10-25T08:00:06] Working
[2010-10-25T08:00:06] Looking up IP addresses for dag01.
[2010-10-25T08:00:06]   dag01 = [ 192.168.2.45 ].
[2010-10-25T08:00:06] Looking up IP addresses for EX01.tnx.cz.
[2010-10-25T08:00:06]   EX01.tnx.cz = [ 192.168.2.42 ].
[2010-10-25T08:00:06] Updated Progress 'Validating removal of mailbox server 'ex01' from 'dag01'.' 0%.
[2010-10-25T08:00:06] Working
[2010-10-25T08:00:06] CheckClusterStateForDagServerRemoval entered. m_removeNode=False, m_destroyCluster=False
[2010-10-25T08:00:06] Dumping the cluster by connecting to: <existing cluster handle>.
[2010-10-25T08:00:06] The cluster's name is: dag01.
[2010-10-25T08:00:06] Groups
[2010-10-25T08:00:06]     group: Cluster Group [Cluster Main Group]
[2010-10-25T08:00:06]         OwnerNode: ex02.tnx.cz
[2010-10-25T08:00:06]         State: Online
[2010-10-25T08:00:06]             Resource: File Share Witness (\\DC02.TNX.CZ\dag01.TNX.CZ) [Online, type = File Share Witness, PossibleOwners = EX01,EX02 ]
[2010-10-25T08:00:06]             Resource: Cluster Name [Online, type = Network Name, PossibleOwners = EX01,EX02 ]
[2010-10-25T08:00:06]                 NetName = [DAG01]
[2010-10-25T08:00:06]             Resource: IPv4 Static Address 1 (Cluster Group) [Online, type = IP Address, PossibleOwners = EX01,EX02 ]
[2010-10-25T08:00:06]                 Address = [192.168.2.45]
[2010-10-25T08:00:06]                     EnableDhcp = [0]
[2010-10-25T08:00:06]                     Network = [Cluster Network 1]
[2010-10-25T08:00:06]     group: Available Storage [not a CMS]
[2010-10-25T08:00:06]         OwnerNode: ex02.tnx.cz
[2010-10-25T08:00:06]         State: Offline
[2010-10-25T08:00:06] Nodes
[2010-10-25T08:00:06]     node: EX01.tnx.cz [ state = Down ]
[2010-10-25T08:00:06]     node: ex02.tnx.cz [ state = Up ]
[2010-10-25T08:00:06] Subnets
[2010-10-25T08:00:06]     Name(Cluster Network 1), Mask(192.168.2.0/24), Role(ClusterNetworkRoleInternalAndClient)
[2010-10-25T08:00:06]         NIC 192.168.2.42 on Node ex01 in State=Unavailable
[2010-10-25T08:00:06]         NIC 192.168.2.44 on Node EX02 in State=Up
[2010-10-25T08:00:06]     Name(Cluster Network 2), Mask(10.0.0.0/24), Role(ClusterNetworkRoleInternalUse)
[2010-10-25T08:00:06]         NIC 10.0.0.1 on Node ex01 in State=Unavailable
[2010-10-25T08:00:06]         NIC 10.0.0.2 on Node EX02 in State=Up
[2010-10-25T08:00:06] Trying to open the node on the cluster.
[2010-10-25T08:00:06] Node.GetState( EX01.tnx.cz ) reports that it is Down.
[2010-10-25T08:00:06] Successfully resolved servers based on the AD configuration.
[2010-10-25T08:00:06] The following servers are in the StartedServers list:
[2010-10-25T08:00:06] The following servers are in the StoppedServers list:
[2010-10-25T08:00:06] Verifiying that the members of database availability group 'dag01' are also members of the cluster.
[2010-10-25T08:00:06] Verifying that the members of cluster 'dag01' are also members of the database availability group.
[2010-10-25T08:00:06] There are 2 nodes in the cluster.
[2010-10-25T08:00:06] DAG dag01 has 2 servers:
[2010-10-25T08:00:07] DAG dag01 contains server ex02.
[2010-10-25T08:00:07] DAG dag01 contains server ex01.
[2010-10-25T08:00:07] Reopening a handle to the cluster using the names [ex02].
[2010-10-25T08:00:07] CheckClusterStateForDagServerRemoval left. m_removeNode=True, m_destroyCluster=False.
[2010-10-25T08:00:07] Checking if msexchangerepl is running on ex02.
[2010-10-25T08:00:07] The CNO is currently Online.
[2010-10-25T08:00:11] Updated Progress 'Removing server 'ex01' from 'dag01'.' 2%.
[2010-10-25T08:00:11] Working
[2010-10-25T08:00:11] The core cluster group 'Cluster Group' is currently on machine 'ex02'.
[2010-10-25T08:00:11] Updated Progress 'No longer using file share witness share because there are an odd number of members in the database availability group.' 4%.
[2010-10-25T08:00:11] Working
[2010-10-25T08:00:11] Setting cluster quorum resource to the netname resource (i.e. MNS quorum).
[2010-10-25T08:00:11] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2010-10-25_08-00-06.151_remove-databaseavailabiltygroupserver.log".
[2010-10-25T08:00:11] WriteError! Exception = Microsoft.Exchange.Management.Tasks.DagTaskProblemChangingQuorumException: There was a problem changing the quorum model for database availability group dag01. Error: An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"SetClusterQuorumResource() failed with 0x1725. Error: A quorum of cluster nodes was not present to form a cluster"' failed.
[2010-10-25T08:00:11] Updated Progress 'Done!' 100%.
[2010-10-25T08:00:11] COMPLETED
remove-databaseavailabiltygroupserver explicitly called CloseTempLogFile().


I have done some googling, but with no luck. The reason might be that I am not entirely sure what I am looking for. Any hint is highly appreciated. If any other information required, I will post it here.

Thank you
Jan Kovar
Post #: 1
RE: Recovery installation problem - 25.Oct.2010 5:13:06 AM   
Jesper Bernle

 

Posts: 912
Joined: 15.Oct.2007
Status: offline
The steps to recover a failed Exchange 2010 server is clearly stated here: Recover an Exchange Server

Give my best to your wife!

(in reply to tnx)
Post #: 2
RE: Recovery installation problem - 25.Oct.2010 6:01:10 AM   
tnx

 

Posts: 5
Joined: 24.Oct.2010
Status: offline
Hello.
I have seen that. But you can see there a link for recovering DAG member server.
http://technet.microsoft.com/en-us/library/dd638206.aspx
There are more prerequisites in that case. And get stuck at one of those steps.
Best regards
Jan Kovar

(in reply to Jesper Bernle)
Post #: 3
RE: Recovery installation problem - 25.Oct.2010 6:27:01 AM   
Jesper Bernle

 

Posts: 912
Joined: 15.Oct.2007
Status: offline
Is your DAG online?

Try running Get-DatabaseAvailabilityGroup cmdlet and check what ControllingActiveManager is set to.

Get-DatabaseAvailabilityGroup <Your DAG Name> -Status | fl

(in reply to tnx)
Post #: 4
RE: Recovery installation problem - 25.Oct.2010 6:38:24 AM   
tnx

 

Posts: 5
Joined: 24.Oct.2010
Status: offline
Hello.

It is ex02. The whole result follows:

[PS] C:\Windows\system32>Get-DatabaseAvailabilityGroup dag01 -status | fl


RunspaceId                             : f02ddc35-b545-4949-86d6-34b382747edc
Name                                   : dag01
Servers                                : {ex02, ex01}
WitnessServer                          : DC02.TNX.CZ
WitnessDirectory                       : c:\witness
AlternateWitnessServer                 :
AlternateWitnessDirectory              :
NetworkCompression                     : InterSubnetOnly
NetworkEncryption                      : InterSubnetOnly
DatacenterActivationMode               : Off
StoppedMailboxServers                  : {}
StartedMailboxServers                  : {}
DatabaseAvailabilityGroupIpv4Addresses : {192.168.2.45}
OperationalServers                     : {ex02}
PrimaryActiveManager                   : ex02
ThirdPartyReplication                  : Disabled
ReplicationPort                        : 64327
NetworkNames                           : {DAGNetwork01, DAGNetwork02}
AdminDisplayName                       :
ExchangeVersion                        : 0.10 (14.0.100.0)
DistinguishedName                      : CN=dag01,CN=Database Availability Groups,CN=Exchange Administrative Group (FYD
                                        IBOHF23SPDLT),CN=Administrative Groups,CN=TNX HOME,CN=Microsoft Exchange,CN=Se
                                        rvices,CN=Configuration,DC=tnx,DC=cz
Identity                               : dag01
Guid                                   : d2bc2717-2e56-468a-906f-91caabc39a7e
ObjectCategory                         : tnx.cz/Configuration/Schema/ms-Exch-MDB-Availability-Group
ObjectClass                            : {top, msExchMDBAvailabilityGroup}
WhenChanged                            : 10/24/2010 8:52:45 PM
WhenCreated                            : 8/4/2010 1:01:37 PM
WhenChangedUTC                         : 10/24/2010 6:52:45 PM
WhenCreatedUTC                         : 8/4/2010 11:01:37 AM
OrganizationId                         :
OriginatingServer                      : DC01.tnx.cz
IsValid                                : True


Just for sure. I know that witness should not be on the DC, but I cannot affort have separate HUB transport server. Not enough RAM on vmware host. I have added exchange trusted subsystem to domain admins to allow witness on DC before I was creating DAG.
BR
Jan Kovar

(in reply to Jesper Bernle)
Post #: 5
RE: Recovery installation problem - 25.Oct.2010 6:47:29 AM   
Jesper Bernle

 

Posts: 912
Joined: 15.Oct.2007
Status: offline
Cool! Your DAG is online. Please try the -ConfigurationOnly switch to remove EX1 from the DAG.

(in reply to tnx)
Post #: 6
RE: Recovery installation problem - 25.Oct.2010 9:00:39 AM   
tnx

 

Posts: 5
Joined: 24.Oct.2010
Status: offline
Hello.

Thank you for advice. -configurationonly parameter helped to remove mailbox server from dag. I was able to perform recovery installation.
Right now I got stuck when trying to add server back to the dag. Server ex01 is the recovered server. When I try to run the commandlet on the ex01, this happens:
[PS] C:\Windows\system32>Add-DatabaseAvailabilityGroupServer -Identity dag01 -MailboxServer ex01
WARNING: The operation wasn't successful because an error was encountered. You may find more details in log file
"C:\ExchangeSetupLogs\DagTasks\dagtask_2010-10-25_12-25-07.138_add-databaseavailabiltygroupserver.log".
The following servers in the Windows Failover Cluster are not in Active Directory: ex01. This is usually the result of an incomplete membership change (add or remove) of the database availabilty group.
   + CategoryInfo          : InvalidArgument: (:) [Add-DatabaseAvailabilityGroupServer], DagTaskServersInClusterNotIn Ad
   + FullyQualifiedErrorId : C2FEE895,Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityG
  roupServer


When I try it on ex02:
[PS] C:\Windows\system32>Add-DatabaseAvailabilityGroupServer -Identity dag01 -MailboxServer ex01
WARNING: The operation wasn't successful because an error was encountered. You may find more details in log file
"C:\ExchangeSetupLogs\DagTasks\dagtask_2010-10-25_12-40-32.255_add-databaseavailabiltygroupserver.log".
A server-side database availability group administrative operation failed. Error: The operation failed with message: An error occurred while attempting a cluster operation. Error: Node ex01 is already joined to a cluster. [Server: ex02.tnx.cz]
   + CategoryInfo          : InvalidArgument: (:) [Add-DatabaseAvailabilityGroupServer], DagTaskOperationFailedException
   + FullyQualifiedErrorId : 21B40158,Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityG   roupServer


I have found some information on the internet that this happens when the parameter -configurationonly is used when removing dag member. The presented solution was to evic cluster member ex01 in Failover Cluster Manager. But there was no confirmation how it ended. I am not very good in cluster technologies. Is this a good idea?
Thanks
Jan Kovar

(in reply to Jesper Bernle)
Post #: 7
RE: Recovery installation problem - 25.Oct.2010 9:09:04 AM   
Jesper Bernle

 

Posts: 912
Joined: 15.Oct.2007
Status: offline
Eveicting a node from an Exchange cluster isn't all that uncommon.

(in reply to tnx)
Post #: 8
RE: Recovery installation problem - 25.Oct.2010 10:19:12 AM   
tnx

 

Posts: 5
Joined: 24.Oct.2010
Status: offline
Hello.
Thank you very much. Everything works now.
Jan Kovar

(in reply to Jesper Bernle)
Post #: 9
RE: Recovery installation problem - 26.Oct.2010 4:21:06 AM   
Jesper Bernle

 

Posts: 912
Joined: 15.Oct.2007
Status: offline
Great!

(in reply to tnx)
Post #: 10
RE: Recovery installation problem - 28.Oct.2010 2:41:20 AM   
michel700

 

Posts: 1
Joined: 28.Oct.2010
Status: offline
I face this problem number of time that I could download a file on my computer but can not over write or delete and not even copy ??

< Message edited by michel700 -- 28.Oct.2010 2:46:12 AM >


_____________________________

Global Warming Answers

(in reply to Jesper Bernle)
Post #: 11

Page:   [1] << Older Topic    Newer Topic >>
All Forums >> [Microsoft Exchange 2010] >> Installation >> Recovery installation problem Page: [1]
Jump to:

New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts


Follow TechGenix on Twitter