Hi,
Yes, it was a domain rename gone bad. Basically the main server was reinstalled from scratch but our test server, rather than reinstall everything, was forcefully removed off the old domain name and joined to the new domain name.
Here are the dcdiag results from MAIN:
Domain Controller Diagnosis
Performing initial setup:
Done gathering initial info.
Doing initial required tests
Testing server: Default-First-Site-Name\MAIN
Starting test: Connectivity
......................... MAIN passed test Connectivity
Doing primary tests
Testing server: Default-First-Site-Name\MAIN
Starting test: Replications
......................... MAIN passed test Replications
Starting test: NCSecDesc
......................... MAIN passed test NCSecDesc
Starting test: NetLogons
......................... MAIN passed test NetLogons
Starting test: Advertising
......................... MAIN passed test Advertising
Starting test: KnowsOfRoleHolders
......................... MAIN passed test KnowsOfRoleHolders
Starting test: RidManager
......................... MAIN passed test RidManager
Starting test: MachineAccount
......................... MAIN passed test MachineAccount
Starting test: Services
......................... MAIN passed test Services
Starting test: ObjectsReplicated
......................... MAIN passed test ObjectsReplicated
Starting test: frssysvol
......................... MAIN passed test frssysvol
Starting test: frsevent
There are warning or error events within the last 24 hours after the
SYSVOL has been shared. Failing SYSVOL replication problems may cause
Group Policy problems.
......................... MAIN failed test frsevent
Starting test: kccevent
......................... MAIN passed test kccevent
Starting test: systemlog
......................... MAIN passed test systemlog
Starting test: VerifyReferences
......................... MAIN passed test VerifyReferences
Running partition tests on : ForestDnsZones
Starting test: CrossRefValidation
......................... ForestDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom
Running partition tests on : DomainDnsZones
Starting test: CrossRefValidation
......................... DomainDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom
Running partition tests on : Schema
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom
Running partition tests on : Configuration
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom
Running partition tests on : DOMAIN
Starting test: CrossRefValidation
......................... DOMAIN passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DOMAIN passed test CheckSDRefDom
Running enterprise tests on : DOMAIN.local
Starting test: Intersite
......................... DOMAIN.local passed test Intersite
Starting test: FsmoCheck
......................... DOMAIN.local passed test FsmoCheck
Here are the dcdiag results from TEST:
Domain Controller Diagnosis
Performing initial setup:
Done gathering initial info.
Doing initial required tests
Testing server: Default-First-Site-Name\TEST
Starting test: Connectivity
......................... TEST passed test Connectivity
Doing primary tests
Testing server: Default-First-Site-Name\TEST
Starting test: Replications
[Replications Check,TEST] A recent replication attempt failed:
From MAIN to TEST
Naming Context: DC=ForestDnsZones,DC=DOMAIN,DC=local
The replication generated an error (1256):
The remote system is not available. For information about network troubleshooting, see Windows Help.
The failure occurred at 2006-03-21 09:59:11.
The last success occurred at 2006-03-17 12:45:11.
95 failures have occurred since the last success.
[MAIN] DsBindWithSpnEx() failed with error -2146893022,
The target principal name is incorrect..
[Replications Check,TEST] A recent replication attempt failed:
From MAIN to TEST
Naming Context: DC=DomainDnsZones,DC=DOMAIN,DC=local
The replication generated an error (1256):
The remote system is not available. For information about network troubleshooting, see Windows Help.
The failure occurred at 2006-03-21 09:59:11.
The last success occurred at 2006-03-17 13:28:03.
175 failures have occurred since the last success.
[Replications Check,TEST] A recent replication attempt failed:
From MAIN to TEST
Naming Context: CN=Schema,CN=Configuration,DC=DOMAIN,DC=local
The replication generated an error (-2146893022):
The target principal name is incorrect.
The failure occurred at 2006-03-21 09:59:11.
The last success occurred at 2006-03-17 12:45:11.
94 failures have occurred since the last success.
[Replications Check,TEST] A recent replication attempt failed:
From MAIN to TEST
Naming Context: CN=Configuration,DC=DOMAIN,DC=local
The replication generated an error (-2146893022):
The target principal name is incorrect.
The failure occurred at 2006-03-21 09:59:11.
The last success occurred at 2006-03-17 12:50:47.
218 failures have occurred since the last success.
[Replications Check,TEST] A recent replication attempt failed:
From MAIN to TEST
Naming Context: DC=DOMAIN,DC=local
The replication generated an error (-2146893022):
The target principal name is incorrect.
The failure occurred at 2006-03-21 10:00:49.
The last success occurred at 2006-03-17 13:28:32.
1036 failures have occurred since the last success.
REPLICATION-RECEIVED LATENCY WARNING
TEST: Current time is 2006-03-21 10:01:37.
DC=ForestDnsZones,DC=DOMAIN,DC=local
Last replication recieved from MAIN at 2006-03-17 12:45:13.
DC=DomainDnsZones,DC=DOMAIN,DC=local
Last replication recieved from MAIN at 2006-03-17 13:28:05.
CN=Schema,CN=Configuration,DC=DOMAIN,DC=local
Last replication recieved from MAIN at 2006-03-17 12:45:13.
CN=Configuration,DC=DOMAIN,DC=local
Last replication recieved from MAIN at 2006-03-17 12:50:48.
DC=DOMAIN,DC=local
Last replication recieved from MAIN at 2006-03-17 13:28:33.
......................... TEST passed test Replications
Starting test: NCSecDesc
......................... TEST passed test NCSecDesc
Starting test: NetLogons
......................... TEST passed test NetLogons
Starting test: Advertising
Warning: TEST is not advertising as a time server.
......................... TEST failed test Advertising
Starting test: KnowsOfRoleHolders
Warning: MAIN is the Schema Owner, but is not responding to DS RPC Bind.
[MAIN] LDAP bind failed with error 8341,
A directory service error has occurred..
Warning: MAIN is the Schema Owner, but is not responding to LDAP Bind.
Warning: MAIN is the Domain Owner, but is not responding to DS RPC Bind.
Warning: MAIN is the Domain Owner, but is not responding to LDAP Bind.
Warning: MAIN is the PDC Owner, but is not responding to DS RPC Bind.
Warning: MAIN is the PDC Owner, but is not responding to LDAP Bind.
Warning: MAIN is the Rid Owner, but is not responding to DS RPC Bind.
Warning: MAIN is the Rid Owner, but is not responding to LDAP Bind.
Warning: MAIN is the Infrastructure Update Owner, but is not responding to DS RPC Bind.
Warning: MAIN is the Infrastructure Update Owner, but is not responding to LDAP Bind.
......................... TEST failed test KnowsOfRoleHolders
Starting test: RidManager
......................... TEST failed test RidManager
Starting test: MachineAccount
......................... TEST passed test MachineAccount
Starting test: Services
......................... TEST passed test Services
Starting test: ObjectsReplicated
......................... TEST passed test ObjectsReplicated
Starting test: frssysvol
......................... TEST passed test frssysvol
Starting test: frsevent
There are warning or error events within the last 24 hours after the
SYSVOL has been shared. Failing SYSVOL replication problems may cause
Group Policy problems.
......................... TEST failed test frsevent
Starting test: kccevent
An Warning Event occured. EventID: 0x8000051C
Time Generated: 03/21/2006 09:49:11
Event String: The Knowledge Consistency Checker (KCC) has
An Warning Event occured. EventID: 0x80000632
Time Generated: 03/21/2006 09:59:10
(Event String could not be retrieved)
......................... TEST failed test kccevent
Starting test: systemlog
An Error Event occured. EventID: 0x00000C8A
Time Generated: 03/21/2006 09:08:20
Event String: This computer could not authenticate with
An Error Event occured. EventID: 0x00000423
Time Generated: 03/21/2006 09:45:40
Event String: The DHCP service failed to see a directory server
An Error Event occured. EventID: 0xC0000021
Time Generated: 03/21/2006 09:45:53
(Event String could not be retrieved)
An Error Event occured. EventID: 0xC0000021
Time Generated: 03/21/2006 09:45:53
(Event String could not be retrieved)
An Error Event occured. EventID: 0xC0000021
Time Generated: 03/21/2006 09:45:53
(Event String could not be retrieved)
An Error Event occured. EventID: 0xC0000021
Time Generated: 03/21/2006 09:45:53
(Event String could not be retrieved)
An Error Event occured. EventID: 0xC0000021
Time Generated: 03/21/2006 09:45:53
(Event String could not be retrieved)
An Error Event occured. EventID: 0x00000423
Time Generated: 03/21/2006 09:46:08
Event String: The DHCP service failed to see a directory server
An Error Event occured. EventID: 0x40000004
Time Generated: 03/21/2006 09:50:04
Event String: The kerberos client received a
An Error Event occured. EventID: 0x40000004
Time Generated: 03/21/2006 09:53:17
Event String: The kerberos client received a
An Error Event occured. EventID: 0x40000004
Time Generated: 03/21/2006 09:55:27
Event String: The kerberos client received a
An Error Event occured. EventID: 0x40000004
Time Generated: 03/21/2006 10:01:37
Event String: The kerberos client received a
An Error Event occured. EventID: 0x40000004
Time Generated: 03/21/2006 10:01:37
Event String: The kerberos client received a
......................... TEST failed test systemlog
Starting test: VerifyReferences
......................... TEST passed test VerifyReferences
Running partition tests on : ForestDnsZones
Starting test: CrossRefValidation
......................... ForestDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom
Running partition tests on : DomainDnsZones
Starting test: CrossRefValidation
......................... DomainDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom
Running partition tests on : Schema
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom
Running partition tests on : Configuration
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom
Running partition tests on : DOMAIN
Starting test: CrossRefValidation
......................... DOMAIN passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DOMAIN passed test CheckSDRefDom
Running enterprise tests on : DOMAIN.local
Starting test: Intersite
......................... DOMAIN.local passed test Intersite
Starting test: FsmoCheck
......................... DOMAIN.local passed test FsmoCheck