RSS Search

 News  Feeds  Tags  Search Shortcuts

FeedsFarm.com > OAB PDN Changes and Exchange Site Consolidations

OAB PDN Changes and Exchange Site Consolidations

17th Feb 2006, 11:42 GMT

Recently I have been involved with debugging a lot of OAB issues. One of the most interesting issues has been the PDN table change. With Exchange 5.5 coming to the end of it's life cycle, lots of companies have been migrating quickly from Exchange 5.5 to Exchange 2000 or Exchange 2003. One major component that is affected by mixed mode site migrations/consolidation is the OAB. So with that being said I wanted to take a few minutes to share my knowledge with everyone around troubleshooting OAB PDN changes. What is a PDN table? A PDN table is a string table that is maintained by OABGen of PDN’s (Parent Distinguished Names). When the OAB Generation process is generating an address list, it will split the ExchangeLegacyDN and x500 proxy addresses that belong to an object in to two separate parts. PDN – Parent Distinguished Name - /o=organization/ou=site/cn=container/ RDN – Relative Distinguished Name - /cn=dgoldman This PDN table is used as a reference for the client when it builds MAPI entry ids of mail recipients. Since most recipients in the OAB share the same small set of PDNs, a table was used to save space instead of storing them separately for each recipient. Prior to Exchange 2003 SP2, the OAB Generation process had problems dealing with the addition and removal of PDN’s. Once a PDN table change is detected, it will cause the OAB generation process to skip one day’s differential build..OAB Version 2 and OAB Version 3a clients are affected by this and the ramifications are a full download for the client. With the addition of Exchange 2003 SP2, this is no longer a problem for OAB Version 4 clients. Event ID’s that are displayed in the application log when we detect a PDN table change: 1. When generating an address list you might get a combination of the following Events ID’s: 9340's, 9341’s, 9342’s and (9360’s - discussed later on in this document). Event ID: 9340 Category: OAL Generator Source: MSExchangeSA Type: Warning Generated: 11/2/2005 10:06:25 AM Machine: OABGEN-SRV Description: A new parent Legacy Exchange DN container value '/o=organization/ou=site/cn=container/cn=container' was found during generation of the differential update file for offline address list '\Global Address List'. This will force clients using this offline address list to do a full download of the offline address list. - Default Offline Address List Event ID: 9341 Category : OAL Generator Source: MSExchangeSA Type: Warning Generated: 11/2/2005 10:06:25 AM Machine: OABGEN-SRV Description: The parent Legacy Exchange DN container value '/o=organization/ou=site/cn=container/cn=container' was not found during generation of the differential update file for offline address list '\Global Address List'. This will force clients using this offline address list to do a full download of the offline address list. - Default Offline Address List Event ID: 9342 Category: OAL Generator Source: MSExchangeSA Type: Warning (No Previous Diff found) Generated: 11/2/2005 10:06:25 AM Machine: OABGEN-SRV Description: No previous version of an offline address list for '\Global Address List' can be found. No differential update file will be produced. This is expected if this is the first time this offline address list has been generated. - Default Offline Address List Event ID: 9360 Category: OAL Generator Source: MSExchangeSA Type: Error Generated: 11/2/2005 10:06:25 AM Machine: OABGEN-SRV Description: OALGen encountered an error while generating the changes.oab file for version 2 and 3 differential downloads of address list '\Global Address List'. The offline address list has not been updated so clients will not be able to download the current set of changes. Check other logged events to find the cause of this error. The causes for PDN table changes: 1. If the PDN table changes in the OAB by either a new PDN or removal of a PDN then all Outlook cached mode clients using V2 of V3 OAB will also attempt a full download. The PDN table is the set of all PDNs (Parent Distinguished Names) found in the directory. 2. The admin can cause a change in the PDN table in these ways: Manually modifying a legacyExchangeDN in the AD to create a PDN that did not exist before. This most often is done by accident if someone is editing this value and mis-types the value, thus creating a new PDN. 3. With Exchange 5.5 and ADC in place, creating a new container in 5.5 and inserting an object into it, or deleting the last object in a 5.5 container. 4. With Exchange 5.5 and ADC in place, ADC set to replicate the container hierarchy to 5.5, and the administrator creates new mail-enabled objects in a new AD container. The ADC will create the new container in 5.5 and back-replicate the new 5.5 distinguished name as the legacyDN of the AD object creating a new PDN. 5. Add an administrative group. The first mailbox created on a server in this AG will cause a new PDN to show up in the directory. 6. Deleting the last object with a particular PDN in its legacyExchangeDN or proxyAddresses. Example: A few years after consolidating and deleting a site, the last mailbox formerly in that site is finally deleted. The x500 placeholder is gone and reduces the size of the PDN table. 7. Adding/removing/modifying an X500 proxy address with new PDN. This can be done using ADU&C. If the X500 address is in the local org, but the ou and containers are new or mistyped, a PDN will be added or deleted from the table. The reason why this is such a problem is because the pdndex.oab file can not be re-indexed after it has been created on the client side. This can only happen on a full download. To stop Outlook clients from being affected by this, both the client and server must be running SP2. If the client is on SP2 it should automatically update to the version 4 providing that the client is already using a Unicode profile and that they actually connected to the server where the v4 is. New default behavior with Exchange 2003 SP2: Now by default in Exchange 2003 SP2 we’ve changed what OAB Version 2 and OAB Version 3 does when it detects a PDN change (addition or removal). The old behavior (prior to Exchange 2003 SP2) was that we would not generate a differences file, but will still create a full OAB post. Once Outlook clients try to download the OAB diff files, it will notice that there is no diff file and the clients will be forced to do a full download. This can generate a lot of heavy network traffic and cause other issues. The new behavior (with SP2) is to generate neither a diff nor full OAB post. We also have the ability to now add a registry key on the Exchange server to force Exchange to post a full OAB message when a diff failure has occurred to revert to the old SP1 behavior. Example - Exchange 2003 SP1 1. OABGen rebuilds an address list. 2. OABGen connect to the active directory for it's data. 3. Compare the data that was read from the active directory and compared to the post in the public folder information store. 4. A PDN change was detected (an addition or removal of a PDN). 5. Diff generation FAILS, but OABGen posts the data that we have. (This contains the changes). 6. Clients will perform a full download, but the OAB data is current. Example - Exchange 2003 SP2 without the "OAL post full if diff fails" registry key in place. 1. OABGen rebuilds an address list. 2. OABGen connect to the active directory for it's data. 3. Compare the data that was read from the active directory and compared to the post in the public folder information store. 4. A PDN change was detected (an addition or removal of a PDN). 5. Diff generation FAILS, NO OABGen post is made to the public folder store. (The data is discarded and the next time we generate we will compare the same active directory data against the OAB post in the store. This WILL results in the same errors. 6. Clients will perform a full download, but the OAB data is not current. Example - Exchange 2003 SP2 with the "OAL post full if diff fails" registry key in place. 1. OABGen rebuilds an address list. 2. OABGen connect to the active directory for it's data. 3. Compare the data that was read from the active directory and compared to the post in the public folder information store. 4. A PDN change was detected (an addition or removal of a PDN). 5. Diff generation fails, but OABGen is forced to post the data that we have. (This contains the changes). 6. Clients will perform a full download, but the OAB data is current. To enable this functionality You must set the “OAL post full if diff fails” registry value. To do this, follow these steps: Click Start, click Run, type regedit, and then click OK. Locate and then click to select the following registry key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeSA\Parameters On the Edit menu, point to New, and then click DWORD Value. Type OAL post full if diff fails, and then press ENTER. Right-click OAL post full if diff fails, and then click Modify. In the Value data box, type 0x1 (1). Quit Registry Editor. If this registry key is in added an event will be logged once a diff generation fails: Event Type: Warning Event Source: MSExchangeSA Event Category: OAL Generator EventID: 9116 Generated: 11/2/2005 10:06:25 AM Machine: OABGEN-SRV OALGen encountered an error while generating the changes.oab file for differential downloads of address list ‘\Glboal Address List’. Clients will not be able to incrementally update to the new version of the offline address list, they will perform a full download instead. This is normal if this is the first time this offline address list has been generated. Check other logged events to see if this is a serious error. If the registry key does not exist or is set to zero, a new event is logged if a diff generation fails: Event Type: Error Event Source: MSExchangeSA Event Category: OAL Generator Event ID: 9360 Date: 8/15/2005 5:06:52 AM User: N/A Computer: OABGEN-SRV Description: OALGen encountered an error while generating the changes.oab file for version 2 and 3 differential downloads of address list '\Global Address List'. The offline address list has not been updated so clients will not be able to download the current set of changes. Check other logged events to find the cause of this error. How to troubleshoot and Isolation PDN table changes: 1. Check for active directory related issues. If you have made changes and the OAB generation is diff is still failing, it is probably due to the fact that the OAB generation is reading from a domain controller that does not contain the object changes. You can do this by running the following tools: Netdiag with the /v switch Dcdiag with the /v switch Netmon ExBPA - (Link provided at the bottom of this document) OABInteg - (Link provided at the bottom of this document) NOTE: You can run OABInteg to get this information and or take a Netmon trace during OAB generation process to see where the NSPI calls are going. Then use ADSIEdit to check that domain controller first and compare the results to the other domain controllers in the site. 2. If you have Exchange 2003 SP2 installed you can start the generation process and look for the following Event ID – 9117 Event Type: Information Event Source: MSExchangeSA Event Category: OAL Generator Event ID: 9117 Date: 10/26/2005 12:21:06 PM Computer: OABGEN-SRV Description: OALGen successfully opened a connection to Active Directory which will supply the current Address Lists. - Default Offline Address List 3. Do not change OAB generation servers. *The problem may continue to exist*. - Active Directory replication latencies or lack of replication can contribute to this problem and usually do. This is common because when you generate the OAB it reads all the data in a linear fashion from the active directory. You will find the same data if you are connecting to a domain controller that has not yet received the new data. You also have to take in to account different service pack levels on servers. This can lead to different behavior when generating an offline address list. 4. Once the objects have been located try to confirm what changes have been made. This should help you identify what needs to be done to correct it. You will only find objects that contain invalid x500 addresses and invalid legacyExchangeDN’s. 5. If you make any corrections and or changes, you need to make sure that you replicate all of your active directory domain controllers. This will ensure that all domain controllers contain the up to date changes. Site Consolidations and PDN table changes: If you are in a mixed mode environment and you are cross-site migrating users manually, you will want to create a mail enabled object, add X500 addresses to this placeholder object for all the sites that you plan to migrate out of. This has to be done so when the objects are moved we do not cause a PDN change. NOTE 1: Forgetting to add this x500 proxy address to the objects can cause your diff files to failing during the generation process. Otherwise, the diff generation will fail and all OAB Version 2 and OAB Version 3a clients will be making a trip to the server to get the full oab. This traffic will flood your network and this is not a good thing. NOTE 2: If you have Exchange 2003 SP1 you can use the new mixed-mode cross-site move mailbox functionality. Mail enabled objects that have been moved cross-site are associated with the distinguished name of the object that existed before the move. This association is will be maintained because Exchange 2003 SP1 applies an X.500 proxy address on the mailbox or distribution list that references the original mailbox or distribution list object in the source site. NOTE 3: When running in Exchange 2000 native mode, legacyDN’s don’t change when users are moved between admin groups. NOTE 4: PDN changes are not based on the container being removed; it is based on the last recipient being removed. During the OAB generation process we will include x500 addresses to the oab for recipients that belong to the same org (/O=). How to fix the PDN table changes: NOTE 1: You might see different behavior when generating an offline address list on an Exchange 2003 Service Pack 1 and Service Pack 2. Please refer to the section (New default behavior with Exchange 2003 SP2) Situation #1: When you receive an event id 9341 during an OAB generation you will need to do the following: 1. Examine the event id 9341 and look at the legacyExchangeDN value '/o=organization/ou=Site/cn=Recipient' that is referenced. This means that no object in the current offline address list being generated has that value anymore. The object may have been deleted or mail disabled. Or there may still be an object in the active directory that still has a reference to that PDN, however it is not in the current offline address list and is no longer in the PDN table for this OAB. This object can be any mail enabled object (Public Folder, Distribution List, Contact, etc). 2. Create a placeholder object that would belong to the affected offline address list. This object will be in the form of a mail enabled contact. Once the contact is created, make sure the RUS (Recipient Update Service) stamps this user with the default proxy addresses. 3. Add an x500 address to this contact with the PDN that has been referenced in the 9341 event id: '/o=organization/ou=Site/cn=Recipient'. Before you ok this address you will need to add a RDN to it /cn=ContactName. Example of the x500 proxy address. '/o=organization/ou=Site/cn=ContactName’. NOTE: This will add the new PDN to the table and correct this issue. Situation #2: When you receive an event id 9341 for an object that has a temporary legacyExchangeDN. If there is an object in the active directory that does not have a legacyExchangeDN, the NT DS provider will add a temporary legacyExchangeDN that begins with /o=NT5 and this will contain the GUID of the forest. This DN will not show up in the Active Directory using LDAP tools, but is only returned by the Active Directory through its MAPI interface. Event Type: Information Event Source: MSExchangeSA Event Category: OAL Generator Event ID: 9341 Date: 10/26/2005 12:21:06 PM Computer: OABGEN-SRV Description: The parent Legacy Exchange DN container value '/o=NT5/ou=EAEB4348E4B6DB41ADA82A87B993E4BD' was not found during generation of the differential update file for offline address list '\Global Address List'. This will force clients using this offline address list to do a full download of the offline address list. - Default Offline Address List For more information, click http://www.microsoft.com/contentredirect.asp . NOTE: You MUST have Exchange 2003 SP build (6.5. 7569.0) installed in order to use this registry key. We can add a registry key to detect these values: To enable this functionality, you must set the OAL NT5 DN Rejection registry value. To do this, follow these steps: Click Start, click Run, type regedit, and then click OK. Locate and then click to select the following registry key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeSA\Parameters On the Edit menu, point to New, and then click DWORD Value. Type OAL NT5 DN Rejection, and then press ENTER. Right-click OAL NT5 DN Rejection, and then click Modify. In the Value data box, type 0x1 (1). Quit Registry Editor. NOTE: This will let us know if we have temp legacyExchangeDN’s for recipients and will not add them to the OAL. Situation #3: When you receive an event id 9340 indicating that a new PDN has been added to the PDN table. Event ID: 9340 Category: OAL Generator Source: MSExchangeSA Type: Warning (Added PDN) Generated: 11/2/2005 10:06:25 AM Machine: OABGEN-SRV Description: A new parent Legacy Exchange DN container value '/o=organization/ou=site/cn=container/cn=container' was found during generation of the differential update file for offline address list '\Global Address List'. This will force clients using this offline address list to do a full download of the offline address list. - Default Offline Address List NOTE: When you change a PDN you get a 9340 event. If you do not have the “OAL post full if diff fails” registry key set to let the OAB generate a full post , the exchange server will still have the old PDN table before you changed the legacyExchangeDN initially. When you change the legacyDN back to its original state, you will not see the 9341 event id, this is because the new PDN was never saved in the OAB. To enable this functionality, you must set the “OAL post full if diff fails” registry value. To do this, follow these steps: Click Start, click Run, type regedit, and then click OK. Locate and then click to select the following registry key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchangeSA\Parameters On the Edit menu, point to New, and then click DWORD Value. Type OAL post full if diff fails, and then press ENTER. Right-click OAL post full if diff fails, and then click Modify. In the Value data box, type 0x1 (1). Quit Registry Editor. NOTE: This will force the exchange server to save the new changes and post a full OAB message after the failure. Effects that a PDN table change can have on a network: Given that there are several cases listed above that can cause a large number of full OAB downloads you should understand the affect on bandwidth a large OAB download will have on the network. Overall Duration of the OAB Download: If many Outlook clients are attempting to download the Full OAB at the same time, this can take considerable time for all the downloads to complete (Example: If an organization has a 10 MB OAB, with 50 Outlook clients at a remote site, this equates to 500 MB of data to download. Just using the full bandwidth of a 256kbps link (without latency), it would take about 4 ½ hours transfer the download 500 MB). In addition the OAB is downloaded through MAPI and RPC. MAPI and PRC will add small percentage of additional data to the total download, and the latency between Outlook and the Exchange server will limit how much of the overall bandwidth can be used for all the data to be transferred. Overall each client may not take the entire time but between all clients the network will be used for the overall duration of the OAB Download (NOTE: 4 ½ hours was calculated by: 500 MB /32 KBps (32 KBps = 256Kbps) this did not take into account latency, extra traffic due to RPC, or other applications using the link = 4.34 hours). In order to determine duration of OAB downloads, first determine the size of your Full OAB. You can use ESM from Exchange 2003 to determine the size or the OAB. Drill down to the Public Folders node, then right-click, and choose "View System Folders". Once here, you can find the OAB Version 3a folder, and in the tabs on the right-hand side, choose "Content". You will be able to see the last 30 days worth of changes. The larger object with multiple attachments is the Full OAB and the size can be determined by adding up the sized of the attachments. Network Saturation: The Exchange server can easily handle many download requests for the OAB; as a result multiple attempts to download a Full OAB over a slow link can saturate a network. A saturated network is when all the available bandwidth is being utilized. When this happens, there are two significant side affects. First: Applications that need to use the WAN will perform slowly because as they wait for their network request to traverse the saturated WAN link. Second: The actual traffic needed on the WAN will increase because individual network requests may time-out resulting in additional requests being made. When the network becomes saturated the latency increases, not only the time that each client takes to download the OAB but can increase the overall duration. When the network is saturated the latency will increase, normally this just means that the data rate for each client is reduced, however if the latency increases too much, RPC packets will time out, causing additional RPC requests for the same data to be retrieved. Worse, if someone is using Outlook and the download is canceled or fails, Outlook will delete what has downloaded and re-attempt the entire download. As a result more data will be requested and increase the overall duration of a large set of OAB downloads. How can all of the outlook downloads saturate a WAN? When Outlook downloads the OAB from the Exchange server, it will download the OAB through a series of RPC packets. Each packet is received, acknowledged, and then the next one is sent. Based on the latency between Outlook and Exchange a single Outlook client is limited on how fast it can receive, and acknowledge each packet. Because of this delay a single Outlook client may not be able to saturate a network link, however as more outlook clients begin to download the OAB the combined download rate of all clients could saturate the link. The link will remain saturated until the Full OABs have been downloaded. The relationship is linear, in that the larger the latency between the Outlook client and the Exchange server, the fewer packets can be received, and the more clients can download an OAB before a slow line is saturated. The reverse is also true, if latency is low, then fewer clients are needed to saturate a slow link. The number of Outlook clients that can download the OAB simultaneously without saturating the WAN will increase as either network latency increases or network bandwidth increases. Minimizing the side effects of Full OAB Downloads caused by PDN table changes: If your organization needs to minimize the affects of the full OAB downloads across a WAN here are the recommended options available in Exchange 2003 SP1. Limit Large Sets of Full OAB download The first option is to limit large sets of full OAB downloads as much as possible. Above is a list of conditions that will cause Outlook to pull down a full OAB, either through mailbox moves, large changes in the directory, or changes to the PDN table. Review these conditions and consider if there are practices in your organization that can be put into place to limit the cases that cause a Full OAB Download. Several Examples include: 1. Consider how and when groups of users are moved between servers. 2. Consider when large changes are being made to the active directory (such as applying a new area code, etc.) 3. When in a mixed 5.5 / 2000 or 2003 environment, consider where new mailboxes are created and can the growth of the PDN table be limited. As much as the items that trigger a full download can be limited in an organization, then the need for a full OAB download can also be limited. Limit Impact of a full OAB download There are several ways that you can also limit the Impact that a large set of full OAB download will have on the WAY an organization: Limit the Size of the OAB: You can limit the OAB in Exchange 2003 by: 1. Upgrade to Exchange 2003 Sp2: There have been several fixes to the Exchange Offline Address Book service to filter out un-need attributes, including extra certificates that are not used by Outlook. 2. Consider Certificates: Certificates are the largest single attribute stored in the OAB, expired certificates or un-used certificates can be removed from the directory. Consider using the No Details OAB for Remote Desktop clients: The No-Details OAB is an option for remote Outlook clients to just have a minimal OAB. This OAB version is extremely small and only contains the display name, email addresses, and office location. Benefits: The advantage of the No-Details OAB is that it is small, so the cost of the download is limited. Limitations: Any time Outlook tries to pull up details information on an address, then Outlook will perform an on-line request directly to the AD for the details. Offline access has very limited information so for Laptop users that are primarily offline this is not an option. Consider a Remote OAB Only Server for Remote Outlook clients: An Exchange PF ONLY server can be installed at a remote site with an OAB. All remote clients at this remote site would download the OAB from the local Exchange PF server. Benefits: Downloads of the Full OAB do not impact the WAN, a full mailbox server is not required so mailbox servers can still be consolidated to a central location. Limitations: An extra server is required at the remote sits 3. Limit number of users that access Exchange across a remote link: Consider a Remote OAB Only Server for Remote Outlook clients: An Exchange PF ONLY server can be installed at a remote site with an OAB. All remote clients at this remote site would download the OAB from the local Exchange PF server. Benefits: Downloads of the Full OAB do not impact the WAN, a full mailbox server is not required so mailbox servers can still be consolidated to a central location. Links to tools: 1. OABInteg – KB Article ID: Q907792 - Description of the Offline Address Book Integrity (OABInteg) tool - http://support.microsoft.com/default.aspx?scid=kb;EN-US;907792 2. ExBPA (Microsoft Exchange Server Analyzer Tools - http://www.microsoft.com/technet/prodtechnol/exchange/downloads/2003/analyzers/default.mspx Thanks to Neil Shipp for his review of all this!

View full story at blogs.msdn.com

OAB PDN Changes and Exchange Site Consolidations related news:

Latest news from Dgoldman's WebLog: