Troubleshooting packages that just won’t install on DP’s

You may have seen in my earlier blog post instructions on how to deploy many packages to DP’s where you know they have a small link or where you’ve had to limit the rate used by SCCM to a very small %. Even where you have followed them, you’ll still probably find some stragglers that never get to the DP without a little bit more….persuasion!

Below you’ll find the process I use to clean up that active packages report:

Step 1 – Identify problem packages

1. Firstly open up the active packages report in SCCM reporting and sort by DP, so you have a list of all packages that are waiting to install on each of the DP’s. Choose a specific DP to troubleshoot.
2. Check the sender.log on the parent primary, to see if anything is being sent to the DP. If it is, then files are still being sent to the DP and you should leave this DP for now and move on to the next.
3. RDP on to the DP you want to troubleshoot (if the server that hosts the DP is just that and not a Pri/Sec site, then you will need to RDP to the server that is the site server rather than just the DP)
4. Check the root of the drive that SCCM is installed on, for any “_XXXXX.TMP” folders as this will indicate if the DP is still working on uncompressing the PCK files. If you see any folders similar to the above then leave this DP for now and move on to the next. If you don’t, then a good final check for whether the server is doing anything, is to open up the distmgr.log file (I usually open this up in Trace32 as it makes much easier reading) and if you see the message:
   Used 0 out of 3 allowed processing threads.
   then chances are its not going to uncompress any more without a little help from you.

Step 2 – Refresh the packages

1. Before you do anything else, just try to simply refresh the packages on the affected DP. When doing this ensure the parent primary site is the “owner” of the package for the secondary site DP you are troubleshooting. If you are unsure, connect to the SCCM console of the parent primary site of the affected secondary site and go to the affected package > Distribution Points > if the secondary site DP has a padlock next to it then the “owner” will be another site. To change it go to the SQL server where the SCCM DB is located, open up SQL Server Management Studio (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing) and create a new query for the SCCM DB.
2. Run the following query, ensuring to fill in your own information:
   update PkgServers set SourceSite=’parent primary site code’ where SiteCode=’affected secondary site code’ and PkgID=’affected packageID’
   So if your parent primary site code was XXX and package XXX00001 has a padlock for secondary site code YYY, then you would run the following:
   update PkgServers set SourceSite=’XXX’ where SiteCode=’YYY’ and PkgID=’XXX00001′
3. Once done, go to the affected package and right click over the Distribution Points node > select refresh and the padlock should have disappeared.
4. Now simply refresh the package for secondary site DP and monitor the distmgr.log file on the secondary site server for a short time.
5. If the package is successfully uncompressing, you will see the entries similar to following:
   Start updating the package XXX00001…
   Attempting to add or update a package on a distribution point.
   Validating the compressed file D:\SMSPKG\XXX00001.PCK for package 0GL000DC
   Used 1 out of 3 allowed processing threads.
   The package file requires a NTFS drive to decompress to.
   Decompressing package D:\SMSPKG\XXX00001.PCK to D:\_S Mq0j8.TMP
   then after a while:
   Created directory D:\SMSPKGD$GL001B107045a9-91fe-4a47-aaf3-3a8c0cfe5b59
   Copying D:\_S Mq0j8.TMP\XXXXX to D:\SMSPKGD$\XXX00001\XXXXX, OK
   Once its starts the copying you can generally be sure the package will uncompress and get removed from the active packages report
6. If after a reasonable amount of time the package doesn’t start to uncompress or you see that the package ID has been processed but for some reason hasn’t uncompressed move on to “Step 3 – PreloadPkgOnSite tool”

Step 3 – PreloadPkgOnSite tool

1. If the refresh hasn’t worked you should now try the PreloadPkgOnSite.exe tool to uncompress the outstanding packages. To do this copy the PreloadPkgOnSite.exe tool (can be found online for SCCM) to the server (I usually copy it to %ConfigMgrInstall%\bin\i3860000409), open a command prompt (as admin if using WS2008) and browse to the folder location where PreloadPkgOnSite.exe is saved and use the following command:
   PreloadPkgOnSite.exe XXX00001 (or what ever the package ID is)
2. A lot of the time, you will see:
   D:\ConfigMgr\bin\i3860000409>PreloadPkgOnSite.exe XXX00001
   ****** Preload Package On Site ******
   Forward package status for pkg XXX00001 to site XXX
   ****** Successfully set the Compressed Package Path on this site ******
   ****** Successfully forwarded the information up the hierarchy ******
   This generally means the package is now going to be uncompressed and the information will be forwarded up to the parent primary and removed from the active packages report.
3. Again you can confirm if it is successfully uncompressing by following the distmgr.log and looking for the same entries as Step 2 > Action 5
4. If it doesn’t uncompress or you receive an error when running the PreloadPkgOnSite tool then move on to “Step 4 – Other problems”

Step 4 – Other problems
The above works a lot of the time but sometimes you will find other error messages are returned. See below for fixes I have previously used, for some of these.

Problem 1
Either of the following messages are returned by the PreloadPkgOnSite tool:
Failed to get the specified package XXX00001 in the database. Please check if you have instance rights to that package.
OR
The compressed package path for package XXX00001 is already set in the database

1. Ensure the PCK file for the package has been copied across to the SMSPKG folder.
2. If not, copy across manually (from another site) and then run PreloadPkgOnSite tool again for that package. If it returns successful then it will normally begin to uncompress.
3. If it doesn’t, go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
4. You now need to edit the SCCM DB (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing). I’m presuming the DP you are trying to add these packages to is a secondary site. If so on its direct parent primary site, open up SQL Server Management Studio and create a new, compltely blank View for your site DB.
5. In the view, copy the below:
   Select * from PkgStatus where SiteCode = ‘Site Code of DP with troublesome packages’
6. Scroll to the row where the ID matches the package ID and change the SourceVersion to 0 (in some cases there may be more than one row for each ID – change both)
7. Refresh the package for that specific DP (following the instructions in step 2)
8. After refreshing the package or packages, new PKG files should get sent to the %ConfigMgrInstall%\inboxes\distmgr.box.
9. If, after the new PKG files appear, the package doesn’t automatically begin to uncompress then run PreloadPkgOnSite tool with the package ID

Problem 2
When running the PreloadPkgOnSite tool it states as successful but in distmgr.log you recieve the following:
The compressed files for package XXX00001 hasn’t arrived from site 0GL yet, will retry later.
Sent package server list to parent site XXX for package XXX00001
StoredPkgVersion (4) of package XXX00001. StoredPkgVersion in database is X.
SourceVersion (4) of package XXX00001. SourceVersion in database is X.

1. Ensure the PCK file for the package has been copied across to the SMSPKG folder.
2. If not, copy across manually (from another site) and then run PreloadPkgOnSite tool again for that package. If it returns successful then it will normally begin to uncompress.
3. If it doesn’t, go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
4. You now need to edit the SCCM DB (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing). I’m presuming the DP you are trying to add these packages to is a secondary site. If so on its direct parent primary site, open up SQL Server Management Studio and create a new, compltely blank View for your site DB.
5. In the view, copy the below:
   Select * from PkgStatus where SiteCode = ‘Site Code of DP with troublesome packages’
6. Scroll to the row where the ID matches the package ID and change the SourceVersion to 0 (in some cases there may be more than one row for each ID – change both)
7. Refresh the package for that specific DP (following the instructions in step 2)
8. After refreshing the package or packages, new PKG files should get sent to the %ConfigMgrInstall%\inboxes\distmgr.box.
9. If, after the new PKG files appear, the package doesn’t automatically begin to uncompress then run PreloadPkgOnSite tool with the package ID

Problem 3
You receive the following message in the distmgr.log file:
The compressed package file D:\SMSPKG\XXX00001.PCK is either missing or corrupted, CTool return code = 11

1. Delete the current PCK file for the package that is having this problem from the SMSPKG folder. Copy a new version of the PCK file from another location.
2. Once copied, run PreloadPkgOnSite tool with the package ID

Problem 4
You receive the following message in the distmgr.log file:
A newer version (1) of package XXX00001 has already arrived, delete the replication file D:\ConfigMgr\inboxes\distmgr.box\INCOMING\RFVI26N1.PKG for version 0

1. Go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
2. Refresh the package for that specific DP (following the instructions in step 2)
3. If the package doesn’t automatically begin to uncompress after a refresh then run PreloadPkgOnSite tool with the package ID

Problem 5
You receive the following message in the distmgr.log file:
The compressed files for package XXX00001 hasn’t arrived from site XXX yet, will retry later.

1. Ensure the PCK file for the package has been copied across to the SMSPKG folder. If not, copy across manually (from another site).
2. Run the PreloadPkgOnSite tool with the package ID.

Problem 6
The following message is returned by the PreloadPkgOnSite tool:
Failed to get the compressed file of package XXX00001

1. The PCK file is missing from the SMSPKG folder. Check if this is the case and if so, copy across manually (from another site).
2. Run the PreloadPkgOnSite tool with the package ID.

Problem 7
The following message is returned by the PreloadPkgOnSite tool:
Cannot connect to the inbox source, please try again after 5 mins. Failed to get the specified package XXX00001 in the database. Please check if you have instance rights to that package.

1. Chances are this is a Server 2008 machine and so the Command Prompt must be run as administrator
2. Right click over “Command Prompt” in Start Menu and select “Run as administrator”

If all else fails…
If you’ve done all of the above and it nothing has worked then doing the following should always work:

1. Delete the current PCK file, for the package you are troubleshooting, from the SMSPKG folder.
2. Go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
3. You now need to edit the SCCM DB (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing). I’m presuming the DP you are trying to add these packages to is a secondary site. If so, on all parent primary sites, open up SQL Server Management Studio and create a new query for your site DB.
4. In the query, copy the below:
   select * from pkgstatus where SiteCode = ‘Site Code of DP with troublesome package’ and ID = ‘Package ID’
   select * from PkgServers where SiteCode = ‘Site Code of DP with troublesome package’ and PkgID = ‘Package ID’
5. Double checking to make sure the only results relate to the site and package that you are having problems with, you then need to delete these database entries using the below:
   DELETE from pkgstatus where SiteCode = ‘Site Code of DP with troublesome package’ and ID = ‘Package ID’
   DELETE from PkgServers where SiteCode = ‘Site Code of DP with troublesome package’ and PkgID = ‘Package ID’
6. Once the delete command has completed, you now need to redeploy the package to the DP. Go into the SCCM console and check the DPs for the package – you should see that the troublesome DP is no longer there. Re-add the DP to the package.
7. As the entries have been removed from database, SCCM thinks it is deploying the package to a brand new DP and will send the PCK file using the normal method. Watch the sender.log to monitor the package being sent.
8. On the DP open the distmgr.log and wait for the package to be processed.
9. If you see the package get processed but not uncompressed, simply refresh the package for that specific DP again.
10. After refreshing the package, the software should uncompress automatically.