This post talks about the steps to take traces for a BizTalk host as well as memory leak traces for host process. Memory leak traces applies to any windows process. These traces are normally required when you are having performance issues especially due to memory leaks. Pssdiag trace for BizTalk is provided by Microsoft support team usually to troubleshoot issues. Normally, you will work with Microsoft support to take traces and provide them to analyze the traces. These steps are just a guideline and help you prepare better in some cases beforehand.
- Make sure host that has memory leak issue is not currently consuming high memory (restart and wait for 5 min if needed)
- Start BTS pssdiag (with BTStrace, MBV, and perfmon) (See below)
- Trigger 1 manual userdump during normal memory usage (See below))
- Set up Debugdiag leaktracker with memory dumps (see below)
- Wait for memory to get high
- Trigger 2 manual memory dumps (even if leaktracker already took some)
- Download and install DebugDiag from here – https://www.microsoft.com/en-us/download/details.aspx?id=49924
Steps for PSSDiag:
- Extract the PSSDiag zip on all the BizTalk servers in the group. Follow below steps on all machines.
- Open Initialize.exe as an administrator and have the following settings in place. Change the “Host Instance To Track” to host for which you see the issue. Don’t select the WCF trace option.
- Open up a command prompt as an administration and go to the PSSDiag folder.
- Run “Pssdiag /c” to start the trace. Wait for the collection started message to appear.
- Wait for the issue to happen. Stop the trace once you start seeing the issue in your environment. Make sure you capture at least one occurrence of the issue.
- Stop the PSSDIag trace by pressing Ctrl + C. Wait for the stop to complete.
- Provide the Output folder present in the PSSDIag folder.
Debugdiag leaktracker with memory dumps:
- Open DebugDiag, Add Rule, and choose Native (non-.NET) Memory and Handle Leak and click Next
- Select the BizTalk process that has the memory issue and click Next
- Under Userdump generation, click the Configure button
- In the Configure userdumps window, make sure Auto-create is checked
- Check Generate userdump when private bytes reach 700mb and each additional 200mb
- Check Generate userdump when virtual bytes reach 900mb and each additional 300mb
- Click Save & Close and click Yes to the “Are You Sure” message
- Click Next and select a Userdump Location with plenty of space (preferably over 20gb of free space) and click Next
- Choose Activate the rule now and click Finish
- If you get a popup asking about setting Symbol Search Path, click Yes
The above action plan assumes the normal memory usage is below 700mb for this host and it goes above 900mb before the perf issue happens (since we want atleast 2 dumps).
If the above statement is not true, we may need to tweak these settings.
If we’re not sure about what memory usage looks like before and during the issue, we can go with the above settings on the understanding that we may need to tweak them in the future if the necessary dumps are not captured on failure.
Collect Manual Userdump
- Open DebugDiag collection.
- Hit cancel on the first screen.
- Click on the “Processes” tab on the next screen.
- Select the BizTalk process from the list of processes.
- Right click the process and select “Create Manual Dump” option.