Debugging OpenFOAM Applications with TotalView

This blog post describes how to compile and debug an OpenFOAM example and how the TotalView C++ View (CV) facility can be used to improve the debugging experience for users. We will use the OpenFOAM simpleMatrix example provided with the full OpenFOAM installation in this tutorial and write a transformation for the OpenFOAM vector of doubles type.

What is OpenFOAM?

OpenFOAM is a free, opensource CFD software developed by Open CFD Ltd since 2004. It has a large user base across most areas of engineering and science, from both commercial and academic organisations. OpenFOAM has an extensive range of features to solve anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to acoustics, solid mechanics, and electromagnetics.

More information can be found on the OpenFOAM website here.

What is TotalView C++ View?

C++View (CV) is a facility that allows you to format program data in a more useful or meaningful form than the concrete representation that you see in TotalView when you inspect data in a running program. To use C++View (CV), you must write a function for each type whose format you would like to control.

More information can be found in the TotalView Reference Guide for C++View here. 

Requirements

This blog post uses OpenFOAM v2112, Centos 7 and TotalView 2022.1.11

You can request a download for TotalView Debugger Free Trial here.

Writing a C++ View Data Display Function

In order for C++View to work correctly, the code you write and TotalView must cooperate.

There are two key issues:

• The first issue is registering your function so that TotalView can find it when it needs to format data for display. This is straightforward, all you need to do is to define your function to have the right name and prototype. When TotalView needs to format the data of type T, it will look for a function with this signature:

int TV_ttf_display_type ( const T * );

The const is deliberate to remind you that changes should not be made to the object being formatted for display. Many real-world applications are not entirely const-correct, and in cases where you must cast away the const, extreme caution is advised.

You will need to define a TV_ttf_display_type function for each type you want to format. A TV_ttf_display_type function may be at global scope, or it may be a class (static) method. It cannot be a member function.

• The second issue concerns how the TV_ttf_display_type function which you will write communicates with TotalView. The API you will need to use is given in the header file tv_data_display.h included with your TotalView distribution in the <totalview-installation>/src directory.

Your TV_ttf_display_type will use the provided function TV_ttf_add_row to tell TotalView what information should be displayed. Its prototype is:

int TV_ttf_add_row ( const char *field_name,
const char *type_name,
const char *address );

The field_name parameter is the descriptive name of the data field being computed. It will be shown by TotalView in a form similar to that of the name of a structure's field.

The type_name parameter is the type of the data to be displayed. It must be the name of a legal type name in the program, or one of TotalView's types.

Steps (without C++ View)

1. Copy the OpenFOAM simpleMatrix project to a local user workspace:

cp -r /usr/lib/openfoam/openfoam2112/applications/test/simpleMatrix.

2.​ ​​​​​​ Enable Debugging for the OpenFOAM build:

export WM_COMPILE_OPTIONS=Debug

3.  Build the the simpleMatrix project:

wmake

4.  Start TotalView on the simpleMatrix example:

totalview ./Test-simpleMatrix

5. Set a breakpoint at line 54 Info << hmm << endl;
6. Press GO to run to line 54.
7. Add the hmm data structure to the Data View by dragging and dropping from the Local Variables or right clicking on the hmm variable and selecting “Add to data view."
8. TotalView displays the hmm data structure in the Data View window.

9. Expand out the hmm data structure.

At the lowest level we see the double[3] array but not the values stored in the vector of doubles.

We can improve on this by writing a custom C++ View transformation and adding this to the Test-simpleMatrix.C file.

Steps (using C++ View)

1. In order to use the TotalView C++ View facility we must build the tv_data_display shared library which we will link to when building the simpleMatrix example.

TotalView provides the tv_data_display.c and tv_data_display.h files in the TotalView installation in the /usr/toolworks/totalview.2022.1.11/src/ and /usr/toolworks/totalview.2022.1.11/include/ directories respectively.

2. Build the shared library tv_data_display.so and place it in the TotalView lib directory:

g++ -g -I/usr/toolworks/totalview.2022.1.11/include 
/usr/toolworks/totalview.2022.1.11/src/tv_data_display.c -fPIC -c -o 
/usr/toolworks/totalview.2022.1.11/src /tv_data_display.o

g++ -g -shared -o /usr/toolworks/totalview.2022.1.11/lib/libtv_data_display.so 
/usr/toolworks/totalview.2022.1.11/src /tv_data_display.o

3. Copy the OpenFOAM simpleMatrix project to a local user workspace:

cp -r/usr/lib/openfoam/openfoam2112/applications/test/simpleMatrix.

4. Edit the simpleMatrix options file to include the TotalView include and lib directories and the tv_data_display shared library:

EXE_INC = -I/usr/toolworks/totalview.2022.1.11/include/
EXE_LIBS = -L/usr/toolworks/totalview.2022.1.11/lib -ltv_data_display

5. Enable Debugging for the OpenFOAM build:

export WM_COMPILE_OPTIONS=Debug

6. Add the Views C++ transformation for OpenFOAM Foam::List<<Foam::vector<double> > to the top of the Test-simpleMatrix.C file.

7. Build the simpleMatrix project:

wmake

8. Add the TotalView lib directory to the LD_LIBRARY_PATH environment variable:

export LD_LIBRARY_PATH=/usr/toolworks/totalview.2022.1.11/lib:$LD_LIBRARY_PATH 

9. Start TotalView on the simpleMatrix example:

totalview ./Test-simpleMatrix

10. Set a breakpoint at line 54 Info << hmm << endl;
11. Press GO to run to line 54.
12. Add the hmm data structure to the Data View by dragging and dropping from the Local Variables or right clicking on the hmm variable and selecting “Add to data view."
13. Expand out the hmm data structure and examine the source_ structure.

TotalView displays the vector of doubles as a result of the C++ View transformation.

Next Steps

This step by step guide showed you how to debug OpenFOAM applications using TotalView C++ View. 

Start your TotalView free trial and test its debugging capabilities for yourself.

Try TotalView Free