by Patrick Godlonton
I’m quite sure a few developers will be raising an eyebrow at this title. What’s the basis for this controversial statement? To answer this we’ll need to take a look under the hood of the Openlink Endur Scripting API and in particular, the conventions used by the standard library of the C programming language for dynamic memory allocation. What is dynamic memory allocation? It’s a mechanism for writing software to make use of quantities of memory that can’t be known until the program is running. This allows us to write flexible programs such as text editors that can handle any amount of typed text; without these mechanisms there would be an upper limit on the quantity of text that could be typed. So, let’s take a closer look at these mechanisms as they exist in the C programming language.
Nearly all high-level programming languages make use of a section of the process address space called the heap. This is an area of memory that can grow or shrink according to the demands of the running program. When a program is first launched the operating system loader creates a process to hold the program and all its associated data as it runs. This includes a heap that has a default size of 1 megabyte on Windows machines. How does the program access the heap? The standard library of the C programming language provides two functions called malloc and free (among thousands) that are used to allocate and release memory in heap respectively. Malloc requires a single argument that is the number of bytes to be allocated from the heap and it returns the memory address of the starting byte of the allocated block back to the caller. It is also responsible for calling the operating system, referred to a system call, when the existing resources in the heap have been exhausted.
So, you might be asking why the program doesn’t call the operating system directly. What is the point in having this extra layer? It can call the operating system directly; however, the malloc and free functions are portable; that is, source code using them can be compiled for any system whereas the operating system primitives differ from platform to platform. Secondly, these primitives dispense memory in the form of virtual memory pages; which for Intel-based Windows machines is 4 kilobytes. Malloc allows memory to be allocated one byte at a time. To support this granularity an internal list of free memory blocks is maintained to prevent the same blocks being dispensed more than once. The free function places memory blocks back onto this list so that they can be dispensed by future calls to malloc. This recycles the memory that the program has no further use of and prevents the program from exhausting all the computer’s available memory over time. Once a memory block has gone through the cycle of allocation and release it is important to discard the memory pointer. The standard practice to discard the pointer is to set it equal to null. When it is not nullified this is referred to as a dangling pointer and is considered a programming error.
In the context of Openlink scripting, the TablePtr variables available in AVS and Table objects available in JVS are pointers to an allocated memory block. The Openlink API routines that create blank tables; that is, Table_TableNew, call malloc under the hood to reserve space for the data representing the table structures. I have witnessed scripts where after creating a table, writing the table’s memory address to a pointer variable, destroying it and then creating another table, the dangling pointer variable is found to be pointing to the second table; not the dead carcass of the first table as we would expect. This is due to the inherent recycling in malloc and free as discussed above. The crux of the issue is that the same memory address was returned by both calls to malloc and hence from Table_TableNew for the first and second tables. By testing the validity of this dangling pointer using Table_IsTableValid the validity of the second table is being tested; not the first. This may cause issues in the script engine depending on the script logic, such as the destroying the same table twice by calling Table_Destroy with two pointer variables holding the same value. As discussed above this will corrupt the validity of the internal free memory block records maintained by malloc and free; and eventually cause the script engine to crash.
I have personally seen calls to Table_IsTableValid verify the validity of tables that are known to have been destroyed. It is fundamentally flawed as it relies on verifying pointers where the structures pointed to will change as the program runs. This will result in sporadic and unrepeatable script engine crashes by not abiding by the malloc and free usage contract. Try to avoid using it.