macOS How to overload 'malloc'?

[YuriV]

I am trying to use strdup/free in the environment where 'malloc' and 'free' are overloaded in dynamic library libtcmalloc_minimal.dylib from google-perftools.

My program brakes with invalid pointer in 'free' since even though /opt/local/lib/libtcmalloc_minimal.0.dylib is loaded and has 'malloc' but 'strdup' in
/usr/lib/libSystem.B.dylib still uses and old 'malloc' in libSystem.B.dylib.

Someone previously suggested using -Wl,-bind_at_load in link comamnd, but it doesn't help.

What is the correct way to completely overload 'malloc'/'free'?

-- tst.C --
#include <string.h>
#include <stdlib.h>

main() {
char *d = strdup("abc");
free(d);
}

-- link command --
g++ -g -o tst tst.C -L/opt/local/lib -ltcmalloc_minimal

--- error from tst ---
src/tcmalloc.cc:372] Attempt to free invalid pointer: 0x1002a0
Abort trap

 

[willieva]

Have you tried setting the environment variable
DYLD_LIBRARY_PATH
to point to the new malloc library? man dyld should give you some help.

 

[YuriV]

No, DYLD_LIBRARY_PATH doesn't help. DYLD_LIBRARY_PATH allows to set path where to look for dynamic libraries. In this case all libraries are found, but wrong symbols are bound since there is more than one choice for malloc/free symbols.

 

[jared_kipe]

What is the purpose of the new malloc/free? What are you building for, why the limitations of the "old" malloc/free?

If you have to use the "new" malloc/free, why not make a new function to malloc/duplicate a string.

AKA don't use <string.h> to duplicate strings, write your own function.

 

[YuriV]

Limitations of system malloc:
1. Unable to return total size of allocated memory.
2. Unable to profile memory (how much memory is at the particular point in a program allocated by which procedures)
3. Allows to return memory back to the system on client request.

All these limitations are solved by google-perftools. It also often is faster than system malloc, and even if not anything else, its good to be able to have an alternative to system malloc to be able to choose from and to experiment with.

strdup/free combination is explicitly described in strdup(3), and such cross-allocation also happens with a lot of other APIs.

"Write your own function" advice doesn't work. Many APIs are predefined the way they are and I can't be rewriting them. strdup is just a simple illustration of the problem.

 

[jared_kipe]

Limitations of system malloc:
1. Unable to return total size of allocated memory.
2. Unable to profile memory (how much memory is at the particular point in a program allocated by which procedures)
3. Allows to return memory back to the system on client request.

All these limitations are solved by google-perftools. It also often is faster than system malloc, and even if not anything else, its good to be able to have an alternative to system malloc to be able to choose from and to experiment with.

strdup/free combination is explicitly described in strdup(3), and such cross-allocation also happens with a lot of other APIs.

"Write your own function" advice doesn't work. Many APIs are predefined the way they are and I can't be rewriting them. strdup is just a simple illustration of the problem.

So what you are saying is that you want to use this library with other libraries that conflict with the first library. Well to the best of my knowledge you are SOL, short of getting the source for the other libraries, there is no way to make some code link with one version of a function, and other code link with different versions.

In Objective-C it is POSSIBLE for category method replacements to call the original method, but it isn't particularly pretty. I know of nothing similar in C, not that it would really help in this circumstance.

 

[YuriV]

So what you are saying is that you want to use this library with other libraries that conflict with the first library. Well to the best of my knowledge you are SOL, short of getting the source for the other libraries, there is no way to make some code link with one version of a function, and other code link with different versions.

No, I want all code to link with the same version of the function. I want to overload malloc and free 100%, in all uses, even in the same library where the original malloc is defined (where strdup also happens to reside).

 

[jared_kipe]

No, I want all code to link with the same version of the function. I want to overload malloc and free 100%, in all uses, even in the same library where the original malloc is defined (where strdup also happens to reside).

Right, so since the library that has strdup already compiled and linked, you cannot use your new malloc. Thus making a new strdup or building that library from scratch linking against your new malloc is really the only option I see.

 

[cqexbesd]

What is the correct way to completely overload 'malloc'/'free'?

I'm not sure there is a "correct" way to do this as I'm not sure it was ever intended to be something you would do. 2 ideas come to mind though:

a) making sure the symbols are loaded from your library first. This is platform specific and I'm no expert on OSX linking. Random thoughts though:

Keep in mind there may be a difference between runtime and linktime ordering.

You might need to turn off prebinding...or at least control it to prebind to what you want.

You may need to resort to -nostdlib and manually specifying all libraries yourself so you have complete control over linking order. Don't know if there is anyway to keep that order at runtime however.

LD_PRELOAD might get your library to load first.

b) use wrap. e.g. -Wl,-wrap,malloc. This allows you to provide your own __real_malloc function from which you could call the new malloc function. See Google for more info.

HTH but I haven't tried any of it myself...perhaps it will give you something to Google on.

Andrew