Introduction

The following article is my response to “John” comment on my other post about native C++/C# interop using C++/CLI.
The initial request was rather simple: for whatever reason John wanted to use the C++/CLI wrapper from Python and not from native C++.

It seemed technically feasible because Python has a remarkable tool to interact with native code: the ctypes module.
The only issue is that ctypes only supports C interfaces not C++ classes so in this case it can’t directly use the YahooAPIWrapper class.

In fact it’s a minor issue because this kind of situation is well known and a well documented pattern exists to circumvent it: building a C wrapper around the C++ API.

This looks a little crazy because you now have 2 layers between the Python client code and the C# Yahoo API:

Python -> C wrapper -> C++/CLI wrapper -> C# API

So, while I don’t think this layering could have any usefulness in real-life, this was a challenging and interesting question.

Looks simple no? Well, as you know when you start to pile up heterogeneous layers unexpected issues can appear and this is exactly what happened there, and it has revealed one that is worth talking about.
So keep reading!

A naive implementation

I first started with this simple C wrapper:

extern "C"
{
    __declspec(dllexport) void* YahooAPIWrapper_New()
    {
        return new(std::nothrow) YahooAPIWrapper();
    }
	
    __declspec(dllexport) void YahooAPIWrapper_Delete(void* wrapper)
    {
        delete wrapper; 
    }
	
    __declspec(dllexport) double YahooAPIWrapper_GetBid(void* wrapper, const char* symbol)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetBid(symbol);
    }
	
    __declspec(dllexport) double YahooAPIWrapper_GetAsk(void* wrapper, const char* symbol)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetAsk(symbol);
    }
	
	__declspec(dllexport) const char* YahooAPIWrapper_GetCapitalization(void* wrapper, const char* symbol)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetCapitalization(symbol);
    }
	
	__declspec(dllexport) const char** YahooAPIWrapper_GetValues(void* wrapper, const char* symbol, const char* fields)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetValues(symbol, fields);
    }
}

Nothing special: it just forwards the calls to the C++ class, including one function for creating a new instance, YahooAPIWrapper_New, and another for deleting an existing instance, YahooAPIWrapper_Delete.

The Python script

Here is the Python script, “test.py”, that mimics the behavior of the native C++ program from the original article:

import ctypes

YahooAPIWrapper = ctypes.CDLL('YahooAPIWrapper.dll')

YahooAPIWrapper_New = YahooAPIWrapper['YahooAPIWrapper_New']

YahooAPIWrapper_Delete = YahooAPIWrapper['YahooAPIWrapper_Delete']

YahooAPIWrapper_GetBid = YahooAPIWrapper['YahooAPIWrapper_GetBid']
YahooAPIWrapper_GetBid.restype = ctypes.c_double

YahooAPIWrapper_GetAsk = YahooAPIWrapper['YahooAPIWrapper_GetAsk']
YahooAPIWrapper_GetAsk.restype = ctypes.c_double

YahooAPIWrapper_GetCapitalization = YahooAPIWrapper['YahooAPIWrapper_GetCapitalization']
YahooAPIWrapper_GetCapitalization.restype = ctypes.c_char_p

YahooAPIWrapper_GetValues = YahooAPIWrapper['YahooAPIWrapper_GetValues']
YahooAPIWrapper_GetValues.restype = ctypes.POINTER(ctypes.c_char_p)

wrapper = YahooAPIWrapper_New()

stock = ctypes.c_char_p(b"GOOG");

bid = YahooAPIWrapper_GetBid(wrapper, stock)
ask = YahooAPIWrapper_GetAsk(wrapper, stock)
capi = YahooAPIWrapper_GetCapitalization(wrapper, stock)

bidAskCapi = YahooAPIWrapper_GetValues(wrapper, stock, ctypes.c_char_p(b"b3b2j1"));

print("Bid: " + str(bid))
print("Ask: " + str(ask))
print("Capi: " + capi.decode())

print("BidAskCapi[0]: " + bidAskCapi[0].decode())
print("BidAskCapi[1]: " + bidAskCapi[1].decode())
print("BidAskCapi[2]: " + bidAskCapi[2].decode())

YahooAPIWrapper_Delete(wrapper)

If you’re not familiar with ctypes here is what it does:

• load the DLL, and specify it uses the cdecl calling convention (it’s the default for C++)
• get a proxy for each of the exported functions, and specify their return types: this is important because by default ctypes considers the return type as int
• call the functions taking care of marshalling and unmarshalling the strings

There is not a lot of plumbing as ctypes manages all the hard work for us.

The issue: assemblies probing

But running the script for the first time gave:

>python.exe test.py
Traceback (most recent call last):
  File "test.py", line 21, in 
    wrapper = YahooAPIWrapper_New()
OSError: [WinError -532462766] Windows Error 0x%X

Ouch! Without more information it’s hard to find the source…

But by playing with the code I’ve found that the issue was not related to Python or C++/CLI but to the C# library, more exactly to the way it is probed by .Net.

Indeed, remember “YahooAPI.dll” is a .Net assembly, and .Net does not load all the dependencies at startup but loads them on the fly, only when it encounters a type which is in an assembly not already loaded.
And as in the YahooAPIWrapper constructor we need the YahooAPI type, the first time it encounters it, the .Net runtime, the CLR, searches the assembly it knows contains the YahooAPI type (thanks to the metadata emitted during the compilation): “YahooAPI.dll“.

It first looks into the GAC, doesn’t find it, then looks in the directory of the current process executable, and that’s precisely the issue: the current program is the Python interpreter, “python.exe“; so the CLR was searching for the YahoAPI.dll assembly into the Python install path; well you guess it: it doesn’t found anything useful there and expressed its vexation by crashing.

Note that this issue doesn’t exist when the code calling the C++/CLI wrapper is a native C++ application, as in the original article, because the native executable is typically run from the same directory as the assembly.

The fix: AssemblyResolve to the rescue

This is a classic issue with .Net when you use it from another context, e.g. when you write some Excel addins; and the fix is quite simple and standard: you just have to help the CLR.
Indeed when the CLR is not able to find an assembly it can ask you to do the job by raising an event: AssemblyResolve.
You only need to provide an handler and try your best to find the assembly and returns it to the CLR so that it can load its types and continue running.
In other words we have to add one more pipe to complete the plumbing.

I’ve used a direct approach: using the currently executing assembly (“YahooAPIWrapper.dll“) path as a hint.

Here is the AssemblyResolve handler:

static Assembly^ AssemblyResolve(Object^ Sender, ResolveEventArgs^ args)
{
	AssemblyName^ assemblyName = gcnew AssemblyName(args->Name);

	if (assemblyName->Name == "YahooAPI")
	{
		String^ path = Path::Combine(Path::GetDirectoryName(Assembly::GetExecutingAssembly()->Location), "YahooAPI.dll");
		
		return Assembly::LoadFile(path);
	}
		
	return nullptr;
}

So with words here is what it does:

• get the current assembly path (e.g. “C:\temp\wrapper\YahooAPIWrapper.dll“)
• extract the directory (“C:\temp\wrapper\“)
• build the “YahooAPI.dll” assembly full path (“C:\temp\wrapper\” + “YahooAPI.dll“)
• load the assembly
• return the assembly to the caller (the CLR)

Simple and IMHO reliable.

The handler is registered in an additional “Initialize” method:

void Initialize()
{
	if (!isInitialized)
	{
		AppDomain::CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(AssemblyResolve);
		
		isInitialized = true;
	}
}

It is registered only once thanks to the isInitialized safeguard.

This “Initialize” method is itself called at each new instantiation of a YahooAPIWrapper:

__declspec(dllexport) void* YahooAPIWrapper_New()
{
    Initialize();
	
    return new(std::nothrow) YahooAPIWrapper();
}

So to sum it up:

• the first time YahooAPIWrapper_New is called it calls Initialize which registers for the AssemblyResolve event,
• later when the CLR sees the YahooAPI type, so for the first time, it tries to load the YahooAPI.dll assembly but fails,
• as a last resort it raises the AssemblyResolve event,
• our AssemblyResolve event handler is called, gets the requested assembly, and send it back to the CLR,
• the CLR is happy, loads the assembly and continues running, executing the YahooAPIWrapper constructor
• the next times we call YahooAPIWrapper_New, Initialize returns immediately because it knows, thanks to the isInitialized flag, that the assembly has already been loaded

Source code

So here are the different parts:

the C++/CLI wrapper header, YahooAPIWrapper.h:

class YahooAPIWrapperPrivate;

class __declspec(dllexport) YahooAPIWrapper
{
    private: YahooAPIWrapperPrivate* _private;

    public: YahooAPIWrapper();
	
    public: ~YahooAPIWrapper();
    
    public: double GetBid(const char* symbol);

    public: double GetAsk(const char* symbol);
    
    public: const char* GetCapitalization(const char* symbol);
    
    public: const char** GetValues(const char* symbol, const char* fields);
};

And the new wrapper implementation in YahooAPIWrapper.cpp:

#using "YahooAPI.dll"

#include <msclr\auto_gcroot.h>
#include <new>

#include "YahooAPIWrapper.h"

using namespace System; // Object
using namespace System::IO; // Path
using namespace System::Reflection; // Assembly
using namespace System::Runtime::InteropServices; // Marshal

class YahooAPIWrapperPrivate
{
    public: msclr::auto_gcroot<YahooAPI^> yahooAPI;
};

static Assembly^ AssemblyResolve(Object^ Sender, ResolveEventArgs^ args)
{
	AssemblyName^ assemblyName = gcnew AssemblyName(args->Name);

	if (assemblyName->Name == "YahooAPI")
	{
		String^ path = Path::Combine(Path::GetDirectoryName(Assembly::GetExecutingAssembly()->Location), "YahooAPI.dll");
		
		return Assembly::LoadFile(path);
	}
		
	return nullptr;
}

YahooAPIWrapper::YahooAPIWrapper()
{
	_private = new YahooAPIWrapperPrivate();
	_private->yahooAPI = gcnew YahooAPI();
}

double YahooAPIWrapper::GetBid(const char* symbol)
{
	return _private->yahooAPI->GetBid(gcnew System::String(symbol));
}

double YahooAPIWrapper::GetAsk(const char* symbol)
{
	return _private->yahooAPI->GetAsk(gcnew System::String(symbol));
}

const char* YahooAPIWrapper::GetCapitalization(const char* symbol)
{
	System::String^ managedCapi = _private->yahooAPI->GetCapitalization(gcnew System::String(symbol));

	return (const char*)Marshal::StringToHGlobalAnsi(managedCapi).ToPointer();
}

const char** YahooAPIWrapper::GetValues(const char* symbol, const char* fields)
{
	cli::array<System::String^>^ managedValues = _private->yahooAPI->GetValues(gcnew System::String(symbol), gcnew System::String(fields));
	
	const char** unmanagedValues = new const char*[managedValues->Length];
	
	for (int i = 0; i < managedValues->Length; ++i)
	{
		unmanagedValues[i] = (const char*)Marshal::StringToHGlobalAnsi(managedValues[i]).ToPointer();
	}
	
	return unmanagedValues;
}

YahooAPIWrapper::~YahooAPIWrapper()
{
	delete _private;
}

extern "C"
{
	bool isInitialized = false;
	
	void Initialize()
	{
		if (!isInitialized)
		{
			AppDomain::CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(AssemblyResolve);
			
			isInitialized = true;
		}
	}

	__declspec(dllexport) void* YahooAPIWrapper_New()
	{
		Initialize();
	
		return new(std::nothrow) YahooAPIWrapper();
	}
	
	__declspec(dllexport) void YahooAPIWrapper_Delete(void* wrapper)
	{
		delete wrapper; 
	}
	
	__declspec(dllexport) double YahooAPIWrapper_GetBid(void* wrapper, const char* symbol)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetBid(symbol);
    }
	
	__declspec(dllexport) double YahooAPIWrapper_GetAsk(void* wrapper, const char* symbol)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetAsk(symbol);
    }
	
	__declspec(dllexport) const char* YahooAPIWrapper_GetCapitalization(void* wrapper, const char* symbol)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetCapitalization(symbol);
    }
	
	__declspec(dllexport) const char** YahooAPIWrapper_GetValues(void* wrapper, const char* symbol, const char* fields)
    {
        return reinterpret_cast<YahooAPIWrapper*>(wrapper)->GetValues(symbol, fields);
    }
}

Compilation and results

Compilation is the same as for the original wrapper:

>cl /clr /LD YahooAPIWrapper.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.40219.01
for Microsoft (R) .NET Framework version 4.00.30319.18047
Copyright (C) Microsoft Corporation.  All rights reserved.

YahooAPIWrapper.cpp
Microsoft (R) Incremental Linker Version 10.00.40219.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:YahooAPIWrapper.dll
/dll
/implib:YahooAPIWrapper.lib
YahooAPIWrapper.obj
   Creating library YahooAPIWrapper.lib and object YahooAPIWrapper.exp

And this time when running the Python script we get the expected results:

>python.exe test.py
Bid: 879.5
Ask: 880.0
Capi: 292.3B
BidAskCapi[0]: 879.50
BidAskCapi[1]: 880.00
BidAskCapi[2]: 292.3B

Application to WPF

Following a comment on this post here is how you could call a WPF component from Python.

The WPF components

Here is the pair of .Net components, a WPF window and a public API:

using System;
using System.Windows;
using System.Windows.Controls;
using System.Threading;
using System.Windows.Threading;

public class SliceServerWindow : Window
{
    public TextBlock TextOutput { get; private set; }

    public SliceServerWindow()
    {
        Width = 400;
        Height = 400;

        TextOutput = new TextBlock();

        Content = TextOutput;
    }
}

public class SliceServer
{
    Dispatcher dispatcher;
    SliceServerWindow window;

    private void ThreadStart()
    {
        window = new SliceServerWindow();
        Application application = new Application();
        dispatcher = application.Dispatcher;
        application.Run(window);
    }

    public void Run()
    {
        Thread thread = new Thread(ThreadStart);
        thread.SetApartmentState(ApartmentState.STA);

        thread.Start();
    }

    public void Send(string text)
    {
        MessageBox.Show(text);

        dispatcher.BeginInvoke((Action)(() => window.TextOutput.Text += text + "\n"));
    }
}

Some explanations:

• the WPF window is run in a dedicated thread for at least 2 reasons:
  • it can’t be the calling Python thread as it would make it run the Windows message loop until the WPF application is finished, making it unavailable to run the Python code,
  • it needs some special metadata: the STA attribute which is a common constraint in UIs where components can only be accessed from a single thread,
• we interact with the window from this WPF UI thread only, as it is the one that created it, using a dispatcher that acts as a pipe between the calling Python thread and this thread.

I’ve compiled it using Visual Studio (in x86/32-bit mode) as it is simpler to handle WPF dependencies (WindowsBase.dll, PresentationCore.dll…).

The C++/CLI wrapper with the C DLL interface

The plumbing is similar to the one used for the original sample:

#using "SliceServer.dll"
 
#include <msclr\auto_gcroot.h>
#include <new>
 
#include "SliceServerWrapper.h"
 
using namespace System; // Object
using namespace System::IO; // Path
using namespace System::Reflection; // Assembly
using namespace System::Runtime::InteropServices; // Marshal
 
class SliceServerWrapperPrivate
{
    public: msclr::auto_gcroot<SliceServer^> sliceServer;
};
 
static Assembly^ AssemblyResolve(Object^ Sender, ResolveEventArgs^ args)
{
    AssemblyName^ assemblyName = gcnew AssemblyName(args->Name);
 
    if (assemblyName->Name == "SliceServer")
    {
        String^ path = Path::Combine(Path::GetDirectoryName(Assembly::GetExecutingAssembly()->Location), "SliceServer.dll");
         
        return Assembly::LoadFile(path);
    }
         
    return nullptr;
}
 
SliceServerWrapper::SliceServerWrapper()
{
    _private = new SliceServerWrapperPrivate();
    _private->sliceServer = gcnew SliceServer();
}

void SliceServerWrapper::Run()
{
    return _private->sliceServer->Run();
}
 
void SliceServerWrapper::Send(const char* text)
{
    return _private->sliceServer->Send(gcnew System::String(text));
}
 
SliceServerWrapper::~SliceServerWrapper()
{
    delete _private;
}
 
extern "C"
{
    bool isInitialized = false;
     
    void Initialize()
    {
        if (!isInitialized)
        {
            AppDomain::CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(AssemblyResolve);
             
            isInitialized = true;
        }
    }
 
    __declspec(dllexport) void* SliceServer_New()
    {
        Initialize();
     
        return new(std::nothrow) SliceServerWrapper();
    }
     
    __declspec(dllexport) void SliceServer_Delete(void* wrapper)
    {
        delete wrapper;
    }
    
	__declspec(dllexport) void SliceServer_Run(void* wrapper)
    {
        reinterpret_cast<SliceServerWrapper*>(wrapper)->Run();
    }
	
    __declspec(dllexport) void SliceServer_Send(void* wrapper, const char* text)
    {
        reinterpret_cast<SliceServerWrapper*>(wrapper)->Send(text);
    }
}

Compilation:

>cl /clr /LD SliceServerWrapper.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.40219.01
for Microsoft (R) .NET Framework version 4.00.30319.34209
Copyright (C) Microsoft Corporation.  All rights reserved.

SliceServerWrapper.cpp
Microsoft (R) Incremental Linker Version 10.00.40219.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:SliceServerWrapper.dll
/dll
/implib:SliceServerWrapper.lib
SliceServerWrapper.obj
   Creating library SliceServerWrapper.lib and object SliceServerWrapper.exp

The Python script

A Python script that sends 3 messages to the WPF component:

import ctypes
 
SliceServerDLL = ctypes.CDLL('SliceServerWrapper.dll')
 
SliceServer_New = SliceServerDLL['SliceServer_New']
SliceServer_Delete = SliceServerDLL['SliceServer_Delete']
SliceServer_Run = SliceServerDLL['SliceServer_Run']
SliceServer_Send = SliceServerDLL['SliceServer_Send']
 
server = SliceServer_New()

SliceServer_Run(server)
 
text = ctypes.c_char_p(b"HELLO")
SliceServer_Send(server, text)
 
text = ctypes.c_char_p(b"WORLD")
SliceServer_Send(server, text)
 
text = ctypes.c_char_p(b"!!!")
SliceServer_Send(server, text)
 
SliceServer_Delete(server)

input("PRESS ENTER TO END")

Results

After running it from a 32-bit Python interpreter, here is the result when sending the last message:

Note that in terms of bitness all the chain is consistent in 32-bit.

Conclusion

I don’t know if the whole plumbing could have any practical application, because to interact with C# from Python COM seems a better approach.

Anyway all this plumbing illustrates some interesting points:

• integration with the .Net framework and its assembly loading and probing policy,
• implementation of a simple assemblies resolver,
• creation of a C wrapper for a C++ API,
• interop between Python and C with ctypes

And it confirms a fundamental principle of programming: if your plumbing doesn’t work you’re just not using enough pipes.

There is of course a lot of things to improve to make this production ready, but for learning and playing, this is enough. 🙂

If you catch some typo or mistakes, or have additional questions, or think it could be useful in any context (well really unlikely except legacy ;)) feel free to let a comment.

---

To follow the blog please subscribe to the RSS feed