Vicon Nexus SDK: Streaming Markers to MAYA

While Vicon Blade is known for its ease of integration with software such as Motion Builder and Maya, Nexus is most known for its application in Biomechanics as it has the capacity to record numerous sensors such as EMG as well as marker positions.   I wanted to look into art driven by motion capture markers, but I also wanted to use the same software as others in the lab for handling the motion capture, which is nexus.

My strategy was to use the Vicon realtime SDK which can get markers from the TCP/IP of nexus and then send that to MAYA by the same approach.  As a few people online have demonstrated ways to make MAYA receive information over tcp/ip using python, and since python is much faster for me to demonstrate a proof of concept, I decided to reuse some old code that was made for streaming from arduino to maya.  Although I use nexus, the Vicon SDK is supposed to be for both blade and nexus, so this method should work for either.

To do this, you must first install the Vicon realtime SDK.  You should be able to download it from their website after creating a user login on the support page.

Next, load visual studio (in this case i used 2012 express, its free) and make sure to include your .lib and .dll files.    Before editing anything I suggest you compile and test to make sure you have setup everything correctly.

The example file they give for C++ displays all the outputs from nexus however I just want the markers.  If you go down to line 420 you should see:

// Count the number of segments
unsigned int SegmentCount = MyClient.GetSegmentCount( SubjectName ).SegmentCount;
output_stream << "    Segments (" << SegmentCount << "):" << std::endl;

Right after this is some stuff on segments which we dont want, so comment it out until around line 576.  The first uncommented line should be:

unsigned int MarkerCount = MyClient.GetMarkerCount( SubjectName ).MarkerCount;
output_stream << "    Markers (" << MarkerCount << "):" << std::endl;

And then the for loop begins.  This is the heart of what we want in Maya.  First Vicon demonstrates how to get the marker name and translation and output it to the console:

// Get the marker name
std::string MarkerName = MyClient.GetMarkerName( SubjectName, MarkerIndex ).MarkerName;

// Get the marker parent
std::string MarkerParentName = MyClient.GetMarkerParentName( SubjectName, MarkerName ).SegmentName;

// Get the global marker translation
Output_GetMarkerGlobalTranslation _Output_GetMarkerGlobalTranslation =
    MyClient.GetMarkerGlobalTranslation( SubjectName, MarkerName );

output_stream << "      Marker #" << MarkerIndex            << ": "
    << MarkerName             << " ("
    << _Output_GetMarkerGlobalTranslation.Translation[ 0 ]  << ", "
    << _Output_GetMarkerGlobalTranslation.Translation[ 1 ]  << ", "
    << _Output_GetMarkerGlobalTranslation.Translation[ 2 ]  << ") "
    << Adapt( _Output_GetMarkerGlobalTranslation.Occluded ) << std::endl;

 

As vicon has already shown how to get the values and print to screen, we will just modify this method to store the value instead of printing to the screen.  We still want to get the Marker Name and Marker Number, so keep:

// Get the marker name
std::string MarkerName = MyClient.GetMarkerName( SubjectName, MarkerIndex ).MarkerName;
// Get the marker name
std::string MarkerName = MyClient.GetMarkerName( SubjectName, MarkerIndex ).MarkerName;

Now to get the translation data we will store the data in the Output_GetMarkerGlobalTranslation type:

Output_GetMarkerGlobalTranslation valueToSend = MyClient.GetMarkerGlobalTranslation( SubjectName, MarkerName );

For clarity I created three variables to store the X Y and Z position, then create an array to store the values:

double markerxPOS = valueToSend.Translation[0];
double markeryPOS = valueToSend.Translation[1];
double markerzPOS = valueToSend.Translation[2];
double markerXYZ[] = {markerxPOS,markeryPOS,markerzPOS};

To handle some string problems with sending over tcp/ip to maya, we will do some conversions to the marker name:

char *cstr = new char[MarkerName.length() + 1];
strcpy(cstr, MarkerName.c_str());
char *Marker_Name = cstr;

Just before sending the data to our soon-to-be-written function I want to clarify the MarkerIndex is really the Marker number which I just call:

int Marker_Num = MarkerIndex;

Now I will call a function senddata, which we will write next, passing it our marker information:

senddata(Marker_Num,Marker_Name,markerXYZ);

Thats it for the marker part. Before going on, there is some extra data vicon is getting that we dont need. A few lines down from our marker function is the unlabeled markers, if you want this for debugging you can keep it but below this is the devices and force plate functions. Right after:

output_stream << "  Devices (" << DeviceCount << "):" << std::endl;

Comment out the for loop until you reach the commented // Output eye tracker information.
Before moving on the the custom function we need to include a header file to let the too functions communicate. At the top, under #include “Client.h” , add #include “myheader.h”

Create a new header file called myheader.  This will be the link to the custom function. There only needs to be one line:

int senddata(int MarkerNumber, char *MarkerName,double args[]);

Save this and create a new CPP file. In my case I call it SendToMaya.CPP. As the goal is to communicate with Maya over tcp/ip, I started with the basic microsoft example of the send function. First include the header file, myheader.h and then some extras for communication:

#include "myheader.h"

#ifndef UNICODE
#define UNICODE
#endif

#define WIN32_LEAN_AND_MEAN

#include winsock2.h
#include Ws2tcpip.h
#include stdio.h
#include 
//#include 

// Link with ws2_32.lib
#pragma comment(lib, "Ws2_32.lib")

#define DEFAULT_BUFLEN 512
#define DEFAULT_PORT 27015

Next create the function senddata that was used in the vicon sdk file.

int senddata (int MarkerNumber, char* MarkerName, double args[])
{

The first part of the function will define some variables to be used with the socket connections.

    int iResult;
    WSADATA wsaData;

    SOCKET ConnectSocket = INVALID_SOCKET;
    struct sockaddr_in clientService; 

    int recvbuflen = DEFAULT_BUFLEN;

Now the marker information needs to be converted to strings in order to send over the socket. Although this is probably not the best way, it seemed to work as a prototype:

	double MarkNum = MarkerNumber;    // number to be converted to a string
	std::string ResultMN;          // string which will contain the result
	std::ostringstream convertMN;   // stream used for the conversion
	convertMN << MarkNum;      // insert the textual representation of 'Number' in the characters in the stream
	ResultMN = convertMN.str(); // set 'Result' to the contents of the stream
	char *cstrMN = new char[ResultMN.length() + 1];
	strcpy(cstrMN, ResultMN.c_str());
	char* s_MarkNum = cstrMN;

	double x_pos = args[0];    // number to be converted to a string
	std::string Resultx;          // string which will contain the result
	std::ostringstream convertx;   // stream used for the conversion
	convertx << x_pos;      // insert the textual representation of 'Number' in the characters in the stream
	Resultx = convertx.str(); // set 'Result' to the contents of the stream
	char *cstrx = new char[Resultx.length() + 1];
	strcpy(cstrx, Resultx.c_str());
	char* s_xPOS = cstrx;

	double y_pos = args[1];    // number to be converted to a string
	std::string Resulty;          // string which will contain the result
	std::ostringstream converty;   // stream used for the conversion
	converty << y_pos;      // insert the textual representation of 'Number' in the characters in the stream
	Resulty = converty.str(); // set 'Result' to the contents of the stream
	char *cstry = new char[Resulty.length() + 1];
	strcpy(cstry, Resulty.c_str());
	char* s_yPOS = cstry;

	double z_pos = args[2];    // number to be converted to a string
	std::string Resultz;          // string which will contain the result
	std::ostringstream convertz;   // stream used for the conversion
	convertz << z_pos;      // insert the textual representation of 'Number' in the characters in the stream
	Resultz = convertz.str(); // set 'Result' to the contents of the stream
	char *cstrz = new char[Resultz.length() + 1];
	strcpy(cstrz, Resultz.c_str());
	char* s_zPOS = cstrz;

	//std::cout<<"XYZ: "<<s_xPOS<<","<<s_yPOS<<","<<s_zPOS<<std::endl;
	char xyzString[200];   // array to hold the result.

	strcpy(xyzString,s_MarkNum);
	strcat(xyzString,":");
	strcat(xyzString,MarkerName);
	strcat(xyzString,":");
	strcat(xyzString,s_xPOS); // copy string one into the new string.
	strcat(xyzString,",");
	strcat(xyzString,s_yPOS); 
	strcat(xyzString,",");
	strcat(xyzString,s_zPOS);

Set the *sendbuf variable to the newly created xyzString:

	char *sendbuf = xyzString;

Do some initialization and connections:

	char recvbuf[DEFAULT_BUFLEN] = "";
    //----------------------
    // Initialize Winsock
    iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
    if (iResult != NO_ERROR) {
        wprintf(L"WSAStartup failed with error: %d\n", iResult);
        return 1;
    }

    //----------------------
    // Create a SOCKET for connecting to server
    ConnectSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    if (ConnectSocket == INVALID_SOCKET) {
        wprintf(L"socket failed with error: %ld\n", WSAGetLastError());
        WSACleanup();
        return 1;
    }

This is the part where you define the IP and port. For local computer it is probably 127.0.0.1 and the port can be anything, pick a high number which will usually not conflict with anything:

    //----------------------
    // The sockaddr_in structure specifies the address family,
    // IP address, and port of the server to be connected to.
    clientService.sin_family = AF_INET;
    clientService.sin_addr.s_addr = inet_addr( "127.0.0.1" );
    clientService.sin_port = htons( 1188 );

And that is mostly it for the C++ side. The rest is more of the standard communication protocols found in the microsoft examples:

    //----------------------
    // Connect to server.
    iResult = connect( ConnectSocket, (SOCKADDR*) &clientService, sizeof(clientService) );

    if (iResult == SOCKET_ERROR) {
        wprintf(L"connect failed with error: %d\n", WSAGetLastError() );
        closesocket(ConnectSocket);
        WSACleanup();
        return 1;
  }

    //----------------------
    // Send an initial buffer
    iResult = send( ConnectSocket, sendbuf, (int)strlen(sendbuf), 0 );
	//wprintf(L"sending the data that i got: %d\n",args);

    if (iResult == SOCKET_ERROR) {
        wprintf(L"send failed with error: %d\n", WSAGetLastError());
        closesocket(ConnectSocket);
        WSACleanup();
        return 1;
    }

    printf("Bytes Sent: %d\n", iResult);

    // shutdown the connection since no more data will be sent
    iResult = shutdown(ConnectSocket, SD_SEND);
    if (iResult == SOCKET_ERROR) {
        wprintf(L"shutdown failed with error: %d\n", WSAGetLastError());
        closesocket(ConnectSocket);
        WSACleanup();
        return 1;
    }

    // Receive until the peer closes the connection
    do {

        iResult = recv(ConnectSocket, recvbuf, recvbuflen, 0);
        if ( iResult > 0 )
            wprintf(L"Bytes received: %d\n", iResult);
        else if ( iResult == 0 )
            wprintf(L"Connection closed\n");
        else
            wprintf(L"recv failed with error: %d\n", WSAGetLastError());

    } while( iResult > 0 );

    // close the socket
    iResult = closesocket(ConnectSocket);
    if (iResult == SOCKET_ERROR) {
        wprintf(L"close failed with error: %d\n", WSAGetLastError());
        WSACleanup();
        return 1;
    }

    WSACleanup();
    return 0;
}

Now that the C++ is done the Maya file must be created.   The current method has some issues because of the speed of python vs c++ but it is a good way to start getting the markers in maya.  To solve many of these issues the C++ api can be used instead.

To get started we need to import the maya commands and mel, then setup a melproc:

# Take in Data (This case Markers) and Do a Transform
import maya.cmds as mc
import maya.mel as mm
#num = 0
# Our mel global proc.
melproc = """
global proc portData(string $arg){
    python(("portData(\\"" + $arg + "\\")"));
}
"""
mm.eval(melproc)

Next create a function portData that will take in the port information and do something with it. This will be called later in the script. This example will call another function that will parse the data out, then it will try to call cmds.xform, using the second value of the array as the name of the object to move.c

def portData(arg):
    """
    Reads streaming data passed from TCP/IP
    """
    #print "Recieved!: ", arg

    values = get_vals(arg)    
    try:
        cmds.xform(values[1],t=(values[2],values[3],values[4]))
        cmds.refresh()
    except:
        pass

The get_vals() function takes in the data string and parses it. Since the length of the string is unknown as marker names and position values could be different lengths, we create some variables to count when a space exists. When each space is found it will separate the variables:

def get_vals(arg):
    i=0
    marker_num=[]
    marker_name=[]
    marker_XP=[]
    marker_YP=[]
    marker_ZP=[]

    while (arg[i]!=":"):
        i=i+1   
    marker_num=arg[0:i]
    marker_num=float(marker_num)
    i=i+1
    j=i
    while (arg[i]!=":"):
        i=i+1
    marker_name=arg[j:i]
    i=i+1
    k=i
    while (arg[i]!=","):
        i=i+1    
    marker_XP=arg[k:i]    
    marker_XP=float(marker_XP)
    i=i+1
    l=i
    while (arg[i]!=","):
        i=i+1   
    marker_YP=arg[l:i]    
    marker_YP=float(marker_YP)          
    i=i+1
    m=i     
    try:
        while (arg[i]):
            i=i+1    
    except:
        pass
    marker_ZP=arg[m:i]    
    marker_ZP=float(marker_ZP) 

    values = [marker_num,marker_name,marker_XP,marker_YP,marker_ZP]

    return values

The very last step in the script is opening the command port for maya to listen for data.  The name should be equal to the ip and port used in the cpp file:

# Open the commandPort.  The 'prefix' argument string is calling to the defined
# mel script above (which then calls to our Python function of the same name):
mc.commandPort(name="localhost:1188", echoOutput=False, noreturn=False,
               prefix="portData", returnNumCommands=True)

Thats it! Remember that if you are trying to actually move some markers in Maya you need to create some spheres and name them the same as the marker names that are streaming.

  1. Kate

    Hi there,

    First of all thank you for posting this, I’m trying to stream to python and this has been a huge help.
    I was just wondering what you include right at the top of the new cpp file (sendtomaya here)?
    You’ve got some #include lines (just below #define WIN32_LEAN_AND_MEAN) missing what it is you actually include, I’ve put in
    #include
    #include
    #include
    (and of course #include “myheader.h”)
    but I’m afraid I might be missing something, otherwise I can’t think of what the issue could be with this part, the python side is fine..

    Thanks!

    4 years ago

    Reply
  2. Kate

    ok, these comments don’t like some characters..
    I’ve included WinSock2.h, string and sstream

    4 years ago

    Reply
  3. admin

    wow thanks for pointing that out. The website deleted everything inside the “< " and ">” try including winsock2.h Ws2tcpip.h and stdio.h and let me know if that helps

    4 years ago

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *