Add new type to the TypeSystem

The type-system is there to assure the communication between the individual annotators. New types are to be defined in the packages’ descriptors/typesystem/ folder in dedicated xml files.

Adding the new type

Go to the newly created rs_test package and navigate to the upper mentioned folder:

roscd rs_test
cd descriptrs/typesystem

Here you will find a file called all_types.xml. Don’t pay attention to this file yet. We’ll get back to it later. Create a new xml file where your new types are going to reside:

touch test_types.xml

We will create a new type of annotation called MyFirstAnnotation and an atomic type called Centroid having the x,y,z as the parameters. Open the test_types.xml file with your favorite editor and add the following content to it:

<?xml version="1.0" encoding="UTF-8"?>
<typeSystemDescription xmlns="http://uima.apache.org/resourceSpecifier">
<name>test</name>
<description/>
<version>1.0</version>
<vendor/>
<imports>
</imports>
<types>
  <typeDescription>
    <name>rs_test.test.Centroid</name>
    <description/>
    <supertypeName>uima.cas.TOP</supertypeName>
    <features>
      <featureDescription>
        <name>x</name>
        <description/>
        <rangeTypeName>uima.cas.Float</rangeTypeName>
        <multipleReferencesAllowed>false</multipleReferencesAllowed>
      </featureDescription>
      <featureDescription>
        <name>y</name>
        <description/>
        <rangeTypeName>uima.cas.Float</rangeTypeName>
        <multipleReferencesAllowed>false</multipleReferencesAllowed>
      </featureDescription>
      <featureDescription>
        <name>z</name>
        <description/>
        <rangeTypeName>uima.cas.Float</rangeTypeName>
        <multipleReferencesAllowed>false</multipleReferencesAllowed>
      </featureDescription>
    </features>
  </typeDescription>

  <typeDescription>
    <name>rs_test.test.MyFirstAnnotation</name>
    <description>Centorid of a cluster</description>
    <supertypeName>rs.core.Annotation</supertypeName>
    <features>
      <featureDescription>
        <name>centroid</name>
        <description></description>
        <rangeTypeName>rs_test.test.Centroid</rangeTypeName>
      </featureDescription>
      <featureDescription>
        <name>clusterId</name>
        <description></description>
        <rangeTypeName>uima.cas.Integer</rangeTypeName>
      </featureDescription>
    </features>
  </typeDescription>
</types>
</typeSystemDescription>

Call catkin_make in order to generate the container classes for the type-sytem, and resolve the dependencies of defined types (e.g. rs.core.Annotation). Notice that after compilation terminates, the following lines appear in your xml:

<!-- THESE IMPORTS WILL BE AUTOMATICALLY GENERATED BY A SCRIPT -->
<import location="../../../rs_refactored/descriptors/typesystem/core_types.xml"/>

These imports get generated by one of the helper scripts during compilation, and import the definitions of other types, in our case core_types.xml, where the type rs.core.Annotation is defined. The script also edits the all_types.xml (this was the xml the newly created annotator imported in the previous tutorial used), adding the path to our new type descriptor. You can also have a look at the container classes that got generated in include/rs_test/types/.

Using it in the code

Now that a new type has been created, you can start using it from the annotators. For this purpose we will edit the source code of MyFirstAnnotator. Add these lines to the process function:

rs::Scene scene = cas.getScene();
std::vector<rs::Cluster> clusters;
scene.identifiables.filter(clusters);
int idx = 0;
for (auto cluster:clusters)
{
    rs::Cluster &c = cluster;
    pcl::PointIndices indices;
    rs::conversion::from(((rs::ReferenceClusterPoints)c.points()).indices(),indices);
    outInfo("Cluster has "<<indices.indices.size()<<" points");

    Eigen::Vector4d pCentroid;
    pcl::compute3DCentroid(*cloud_ptr,indices, pCentroid);
    rs_test::MyFirstAnnotation annotation  = rs::create<rs_test::MyFirstAnnotation>(tcas);
    rs_test::Centroid centroid = rs::create<rs_test::Centroid>(tcas);
    centroid.x.set(pCentroid[0]);
    centroid.y.set(pCentroid[1]);
    centroid.z.set(pCentroid[2]);
    annotation.centroid.set(centroid);
    annotation.clusterId.set(idx++);
    c.annotations.append(annotation);
}

Don’t forget to include the necessary header files:

#include <rs_test/types/all_types.h>
#include <pcl/common/centroid.h>

Compile and run. If you have not modified your analysis engine nothing will happen. That is because the MyFirstAnnotator is placed right after the ImagePreprocessor in my_demo.xml. Move it down right after the ClusterMerger node. Run it now. You will see it outputting the number of points in each cluster it found. The annotator will store each clusters centroid as an annotation of our new type, allowing the retrieval of it from other components. For the sake of simplicity we will retrieve it from within the same annotator. Before the for loop ends add the following lines and recompile:

std::vector<rs_test::MyFirstAnnotation> testAnnotations;
c.annotations.filter(testAnnotations);
if(testAnnotations.empty())
  continue;
outInfo("Cluster "<<idx-1<<" has "<<testAnnotations.size()<<" annotation of type MyFirstAnnotation");
outInfo("ID is: "<<testAnnotations[0].clusterId() );
outInfo("x="<<testAnnotations[0].centroid().x());
outInfo("y="<<testAnnotations[0].centroid().y());
outInfo("z="<<testAnnotations[0].centroid().z());

Compile and run it again. You will see in the output the cluster IDs and their respective centroids.

This is of course a very simple example. The purpose of the type system is mainly to have higher level annotations that can come from different sources(annotators) represented in a unique manner, so that the results from multiple similar experts can be easily compared, ranked etc.