Performing Queries

The main purpose of fort-myrmidon is to allow user to perform queries [1], i.e. computations on the whole set of tracking data. These queries can only be performed once Ant are properly defined and in some case (such as for collisions and interactions) their wanted virtual shape are defined.

Note

Here the python and C++ API quite diverges from the R API. While the two first reports list or std::vector of object, the latter only reports data.frame or slist of data.frame with only native R types.

Instantaneous Queries

Identifying Frames

The simplest query that could be performed on a tracking data set is identifying frames [2]. It is a simple translation of the raw tracking data from the Identification. It simply returns for each acquired video frame the position and orientation of each ant, taking into account the geometry relation between a tag position and orientation and the virtual ant desired position and orientation.

This query return a list of IdentifiedFrame [6].

# Python
frames = fm.Query.IdentifyFrames(e)
for f in frames:
    print("Got Frame at %s in space %d" % (f.FrameTime,f.Space))
    print("AntID, X, Y, Angle, Zone")
    print(f.Positions)

# R
d = fmQueryIdentifyFrames(e);
d$frames # a data frame summarizing for each frame its dimension, space and acquisition time
d$positions # a list of data frame with each ant position.

// C++
fort::myrmidon::IdentifyFramesQueryArgs args;
// modify here default args;
std::vector<fort::myrmidon::IdentifiedFrame::Ptr> frames;
fort::myrmidon::Query::IdentifyFrame(e,frames,args);

for ( const auto & f : frames ) {
    std::cerr << "Got frame at " << f->FrameTime
              << " in space " << f->Space << std::endl
              << "AntID, X, Y, Angle, Zone" << std::endl
              << f->Positions << std::endl;
}

The main parameters for this query are:

• the starting and ending time of the query: in order to select only a range of the acquired tracking data. Since dataset tends to grow larger than the available RAM, it is important to only work on part of the dataset at a time to keep the memory usage low enough.

• Zone reporting: Choose if the actual zone the ant lies should be reported, see Segmenting Spaces in Zones

Collision Detection

Instead of just reporting the ant position, fort-myrmidon can also report any collision detection between ant shapes by colliding frames [3].

This query returns a collection of IdentifiedFrame and CollisionFrame [7].

#python
frames = fm.Query.CollideFrame(e)
for identified,collided in frames:
    print("Collisions at $s in Space %d." % ( collided.FrameTime, collided.Space) )
    for c in collided.Collisions:
        print("Ants %s and %s collides." % ( fm.FormatAntID(c.IDs[0]), fm.FormatAntID(c.IDs[1])))
                print(c.Types)

#R
d = fmQueryCollideFrames(e)
d$frames # a data.frame that summarizes frame time, dimensions and space
d$positions # a list of data.frame with the ant position for each frame
d$collisions # a data.frame of all collisions in the selected time range. the field frame_row_index refers to the row in frames or the index in positions.

//C++
auto args = fort::myrmidon::Query::CollideFramesArgs();
// modify here default arguments;
std::vector<std::pair<fort::myrmidon::IdentifiedFrame::Ptr,
                      fort::myrmidon::CollisionFrame::Ptr>> frames;
fort::myrmidon::CollideFrames(e,frames,args);

for ( const auto & [identified,collided] : frames ) {
    std::cerr << "Collisions at " << collided->FrameTime
              << " in space " << collided->Space
              << ": " << std::endl;
    for ( const auto & c : collided->Collisions ) {
        std::cerr << "Ants " << fort::myrmidon::FormatAntID(c.IDs.first)
                  << " and " << fort::myrmidon::FormatAntID(c.IDs.second)
                  << " collides." << std::endl
                  << c.Types << std::endl;
    }
}

The parameters are the same than for Identifying Frames, excepted than zone are always reported. Indeed zoning is used to prune undesired collisions as described in Segmenting Spaces in Zones.

Note

By convention, the order of the AntID in the IDs field is always sorted from the smallest to the highest to ensure uniqueness.

Interaction Types

As capsules are typed, for each collision the InteractionTypes [12] are reported. In Python and C++ it is a Nx2 matrix with each column representing the type of each ant which are colliding with the other one.

For example the matrix:

\[\begin{split}\begin{bmatrix} 1 & 1\\ 2 & 1 \end{bmatrix}\end{split}\]

Means body type 1 for the second Ant touches both body type 1 and 2 of the first ant, but the body type 2 of the second ant collide with no other parts.

In R, this matrix will be returned as a character: '1-1,2-1'.

Time-spanning Queries

The next two main queries, instead of report data for a single time, reports data over a time range. This time range is not predetermined and is found as the tracking data is parsed: Indeed as it is not possible to avoid detection losses, each of these queries accepts a maximumGap parameter. If tracking is lost for a duration larger than this parameter, the considered result (i.e. the trajectory or interaction) is considered done, and any latter detection or collision found later will become part of a new result.

Note

If this behavior of segmenting results according to the maximumGap parameter wants to be avoided, one can set it to a large value, e.g. one year.

Ant Trajectories

Instead of reporting instantaneous detection, computing ant trajectories [4] will report for each individual the time ranges where it was continuously detected, accordingly to the maximumGap parameter.

Results are a list of AntTrajectory [8].

# Python
trajectories = fm.Query.ComputeAntTrajectories(e,maximumGap = 500 * fm.Duration.Millisecond)
for t in trajectories:
    print("Ant %s Trajectory from %s to %s in space %d." % (fm.FormatAntID(t.Ant),t.Start,t.End(),t.Space))
    print(t.Positions)

# R
d = fmQueryComputeAntTrajectories(e, maximumGap = fmMillisecond(500))
d$trajectories_summary # a data.frame with the ant, space and starting time of each trajectories
d$trajectories # a list of data.frame with each ant position for each trajectories

// C++
auto args = fort::myrmidon::Query::ComputeAntTrajectoryArgs();
// modify here default arguments
std::vector<fort::myrmidon::AntTrajectory::Ptr> trajectories;
fort::myrmidon::Query::ComputeAntTrajectories(e,trajectories,args);

As I the case of Identifying Frames, the user can choose if the ant zone should be reported with the computeZone argument.

Ant Interactions

Instead of just looking at iNstantaneous collision, one can compute ant interactions [5], i.e. time ranges where ant collides.

Both AntTrajectory and AntInteraction [9] may be reported.

# Python
trajectories,interactions = fm.Query.ComputeAntInteractions(e)
for i in interactions:
    print("Ant %s and %s interacts from %s to %s" % (fm.FormatAntID(i.IDs[0]),fm.FormatAntID(i.IDs[1]),i.Start,i.End))

# R
d = fmQueryComputeAntInteractions(e)
d$trajectory_summaries # the summary of trajectory like for fmQueryComputeAntTrajectories
d$trajectories # the trajectory positions like for fmQueryComputeAntTrajectories
d$interactions # a data.frame with all interactions.

// C++
auto args = fort::myrmidon::Query::ComputeAntInteractionArgs();
// modify here default arguments;
std::vector<fort::myrmidon::AntTrajectory::Ptr> trajectories;
std::vector<fort::myrmidon::AntInteraction::Ptr> interactions;
fort::myrmidon::Query::ComputeAntInteractions(e,trajectories,interactions,args);

for ( const auto & i : interactions ) {
    std::cerr << "Ant " << fort::myrmidon::FormatAntID(i->IDs.first)
              << " and " << fort::myrmidon::FormatAntID(i->IDs.second)
              << " interacts from " << i->Start
              << " to " << i->End
              << std::endl;
}

Full vs Summarized Trajectory Report

As shown by the API, by default, the ComputeAntInteractions query returns both interactions and trajectories. In that case each AntInteraction object contains a pair of AntTrajectorySegment [10] that points to a part of the AntTrajectory corresponding to the current interaction.

Alternatively to simplify the results and to save memory, one can disable the report of the full trajectory with the reportFullTrajectories parameter. In that case the list of returned trajectory will be empty, and instead AntTrajectorySummary [11] will be reported. These object contains the mean position of the ant, and the list of zone traversed during the interaction.

The case of R is a bit more complex. If full trajectories are reported, the $interactions data.frame will report the $trajectories index and first and last rows corresponding to the trajectory segment. Otherwise, only a single data.frame is returned, with the mean position and zone for each ant.

Building Complex Queries with Matchers

Time-spanning queries accepts a matcher [13] parameter that could be used to customize the behavior of these queries.

For example one can select only 1-1 interactions type:

# Python
trajectories,interactions = fm.Query.ComputeAntInteractions(e,matcher = fm.Matcher.InteractionType(1,1))

# R
d = fmQueryComputeAntInteractions(e,matcher = fmMatcherInteractionType(1,1))

// C++
auto args = fort::myrmidon::ComputeAntInteractionsArgs();
args.Matcher = fort::myrmidon::Matcher::InteractionType(1,1);
std::vector<fort::myrmidon::AntTrajectory::Ptr> trajectories;
std::vector<fort::myrmidon::AntInteraction::Ptr> interactions;
fort::myrmidon::Query::ComputeAntInteractions(e,trajectories,interactions,args);

Or one can prune out interactions were one of the ant have a large displacement over a given duration:

# Python
trajectories,interactions = fm.Query.ComputeAntInteractions(e,matcher = fm.Matcher.AntDisplacement(under = 100, minimumGap = 500 * fm.Duration.Millisecond))

# R
d = fmQueryComputeAntInteractions(e,matcher = fmMatcherAntDisplacement(under = 100, minimumGap = fmMillisecond(500)))

// C++
auto args = fort::myrmidon::ComputeAntInteractionsArgs();
args.Matcher = fort::myrmidon::Matcher::AntDisplacement(100,500 * fort::Duration::Millisecond);
std::vector<fort::myrmidon::AntTrajectory::Ptr> trajectories;
std::vector<fort::myrmidon::AntInteraction::Ptr> interactions;
fort::myrmidon::Query::ComputeAntInteractions(e,trajectories,interactions,args);

More complex matcher can be combined together. Here we only look at interactions with ant 001 and the two above matcher combined:

# Python
m = fm.Matcher.And(fm.Matcher.AntID(1),fm.Matcher.InteractionType(1,1),fm.Matcher.AntDisplacement(100,500 * fm.Duration.Millisecond))
trajectories,interactions = fm.Query.ComputeAntInteractions(e,matcher = m)

# R
m <- fmMatcherAnd(list(fmMatcherAntID(1),fmMatcherInteractionType(1,1),fmMatcherAntDisplacement(100,fmMillisecond(500))))
d <- fmQueryComputeAntInteractions(e,matcher = m)

// C++
auto args = fort::myrmidon::ComputeAntInteractionsArgs();
args.Matcher = fort::myrmidon::Matcher::And({fort::myrmidon::Matcher::AntID(1),
                                             fort::myrmidon::Matcher::InteractionType(1,1),
                                                                                             fort::myrmidon::Matcher::AntDisplacement(100,fmMillisecond(500))});
std::vector<fort::myrmidon::AntTrajectory::Ptr> trajectories;
std::vector<fort::myrmidon::AntInteraction::Ptr> interactions;
fort::myrmidon::Query::ComputeAntInteractions(e,trajectories,interactions,args);

More matchers can be found in the Matcher class definition [13].

Miscellaneous Queries

TODO

[1]

Python: py_fort_myrmidon.Query, R: fmQuery, C++: fort::myrmidon::Query

[2]

Python: py_fort_myrmidon.Query.IdentifyFrames(), R: fmQueryIdentifyFrames(), C++: fort::myrmidon::Query::IdentifyFrames()

[3]

Python: py_fort_myrmidon.Query.CollideFrames(), R: fmQueryCollideFrames(), C++: fort::myrmidon::Query::CollideFrames()

[4]

Python: py_fort_myrmidon.Query.ComputeAntTrajectories(), R: fmQueryComputeAntTrajectories(), C++: fort::myrmidon::Query::ComputeAntTrajectories()

[5]

Python: py_fort_myrmidon.Query.ComputeAntInteractions(), R: fmQueryComputeAntInteractions(), C++: fort::myrmidon::Query::ComputeAntInteractions()

[6]

Python: py_fort_myrmidon.IdentifiedFrame, R: see return value of fmQueryIdentifyFrames(), C++: fort::myrmidon::IdentifiedFrame

[7]

Python: py_fort_myrmidon.CollisionFrame, R: see return value of fmQueryCollideFrames(), C++: fort::myrmidon::CollisionFrame

[8]

Python: py_fort_myrmidon.AntTrajectory, R: see return value of fmQueryComputeAntTrajectories(), C++: fort::myrmidon::AntTrajectory

[9]

Python: py_fort_myrmidon.AntInteraction, R: see return value of fmQueryComputeAntInteractions(), C++: fort::myrmidon::AntInteraction

[10]

Python: py_fort_myrmidon.AntTrajectorySegment, R: see return value of fmQueryComputeAntInteractions(), C++: fort::myrmidon::AntTrajectorySegment

[11]

Python: py_fort_myrmidon.AntTrajectorySummary, R: see return value of fmQueryComputeAntInteractions(), C++: fort::myrmidon::AntTrajectorySummary

[12]

Python: py_fort_myrmidon.Collision.Types, R: N.A., C++: fort::myrmidon::InteractionTypes

[13] (1,2)

Python: py_fort_myrmidon.Matcher, R: fmMatcher, C++: fort::myrmidon::Matcher