Finding VHIST Files

Introduction
Example Queries
Syntax

Introduction

vhistzard contains a tool for searching a directory for VHIST files matching some requirements pattern. These requisitions can be as simple as

The VHIST file must contain the word Co-Registration

and as complex as

The VHIST file must contain a workflowstep with the title
Co-Registration, as well as two files with filenames
matchin the patterns patient_*.img and patient_*.img.hdr.
The first file should be larger than 1.5MB and both files
must be part of the workflowstep. Moreover, the workflowstep
must be newer than 2010-05-06.

The prerequisites are described in the form of a simple query language, which is explained in the following sections.

Example Queries

To begin with, we will discuss a few example queries to understand the syntax of the queries.

The query

Co-Registration

searches for all VHIST files, which contain the word "Co-Registration" somewhere. If you want to search for VHIST files, which contain both, the word "Co-Registration" as well as the word "Filtering", simply type

Co-Registration Filtering

You can also write

Co-Registration && Filtering

instead. If only one of the two terms must appear in the VHIST file to match the requirements, the sign || can be used:

Co-Registration || Filtering

To search for a term which contain multiple words, place the words within single or double quotes.

"Filtered Image" 'Filter Method'

will search for VHIST files, which contain the phrases "Filtered Image" and "Filter Method".

Sometimes, you want to query the values of specific attributes inside a workflowstep or file within the VHIST file. For example, you may want to find all VHIST files containing a section with the title "Learn Parameters". This prerequisite can be described with the following query:

:title == "Learn Parameters"

Several requirements of this nature can be combined with ||, && as mentioned before. The && sign can be omitted.

:title == "Learn Parameters" && :filename == parameters.txt

This query searches for VHIST files which contain a workflowstep with the title "Learn Parameters" as well as a file with the name parameters.txt. If several properties should be true for the same section or file, you can place a variable name in front of the colon:

x:title == "Learn Parameters" x:comment == "Method X was used"

This query will find all VHIST files which contain a workflowstep with the title "Learn Parameters" and the comment "Method X was used".

There are several ways to compare the value of a property with some other value. The operator %contains tests if a property contains a given string, whereas the operators < and > can be used to compare numbers, filesizes and dates:

x:title %contains Evaluation && x:timestamp > 2010-03-13 && y:filename == results.txt && y:filesize < 2mb

In fact, the first query described in this section is just a shorter, more convenient form for

:any %contains Co-Registration

A detailed description of all properties and comparisson operators can be found in the following sections.

Syntax

This section describes the syntax of vhistzard find queries.

Variables

A variable is a placeholder for one workflowstep or file within a VHIST file. You can use a variable to access and compare all attributes of a file or workflowstep in the VHIST file. A variable name can contain any character from the latin alphabet. Valid variable names are x, foo or SomeVariableName, but not @xyz, file12 or äöü. A query can contain any number of variables. It is important to note that two variables in one query will never refer to the same workflowstep or file. This means that the query x:title = 'foo' && y:title = 'foo' requires a VHIST file to contain two distinct workflowsteps, both having the title 'foo'. Also keep in mind that variable names are case insensitive, so abc, ABC and aBc all denote the same variable.

Attributes

Variables are always used in combination with an attribute which will specify the property of the file or workflowstep you want to query. Variable and attribute are joined by a colon (:). Attributes can contain the same characters allowed in variables as well as numbers. Valid variable/attribute pairs are x:title, foo:embedded or vAr:AtTrIbUt123. Like variables, attributes are case insensitive.

Pre-defined Attributes

There is a list of pre-defined attributes for both workflowsteps and files. The second column of the following tables denote the datatypes of the attributes. A description of these types can be found at Datatypes.

Name	Datatype	Description
tagid	number	A unique id for the workflowstep
vhistrootfile	string	The name of the `VHIST` root file
timestamp	string	The date when the workflowstep was appended to the `VHIST` file
title	string	The title of the workflowstep
description	string	The description of the workflowstep
comment	string	The comment of the workflowstep
command	string	The commandline command used during workflowstep
tool	string	The tool used during the workflowstep
toolpath	string	The path to the tool used
host	string	The name of the machine on which the workflowstep was performed
user	string	The name of the user who performed the workflowstep
creator	string	The tool which created the `VHIST` workflowstep
iswfs	boolean	Always `true`
isfile	boolean	Always `false`
userattrs	string	A newline-separated list of all user attribute keys
any	string	Contains the values of all non-boolean attributes. Can be used in combination with %contains

Name	Datatype	Description
tagid	number	A unique id for the file
filename	string	The name of the file without the path
filepath	string	The path of the file
filetype	string	The type of the file
description	string	The description of the file
comment	string	A comment about the file
lastmodified	string	The date when the file was last modified
embedded	boolean	`true` if the file has been embedded in the VHIST file
compressed	boolean	`true` if the file has been embedded in compressed form
filepurpose	string	Either Infile or Outfile
filesize	number	The size of the original file
cfilesize	number	The size of the file after compression
md5	string	The md5 sum of the original file
cmd5	string	The md5 sum of the compressed file
iswfs	boolean	Always `false`
isfile	boolean	Always `true`
parentid	number	The tagid of the workflowstep containing this file
userattrs	string	A newline-separated list of all user attributes
any	string	Contains the values of all non-boolean attributes. Can be used in combination with %contains

User-defined Attributes

Next to the pre-defined attributes, users can define their own attributes when creating VHIST files. User defined attributes can be accessed within a vhistfile expression by prefixing them with user:. Examples of user-defined attributes might be x:user:kernelsize or y:user:parameter1. If a user defined attribute contains spaces or special characters, you must use the slightly more complicated syntax x:user{filter method}. All user-defined attributes are handled as strings. However, vhistfind allows you to use number and date comparisson operators as well. Keep in mind, that these operations may fail if your user-defined attribute's value can not be converted to a number/date.

Datatypes

vhistfind expressions support three different kinds of datatypes: strings, numbers and booleans.

Strings can contain any kind of text. Most attributes are represented by strings. If a string is used within a predicate which expects numbers or dates, the string will be automatically converted into the appropriate datatype. If the conversion fails, the predicate will fail as well.
Numbers represent integers or real numbers. Numbers are a special subset of strings, therefore all string predicates can be used with numbers. Additionally, numbers can be compared using <, <=, >, >= as well as a fuzzy comparisson for real numbers. Numbers support the three postfixes kb, mb and gb, which are useful when comparing filesizes, e.g. x:filesize < 10mb.
Dates refer to a point in time and can be used to compare timestamps. A date either has the form 2010-03-14, which refers to the 14th of March in 2010, or may also contain a specific time, like "2010-03-14 20:34:00". Keep in mind that a date with a time must be quoted, otherwise, the date will be split at the space character into two strings. Dates can be compared to other dates using the operators <, <=, > and >=. Note that the date 2010-03-14 is equivalent to "2010-03-14 00:00:00". This means, that the comparisson 2010-03-14 < "2010-03-14 00:00:01" yields true as a result.
Booleans contain one of the two values true or false. Booleans are returned by all predicates and used as input for boolean connectives. Booleans can not be converted into strings and vice versa. If you want to turn a string into a boolean, use an expression like this: x:user:normalized = true or x:user:normalized = 1, depending on the value you assigned to the attribute normalized.

Predicates and Connectives

Both, the equality operator (=) as well as the *inequality operator* (!) are used to test two string values for equality or inequality.
Examples:
```
x:title == "my workflowstep"
```
The ordering predicates <, <=, > and >= can be used to compare numbers or dates.
Examples:
```
x:filesize <= 10mb, x:user:kernelsize > 5, y:timestamp < 2010-03-04.
```
%isapprox can be used to test if two real numbers are approximately equal. Note that comparing two numbers with = will perform a string comparisson. Therefore 1.2 = 1.200 will return false, whereas 1.2 %isapprox 1.200 will return true
Examples:
```
x:user:pi %isapprox 3.1415
```
%contains tests if a string contains another string.
Examples:
```
x:userAttrs %contains "filtertype"
```
%matches tests if a string matches a wildcard pattern.
Examples:
```
x:filename %matches *.txt
```
Connectives and (&&) and or (||) can be used to check if all or any subexpression evaluates to true.
Examples:
```
x:filename == "foo" && x:comment != ""
x:filename == "logfile.txt" || x:filename == "logfile.log"
```
The negation operator not (!) negates the value of a boolean value.
Examples:
```
!x:embedded
```