10.9 Column and Keyword Filtering Specification

The optional column/keyword filtering specifier is used to modify the column structure and/or the header keywords in the HDU that was selected with the previous HDU location specifier. This filtering specifier must be enclosed in square brackets and can be distinguished from a general row filter specifier (described below) by the fact that it begins with the string 'col ' and is not immediately followed by an equals sign. The original file is not changed by this filtering operation, and instead the modifications are made on a copy of the input FITS file (usually in memory), which also contains a copy of all the other HDUs in the file. This temporary file is passed to the application program and will persist only until the file is closed or until the program exits, unless the outfile specifier (see above) is also supplied.

The column/keyword filter can be used to perform the following operations. More than one operation may be specified by separating them with semi-colons.

• Copy only a specified list of columns columns to the filtered input file. The list of column name should be separated by semi-colons. Wild card characters may be used in the column names to match multiple columns. If the expression contains both a list of columns to be included and columns to be deleted, then all the columns in the original table except the explicitly deleted columns will appear in the filtered table (i.e., there is no need to explicitly list the columns to be included if any columns are being deleted).

• Delete a column or keyword by listing the name preceded by a minus sign or an exclamation mark (!), e.g., '-TIME' will delete the TIME column if it exists, otherwise the TIME keyword. An error is returned if neither a column nor keyword with this name exists. Note that the exclamation point, '!', is a special UNIX character, so if it is used on the command line rather than entered at a task prompt, it must be preceded by a backslash to force the UNIX shell to ignore it.

• Rename an existing column or keyword with the syntax 'NewName == OldName'. An error is returned if neither a column nor keyword with this name exists.

• Append a new column or keyword to the table. To create a column, give the new name, optionally followed by the datatype in parentheses, followed by a single equals sign and an expression to be used to compute the value (e.g., 'newcol(1J) = 0' will create a new 32-bit integer column called 'newcol' filled with zeros). The datatype is specified using the same syntax that is allowed for the value of the FITS TFORMn keyword (e.g., 'I', 'J', 'E', 'D', etc. for binary tables, and 'I8', F12.3', 'E20.12', etc. for ASCII tables). If the datatype is not specified then an appropriate datatype will be chosen depending on the form of the expression (may be a character string, logical, bit, long integer, or double column). An appropriate vector count (in the case of binary tables) will also be added if not explicitly specified.

  When creating a new keyword, the keyword name must be preceded by a pound sign '#', and the expression must evaluate to a scalar (i.e., cannot have a column name in the expression). The comment string for the keyword may be specified in parentheses immediately following the keyword name (instead of supplying a datatype as in the case of creating a new column).

• Recompute (overwrite) the values in an existing column or keyword by giving the name followed by an equals sign and an arithmetic expression.

The expression that is used when appending or recomputing columns or keywords can be arbitrarily complex and may be a function of other header keyword values and other columns (in the same row). The full syntax and available functions for the expression are described below in the row filter specification section.

For complex or commonly used operations, one can also place the operations into an external text file and import it into the column filter using the syntax '[col @filename.txt]'. The operations can extend over multiple lines of the file, but multiple operations must still be separated by semicolons. Any lines in the external text file that begin with 2 slash characters ('//') will be ignored and may be used to add comments into the file.

Examples:

   [col Time;rate]               - only the Time and rate columns will
                                   appear in the filtered input file.

   [col Time;*raw]               - include the Time column and any other
                                   columns whose name ends with 'raw'.

   [col -TIME; Good == STATUS]   - deletes the TIME column and
                                   renames the status column to 'Good'

   [col PI=PHA * 1.1 + 0.2]      - creates new PI column from PHA values

   [col rate = rate/exposure]   - recomputes the rate column by dividing
                                   it by the EXPOSURE keyword value.