Notes on Adding a New Sensor

This set of notes resulted from the incorporation of a new sensor representation for a Radar Homing and Warning (RHAW) for the Horizontal Fusion Investigation (HFI). It started as a task list for the programmers and retains much of that flavor; I trust this will not distract from the value of the document to future users.

The HFI project required an alternate RHAW representation from the ESM model previously used. In particular, it was assumed that the receiver specifications would be inadequate to populate the data required for the original ESM model. On the other hand, new inputs for error rates and types were required to provided realistically inaccurate input data to the sensor fusion engine. Thus, the new model had points of both higher and lower fidelity than the original model. It wasn’t better or worse in a general sense, just different (and in this case better suited to the study). This is common when new representations are added. For this reason it was decided to create a new RHAW representation while keeping the old representation for other studies where the original model was better suited.

It should be noted that developing an additional RHAW representation while maintaining the old representation resulted in permitting new code to be developed and tested without disabling previous functionality and scenario elements. It is therefore generally recommended to develop an additional representation that can be incorporated side-by-side with the old representation whenever the new representation is not clearly superior. When the new representation is clearly better and it’s decided to update the old model the programmer must remember to develop and distribute a database patch to accompany to adaptation of the new model so users can continue to operate without data failures.

In this study not only was a new sensor model desired, but it was decided to develop a new sensor message family. In JFORCES the sensor messages are supported in separate code from the sensor representation. So this project had the following upgrades:

  1. A new sensor representation module

  2. A new sensor message family API.

These will be discussed separately (as they were coded).

Developing a New Sensor Representation.

After defining the requirements for the new sensor representation the following steps must be addressed to add a representation:

  1. Add new constants to the Autogen files

  2. Alter DB schema as required

  3. Add any ancillary data to the database(s)

  4. Generate DB patch files

  5. Alter database prototyping to accommodate defining the new sensor

  6. Alter scenario generation as required to accommodate new sensor tasking

  7. Alter DB support utilities as required to fully support the new sensor. This includes altering the utility to copy, archive and restore subsystem and platform definition.

  8. Create runtime structures to maintain the sensor characteristics and operational tasks as required

  9. Alter the data analysis database schema (or other output formats) as required to support the new sensor.

  10. Create a routine to initialize the runtime environment from the database specifications. This includes sensor characteristics, operational tasks, and any instance-specific information.

  11. Create a runtime routine to actually model the sensor and generate sensor-specific analysis outputs as required.

  12. Link the sensor representation code into the simulation processing flow.

  13. Update Documentation!!!!

Each of these is addressed below.

Add new constants to the Autogen files

Add a new mnemonic with to the “Sensor Types” constants category as defined in the C_FOR_DECLARATIONS table. The associated integer value must be the next available value within this category. You’ll also want to check the value of the “maxsensortypes” variable in this table to verify the new sensor type won’t exceed (or equal) this value.

Alter DB schema as required (continued on next page)

Figure 1 – JFORCES Sensor DB Schema

Figure 1 is the JFORCES scenario DB schema as it existed prior to this effort. The basic logic is:
!) the master reference table is subsyschar. This is a table for all subsystems, not just sensors. If the type field in this table is set to ‘SENSOR_SS’ (a.k.a. 1) then the row is known to reference a sensor, so the top-level data for the sensor is retrieved from sensorchar based on a matching subsystemname. Note that there are equivalent table types for all other non-trivial subsystem types (e.g. armamentchar). The information is further broken down according to the type of sensor (the sensortype flag in the sensorchar table – this must be set to the values defined in the “Sensor Types” constants category in step 1). So any of these ancillary tables can be ignored for a new sensor type – just create your own table (or series of tables as appropriate). Note that the principle key field in any new sensor-specific table MUST be named “subsystemname” and correspond to the like-named field in sensorchar. Some of the DB utilities rely on this. That’s why the sensor-specific tables all contain a field with this name. Those tables with a principle key field name of subtab are generic table look-up tables that are optionally used by the sensors.

Note that the current sensor schemas can be looked up on up-to-date development machines by entering “pgaccess tactical” and then going to schemas. It is CRITICAL to update this schema display and broadcast it after the schema itself is updated. It is also important to update the ddtablelist table in this database (which describes the tables) and the ddcollist table also in this database (this table describes each column) and broadcast the updates to these tables. To date there’s no automated way to do this.

!! NOTE !! it is imperative to update the ENTITY_RELATIONSHIPS table with the new schema as many of the database utilities rely on the contents of this table! You should at least have a row specifying sensorchar as the master and any new tables as the slave.

Add any ancillary data to the database(s)

Always add a row to the sensortypes table in the scenario database. This is generally all that is required. But some sensors (e.g. the legacy E3 radar model) requires special data that should be stored in the database to simplify maintenance. These data should be identified and populated as required.

Generate DB patch files

After the DB schema changes are made a patch file consisting of both DB schema changes and any supporting data (e.g. the atmospheric absorption table for the legacy E3 radar model). Make sure this includes the new row in the sensortypes table (make sure to do cleanup so you don’t end up with multiple identical rows in the sensortypes table when the patch is run multiple times). When creating this patch you should try to:

  1. Avoid corrupting any prior scenario data in ANY user’s database. Since you will not have access to all users’ databases you will have to plan this carefully.

  2. Try to make the patch run successfully even when repeated.

  3. Design the patch to use the pg_apply_patch utility

Since user data integrity is paramount, always first test your script against a local test database (e.g. test) that you won’t mind losing, and then test the patch using the pg_apply_patch utility against all of your databases. Perform some regression tests (e.g. running preexisting scenarios to verify they still run correctly).

While not technically a patch file, verify the C_FOR_DECLARATIONS, C_FOR_STRUCTS and ENTITY_RELATIONSHIPS files in $SCRIPT_DIR are correct and checked into CVS. The updates will not work correctly until users update and load these files.

Alter database prototyping to accommodate defining the new sensor

Before starting, the basic point to the next steps is to make certain that any functionality supported generically for sensor types is supported for the new sensor type. The programmer should try to identify this functionality by doing a grep through the code and script directories (e.g. “cd; grep -lR IRDETECT * and then look at any results and evaluate what should be done to support the new sensor type.) What follows covers the items that needed to be accommodated at the date of this document, but the grep might find functionality added since that time.

To add a new sensor to the DB Prototyping interface for creation, editing and deletion create new branches in the $TCL_LIB/dbp_sensor.tcl file. Mimic the code developed for the IRDETECT sensor type (modified as appropriate).

Alter scenario generation as required to accommodate new sensor tasking

This probably doesn’t apply to most new sensor types but some (e.g. space based sensors) might have unusual tasking limitations. In this case modify or add a new operational task as required. To do this modify the following:

  1. If required, add a new operational task ID in the “Operational Tasks” section of constants. This name should start with OPTASK_ (e.g. OPTASK_SENSOR_TASK).

  2. Create a new operational task structure in the opertask files defined in C_FOR_STRUCTS. Reference the sensor_task structure for ideas.

  3. Create any required tables to define the operational task. Reference table sensor_tasking for ideas.

  4. Add scenario generator interface code permitting user to create, edit and delete new tasks of this type in the $TCL_LIB/ps_optasks.tcl file. Again, reference the use of the sensor_tasking table for ideas.

  5. Create a routine to read and store any operational tasks from the database in the simulation in the $SIM_DIR/initial/init_optasks.c file.

  6. Create the tasking routine itself.

  7. Link the tasking routine into the simulation process through one or more of the following (as appropriate to the task):

    1. For prescheduled events stack the event at initialization (see init_sensor_tasking in $SIM_DIR/initial/init_optasks.c)

    2. For events to occur at a specific location put an entry in $SIM_DIR/objutil/optaskcheck.F

    3. Any other cases you’ll have to handle on a case-by-case basis.

  8. Create and link and data collection procedures and associated tables as required

Alter DB support utilities as required to fully support the new sensor.

Currently the utilities will not require any modification as long as the ENTITY_RELATIONSHIP table is updated. But look around to make sure. To date the key utilities are the DB Prototype archive and restore utility, found in the $TCL_LIB/dbp_copy.tcl file.

Create runtime structures to maintain the sensor characteristics and operational tasks as required

Generally you’ll want to create a structure to contain the sensor characteristics in $COMMON_DIR/ This information is equivalenced to the gain array in $COMMON_DIR/sensor.cmn, so (assuming you don’t need to use this gain array directly) you have at least 720 bytes of storage shared between C and Fortran. Trace the use of the ir_sensor_params through $SIM_DIR/*/*.c to see how it’s used. You might also want to create fortran names for the structural elements, in which case check out the equivalence statements in $COMMON_DIR/sensor.cmn for examples.

Alter the data analysis database schema as required to support the new sensor.

This is not usually required, but as part of the checklist see if you want to save any new data in either the database for data analysis or in flat files and create structures to do this as required. Remember that any changes the data analysis tables need to be reflected in the $SCRIPT_DIR/DA_TABLE_LIST and $SCRIPT_DIR/DA_TABLE_FIELDS files as some of the autogeneration procedures and other utilities rely on these files (and the loaded DB tables) being accurate.

Create a routine to initialize the runtime environment from the database specifications.

All of this code should go in $SIM_DIR/initial/init_sensor_ss.c file. First, put in a case for the new sensor in the initsenssys routine in this file. This should branch to a new procedure, which the programmer writes as required for the sensor specifics. See any of the branch routines (e.g. init_los_radar) in this file for templates.

Create a runtime routine to actually model the sensor and generate sensor-specific analysis outputs as required.

This is the heart of the application – determining if sensor A can see target T. There are two call procedures here. If the routine is called on a cyclical basis you’ll find it easiest if to use one of the established routines found in $SIM_DIR/sensor/detection.c as a template. In this case you’ll also want to start off using the “STD_CALL” arguments (defined in this file) as the core call arguments, though this list can be added to. Or you can use any other arguments if you wish. The created detection routine should return an integer value with 1 indicating both a detection and correct communications. The created routine is responsible for calling the “sensormsgs_” routine whenever a detection occurs; this routine will alter the detection flag from 1 to a negative value if communications fail. The routine should return a 0 if the target is outside field of view. Any other integer result is interpreted as a failure. If you use the following default result codes (where appropriate to the sensor) the data analysis package will not require any modification to correctly reflect the success of the sensor:

9 terrain masking
8 in velocity notch
7 combined noise too high
6 small signature denied detection
5 nuclear detonation noise denied detection
4 continuous wave jamming noise denied detection
3 wx denial of detection
2 smart jammer denied detection
1 detection
0 outside field of view
-2 Comm Error - low probability of correct message reception
-3 Comm Error - exceeded bandwidth

You’ll probably want to use one of the established 1-on-1 detection routines in $SIM_DIR/detection.c (e.g. losradardet) if you’re modeling a cyclical sensor detection attempt (like a surveillance radar). By using this structure a number of fast filters are already implemented and data collection maintenance will be prepared and completed by higher level routines.

Link the sensor representation code into the simulation processing flow.

There are at least two categories of sensors detection types that each have their own link procedures. In the more common sensor detection technique detection attempts are made on a cyclical basis. In general the detection for any sensor of this type is based upon stacked events per sensor type. The first of these calls is set up in the $SIM_DIR/initial/init_sensor_ss.c file in the initsenssys routine when a new sensor type is created. The invocation of this event generates a call to prelimsensor.F, which then:

  1. Determines which sensors are currently on

  2. Determines which way the sensors are pointing. This is how sensor tasking is handled; there’s no need to write a non-cyclical detection flow for tasked sensors. The sensors’ field of regard and operational status will be updated automatically.

  3. Performs data collection maintenance.

  4. Creates a list of targets within field of view.

  5. Calls the detection routine for each 1 sensor versus 1 target pair

  6. Restacks the detection attempt routine for subsequent calls.

In step 5 the detection routine in $SIM_DIR/sensor/detection.c is called. This routine calls the appropriate phenomena-specific detection routine for each sensor type. To add a new sensor type just add a call to the sensor-specific detection routine just add a new case in the switch found in this procedure.

There’s another case of sensor detections where the detection attempt can only occur when a specific event occurs. An example would be a fire-finder radar attempt, where the detection attempt can only occur when somebody fires. In this case the 1-v-1 detection routine needs to be called by the event model. The call can either be direct or via a message, though a message call is preferred as the representative model can then be distributed, permitting confederation with other simulations. In this example the fire finder model (routine detectenemyfire_ in $SIM_DIR/sensor/detectenemyfire.F) is called as a result of a message stacked in $SIM_DIR/engment/artillerysalvo.F for every shot. The particulars of the call need to be resolved on a case-by-case basis, but it should be noted that special care should be employed to make sure the data collected for analysis is correctly initialized and recorded since the maintenance utilities incorporated in prelimsensor.F are unavailable.

Update Documentation!!!!

This is key, especially since so much of our code relies on scripts employing what in most systems is considered static documentation.

Here are the key rules and points:

  1. USE PSEUDOCODE! The standard is published on the JFORCES website under the programmer area. Use it!

  2. Verify the following tables are updated:

    1. C_FOR_DECLARATIONS (constant values)

    2. C_FOR_STRUCTS (structure definitions)

    3. ENTITY_RELATIONSHIPS (table mapping – CRITICAL!)

    4. DA_TABLE_LIST (data analysis table descriptions)

    5. DA_TABLE_FIELDS (data analysis table field-level descriptions)

    6. DDTABLELIST (internal DB table descriptions)

    7. DDCOLLIST (internal DB column descriptions)

  3. The database schemas found in pgaccess. Under this effort we will try to come up with a way to distribute the schema to users via CVS or email.

  4. In addition, there’s significant programmer-level documentation for sensors. Check out the "Programmer's Area" section of the website. The core of this document references the 1990 baseline manual, so I don’t consider updating this as important as the above items, though it’s nice to have BUT, if anything unusual is done and you think its worthy of special attention, please write up a description and put it on the website under programmers area->algorithm notes (or other sub section you deem appropriate). Note that there’s already a writeup for the Detailed SBIRs sensor. You might want to review this to see what I deemed important for this sensor type.

Developing a New Sensor Message Family Representation

In addition to developing a new sensor model, a new sensor message representation was required for this effort. Fortunately, this is easier than generating a new sensor representation. The steps to implementing a new sensor message family are:

  1. Create a message mnemonic in the “Sensor Formats” constants in C_FOR_DECLARATIONS. Make sure the value is unique within the “Sensor Formats” category.

  2. Create a patch file to insert a new line into the sensormsgs DB table (first deleting any prior references that match on name or value). Distribute this file

  3. Create a routine to generate a message of the right format. This routine can contain more than one type of message. See $SIM_DIR/sensor/msgjssfrmt.F for a template. In this case a decision was made to use data corresponding the actual message format but not packed identically. The routine can use this approach or accurately format and pack the message as appropriate to the application.

  4. Put a case for the new message family in $SIM_DIR/sensor/sensormsgs.F.

  5. Verify that the data required to generate the message is available in the new routine; fix as required.