Data Manager

The Data Manager is used to enter new data, edit existing data, verify data using a double entry method, and print data using a form created with the Study Design program. All of the buttons on the opening screen will be disabled until a data file has been loaded.

In order to use the Data Manager, you must have first created a data manager form. It is not possible to enter new data or edit existing data without a form.

The form provides the data entry template (i.e., what the data entry person will see on the screen). The Data Manager will attempt to show the current field near the middle of the screen. This means that during data entry, the cursor will appear to remain relatively stationary near the middle of the screen, and the form will scroll after each field is entered.

There are two ways to run the Data Manager. If the data file already exists, and you want to add, edit, or delete records, select File, Open, Data File. If a data file does yet exist, you can select Data, Run Data Manager.

The following are the basic keyboard functions for record and field selections:

To create a new data file, first run the Data Manager. Then select File, Open, and type a new file name. In nearly all cases, the data file name will be the same as the codebook and form names. The program will first ask for the name of the form you want to use as your data entry template. Then it will ask for the name of the data file. The form will load and you will be able to begin entering data.

To edit or add records to an existing data file, first run the Data Manager. Then select File, Open and select the data file. If there is a form with the same name (most situations), it will be loaded and you will be able to edit or enter data. If there is not a form with the same name, you will be asked to enter the name of the form.

It is not necessary to close the current data file before choosing to edit a different data file. To change data files select File, Open and select the data file. The current data file will be closed automatically before the program opens the new file.

The current field is the one you are ready to enter data into. It will be highlighted on the form using the color scheme developed during the design of the form (although the colors can be changed with the Data Manager program).

During normal data entry, the current field will change automatically following skip patterns if necessary. The user, however, is free to manually change to any field at any time. There are many ways to change fields.

Clicking the mouse on any field will make it the current field. Clicking the mouse on the Previous Field Button or the Next Field Button will change to the previous or next field.  You can also type a field number into the Current Field window and press enter.

If the Variable List window is displayed, clicking on a variable will make that variable the current field. If the Variable Detail window is displayed, selecting a new variable will make it the current field. The Variable List and Variable Detail windows can be displayed by selecting View.

Finally, the following keys can also be used to manually change fields.

.

The Automatic Record Advance option controls whether the current record will be automatically incremented when data entry person reaches the last field in the current record.

If the Automatic Record Advance option is set, the program will automatically advance to the next record when the last field of the current record has been entered. This way, it is possible to enter a large number of records without clicking on the New Record Burton for each record. To change the Automatic Record Advance option, select Options, Data Input Settings.

There are several ways to manually change records. Clicking on the Previous Record Button or the Next Record Button will change to the previous or next record.  You can also type a record number into the Current Record window and press enter.  Finally, the following keys can also be used to manually change records.

.

There are two ways to enter a new data record. The first is to click on the New Record Button, and the second is to select Edit, Begin New Record. The current record number window will be updated to show the record number for the new record. If you do not enter any data for the new record, it will not be saved. If any data is entered, the new record will be added to the end of the data file. It is not possible to insert a new record into the middle of and existing data file.

There are two ways to view the data for a record with a known record number. The first is to type the desired record number into the current record number window and press enter (or click anywhere on the form). The second is to select Edit, Go To Record (or use the Control G shortcut).

 When finished, the selected record will be displayed, and the focus will remain on the currently selected field. The current record number window will be updated to show the record number for the selected record.

StatPac makes it easy to find data records that contain specified data.  To begin a search, click the Find Button, or select Edit, Search Data File (or use the Control F shortcut). The Find dialog window will be displayed.

After a search has been started and a match has been found, you can continue the search by selecting Edit, Continue Search (or use the [F3] shortcut key). The Find dialog window does not have to be displayed to continue a previous search, although if it is displayed, you can also click on Find Next to begin the search. Note that the search often happens so quickly that it seems instantaneous. The user can watch the current record number to confirm the record number that they are viewing following a search.

There are four components in the Find dialog window.

Field To Search

Select the variable (i.e., field) you want to search. The default will be the current variable.

Search For

This is the text or value you want to search for. If you want to search for missing data, leave this field blank.

Search Direction

Select All, Forward, or Backwards.

When the Search Direction is set to All, the search will begin with the record following the current record (e.g., if record 5 is displayed on the screen, the search will start with record 6).  If no match is found by the end of the file, the search will continue with record 1 and continue until all records have been examined.  If no match is found, StatPac will report it, and the current record will continue to be displayed. When the Search Direction is set to Forward or Backward, the search will go only to the end or beginning of the file (respectively).

Search Method

Select Exact Match, Partial Match or Soundex. 

Upper and lower case differences will be ignored for all searches regardless of the method used. If you search for JOHN JONES, John Jones will be found.

If you set the Search Method to an Exact Match, then the data must exactly match the search string (with the exception of case differences). If you search for Jones, StatPac will not find a record that contains John Jones in the search field

If you set the Search Method to a Partial Match, then the search will find records that contain the search string regardless of other text in the field. If you search for Jones, StatPac will find records that contain John Jones.

A Soundex search is one that uses the sound of the word instead of its exact spelling.  This makes it possible to find text even when there are spelling or data entry errors.  If you set the Search Method to a Soundex Search, then the search will find records that contain the sound of the search string regardless of other text in the field. If you search for Jonathan, StatPac will find records that contain John Jonethon, Fred Johnathon, Mary Ann Jonathon, etc.  

It is sometimes desirable to be able to repeat data from the previous record. For example, assume one of the variables in your study is the full city name. During data entry you would type the city name for the first record. When entering data for the city field in the second (and subsequent) records, you could duplicate the response from the previous record. The duplicate field function is disabled during double entry verification.

There are two ways to duplicate the data from the previous record. The first is to click on the DUP Button, and the second is to select Edit, Duplicate Field From Previous Record (or use the [F5] shortcut key). The duplicate function will copy the contents of the previous record to the current record for the current field.

Deleting a record is a fairly common procedure because duplicate records are often discovered in a data file.  There are two ways to delete a record. The first is to click on the DEL Button and the second is to select Edit, Delete Current Record.

StatPac will not actually delete the record at this time. Instead, the contents of all variables in the record are set to blanks (missing). In this way, the record is marked for deletion rather than actually being deleted. The rationale being that deleting a record would cause all subsequent records to move down in the data file, in effect changing their record numbers.  Since editing is often done by record number, it is important that the record numbers do not change during an editing session.

Records marked for deletion cannot automatically be undeleted.  When you mark a record for deletion, it is actually stored as blanks in the data file, so there is no way to recover the information. You can, however, re-enter the data for that record.  If you enter any data in a record marked for deletion, it will not be deleted since it now contains data.

When you exit the Data Manager, you will have the opportunity to compact the file (i.e., eliminate the deleted records).  This will change the record numbers for future editing sessions.  If you want to preserve the record numbers for the next editing session, don't compact the file.

You can also manually compact the data file at any time by selecting Options, Compact Data File. You will be asked to confirm your selection because the procedure will change the record numbers. The delete record function is disabled during double entry verification.

If you have deleted records, it is important to compact the data file before performing any statistical analysis.  Otherwise, the deleted records would be counted as missing data in the analysis.

Most data entry control parameters are specified in the Study Design program. However, a few parameters can be altered by the data entry person. Select Options, Data Input Settings to change the settings.

The Automatic Record Advance option controls whether the current record will be automatically incremented when data entry person reaches the last field in the current record.

If the Automatic Record Advance option is set, the program will automatically advance to the next record when the last field of the current record has been entered. This way, it is possible to enter a large number of records without clicking on the New Record Burton for each record.

The Disable Skips During Editing option controls whether skip codes should be active when you are editing an existing record. When this option is set, skip codes will work when you are entering a new record and will not work when editing an existing record. If you are entering a new data record, and temporarily return to a previous record, and then back to the new record you were working on, then you will no longer be entering a “new” data record. In other words, a record becomes permanent as soon as you change records (even if only some of the information was entered for that record). Skip codes will be disabled when you return to the “new” record that you were entering unless the Disable Skips During Editing is unchecked.

The final Data Input Settings let you change the colors for the fields.

When you delete a record with the Data Manager,  StatPac makes it a blank record without actually deleting the physical record from the data file. If you have deleted records, it is important to compact the file by deleting the blank records. Otherwise, the deleted records would be counted as missing data in the analysis.  To compress a file, select Options, Compact Data File. See Deleting A Record. The compact data file function is disabled during double entry verification.

StatPac performs validity and range checking on all data entered. However, some people prefer to use a Double Entry Verification method to further reduce data entry error. Using this method, all the data is first entered into a data file. Then the data is entered again and compared against the first data during the second data entry process. Discrepancies are brought to the attention of the data entry person and they are resolved in real time during data entry..

To begin Double Entry Verification open the data file to be verified and select Options, Double Entry Verification. All the existing data will be "hidden" from the user, and it will appear that you are ready to begin entering data beginning with the first record. As each field is entered it is compared with the existing data. Skip patterns will be followed as if this were the same as entering a new data record. Discrepancies will be highlighted and you will be able to specify whether to use the new data or original data. Once a field has been verified, it's data will no longer be hidden.

Obviously, Double Entry Verification works only if surveys are entered in exactly the same order as the original data entry. When operating in the Double Entry Verification mode, each time a new record is displayed, the field data will appear blank even if that record was already verified. Thus, if the data entry person stops midway through the verification process, it is important they note the record number they were working on when they stopped, so they can begin at that record when they return.

To print a data record select File, Print. The current record will be printed using the form as a template.

The Variable List window allows the user to view the variable names and variable numbers while entering data. Clicking on a variable in the Variable List window will make that variable the current variable. Double clicking on a variable in the Variable List window will activate the Variable Detail window.

The Variable Detail window give complete information on the current variable. It can be turned on or off by selecting View, Variable Detail. Double clicking on a field will also activate the Variable Detail window.

The Variable Detail window can be dragged around the screen by clicking and holding the left mouse button on any gray area in the Variable Detail window.

StatPac stores its data files in sequential ASCII fixed format with a carriage return and line feed at the end of each record. This is sometimes referred to as a flat ASCII file. The data file also contains an end of file mark (Ctrl Z or ASCII 26) at the end  of the file.  This is the most universally accepted data file format, and many data base managers refer to it as SDF format

When using data created by another program, you can determine if it is an ASCII file by loading into a text editor. If the data does not appear as garbage, it's probably ASCII.  If all records appear to contain the same number of characters, it's probably fixed format.  Data in any other format must be imported into StatPac.

A "record" or "case" in StatPac is defined as a string of characters terminated with a carriage return and line feed.  Fixed format means that all records are exactly the same length (contain the same number of characters) as all other records.

Two different fixed format data files are shown as follows:

and so on ....

Notice that a data file is just a series of numbers (or letters).  All the records are always the same length.  In this example, the single variable file uses two columns per record and the multiple variable file uses ten columns per record (with an unknown number of variables).  The end of each data record contains an invisible carriage return and line feed, so each record actually contains two more characters per record (one for the carriage return and one for the line feed).

If you already have a data file stored in sequential ASCII format, you can use it with StatPac by giving it a .dat extension.  You only need to set up a codebook to match the format of the data file.  No other changes are necessary.

Most data downloaded from a mainframe, read-in from a scanner, or received from a data entry company will be in sequential ASCII format.  To use this data with StatPac, perform the following three steps:

1. Set up a study design with StatPac that exactly matches the format of the data file.  The format of the study design must exactly match the format of the data you will be using.  If the data file contains unused columns, dummy variables should be created in StatPac to "pass-over" the blank portions of the data record.

2. Copy the data file to your work subdirectory. Then rename it to the same name as the codebook except with a .dat extension.

If you need to use a data file that is not stored in sequential ASCII  format, you must import it into StatPac. The Analysis program has a utility to import most foreign data files into StatPac's format.  This makes it easy to exchange data with other software packages, including most data base managers and spreadsheets.