The Reasons For Having Variable-Length Records And The Use Of Separator Characters

There may be many reasons for a file having variable-length records, as described by Elmasri & Navathe (2015). They are described as one field that may need to be a varying size, a field that may contain a repeating group or value, a field that may be identified as an optional field, or a file that may contain more than one record type.

The variable length record was common before relational databases were available. Before relation database management systems were invented, one used COBOL to program to read and write files system data files. In the Mainframe era, IBM invented VSAM filesystem architecture to access the data in a file. VSAM has Key Sequence data sets(KSDS), Entry sequence data sets (ESDS) , Relative record sets (RRDS), and Liner data sets (lDS) as described by IBM (2010). The description Elmasri & Navathe (2015) lays out in 16.4.2, titled “ Files, Fixed-Length Records, and Variable-Length Records,” describes how VSAM and COBOL were used. In the mainframe era, disk space was expensive, so incorporating variable-length records saved disk space was necessary. Today, the cost of a disk is very inexpensive, so these types of variable-length records may not be needed. Micro Focus(nd) discusses that variable length records are used to enable disk space to be saved. They discuss that if an application generates many small records with occasional long ones, using a fixed-length record wastes disk space, so the variable-length record would be better.

Examples Elmasri & Navathe (2015) provide author discuss the EMPLOYEE field as a variable-length field. Arrays would be an example of a repeating group that might be stored in a field. An array stored in a field would be an example of a repeating group that could grow larger or smaller in a variable-length field. An example of an optional field might occur if a check box on a user interface is checked, additional fields are used in the record, but if it is not checked, then any fields are populated. This same example could hold true for different record types. If a checkbox is checked in the user interface (UI), create a smaller record that holds the information.

A type separator character is needed with variable length records.

This way, one can identify the start and end of each record. An example of a separator charter used is a comma-separated value file or CSV. A command separates the fields, and the end of the row contains an end-of-line character.

A comma today is too standard, and this might be replaced with a tilde ~ or a pipe | that is not used as much in a text file.

References

Elmasri, R., Navathe, S. B. (2015). Fundamentals of Database Systems, 7th Edition [VitalSource Bookshelf version]. Retrieved from https://bookshelf.vitalsource.com/books/9781323139899

IBM. (2010). What Is VSAM?. Retrieved from https://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zconcepts/zconcepts_169.htm

Micro Focus. (nd). Micro Focus Developer. Retrieved from http://documentation.microfocus.com/help/index.jsp?topic=%2FGUID-0E0191D8-C39A-44D1-BA4C-D67107BAF784%2FBKFHFHORGSS014.html