When selecting a Flat File type, a number of fields become available on the main screen that need populating.

A Flat File type File Definition can be used to interact with a text file that is in the style of a .CSV (comma separated value) file. In particular, these files have values across one record separated by a comma or other character, and each record is ended by a line break such as a CR/LF. Statelake Flat File Definitions provide control over the characters to be read or written as field separators and line delimiters.

This allows both single record, and hierarchical flat files. Flat Files can be either fixed length or delimited.

image-20240317-002758.png

Type Specific Fields

Field Name

Description

Record delimiter

Select the method for terminating each record in the Flat File.

Auto-detect CR/LF

Only used when reading from a source file. Automatically checks the source file for Windows-style or Unix-style delimiters (line breaks). This might override the Record Delimiter setting on this definition if a different style delimiter is found.

Separated Fields?

Select to enable delimited fields, otherwise the fields are treated as fixed length.

Field Separator

Specify the ASCII character to use as the field delimiter. Enter the ASCII character code.

Use Quoted Strings

Disable the automatic trimming of blank spaces from the start and end of strings.

Quote Char

Specify the ASCII character to use as the quote character. Enter the ASCII character code.

Use Null Conversion

Select to enable null conversion that will convert between null values in the in-memory data definition and the specified Null String in the file.

Null String

Specify the string that null values will be converted to when written to a file, or converted from when reading from the file.

Top Lines Excluded

Enter the number of lines to exclude from the top of the file when loading into the File Definition. This can be used to ignore header rows.

File Encoding

Specifies the type of encoding.

Bottom Lines Excluded

Enter the number of lines to exclude from the bottom of the file when loading into the File Definition. This can be used to ignore footer rows or trailing markers.

Write BOM

Byte Order Mark. Controls whether a file is being written as “little-endian” or “big-endian”.

Add Record Delimiter to last line

When outputting to file you may need to add a record delimiter to the last line, essentially creating a blank row on the end.

Print Header Row

Select to print a header row containing the names of the fields when outputting to file.

Use loose quotes string checking

Select to enable less strict processing of files with quote characters. You should always attempt to process quoted files with this deselected first.

Ignore Invalid Lines

Select to ignore processing of invalid lines. Invalid lines will be dropped from the file but the rest of the data will load successfully. Use with caution.

Allow quoting of any data type

Allow any data type to be quoted rather than just string data types.

Ignore Extra Fields

Select to ignore extra fields on the end of the row. If you have only defined 5 fields but there is 11 in the file this will ignore the last 6 fields without error, otherwise an error will be raised. Use with caution.

Ignore undefined records

When loading a hierarchal file into memory of the File Definition if it encounters a record that has an ID that is not defined in the Designer it will ignore the record, otherwise it will add the record to the first dataview. You should not use this option with a single record type file that does not have a record id.

Ignore Invalid Numbers

Ignore fields with invalid numbers. Any field that is defined as a number that cannot be read from the file as such will be defaulted to 0, otherwise an error will be raised. Use with caution.

Ignore null/empty records

Tick the box to skip any blank or empty records.

Ignore truncated field data

Tick the box to turn off error generation for data values in the file that are bigger than the size that has been defined in the definition.

Ignore warnings for date/time fields with invalid mask format

Tick the box to turn off error generation where the data values in the file for date/time fields do not match the mask format (Display Format setting) specified in the definition.