Data Definitions
A Statelake Definition defines a data structure for use as input or output. The data structure will include at least one record and the field or fields in each record.
A Statelake Definition can define complex structures that include multiple records in various relationships, such as invoice header and associated invoice lines.
A Statelake Definition also handles the conversion of records and fields to and from data formats such as CSV, XML, SQL, and JSON.
A Data Definition can be changed easily by you as your data structure changes as a result of system upgrades and customisation.
A Data Definition is an in-memory representation of the data you are going to read or write.
When reading data, the definition is opened against the data structure and data read from the structure into memory. The data is retained in memory until the definition is closed.
When writing data, the definition is opened against the structure but no data is read. This provides an in-memory representation of what the data structure is. You then write data into the definition using a map, and this is subsequently converted to the appropriate format when writing the data into the structure. Even after writing the data, the data is retained in memory until the definition is closed.
DB (Database) Definitions Overview
DB Definitions complement DB Connections. Once the data source and/or destination locations have been specified through the DB Connections module, Statelake needs to know what the data structure is, to ensure that the proper fields can be accessed and the data correctly either extracted from the database, or written to the database. This is achieved through the DB Definitions module.
Sitting under the Data Definitions umbrella, Statelake uses the DB Definitions module to define these data structure components, and to design the database queries for both reading the data out of, and writing in to, the databases.
DB Definitions allow you to describe the structure of your data to the Statelake application. This means defining the exact information to be extracted, and where appropriate, also identifying how the output file is to be constructed. This allows Statelake to understand your particular data structure and read and write to it. This gives you the flexibility to interact with any data structure as you are in complete control over the definition.
A DB Definition is an in-memory representation of the data you are going to read or write from the database.
When reading data the definition is opened against the database and data read from the database into memory. The data is retained in memory until the definition is closed.
When writing data the definition is opened against the database but no data is read. This provides an in-memory representation of what the data structure is. You then write data into the definition using a Map - this is subsequently converted to appropriate Insert/Update statements when writing the data into the database. Even after writing the data to the database, the data is retained in memory until the definition is closed. If the act of writing to the database has caused auto-generated fields to create values, these values can be returned to the in-memory dataset.
You can create a single DB Definition and use it against multiple similar databases simply by assigning a different or unique DB Connection to the DB Definition.
When assigning a DB Connection to a DB Definition, it is considered to be a design-time assignment.
When coming to use the DB Definition in an Action you have the chance to re-assign the DB Connection which is known as a run-time assignment. This allows you to use the same DB Definition in multiple Actions each with their own DB Connection.
Please refer to DB (Database) Definitions for more information.
There are a few differences in the behaviour of a DB Definition that is used to extract or import data as a source definition, when compared to a DB Definition used to write or export data out as a destination.
So it is most important when designing your definition that the purpose is known and understood. The DB Definition should be designed at the outset as use for either a source definition or a destination definition.
File Definitions Overview
The File Definitions module is used to design the structure of the files that are either read from, or written to the Statelake configuration.
File Definitions allow you to describe the structure of your files to the Statelake application, thereby allowing Statelake to understand your particular file structure to correctly read data from it, and write data to it. This gives you the flexibility to interact with any file, as you are in complete control over the definition.
A File Definition is an in-memory representation of the data you are going to read from, or write to, the file.
When reading data the definition is opened and the data loaded from the file into memory. The data is retained in memory until the definition is closed.
When writing data, the definition is opened against the database but no data is read from the file. This provides an in-memory representation of what the data structure is. You then write data into the definition using a Map, and this is subsequently output in the appropriate format to the file at the specified location. Even after writing the data to the file, the data is retained in memory until the definition is closed. If the act of writing to the file has caused auto-generated fields to create values, these values can be returned to the in-memory dataset.
There are four types of files that can be specified in this module:
Flat text files - any structured text file either fixed length or delimited.
XML files - any XML based file that can be described using XML Schema (XSD).
Excel files - binary Excel files.
EDI (EDIFACT) type files - any UN/EDIFACT or ANSI based files.
You can create a single File Definition and use it against multiple similar files simply by assigning a different File Connection to the File Definition.
When assigning a File Connection to a File Definition, it is considered a design-time assignment.
When coming to use the File Definition in an Action, you have the chance to re-assign the File Connection - this is a run-time assignment. This allows you to use the same File Definition in multiple Actions each with their own File Connection.
Please review File Definitions for more information.
SOAP (Web) Definitions Overview
SOAP Definitions allow you to describe the structure of web services to the Statelake application.
This allows Statelake to understand your particular web service structure to read from it, and write data to it. This gives you the flexibility to interact with any web service as you are in complete control over the definition.
A SOAP Definition can be used to describe web services that are based on Web Service Definition Language, or WSDL.
A SOAP Definition is an in-memory representation of the data you are going to read from or write to the web service.
A web service is a special case in comparison to DB Definitions and File Definitions - a SOAP (Web) Definition has both a request and a response definition. Both the request and response data is retained in memory until the definition is closed.
When reading data you define the request which is sent to the web service, and the resulting response is loaded from the web service into memory. In this case the request is generally a set of parameters that determine the data to be responded.
When writing data you again define the request which is sent to the web service, and the resulting response is loaded from the web service into memory. But in this case the request is generally the data to be written to the web service and the response is the result of writing that data. You write data into the request definition using a Map, and the data is output in the appropriate format for the web service. Even after writing the data to the web service, both the request and response data is retained in memory until the definition is closed.
You can create a single SOAP Definition and use it against multiple similar web services simply by assigning a different SOAP Connection.
When assigning a SOAP Connection to a SOAP Definition it is considered a design-time assignment.
When coming to use the SOAP Definition in an Action, you have the chance to re-assign the SOAP Connection which is called a run-time assignment. This allows you to use the same SOAP Definition in multiple Actions each with their own SOAP Connection.
For more information, please see SOAP (Web) Definitions.
JSON Definitions Overview
JSON is an acronym for JavaScript Object Notation - a set of rules for encoding a data structure as a string of text characters.
Specific characters, such as square brackets [ and ], and braces { and }, are used to mark the structural aspects within the string.
The advantage of encoding data into a relatively plain string text format of characters such as JSON is that it can be passed easily between a web browser and web server, or can be read from or saved to a text file on disk, making it an ideal data‑interchange language.
More information can be found at JSON Definitions.