Description Rules for Format Definition File¶
Table of contents
Common notation for format definition files¶
This section describes the common description rules for format definition.
Character code¶
The character code of the format definition file is UTF-8
.
Literal notation¶
When a literal is configured in the configuration value, follow the rules below.
Literal type | Description |
---|---|
String | Enclose the value with Note that Unicode escapes or octal escapes are not supported.
|
Decimal integer | Described in the same way as a Java numeric literal. Note that decimals are not supported.
|
Boolean | Configure as true or false . (Uppercase letters are allowed) |
Structure of the format definition file¶
The format definition file is mainly composed of the following two sections.
Directive declaration section: | |
---|---|
This section defines common configuration such as the data format (fixed-length or JSON etc.) used and encoding. For details, see Definition of the directive declaration section. |
|
Record format definition section: | |
Defines contents of the record. Specifically, the field definition in record, data type and data conversion rule for each field are defined. For details, see Record format definition section. |
Definition of the directive declaration section¶
Directive list that can be used in common¶
Directive definitions used in all data formats are as follows.
Directive | Description |
---|---|
file-type required |
Specify the data format The following data formats can be specified as standard.
|
text-encoding required |
Specify the encoding to use when reading and writing string fields. Only character encodings that are available to JVM used can be specified. For example, specify If JSON is specified for file-type, only the following encodings can be specified
If XML is specified for file-type, the encoding specified in the XML declaration section takes precedence over this configured value. |
record-separator optional |
Specify the record end character (Carriage return and Line feed). This is This configuration value is not used when the file-type is JSON or XML. |
List of directives that can be specified in fixed (fixed-length) format¶
The directives used for data in the fixed (fixed-length) format are as follows.
Directive | Description | ||||||||
---|---|---|---|---|---|---|---|---|---|
record-length required |
Specify the byte length of one record. | ||||||||
positive-zone-sign-nibble optional |
Specify the plus sign to be configured in the zone section of the signed zoned decimal as a string in hexadecimal notation. By default, the following values are used depending on the value of text-encoding.
|
||||||||
negative-zone-sign-nibble optional |
Specify the minus sign to be configured in the zone section of the signed zone numeric as a string in hexadecimal notation. By default, the following values are used depending on the value of text-encoding.
|
||||||||
positive-pack-sign-nibble optional |
Specify the plus sign to be configured in the sign bit of the signed pack numeric as a string in hexadecimal notation. By default, the following values are used depending on the value of text-encoding.
|
||||||||
negative-pack-sign-nibble optional |
Specify the minus sign to be configured in the sign bit of the signed packed decimal as a string in hexadecimal notation. By default, the following values are used depending on the value of text-encoding.
|
||||||||
required-decimal-point optional |
Specifies if a decimal point is required for unsigned and signed numeric. If If The default option is to add a decimal point ( |
||||||||
fixed-sign-position optional |
Specifies if the sign position of a signed numeric is to be fixed. If the sign position is fixed ( The default option is fixed (
|
||||||||
required-plus-sign optional |
Specifies if a plus sign is required for the signed numeric. When The default option is not added ( |
An example is shown below.
#
# Directive definition section
#
file-type: "Fixed" # Fixed-length file
text-encoding: "ms932" # Character encoding of the string type field
record-length: 120 # Byte length of each record
positive-zone-sign-nibble: "C" # Plus sign of zone numeric
negative-zone-sign-nibble: "D" # Minus sign of zone numeric
positive-pack-sign-nibble: "C" # Plus sign of pack numeric
negative-pack-sign-nibbleL "D" # Minus sign of pack numeric
required-decimal-point: true # With decimal point
fixed-sign-position: true # Sign at the beginning
required-plus-sign: false # Plus sign is not added
List of directives that can be specified in variable (variable length) format¶
The directives used for data in variable (variable length) format are as follows.
Directive | Description |
---|---|
field-separator required |
Specifies a separator character for the field (item). For example, specify |
quoting-delimiter optional |
Specifies the character used to quote field (item) values. For example, use When the value is configured, all fields (items) are quoted during output. By default, quotes are not used. If the value is configured, the quote characters before and after the field is removed during input. For handling Carriage return and Line feed and quote characters in the fields, see RFC4180. |
ignore-blank-lines optional |
Configures if blank lines should be ignored when reading the data. When configured to By default, blank lines are ignored. |
requires-title optional |
Configures if the first record should be read or written as a title. When configured to By default, the first record is not treated as a title. For the layout definition of the title record, see title-record-type-name directive. |
title-record-type-name optional |
Configures the record type name of the title. If not specified, the record type name of the title will be The title record is edited according to the record format definition associated with the record type name specified in this directive. For the sample format definition file using the title record type, see title record format definition example. For details of record types and record definitions, see Record format definition section. |
max-record-length optional |
Specifies the number of characters in a record that can be read. When data that does not have a record separator character (corrupted data) is read, if all records are expanded on the heap, the process may terminate abnormally due to insufficient heap. Therefore, if the record separator character does not exist even after reading the number of characters configured in this directive, the reading process is aborted, and an exception is thrown as invalid data. By default, configured to 1,000,000 characters. |
An example is shown below.
#
# Directive definition section
#
file-type: "Variable" # Variable length file
text-encoding: "utf-8" # Character encoding of the string type field
record-separator: "\\r\\n" # Carriage return and Line feed
field-separator: "," # CSV
quoting-delimiter: "\"" # Enclose items in double quotes
ignore-blank-lines: true # Ignore blank lines
requires-title: false # No title record
max-record-length: 1000 # Maximum 1000 characters for this csv record
Directive list that can be specified in JSON format¶
There are no directives specific to the JSON data format.
An example is shown below.
file-type: "JSON" # json format
text-encoding: "utf-8" # Character encoding of the string type field
Directive list that can be specified in XML format¶
There are no directives specific to the XML data format.
An example is shown below.
file-type: "XML" # xml format
text-encoding: "utf-8" # Character encoding of the string type field
Record format definition section¶
The record format definition section configures the definition information of the fields (items) that make up the record (such as position and data type in the record).
An example of record format definition is shown below.
- Points
- A record type name, to identify the record, is defined by enclosing with
[
,]
brackets. - Record type name must be unique in the format definition file.
- The record type name defines an optional value.
- The fields (items) in the record are defined from the next line of the record type.
- The field (item) definition are repeated for the number of fields.
- For field definition format, see field definition format.
- A record type name, to identify the record, is defined by enclosing with
[data] # Record type name:data
1 name N(100) # Name
2 age X9(3) # Age
Important
When defining fields in JSON and XML data formats, in multiple fields with the same field name, fields with the field type OB
should not be mixed with fields with other field types.
When mixed, the field type of the field that specifies other than OB is ignored because the definition of OB has priority.
As a result, even though the field is not actually an OB, it is treated as an OB type when reading and writing. This can cause problems where data and format item definitions do not match and data cannot be read and written correctly.
- Inappropriate examples
[order] 1 id N 2 data OB # field type:OB 3 detail OB [data] 1 value N [detail] 1 data N # field type:N <- Inappropriate notation: The field type is considered to be OB.
Field definition¶
Field definition is defined with the following format.
<Field start position> <Field name> <Multiplicity> <Field type> <Field converter>
Details of each element of the field definition are as given below.
Field start position required |
Define the field start position according to the following rules for each data format.
|
||||||||||||
Field name required |
Configure the name to identify the field. The field name is the key of java.util.Map used for input/output by this function. If Important Note that a field name cannot be defined with only numbers. In the case of XML data format, items are treated as an attribute value by adding An example is shown below. [tagName]
@attr
The XML corresponding to the above is as follows. <tagName attr="val">
・・・
</tagName>
|
||||||||||||
Multiplicity optional |
Specifies the number of fields that can be defined. This value can be specified only for JSON and XML data formats.
A specification example is shown below. address [1..3] # 1 to 3 can be defined
address # Only one is possible as it is omitted
address [0..*] # No condition (0 to unlimited)
address [*] # No condition (0 to unlimited)
address [1..*] # 1 or more
In the case of the following xml, the number of definitions in the <person>
<address>Home address</address>
<address>Work address</address>
</person>
In the case of the following JSON, the number of elements in the {
"address" : ["Home address", "Work address", "Delivery address"]
}
|
||||||||||||
Field Type required |
Defines the data type of the field. For field types that can be specified by default, see Field type list. |
||||||||||||
Field convertor optional |
Defines the contents of input/output pre-processing, such as specifying options for field types or data conversion. For field types that can be specified by default, see Field converter list. Multiple field converters can be configured. |
Define a multiformat record¶
For multi-format data, define multiple record formats in the format definition file.
The record format of the input /output data is automatically determined by the value of a specific field. If the input /output target data does not match any record type, it is treated as invalid data and the process is terminated abnormally.
Example of format definition for multi-format is shown below.
- Points
- Defines the record identification field. The record type name is
Classifier
. - The condition to determine a record is defined directly below the record type name of each record definition.
- The field defined in record identification (Classifier) must exist in the record definition.
- Defines the record identification field. The record type name is
file-type: "Fixed" # Fixed-length
text-encoding: "MS932" # Character encoding of the string type field
record-length: 40 # Length of each record
record-separator: "\r\n" # Carriage return and Line feed (crlf)
# Define the record identification condition
[Classifier]
1 dataKbn X(1) # Determine the record type by using the field of the first byte
# Define the header record
[header]
dataKbn = "1" # Header record if dataKbn is "1"
1 dataKbn X(1)
2 data X(39)
# Define the data record
[data]
dataKbn = "2" # Data record if dataKbn is "2"
1 dataKbn X(1)
2 data X(39)
For a sample of the multi-format definition, refer to the following link.
Tip
Since JSON and XML data formats do not have the concept of a record, format definition for multiformat is not supported.
Field type list¶
The data type definition list provided as standard is shown below.
- List of available field types for Fixed (fixed-length) data format
Type Java data types Description X
String Single byte character string (byte length = character string length)
By default, right trim/padding is performed using half-width space.
Argument: Byte length (numeric) required
If the output target value is
null
, the value is converted to an empty character before processing.If the read value is an empty string, it is converted to
null
. Configure convertEmptyToNull tofalse
if conversion of empty string tonull
is not required.N String Double-byte character string (byte length = number of characters/2)
By default, right trim/padding is performed with full-width space.
Argument: Byte length (numeric) required
- If the byte length is not a multiple of 2, a syntax error will occur.
If the output target value is
null
or the read value is an empty string, the handling is the same as single byte string field type.XN String Multibyte string
Specify this field type when handling a field where characters of different byte lengths are mixed, such as UTF-8.
This field type is also used when the padding for the full-width string (double-byte string) uses a half-width space.
By default, right trim/padding is performed with half-width space.
Argument: Byte length (numeric) required
If the output target value is
null
or the read value is an empty string, the handling is the same as single byte string field type.Z
BigDecimal Zone numeric (byte length = number of digits)
By default, left trim/padding is performed with
0
.Argument 1: Byte length (numeric) required
Argument 2: Number of digits after decimal point (numeric) optional
Default:0
If the output target value is
null
, the value is converted to0
before processing.If the number of bytes in the read value is
0
, it is converted tonull
. Configure convertEmptyToNull tofalse
, when the number of bytes is0
and conversion tonull
is not required.SZ BigDecimal Signed zone numeric (byte length = number of digits)
By default, left trim/padding is performed with
0
.Argument 1: Byte length (numeric) required
Argument 2: Number of digits after the decimal point (numeric) optional
Default:0
Argument 3: Plus sign to be configured in the zone section (hexadecimal string) optional
Argument 4: Minus sign to be configured in the zone section (hexadecimal string) optional
Argument 3 and argument 4 are configured when overwriting the plus sign of the signed zone numeric and minus sign of the signed zone numeric.
When the output target value is
null
or the number of bytes of the read value is0
, then the handling is the same as zone numeric field type.P BigDecimal Pack numeric (byte length = number of digits/2 [rounded up])
By default, left trim/padding is performed with
0
.Argument 1: Byte length (numeric) required
Argument 2: Number of digits after the decimal point (numeric) optional
Default:0
When the output target value is
null
or the number of bytes of the read value is0
, then the handling is the same as zone numeric field type.SP BigDecimal Signed pack numeric (byte length = (number of digits + 1)/2 [rounded up])
By default, left trim/padding is performed with
0
.Argument 1: Byte length (numeric) required
Argument 2: Number of digits after the decimal point (numeric) optional
Default:0
Argument 3: Plus sign to be configured in the zone section (hexadecimal string) optional
Argument 4: Minus sign to be configured in the zone section (hexadecimal string) optional
Argument 3 and argument 4 are configured when overwriting the plus sign of the signed pack numeric and minus sign of the signed pack numeric.
When the output target value is
null
or the number of bytes of the read value is0
, then the handling is the same as zone numeric field type.B byte[] Binary string
Padding and trimming are not performed.
Argument: Byte length (numeric) required
The conversion specifications for each application are different when the output value is
null
. Therefore, the value is not converted for this field type even in such a case and InvalidDataFormatException is thrown.When using this field type, configure the value explicitly in the application according to the requirements.
X9 BigDecimal Unsigned numeric string (byte length = number of characters)
Treat single-byte strings (X) in the field as numeric.
By default, left trim/padding is performed with
0
. Decimal point sign (.
) can be included in the string.Argument 1: Byte length (numeric) required
Argument 2: Number of digits after the decimal point (numeric) for fixed decimal point optional
Default:0
When the output value is
null
, then the handling is the same as zone numeric field type.If the read value is an empty string, the handling is the same as single byte string field type.
SX9 BigDecimal Signed numeric string (byte length = number of characters)
Single byte string (X) in the field is treated as a signed numeric. By default, left trim/padding is performed with
0
.Argument 1: Byte length (numeric) required
Argument 2: Number of digits after the decimal point (numeric) for fixed decimal point optional
Default:0
When the output value is
null
, then the handling is the same as zone numeric field type.If the read value is an empty string, the handling is the same as single byte string field type.
To change the sign character (
+
、-
), a project-specific field type can be created to support the implementation of the following classes.For additional field types, see Add a field type.
- List of available field types for Variable (Variable length) data format
Type Java data types Description X
N
XN
X9
SX9String All fields are read and written as strings for variable-length data formats.
The operation does not change no matter what type of identifier is specified. Since there is no concept of field length, arguments are not required.
To read and write as numeric format of character string (BigDecimal), use number converter or signed_number converter.
If the output target value is
null
, the value is converted to a empty character before processing.If the read value is an empty string, it is converted to
null
. Configure convertEmptyToNull tofalse
if conversion of empty string to null is not required.- List of available field types for JSON and XML data formats
Type Java data types Description X
N
XNString String data type
Editing such as padding is not performed.
The value is enclosed in double quotes
"
during output for JSON.If the output value is
null
, the value is not converted in JSON, and is converted to an empty string in XML.X9
SX9String Numeric string type
Data editing such as padding is not performed. During output, the value is output without change.
To read and write as numeric format of character string (BigDecimal), use number converter or signed_number converter.
When the output value is
null
, the handling is the same as field type for string data type.BL String String(
true
orfalse
expressed as a string)Data editing such as padding is not performed. During output, the value is output without change.
When the output value is
null
, the handling is the same as field type for string data type.OB
- Used to specify nested record types.
The record type corresponding to the field name is input/output as a nested element.
When the output value is
null
, the handling is the same as field type for string data type.An usage example is shown below.
- json
{ "users": [ { "name" : "Name", "age" : 30, "address" : "Address" }, { "name" : "Name 1", "age" : 31, "address" : "Address 1" } ] }
- xml
<users> <user> <name> Name </name> <age>30</age> <address> Address </address> </user> <user> <name> Name 1</name> <age>31</age> <address> Address 1</address> </user> </users>
The format definition file for JSON and xml above are as follows:
[users] # Root element 1 user [1..*] OB [user] # Nested element 1 name N # Bottom layer element 2 age X9 3 address N
Field converter list¶
The data converter list provided as standard is shown below.
Converter name | Type conversion specification | Description | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
pad | No type conversion | Configure the characters to pad and trim. Padding and trim positions operate as follows for each field type.
For more information on field types, see Field type list.
|
||||||||||||||||||||
encoding | No type conversion | Configure the character encoding for string type field Configure when overwriting the common configuration (text-encoding) for only specific fields. Can be used only for
|
||||||||||||||||||||
Literal value | No type conversion | Configure the default value for output. Outputs the specified literal value if the value was not configured during output. This configuration value is not used during input. |
||||||||||||||||||||
number |
String <-> BigDecimal | Configure when conversion of numeric string to numeric (BigDecimal) is required.
|
||||||||||||||||||||
signed_number |
String <-> BigDecimal | Configure when conversion of signed numeric string to numeric (BigDecimal) is required. Expect that signs are allowed, the specification is the same as number converter. |
||||||||||||||||||||
replacement |
No type conversion | The character is converted to the conversion destination character and returned for both input and output.
For details, see Replace (collate) the characters. |
Omitting item definitions¶
Describes the behavior when the item definition of the format definition file does not match the item definition of the data.
- For fixed-length and variable length data
- For fixed-length and variable length data, the field definition of the data and the format definition must match exactly. For this reason, an item must be defined in the format definition file even if there are items that are not required by the application.
- For JSON and XML data
- In the case of JSON and XML, items that are not defined in the format definition file will not be read. Therefore, item definition is not required for items that are present in the data but are not required by the application.