Create a Batch registration Function Using Upload

This section describes the function to upload a CSV file for batch registration based on an example application.

Description of the function to be created
  1. Click “Project batch registration (プロジェクト一括登録)” in the header menu.
../../../../../_images/project_upload-link.png
  1. Download the batch registration sample file which generates the validation error from below

    batch_project_registration_validation_error.csv (プロジェクト一括登録_バリデーションエラー.csv)

  2. Upload the sample file and click the “Register(登録)” button.

../../../../../_images/project_upload-invalid_upload.png
  1. Validation error occurs.
../../../../../_images/project_upload-validate.png
  1. Download the batch registration sample file which does not generate validation error from below
project_batch_registration.csv (プロジェクト一括登録.csv)
  1. Upload the sample file and click the “Register(登録)” button.
../../../../../_images/project_upload-valid_upload.png
  1. The contents of the file are registered in the database and the completion message is displayed.
../../../../../_images/project_upload-complete.png

Overview of the business action method to be created

ProjectUploadAction.java
@OnDoubleSubmission
@OnError(type = ApplicationException.class, path = "/WEB-INF/view/projectUpload/create.jsp")
public HttpResponse upload(HttpRequest request, ExecutionContext context) {

    // Acquire upload file
    List<PartInfo> partInfoList = request.getPart("uploadFile");
    if (partInfoList.isEmpty()) {
        throw new ApplicationException(
                MessageUtil.createMessage(MessageLevel.ERROR, "errors.upload"));
    }
    PartInfo partInfo = partInfoList.get(0);

    LoginUserPrincipal userContext = SessionUtil.get(context, "userContext");

    // Loading and validation of upload files
    List<Project> projects = readFileAndValidate(partInfo, userContext);

    // Batch registration to DB
    insertProjects(projects);

    // Add a completion message
    context.setRequestScopedVar("uploadProjectSize", projects.size());

    // Saving a file
    saveFile(partInfo);

    return new HttpResponse("/WEB-INF/view/projectUpload/create.jsp");
}

The processing flow of the business action method is as follows.

  1. Acquire a file
  2. Validate the contents of the CSV file by binding to Bean
  3. Batch registration to DB
  4. Saving a file

The details of each process is described under Implementation of the file upload function and Implementation of the batch registration function.

Implementation of the file upload function

First, how to create the upload part of the batch registration function using upload is explained

  1. Create a file upload screen
  2. Create a business action method to acquire and save a file
Create a file upload screen

Create a screen with a file upload field.

/src/main/webapp/WEB-INF/view/projectUpload/create.jsp
<n:form useToken="true" enctype="multipart/form-data">
    <!-- Omitted -->
    <div class="message-area margin-top">
        <!-- Completion message display part -->
        <c:if test="${not empty uploadProjectSize}">
            <ul><li class="message-info"><n:message messageId="success.upload.project" option0="${uploadProjectSize}" /></li></ul>
        </c:if>
        <!-- Error message display part -->
        <n:errors errorCss="message-error"/>
    </div>
    <!-- Omitted -->
    <h4 class="font-group">Project information file selection</h4>
    <table class="table">
        <!--  Description of screen design is omitted -->
        <tbody>
            <tr>
                <th class="item-norequired" colspan="2">Project information file selection</th>
            </tr>
            <tr>
                <th class="width-250 required">Project information file</th>
                <td >
                    <div class="form-group is-fileinput">
                        <div class="input-group">
                            <n:file name="uploadFile" id="uploadFile"/>
                            <!--  Description of screen design is omitted -->
                        </div>
                    </div>
                </td>
            </tr>
        </tbody>
    </table>
    <div class="title-nav">
        <div class="button-nav">
            <n:button uri="/action/projectUpload/upload"
                      allowDoubleSubmission="false"
                      cssClass="btn btn-raised btn-default">Registration</n:button>
        </div>
    </div>
</n:form>
Key points of this implementation
  • Specify multipart/form-data as enctype attribute of form tag to send multipart file.
  • Create a file upload field using file tag. Specify the registration name of the request object in the name attribute. To acquire the file in a business action, specify this registration name as an argument of HttpRequest#getPart
  • Display upload completed message with message tag, once the upload is completed. In order to include the number of uploads in the completion message, specify the number of uploads configured in the request scope in option0 attribute.
  • Use errors tag to create an area to display the list of validation error messages for the target file. For the output format of the error message list, refer to error message list.
Create a business action method

Describes how to get and save a file in the business action method.

ProjectUploadAction.java
public HttpResponse upload(HttpRequest request, ExecutionContext context)
        throws IOException {

    List<PartInfo> partInfoList = request.getPart("uploadFile");
    if (partInfoList.isEmpty()) {
        throw new ApplicationException(MessageUtil.createMessage(MessageLevel.ERROR,
                 "errors.upload"));
    }
    PartInfo partInfo = partInfoList.get(0);

    // Batch registration process is omitted as it will be described later

    // Saving a file
    saveFile(partInfo);

    return new HttpResponse("/WEB-INF/view/projectUpload/create.jsp");
}

/**
 * Save a file
 *
 * @param partInfo Upload file information
 */
private void saveFile(final PartInfo partInfo) {
    String fileName = generateUniqueFileName(partInfo.getFileName());
    UploadHelper helper = new UploadHelper(partInfo);
    helper.moveFileTo("uploadFiles", fileName);
}
Key points of this implementation
  • Acquire the file HttpRequest#getPart.

  • When the file does not exist (not uploaded), then the size of PartInfo list that is acquired will be zero. This value is used to perform control such as throwing a business exception.

  • The uploaded file is stored in a temporary area by the multipart request handler. Since the temporary area is automatically deleted, if you need to permanently (save) an uploaded file, move the file to an arbitrary directory. However, file transfers are possible only when the file path management is used to manage the input and output of files and directories.

  • Use UploadHelper#moveFileTo method to transfer files. The first argument is the key name of the file storage directory registered in the configuration file. In the Example Application, the configuration is described in the following file.

    filepath-for-webui.xml
    <!-- File path definition -->
    <component name="filePathSetting"
            class="nablarch.core.util.FilePathSetting" autowireType="None">
      <property name="basePathSettings">
        <map>
          <!--Omitted -->
          <!-- Directory to store the upload file -->
          <entry key="uploadFiles" value="file:./work/input" />
        </map>
      </property>
      <!-- Omitted -->
    </component>
    

Implementation of the batch registration function

This section describes how to create the batch registration part of the batch registration function using uploads.

Create a bean to bind the contents of the file

A bean to bind the contents of the file is created.

ProjectUploadDto.java
@Csv(headers = { /** Describe the header **/},
        properties = { /** Properties to bind **/},
        type = Csv.CsvType.CUSTOM)
@CsvFormat(charset = "Shift_JIS", fieldSeparator = ',',ignoreEmptyLine = true,
        lineSeparator = "\r\n", quote = '"',
        quoteMode = CsvDataBindConfig.QuoteMode.NORMAL, requiredHeader = true, emptyToNull = true)
public class ProjectUploadDto implements Serializable {

    // Excerpt of some items only.Getter and setter are omitted

    /** Project name */
    @Required(message = "{nablarch.core.validation.ee.Required.upload}")
    @Domain("projectName")
    private String projectName;

    /** Project type */
    @Required(message = "{nablarch.core.validation.ee.Required.upload}")
    @Domain("projectType")
    private String projectType;

    // Property that holds the line count to process.Setter is omitted.
    /** Line count*/
    private Long lineNumber;

    /**
     * Get line count.
     * @return Line count
     */
    @LineNumber
    public Long getLineNumber() {
        return lineNumber;
    }
}
Key points of this implementation

Tip

The validation error message of a required input item is changed to an appropriate value as per the file upload. For information on how to specify a validation message, refer to configure the input value check rule.

Create a business action method

Create a business action method to register the contents of the uploaded file in the database.

Validate the contents of 1 CSV file by binding to Bean
ProjectUploadAction.java
private List<Project> readFileAndValidate(final PartInfo partInfo, final LoginUserPrincipal userContext) {
    List<Message> messages = new ArrayList<>();
    List<Project> projects = new ArrayList<>();

    // Validate the contents of the file by binding it to the bean
    try (final ObjectMapper<ProjectUploadDto> mapper
             = ObjectMapperFactory.create(
                    ProjectUploadDto.class, partInfo.getInputStream())) {
        ProjectUploadDto projectUploadDto;

        while ((projectUploadDto = mapper.read()) != null) {

            // Validate and configure the result messages
            messages.addAll(validate(projectUploadDto));

            // Create an entity
            projects.add(createProject(projectUploadDto, userContext.getUserId()));
        }
    } catch (InvalidDataFormatException e) {
        // Parsing ends if there is an invalid line in the file format
        messages.add(
            MessageUtil.createMessage(
                MessageLevel.ERROR, "errors.upload.format", e.getLineNumber()));
    }

    // Not registered in the database even if there is one error
    if (!messages.isEmpty()) {
        throw new ApplicationException(messages);
    }
    return projects;
}

/**
 * Validate the project information and store the result in the message list.
 *
 * @param projectUploadDto Project information Bean generated from CSV
 * @return messages         List of validation result messages
 */
private List<Message> validate(final ProjectUploadDto projectUploadDto) {

    List<Message> messages = new ArrayList<>();

    // Single item validation.Execute Bean validation based on the annotation defined in Dto
    try {
        ValidatorUtil.validate(projectUploadDto);
    } catch (ApplicationException e) {
        messages.addAll(e.getMessages()
                .stream()
                .map(message -> MessageUtil.createMessage(MessageLevel.ERROR,
                        "errors.upload.validate", projectUploadDto.getLineNumber(), message))
                .collect(Collectors.toList()));
    }

    // Customer existence check
    if (!existsClient(projectUploadDto)) {
        messages.add(MessageUtil.createMessage(MessageLevel.ERROR,
                "errors.upload.client", projectUploadDto.getLineNumber()));
    }

    return messages;
}
Key points of this implementation
2.Batch registration to DB
ProjectUploadAction.java
public HttpResponse upload(HttpRequest request,ExecutionContext context)
        throws IOException {

    // Execution of validation is described above

    // Batch registration to DB
    insertProjects(projects);

    // Saving a file is described above
}

/**
 * Register multiple project entities to the database in a batch.
 * @param projects List of validated projects
 */
private void insertProjects(List<Project> projects) {

  List<Project> insertProjects = new ArrayList<Project>();

  for (Project project : projects) {
      insertProjects.add(project);
      // Batch insert every 100 records
      if (insertProjects.size() >= 100) {
          UniversalDao.batchInsert(insertProjects);
          insertProjects.clear();
      }
  }

  if (!insertProjects.isEmpty()) {
      UniversalDao.batchInsert(insertProjects);
  }
}
Key points of this implementation
  • Batch registration is executed using UniversalDao#batchInsert.
  • Set an upper limit on the number of registrations per batch registration, because a large number of registrations at a time may result in a deterioration in performance.

This completes the explanation for the batch registration function using upload.

Getting Started To TOP page