Rectangle 27 0

java Converting & validating CSV file upload in Spring MVC?


@RequestMapping(value="/new", method = RequestMethod.POST)
public String newCustomer(@Valid @ModelAttribute("customerForm") CustomerForm customerForm, BindingResult bindingResult) {

    if (bindingResult.hasErrors()) {
        return "NewCustomer"; // only external validation
    } else {

        /* 
           validation has passed, so now we must:
           1) open customerForm.csvFile 
           2) loop through it to validate each line and populate customerForm.customer.sites 
        */

        customerService.insert(customerForm.customer, customerForm.csvFile, bindingResult);
        if (bindingResult.hasErrors()) {
            return "NewCustomer"; // only external validation
        } else {
            return "CustomerList";
        }
    }
}
List<Integer> linesInError = new ArrayList<Integer>();
customerService.insert(customerForm.customer, customerForm.csvFile.getInputStream(), linesInError);
if (! linesInError.isEmpty()) {
    // populates bindingResult with convenient error messages
}
class CsvFile {

    private String name;
    private InputStream inputStream;

    CsvFile(MultipartFile file) {
        name = file.getOriginalFilename();
        inputStream = file.getInputStream();
    }
    // public getters ...
}
class CsvLoader {
@Autowired Verifier verifier;
@Autowired Loader loader;

    void verifAndLoad(InputStream csv) {
        // loop through csv
        if (verifier.verify(myObj)) {
            loader.load(myObj);
        }
        else {
            // log the problem eventually store the line for further analysis
        }
        csv.close();
    }
}
customerService.insert(customerForm.customer, new CsvFile(customerForm.csvFile), linesInError);
customerService.insert(customerForm.customer, customerForm.csvFile, bindingResult);
insert(Customer customer, MultipartFile csvFile, Errors errors) {
    // loop through csvFile.getInputStream populating customer.sites and eventually adding Errors to errors
    if (! errors.hasErrors) {
        // actually insert through DAO
    }
}
  • the file size and mimetype seems Ok (eg : size > 12 && mimetype = text/csv)
  • the validation is global (only real use case, but does not seem to be here)
  • you are sure it will always be very small (and what if a user click on wrong file ?)
  • your application will never be used in a production context under serious load

First, I would split validation in 2. Formal validation is in controller layer and only controls that :

IMHO, it is a bad idea to load the whole CSV in memory unless :

In service class we have

Normally, you avoid directly using Spring classes from business classes. If it is not a concern, the controller directly sends the MultipartFile to a service object, passing also the BindingResult to populate directly the eventual error messages. The controller becomes :

Thanks for your feedback; I have updated my original question to show my MVC config which limits file uploads to 1MB (probably should have included this to start with!). The CSV files are relatively small (5KB on average) and as a result the double-loop shouldn't cause a problem, it just doesn't seem a tidy solution to have to open & read the file twice. Your CsvLoader answer interests me but I'm not sure what you mean by "use a wrapper exposing the InputStream" - can you elaborate please? (More sample code would really help).

That way, your application only uses the memory it really needs, only looping once other the file.

The validation of the content is IMHO a business layer validation and can happen later. In this pattern, SiteCSVFileValidator would only test csv for mimetype and size.

Then the service class only adds line numbers where errors where detected to linesInError but it only gets the InputStream, where it could need say the original file name. You can pass the name as another parameter, or use a wrapper class :

Then you carefully design, code and test a method taking an InputStream as input, reads it line by line and call line by line methods to validate and insert data. Something like

You should either stick to the MultipartFile object, or use a wrapper exposing the InputStream (and eventually other informations you could need) if you do not want to tie your business classes to Spring.

and call

with no direct Spring dependancies

Note