How to Wuickly Read a Large Csv File in Java

In terminal tutorial, you accept learned how to parse Excel file in Java and in this Java tutorial, you lot will larn how to parse CSV file in Coffee. You tin straight parse CSV file in Java without using whatsoever third party library, because ultimately its a text file and y'all can use BufferedReader to read it, only yous tin also take advantage of good open source library similar Apache eatables CSV to parse comma separated values. These library makes developer's life easy and provides rich functionality to parse diverse CSV formats. In real programming world, CSV or comma separated files are used for diverseness of purpose, including for transporting information from ane system to some other e.g. FX rates, importing and exporting records from database etc.

In CSV files entries are separated by comma, and it may or may not contain header. At that place are many ways to parse or read CSV files in Java, and if yous need to do it on your project, its better not to reinvent the wheel and cull commons csv, simply for learning purpose, information technology'south good to know how to do it without using 3rd party library.

In this tutorial, I am going to show yous two means to read CSV files in Java. Start manner is past using java.io.BufferedReader and split() method from java.lang.String course, and second way is past using Apache Eatables CSV library'south CSVParser class. Eatables CSV is new member in rich Apache eatables family and has built in back up to read near common CSV formats e.yard. RFC 4180, Microsoft Excel, MySQL and TDF.

Yous can also create a custom format past using the fluent style API of Apache commons CSV. CSVParser is fully functional parser, which can parse a different kinds of CSV files e.grand. XLS CSV file, CSV file without header or CSV file with header. All you need to practise is to cull different format for CSVParser, which we will learn in this tutorial.

By the way, if you desire to acquire more about reading/writing files in Java and then I advise to join these online core Coffee courses or read one of the good Coffee book like Core Java by Cay Due south. Horstmann or Java: A Beginner'southward Guide past Herbert Schildt.

Maven dependency and JAR file required for CSV Parsing

In order to utilize this library you demand to addcommons-csv-1.i.jar file into your classpath. If you are using Maven you tin also add post-obit dependency in your projection file.

            <dependency>            <groupId>org.apache.commons</groupId>            <artifactId>commons-csv</artifactId>            <version>1.i</version>            </dependency>          

Call back, its better to use Maven for managing dependency because it will too download any other JAR file on which this library is dependent, known equally transitive dependencies. If you add together JAR files manually, y'all make sure to download any dependent JAR.

CSV Parser to read CSV files in Java

Apache Eatables CSV reads and writes files in variations of the Comma Separated Value (CSV) format. For example to parse an Excel CSV file, you need to write following code

            Reader            in            =            ..            .;            Iterable            parser            =            CSVFormat                          .EXCEL            .parse(in);            for            (CSVRecord            record            :            parser) {     ... }

and to read a normal CSV file with header yous need to write :

            Reader            in            =            ..            .;            Iterable            parser            =            CSVFormat                          .DEFAULT            .parse(in);            for            (CSVRecord            record            :            parser) {     ... }

Currently Apache eatables CSV supports following formats :

• DEFAULT to read standard comma separated format, as for RFC4180 only allowing empty lines.
• EXCEL to read Excel file format (both XLS and XLSX) (using a comma as the value delimiter).
• MYSQL to parse default MySQL format used past the SELECT INTO OUTFILE and LOAD Information INFILE operations.
• RFC4180 to read comma separated format equally defined by RFC 4180.
• TDF to parse tab-delimited format, with quote; leading and trailing spaces ignored

It'south more functional, and should be used in real globe project. On the other paw BufferedReader arroyo is pretty direct forward. You open up a CSV file and commencement reading it line by line, since each line contains a blackout separated Cord, y'all need to split them using comma (",") and you volition get an array of Cord containing each column.

But do whatever you wants to exercise with them, if you lot are creating object, as shown in get-go case, then create them, otherwise y'all can merely print them similar in 2d example. You tin even use new Java seven and Java 8 feature to read file more than efficiently.

Java Program to Parse or Read CSV File in Coffee

Here is full code case of how to read CSV file in Java. This program contains two examples, first i read CSV file without using third party library and the second one parse file using Apache commons CSV, a new library for parsing CSV files. Brand sure you include commons-csv-1.1.jar file in your CLASSPATH to run this plan in your PC.

Here is our sample CSV file, which also contains header and has contains countries detail e.g. name of country, its capital and currency.

Our CSV file - countries.txt
NAME,Upper-case letter,CURRENCY
India,New Delhi,INR
USA,Washington,USD
England,London,GBP
Nihon,Tokyo,JPY

At that place are two methods in this plan readCSV() and parseCSV() , former uses BufferedReader to read CSV file. We also have a class Country to represent each line of file, which basically contains land specific data. In first method we read the file line by line and then carve up each line on comma to get a Cord assortment containing individual fields.

Nosotros use this array to create Country object and add them into the List, which is returned by our method. Code of this method is very directly frontwards and cocky explanatory, we take ignored the commencement line considering we know its header.

Second method is interesting every bit it demonstrate how to use apache commons csv library to read csv file. As I said, commons csv supports several csv format direct and nosotros will utilise CSVFormat.DEFAULT , which besides supports header. Hither you create an example of CSVParser by passing it a FileInputStream, which points to your csv file and CSVFormat . This contains several CSVRecord from which you tin can call up individual fields.

            import            java.io.BufferedReader;            import            java.io.FileNotFoundException;            import            java.io.FileReader;            import            coffee.io.IOException;            import            java.util.ArrayList;            import            coffee.util.List;            import            java.util.Scanner;            import            org.apache.commons.csv.CSVFormat;            import            org.apache.commons.csv.CSVParser;            import            org.apache.commons.csv.CSVRecord;            /**  * Java Program to parse and read CSV file using traditional BufferedReader  * approach and by using more than functional CSV parser from Apache Commons CSV  * library. Apache Commons CSV back up different CSV format including default  * ane, with or without header, reading EXCEL or XLS CSV file etc.  *  *              @author              */            public            form            CSVReader            {            individual            static            form            Country            {            individual            String            name;            private            Cord            capital;            individual            String            currency;            public            State(String            proper noun,            Cord            capital,            String            currency) {             this.name            =            proper noun;             this.upper-case letter            =            capital;             this.currency            =            currency;         }            public            String            name() {            return            proper noun;         };            public            Cord            majuscule() {            render            capital;         }            public            Cord            currency() {            return            currency;         }            @Override            public            Cord            toString() {            return            "Country [proper noun="            +            name            +            ", capital="            +            capital            +            ", currency="            +            currency            +            "]";         }     }            public            static            void            main(String            args[])            throws            FileNotFoundException,            IOException            {            System            .out.println("Reading from CSV file using BufferedReader                      and String Split");            List                        nations            =            readCSV();         print(nations);            Organisation            .out.println("Parsing CSV file using CSVParser of                        Apache eatables CSV");         parseCSV();      }            /*      * Java program to read CVS file using BufferedReader and String carve up()      * method      */            public            static            List                        readCSV()            throws            FileNotFoundException,            IOException            {            List                        countries            =            new            ArrayList<>();            BufferedReader            br            =            new            BufferedReader(                          new            FileReader("countries.csv"));            Cord            line            =            br.readLine();            // Reading header, Ignoring            while            ((line            =            br.readLine())            !=            zero            &&            !line.isEmpty()) {            String[] fields            =            line.split up(",");            String            proper noun            =            fields[0];            Cord            majuscule            =            fields[1];            String            currency            =            fields[two];            Country            nation            =            new            Country(name, capital, currency);             countries.add(nation);         }         br.close();            return            countries;     }            /*      * Method to read CSV file using CSVParser from Apache Commons CSV      */            public            static            void            parseCSV()            throws            FileNotFoundException,            IOException            {            CSVParser            parser            =            new            CSVParser(new            FileReader("countries.csv"),            CSVFormat                          .DEFAULT            .withHeader());            for            (CSVRecord            record            :            parser) {            System            .out.printf("%southward\t%southward\t%south\n", tape.get("Proper name"),                     tape.get("Capital"), record.get("CURRENCY"));         }         parser.close();     }            public            static            void            print(List                        countries) {            System            .out.println("========================");            for            (Country            land            :            countries) {            System            .out.println(state);         }            System            .out.println("========================");     }  }            Output            :            Reading            from            CSV            file using            BufferedReader            and            Cord            Carve up            ===            ===            ===            ===            ===            ===            ===            ===            State            [proper name=            India, capital=            New            Delhi, currency=            INR]            State            [name=            United states of america, uppercase=            Washington, currency=            USD]            Country            [name=            England, uppercase=            London, currency=            GBP]            Country            [name=            Nippon, capital letter=            Tokyo, currency=            JPY]            ===            ===            ===            ===            ===            ===            ===            ===            Parsing            CSV            file using            CSVParser            of            Apache            eatables            CSV            India            New            Delhi            INR            USA            Washington            USD            England            London            GBP            Japan            Tokyo            JPY          

You can see output of our plan matches with content of our CSV file. So both of our approach is working properly.

That'south all folks, Enjoy parsing CSV file with Apache commons CSV parser. Another great utility open up-source library from Apache. You lot can likewise report any consequence found while using them. A nice fashion to back up open source projects. I suggest to utilise this library if you have to process CSV files in your project because its tried and tested and rich in functionality. You can use this library to load CSV data into MySQL database or simply create objects from them.

If you similar this tutorial and interested to acquire more about parsing and handling files in Coffee, you tin as well check out post-obit Java IO tutorials from this web log :

• How to parse XML file in Coffee using SAX parser? (guide)
• How to catechumen JSON to Object in Coffee? (example)
• How to read XML file in Java using JDOM parser? (tutorial)
• How to parse big JSON file using Jackson Streaming API? (example)
• How to read a file in 1 line in Coffee viii? (example)
• How to re-create File in Java? (case)
• How to generate MD5 checksum for file in Java? (solution)
• How to read/write RandomAccessFile in Java? (example)

wrennboodgme.blogspot.com

Source: https://javarevisited.blogspot.com/2015/06/2-ways-to-parse-csv-files-in-java-example.html

Share this post