Monday, May 22, 2017

Merging multiple comma separated values CSV files

If there are many comma separated value CSV files with headers and you want to merge them into a single CSV file, it can be a pain having to do it by hand. Fortunately, there are ways to automate the task. One method which I like is to use the tail command from Unix. For Windows, a tail utility can be downloaded from http://tailforwin32.sourceforge.net/;There are others which a Google search can reveal.

Below are screenshots of a few CSV files to merge.


  1. Using a text editor, create a script or batch file. In the editor, type in the tail command to create a new merged file with a header.

    tail -n +1 -q green.csv > merge.csv

    Note: -n +1 means to start from the first line.
    A single > means to output to a new file

  2. Type in the commands to append lines from subsequent files without headers to the output file.

    tail -n +2 -q orange.csv >> merge.csv
    tail -n +2 -q transfer.csv >> merge.csv
    #...etc...
    Note: -n +2 means to start from the 2nd line
    >> means to append to the output file

  3. Run the script or batch file in a Command Prompt.

    The CSV files are merged.

No comments: