Wednesday, April 1, 2009

Example creating polygons from CSV text file with FME 2009

The other day I received some ASCII comma separated value formatted text file (an example is shown below) that stores the vertices of polygons in the first 2 columns and a unique polygon identifier in the 3rd. Each polygon is separated from another with a blank line. I needed to convert the text strings into polygon geometries in a shape file for display in a GIS. There are a myriad of ways to do it but I chose the new SAFE FME 2009 to give the new version a workout. Here's how I created the polygons from the CSV file:


  1. Run the FME Workbench and choose to use the workspace wizard.

    The Create Translation Workspace Wizard appears.


  2. Choose Comma Separated Value (CSV). Click Next. Type in the path and filename of the CSV file e.g. C:\temp\polygons.csv.



  3. Click Settings.

    The Input Settings for Comma Separated Value (CSV) appears.
    Note: change the settings as necessary for your CSV file.


    Click OK.

  4. When prompted for the destination format, select ESRI Shape.



  5. Click Next twice. Click Finish.

    The Workbench shows the source and destination datasets.

  6. Use the Workbench's Transformers pane to add in the StringSearcher transformer. Join this to the source dataset. Open up the StringSearcher Parameters.



    Note: we want to filter out the blank lines demarcating the polygons in the text file. One way of doing it is to specify a regular expression to match empty lines. For instance, the expression ^ means the beginning of the line and $ means the ending of the line. Together ^$ means matching lines with nothing from start to the end.

  7. In the Attribute field, select col0. In the Regular Expression field, type in ^$. Click OK.

  8. Add in the 2DPointAdder transformer and connect it to the StringSearcher transformer's NOT_MATCHED field. Open up the 2DPointAdder Parameters.



    Note: the 2dPointAdder transformer will form a point geometry from the input X and Y strings. The input strings must be from non-blank lines and that is why we connect it to the UN_MATCHED output from the "blank line" StringSearcher transformer.

  9. In the X Value field, select col0. In the Y Value field, choose col1. Click OK.

  10. Add in the PointConnector transformer and connect it to the 2DPointAdder transformer. Open up the PointConnector Parameters.



    Note: the PointConnector transfomer will make polygon geometries from the input point geometries. One way to differentiate from one polygon to another polygon is through the Connection Break Attributes - in this example, the 3rd column stores the polygon identifier and any change in this value indicates a change in the polygon.

  11. In the Connection Break Attributes, click the ... button.

    The Select Connection Break Attributes Items dialog box appears.


  12. Toggle on col2. Click OK. Click OK again.

  13. Connect the PointConnector transformer's POLYGON output to the destination dataset.

  14. Open up the destination dataset's Feature Type Properties.



  15. In the Allowed Geometries field, select shape_polygon. Click OK.

    The workbench process flow may look like this at this point.


  16. Run the translation and display the resultant shape file.



    As you can see from the screen shot above, FME has successfully created the polygons from the input CSV file using the transformers outlined above. 

2 comments:

mehrdad said...

looks interesting. Is it possible to run it from the CMD (command line) with parameters?
Something like: FME example_csv.txt followed by some parameters

thank you
mehrdad

dominoc925 said...

Mehrdad
Yes, you can definitely run it from the command line. You need to create an FME mapping file that defines how you want to map the source and destination data sets. (You can simply save the workbench as a *.fme mapping file). Then from the command line type in

fme yourfile.fme.

But you need to understand the mapping file syntax to get the best use out of this method.