【444】Data Analysis (shp, arcpy)

ABS suburbs data of AUS

1. Dissolve

　　Merge polygons with the same attribute of "SA2_NAME16".

>>> import arcpy
>>> mxd = arcpy.mapping.MapDocument("CURRENT")
>>> df = arcpy.mapping.ListDataFrames(mxd)[0]
>>> lyrs = arcpy.mapping.ListLayers(df)
>>> for lyr in lyrs:
...     arcpy.Dissolve_management(lyr, "Dissolve_" + lyr.name, 'SA2_NAME16', '#', 'MULTI_PART', 'DISSOLVE_LINES')
...

2. Add Centroid XY

　　After using the "Add Geometry Attributes" tool, we should close shp files and add them again and will see the results. (Sometimes it can show directly, WTF!!!)

...
>>> for lyr in lyrs:
...     arcpy.AddGeometryAttributes_management(lyr, "CENTROID")
...

3. Add a state field

　　Before merging those polygons, we should point a specific field storing state info.

　　Before doing this, atrribute window should be closed, or it won't work.

...
>>> for i in range(1, 9):
...     arcpy.AddField_management(lyrs[i], "State", "TEXT")
...     
>>> # file name like "Suburbs_MB_2016_NSW"
>>> # we want to get "NSW"
>>> for i in range(1, 9):
...     cursor = arcpy.UpdateCursor(lyrs[i])
...     fn = lyrs[i].name
...     for row in cursor:
...         row.setValue("State", fn[fn.rfind("_")+1:])
...         cursor.updateRow(row)
...

4. Merge the whole polygons into one

>>> mxd = arcpy.mapping.MapDocument("CURRENT")
>>> df = arcpy.mapping.ListDataFrames(mxd)[0]
>>> lyrs = arcpy.mapping.ListLayers(df)
>>> arcpy.Merge_management(lyrs, "Suburbs_MB_2016_AUS")

5. Export table to csv file

　　Tool: Export Feature Attribute to ASCII

　　ref: Export an attribute table to .txt using arcpy.

>>> arcpy.ExportXYv_stats('Suburbs_MB_2016_AUS', 'SA2_NAME16;CENTROID_X;CENTROID_Y;State;Shape_Area', 'COMMA', r'D:Twitter DataData	est2.csv', 'ADD_FIELD_NAMES')

6. Get specific columns

　　Based on pandas lib.

>>> df = pd.read_csv(r"D:Twitter DataData	est2.csv")
>>> df.head()
       XCoord     YCoord     ...     STATE  SHAPE_AREA
0  117.899601 -35.008360     ...        WA    0.003012
1  118.207172 -34.718972     ...        WA    0.394533
2  115.865812 -31.834866     ...        WA    0.000638
3  115.677976 -31.600241     ...        WA    0.003104
4  115.836085 -32.019166     ...        WA    0.000518

[5 rows x 7 columns]
>>> df.columns
Index(['XCoord', 'YCoord', 'SA2_NAME16', 'CENTROID_X', 'CENTROID_Y', 'STATE',
       'SHAPE_AREA'],
      dtype='object')
>>> df1 = df[['SA2_NAME16', 'CENTROID_X', 'CENTROID_Y', 'STATE', 'SHAPE_AREA']]
>>> df1.columns
Index(['SA2_NAME16', 'CENTROID_X', 'CENTROID_Y', 'STATE', 'SHAPE_AREA'], dtype='object')
>>> df1.head()
                      SA2_NAME16  CENTROID_X    ...      STATE SHAPE_AREA
0                         Albany  117.899601    ...         WA   0.003012
1                  Albany Region  118.207172    ...         WA   0.394533
2  Alexander Heights - Koondoola  115.865812    ...         WA   0.000638
3             Alkimos - Eglinton  115.677976    ...         WA   0.003104
4           Applecross - Ardross  115.836085    ...         WA   0.000518

[5 rows x 5 columns]
>>> df1.to_csv(r"D:Twitter DataData	estSuburbs_AUS.csv", index=False)