On focused dataframes in ArcMap

Q: Why can't I add a north arrow/legend/scale bar to my map?

A: Are you in the layout view (vs. the data view, often the default view)?

Q: Yes, of course. I didn't just install the software this year, you know.

A: OK, I can see I'm dealing with a non-novice here. Let's try something else. Is your layer focused?

Q: Um... it's clear to me, so I guess it's in focus?

A: No, I mean that if you're looking at the data view, is there a diagonal line border around it?

Q: Yes...

A: OK, that means you have that layer focused. It allows you to edit the layer in the layout view. See this thread for more info. To un-focus the layer, hit this button on your toolbar:

For me, it's located a few lines below the word "Selection", but as always the location of the buttons depends on where you have your toolbars anchored.

Can ArcMap calculate area?

Yes, yes it can!

First create a field, then right click your new field and select "calculate geometry". ArcMap will calculate the area based on the projected geography (or another geography if you chose).

Is your layer not projected? Check this post out.

Adding coordinate data to a shapefile and lining up your layers

...or, Why is Maryland floating out in the Atlantic Ocean?

If you find that one (or more) of your layers aren't lining up in your mxd file, it's probably a problem with the geographic and/or projection coordinate systems. Your layers need to be projected in the same system. You can check the projections by right clicking the layer in the table of contents, selecting "properties", and looking at the information listed for both Projected Coordinate System and (by scrolling down) Geographic Coordinate System.

If you need to re-project a layer, see this post (steps 1 - 3) on how to. Remember to add your newly projected layer to your mxd file, and remove your "floating in the ocean" layer.

A problem arises if you have a shapefile that lacks coordinate data. You just can't re-project something that hasn't been projected yet.

For example, the 2000 census zip code shapefiles (available here) lack a coordinate system. When you download the zip file, it does not include the projection (.prj) file for the shapefile. Adding this layer will result in the zip code layer floating in the ocean.

To add a coordinate system, open Arc Catalog (not just the shortcut thru ArcMap). Select the shapefile that corresponds to the layer that is missing the coordinate data by double clicking it in the table of contents so you get a popup window. In this window, go to the XY Coordinate System tab, and click "select" to select a pre-defined coordinate system. Chose the appropriate coordinate system, click "add", and then "ok".

Re-add the layer to your mxd file. Check the properties to make sure the coordinate system has been updated. If the layer is still out of line with the other layers on your map, go through the re-projection steps described above.

Source: ArcGIS Desktop Help

Metadata in ArcGIS10

The metadata file is embedded in the shapefile and can be accessed through ArcCatalog.

Open ArcCatalog 10 (not the shortcut through ArcMap10). Find your shapefile of interest in the table of contents on the left side of the screen. Click on the shapefile to open it. The main window on the right has 3 tabs, "Contents", "Preview", and "Description". Click on the "Description" tab (it may take a minute to load). Scroll down to the bottom of the screen to see if "ArcGIS Metadata" and "FGDC Metadata" is available. If they are, click on these titles to open the metadata. If they are not, or if you are looking for a particular style of metadata, you can change the metadata style by clicking on "Customize" on the very top of the page. Choose "Arc Catalog Options" and open the "Metadata" tab. You can choose the metadata style from the dropdown menu options here.

Uploading files to ftp

This morning, an email requested that I upload sensitive files to a lab ftp. The email gave me the site, user name, and password, and nothing else.

If you ever find yourself in this position, may I suggest starting here:

http://filezilla-project.org/

Download the Client -- it's all you'll need to do a simple file transfer.

Next, hit this up:

http://wiki.filezilla-project.org/Using

The quick guide is both quick and easy to understand.

One big headache I have not been able to overcome is the size limitations on transferred files. I'm trying to upload multiple shapefiles with over 3K records and 20+ fields... anyone out there have a workaround short of breaking up the shapefiles in ArcGIS???

Identifying duplicate records in Excel, ArcGIS

Let me start by saying that I have no doubt there is an easier method. This is just the way I attacked this problem today, and it seemed to work, so I thought I'd share.

There was a question as to why I didn't just use Excel's highlight or delete duplicate action. This is a great addition to the Excel package (I believe it introduced with Excel 2007) and you can read about it here: http://www.lytebyte.com/2008/06/10/highlight-remove-delete-duplicates-excel-2007/. But since I didn't want to blindly delete duplicates, I needed to find a quicker way than eyeballing highlighted cells to investigate and ultimately delete duplicate records.

Problem: I have four layers of patient data, each with over 3000 records. An eyeball review suggests that each layer has duplicate records, defined as records with the same Patient ID and same diagnosis (Dx) date. How do I identify these duplicates so that I can delete them?

Roundabout solution: First, I exported the records for one layer (i.e. 2005 cases) into Excel. I deleted all fields except patient ID, dx date, and STFID. I translated all columns into numbers with no decimal places (select the cells, hit the exclamation point drop down and select "convert to number").

I created a concatenated column named "ID_and_Dx". By using the formula "=CONCATENATE(A2,C2)", I created a record number of patient ID (located in A2) and dx date in numerical form (located in C2).

The important next step: I sorted the data by the concatenated column (ID_and_Dx).

I created a column called "Dup_ID_Dx". This is my indicator variable for duplicate fields. I used the formula "@if(D3=D2,"Dup", "Not Dup"). Filling this formula down through the sheet will mark the first record in a duplicate set as "Dup".

The important next step: I copied the Dup_ID_Dx column, highlighted the next blank column, right click and selected "Paste special", and chose to paste values. This will create a column of "Dup" or "Not Dup" without the formula. I then deleted the formula version column.

One more problem: I noticed that some records have the same patient ID and dx date, but different STFID. I would like to be able to investigate these records later in ArcGIS. How can I denote these kind-of-duplicates in Excel before I bring them back into ArcGIS?

And back to the circuitous solutions: I created another concatenated column and named it "ID_and_Dx_and_STFID". I used the concatenate function as described above to mash up my previously created ID_and_Dx field and my STFID field. Note that if a records couldn't geocode and therefore didn't have an associated STFID, this ID_and_Dx_and_STFID cell only contains the ID_and_Dx value.

I then made another duplicate indicator column, called "ID_Dx_STFID_Dup" and used the formula "@if(F3=F2,"Dup", "Not Dup")" to fill down and identify duplicates. Note that duplicates in this column indicate records that have the same patient ID, Dx date, and STFID. I am actually interested in records that have the same Patient ID and Dx date but different STFID. I will get to these kind-of-duplicates later.

Then it was time to bring it in to ArcGIS10. I created a patient ID field type double and named it "Pat_ID_dbl" on the 2005 case layer on ArcGIS and used the field calculator to populate it with the patient ID. I did this because my duplicate investigation xls document has patient ID listed as a number.

I joined the 2005 layer to the duplicate investigation xls document on the patient ID field. I sorted by the Dup_ID_Dx_ field and selected all duplicate records (using "select by attribute") to calculate my % duplication. I then used "switch selection" to select the non-duplicate records and created a new layer from these selected features (right click on layer name -> data -> export data and make sure you're exporting "selected features"). This will be the layer I will use for my analysis because it contains no duplicate records.

Now, back to the issue of kind-of-duplicates (same patient ID, same Dx date, different STFIDs, indicating that something went quite wrong in the data entry). I used my original, joined layer to select duplicate records (Dup_ID_dx). I created a new layer from this selection (right click layer -> selection -> new layer from selected features). In this layer, I sorted by the kind-of-duplicate field (sort by "ID_Dx_STFID_Dup") so I can investigate the "Not Dup" within these records to try to figure out what's going on with these records. In my case, I noticed all of the patients with the same patient ID and Dx date had the same birthday (DOB). So even though the two kind-of-duplicate records have different home addresses and therefore different STFID's, they seem to indicate that they are true duplicates of the same patient and diagnosis.

Converting latitude and longitude to state plane feet

1. Access the "Project tool"

Click the Arc Toolbox window icon which looks like a small box with a red really-that's-supposed-to-be-a-toolbox icon in the center (for me, it's
directly under the "Customize" menu header, but that will depend on where you root your menus)

2. In the resulting Toolbox window, click Data Management Tools, then Projections and Transformations, then Feature, then Project (double click).

3. Select your input dataset, re-name your output dataset, and choose your output coordinate system

For example, I was projecting file Cases_2002, so I renamed my output
dataset "Cases_2002_proj" and chose "Predefined coordinate system", "Projected
coordinate systems", "State plane", "NAD 1983 (US Feet)", and my state of interest.prj file.

This adds the newly projected layer to your mxd file.

4. Next, open the attribute table of this projected layer, add a field type "Double" and name it "X_St_Pl_ft".

5. Right click the X_St_Pl_ft column and choose "Calculate Geometry".

Sometimes, I randomly encounter an error that I cannot utilize this function unless I'm in editor mode, even if I am in editor mode (enter editor mode by right clicking layer, selecting "Edit" then "Start editing"). When this happens, I save my project, shut it down, and restart ArcGIS 10, which fixes the problem.

6. Select Property: "X Coordinate of Point" and "Use coordinate system of the data source". Select Units: "Feet US [ft]"

7. Repeat for State Plane Y coordinate.

NOTE that if you performed a join on the original layer, the join may not transfer to the projected layer.

Deleting records in ArcGIS10

In order to delete a record, you need to be in "editing" mode:

Highlight your layer of interest

Right click the layer and select "Edit features" then "Start editing"

Now you can open your attribute table, select the records you'd like to delete, right click the records and choose "Delete selected".

When you're done, click the drop down arrow on the editing toolbar and select "Save edits", then "Stop Editing".

Note that the records you have removed have also been removed from its associated files (shp, dbf, etc.).

Adding X, Y coordinate data to a layer in ArcGIS 10

My point pattern data had been previously geocoded but did not include coordinate data, so I added it:

Opened attribute table. Added new field “X” (Type: Double). Right clicked new field, selected “Calculate Geometry” and selected "Property: X Coordinate of Point", "Coordinate System of data source (in this case, GCS: North American 1983), and Units "Decimal Degrees". Repeated the process for "Y" coordinate.

For some reason, the new fields initially populated with "1"'s. I closed the attribute table, removed the layer, re-added the layer, opened the attribute table, and the coordinates magically appeared. This only happened once; further additions of coordinate data to different layers populated correctly.

My unmatched records populated the coordinates with "1".

ArcGIS 10: Exporting part of an attribute table and renaming fields

Q: I want to create a dbf file containing a few columns from my layer's attribute table to send to a collaborator. I also want to rename some of the fields so she doesn't have to translate my shorthand. Please help!

A: Gladly!

First, you want to turn off the fields you are not interested in exporting: Open your attribute table, right click the field of (un)interest, and select "turn field off". This should remove it from view, but don't worry... you can undo this command by clicking the table options in the upper right corner of the attribute table (the icon looks like a bulleted paper)and selecting "Turn all fields on".

Back to the problem at hand... repeat turning off all of the fields you aren't interested in sharing. Now export the data as a dbf (check this post for info). Use Excel to open the dbf file.

Note that you can of course export your entire attribute table to Excel and remove unwanted fields by deleting the columns in Excel instead of hiding them in ArcGIS, but sometimes there are exportation restrictions on sensitive data, so I wanted to outline how to avoid importing prohibited data into a new program.

The file still has your original field names. We didn't change those in ArcGIS 10 because field aliases are lost when you export the data, so now we just change the column names in Excel.

And now for the silliest step... Excel 2007 won't save files as dbf,so we have to save the Excel file in the old .xls format (Save As Excel 97-2003 Workbook) and re-import it into ArcGIS (notes here) to then export it again as dBase (dbf). Horray for hoops!

ArcGIS fields: What's the difference between "address" and "ARC_Street"?

From Getting to Know ArcGIS Desktop, Ormsby et al:

"When you rematch unmatched addresses, you can edit the values in ARC_Street and ARC_Zone to make matches. You would do this, for example, if you found mistakes in the (original data) table. The ADDRESS and ZIP attributes, on the other hand, preserves the original information from the (original data) table."

Q: How do I rearrange my layers in ArcGIS?

A: In your Table of Contents (left side window), switch your list type by clicking the buttons directly below the words "Table of Contents". You probably have the second button selected, "List by Source" (it looks like a water jug sitting on top of a tablecloth -- who does these graphics???). Change it by clicking the first button, "List by Drawing Order", which kind of looks like a bulleted list, which I guess makes more sense, graphically, kinda.

Joining insanity in ArcGIS 10, Excel 2010

Problem: I haven't been able to join the Excel file that contains demographic data by census tract to the census tract shape file.

My specific issue is that when I open the Join window, the joinable field names are not selectable... if the CT shapefile field name is selected, the corresponding field name in the Excel file is not available to select, and vice versa.

I downloaded both sets of info from the census website... so why the technical trubs???

After renaming, resaving, repopulating, and refilling my beer, I finally thought to check the field properties for my problem fields in both layers (open attribute table, highlight column of interest, right click column heading, select "Properties", check "Type"). Surprise! My field of interest in my shapefile layer is type "string", while the corresponding field of interest in my Excel file layer is type "double".

Solution: I created a same-type field to complete the join.

Since I couldn't seem to add a string type field to the Excel layer in ArcGIS, I ended up adding a double type field to the shapefile layer: Open attribute table, select "Add field...", named it "GeoId10d" and type "double" leaving the default for precision and scale. Then I started Editor (button that looks like a pencil connecting the dots), selected the appropriate layer to edit, opened the attribute table, right clicked on my new GeoID10d column, selected "Field Calculator" and just double clicked the "GeoID10" column (telling ArcGIS that GeoID10d = [GeoId10]... in English, setting the new GeoID10d column to copy the values from the old GeoID10 column and store those values as type double).

Once I had a matching type field, the join worked! Horray!

Importing Excel 2007 into ArcGIS 10

Save table as old Excel file .xls. Check for the following (from webhelp@esri):

1. Make sure the first row of the worksheet is properly formatted, since it will be used for the field names in ArcGIS. You should follow these general best practices for field naming, particularly if you want to join an Excel table to another table in ArcMap:

Field names must start with a letter.
Field names must contain only letters, numbers, and underscores.
Field names must not exceed 64 characters.

2. If you have cells with numeric data, dates, and so on, make sure the content is consistently formatted. ArcMap will scan the first eight rows to determine the field type that should be used. If there are other types of data in those rows, the field will be converted to text when the table is in ArcMap.

3. ArcMap can only read the first 255 characters of a cell. If you have more characters than that, ArcMap converts the field to a BLOB type and you won't be able to read its contents.

Exporting ArcGIS 10 attribute table to Excel 2007

Open attribute table. Table options in upper left (button looks like bulleted paper) -> export -> export: all records; output table: save as type dBASE Table in desired folder.

Open folder and manually select Open With: Excel

Entering data field by hand in Arc GIS

Problem: You've opened your attribute table and created a new field. It auto-populated with 0's, and no matter how many times you click those cells, it won't let you edit the attribute table.

Solution: Right click the layer and go to "Edit Features" -> "Start Editing". When you're done, go to the Editor toolbar (if it's not active, click the button around the top center that looks like a pencil connecting dots), then editor dropdown -> "Save Edits" then "Stop Editing".

It seems that you cannot add a field in the editor mode... curious.