Final Website

Please view our final project website, Digital History Methods, to see what we produced in the Spring 2014 semester!

Final Reports

To complete your work for this class, you need to do three things:

  1. Over the weekend, resolve any issues that have been filed in our Github repository for the page(s) you are responsible for on the final website. (UPDATE: Please also comment on this important new issue regarding a statement on our front page about the nature of our sources.) We will consider our final project to be in its finished form at 11:59pm this Sunday.

  2. After the project has been finalized, write a report of approximately 950 to 1,250 words. Your report should assess how well our website meets its objective of demonstrating the possibilities and limits of digital history methods to an audience of historians and scholars interested in digital humanities but new to the field. Be specific about the things that you think work well to meet this objective, and the things that would most need work or expansion in future iterations. At the end of your essay, give the final project a score of up to 30 points (with 30 being the best possible score) based on your assessment of its quality. Your final report grade will be an average of your own score together with a score of up to 30 points that I will assign your report based on how well, and with how much specific evidence, you make the case for your assessment.

  3. Respond to the final "team participation" questionnaire of the semester, which I will email to you all individually.

Thanks for a great semester, and let me know if you have any questions!

Final Project To-Do List

I’ve put the to-do list that we made in class yesterday in the README file for our Drafts repo. For the next week, do all of your work directly on the Github files. Remember to "commit" your changes so you don’t lose work! I recommend drafting your work in a text editor and saving copies on your machine just in case.

Note that if you encounter a message while editing saying that someone else has changed the file while you worked, you should see a red banner message at the top prompting you to review the other person’s changes. Click on the review changes link in that banner, and you should see the new text appear in your editing window along with yours. Now you can commit the file incorporating your changes.

Script for Counting Ads

Some of you expressed an interest in being able to quickly count all the ads in a folder and determine how many were published in a given year, decade, or month (to detect seasonal patterns across the year).

Here is a script that can do that. It is designed to work on Mac or Linux systems.

To use it, you should first download our adparsers repo by clicking on the "Download Zip" button on this page:

Download the adparsers repo as a zip

Download the adparsers repo as a zip

Unzip the downloaded file, and you should then have a directory that contains (among other things) the countads.sh script.

You should now copy the file to the directory that contains the ads you want to count. You can do this the drag-and-drop way, or you can use your terminal and the cp command. (If you forgot what that command does, revisit the Command Line bootcamp that was part of the MALLET homework. Once the script is in the directory, navigate to that directory in your terminal, and then run the command like this:

 ./countads.sh

If you get an error message, you may need to follow the instructions in the comments at the start of the script (which you can read on GitHub) to change the permissions. But if all goes well, you’ll see a printed breakdown of chronological counts. For example, when I run the script in the directory containing all our Mississippi ads, the script returns this:

TOTAL   1632 
     
DEC     ADS
1830s   1118
1840s   178
1850s   133
1860s   4
     
YEAR    ADS
1830    30
1831    54
1832    87
1833    68
1834    143
1835    157
1836    262
1837    226
1838    63
1839    28
1840    16
1841    16
1842    22
1843    33
1844    44
1845    25
1846    14
1847    1
1848    5
1849    2
1850    11
1851    17
1852    19
1853    15
1854    7
1855    9
1856    11
1857    23
1858    13
1859    8
1860    4
     
MONTH   ADS
1   100
2   89
3   103
4   130
5   160
6   161
7   188
8   150
9   150
10  149
11  146
12  86

If you choose, you can also "redirect" this output to a file, like this:

./countads.sh > filename.txt

Now you should be able to open filename.txt (which you can name whatever you want) in Microsoft Excel, and you’ll have a spreadsheet with all the numbers.

The script may seem to have limited value, but the key to its utility lies in first getting an interesting set of ads into a directory. That extends its usefulness. For example, if you wanted only to know the month distribution of ads in a particular year, you could first move all the ads from that year into a directory, and run the script from within it. You’d get lots of zeroes for all the years that you’re not interested in, but you would get the month breakdown that you are interested in. Depending on which ads you put in the directory that you are counting in, you can get a lot of useful data that can then be graphed or added into further calculations.

Final Project Thoughts

Thanks to both groups for the informative presentations in class today. Now that we all know a little bit more about where each part of the project stands, we need to make some rapid decisions about what we want our final product to look like by the end of classes.

Continue reading

Reassembling URLs from Files

As your groups have begun drafting essays for our final product, some of you have asked me how to figure out how to recompose the permalink to a Texas ad using the information in the ad’s txt filename. Here’s a quick tutorial. Continue reading

Progress Report: GeoTeam

Since our last progress report, we have completed the following tasks:

  • Clare revised the rough draft for the close reading essay. You can view the new draft at the bottom of the post.
  • Aaron revised locations_tag.py to merge location entities that are in close proximity in the ad. For example, the raw results of NER for “Sheriff of Pulaski County, Arkansas” are “Pulaski County” and “Arkansas”. The script would convert those terms into a single expression, “Pulaski County, Arkansas”. This makes it easier to generate geo-coordinates for the referenced locations and to trim down the amount of location results. Additionally, we were having problems with incomplete results due to the word “County” being spelled in lowercase and abbreviated. The new version of the script pre-processes the text files to find/replace words such as these.
  • Aaron wrote a script count_states.py to convert the output from locations_tag.py into a mapping between each ad and the states referenced in that ad. It will be used to tally the number of references to each other state in our Texas, Mississippi, and Arkansas datasets.
  • Kaitlyn has been working on example maps using Google Fusion Tables. To generate state counts, she used the Find feature of her text editor to count number of occurrences of known state names (but not initials). Once we have more accurate numbers when count_states.py is extended in functionality, we will be able to create a more accurate map.
  • Kaitlyn also test-drove Palladio. The following is her comments on it:

I was able to take a look at what Palladio has to offer for us, and I think it could be a really interesting tool because of the “point to point” mapping abilities. I quickly learned how to upload spreadsheets to Palladio and extend spreadsheets to certain variables. For example, I created a spreadsheet with columns “Year Ad Published,” “Slave Name,” “Owner Name,” “Owner Location,” “Runaway Location,” “Projected Location,” and “Permalink” and was able to link all of the location variables to a spreadsheet that contained coordinates for each place. Then, using the Palladio mapping tool, I was able to create a map that connected the Runaway Locations to the Projected Locations for each advertisement. Although I only have a few points right now, one can see how this tool could be useful for looking at how connected different places are to each other. If we want to use Palladio, we will need to start expanding the spreadsheet, which is time consuming because it requires manually inputting data. I think Palladio could be a useful tool for showing some of the outliers in our advertisement corpora.

Her comments on creating the fusion tables:

Using basic search functions, I have been taking the data that Aaron collected by running the ads through his tagging script and counting how many times state names are mentioned in each of the state corpora (I have been searching only for whole names right now; eg: “Texas” and not “TX” or “TEX”). This enables me to get a sense of what the google fusion table maps will look like with real data. The main issue that I have come across in doing this is coming up with a scale that will work across the Texas, Arkansas, and Mississippi ads. Because Arkansas and Mississippi have so many more ads than Texas, there is no way right now to line up the scales. Depending on what our final data looks like, it might be a good idea to use percentages instead of raw data. That way, the scale can be consistent as you hover over different states and see each state’s data.

Example Fusion maps:
Texas: Texas Fusion Table

Arkansas: Arkansas Fusion Table

Mississippi: Mississippi Fusion Table

Next Steps
Our next steps are to continue cleaning up our locations data. We need to finish this before we can have final numbers for number of times each state in our data set referenced other U.S. states. To make the data comparable across states and reduce the size of the data set, we will be eliminating pre-1835 ads from the results.

We will be revising our rough draft to ad more citations to back up the claims after we have hard numbers.

We will decide what tool we will use for creating our maps, whether that be Google Fusion Tables or Palladio. Both have their merits.

Rough Draft
Notes from Clare:

Over the past week, I have been going over slave advertisements from Texas and Mississippi in order to close-read and discover trends in geographical patterns or relationships. Based on the suggestions and on reading Team 1’s rough draft, I re-wrote the close-reading as a more general survey, eliminating many of the specific examples and consolidating information into about a paragraph for each state.

Rough Draft 2

Please comment on the rough draft!!

Team 1 4/14 Progress Report Part 1

As we continue to address the question of how similar Texas runaway ads were to those of other states, we have begun focusing on a handful of specific topics within that question. This week we have been working on answering some of the questions we had after the close-reading and initial digital reading of the ads. Daniel and I split up the labour by tackling different questions. Below, find results for the digital reading I conducted on my half of the questions.

Group Runaways?

One of the things I was interested in after completing the close-reading was whether Texas had a higher frequency of group runaway attempts. In TAPoR’s Comparator, a higher occurrence of the plural word “negros” suggests that Texas might have had more multi-person runaway attempts.

19th C. ads seem to have more commonly spelled “negroes” with an “e” however. (Just as a side note, this highlights the importance of checking for variations or abnormalities in spelling when conducting digital research.) Although the raw count for “negroes” is higher in Arkansas, the relative frequency based on document length is higher for Texas, although not by much.

For the word “runaways,” Texas and Arkansas are relatively equal. Just based on these word counts, it is difficult to make a conclusion about group runaways in Texas, but appears that rates of mentions for group runaways in Texas and Arkansas were roughly equivalent.

In comparison to the Mississippi corpus, Texas rates of “negros” and “negroes” are both higher.

However, use of “runaways” is significantly larger in Mississippi.

Curious about why this might be, I clicked through to the word in context section.

This snippet from the use of the word “runaways” in the Mississippi ads reveals that it was often used in relation to jailor’s notices. Even more so than other runaway ads, jailor’s notices tend to follow a very standard format, often following a word-for-word form. In Mississippi, many of them are titled “Runaways In Jail” and conclude with a note about the “law upon the subject of runaways,” as seen above. This is one example of how the presence of jailor’s notices in the mix of runaway ads can skew the trends in one direction or another.

Ultimately, the digital reading of the ads through TAPoR suggest to me that Texas did not have a significantly higher frequency of group runaways than either Arkansas or Mississippi. If we choose to pursue this question more closely, it will be necessary to do a combination of digital and close-reading to confirm one way or the other.

Note: For an in-depth explanation of how TAPoR calculates relative frequency ratios, see this earlier post explaining the calculations behind the figures.

Thieves or accomplices?

In my rough-draft of the close-reading of the ads, I noticed that many runaway ads suspect an accomplice of persuading the slave to run away, or a thief of stealing the slave for resale. These ads provide historical clues about some of the routes of aid that runaways might have had, or perhaps, instances in which slaves were made to “run away” against their will. In ads where the subscriber suspects a thief, it is not possible to know whether those suspicions are accurate or not, but it can give us a sense of the climate in that region. If subscribers are more suspicious of thieves or accomplices in one region vs. another, it may suggest a situation of lawlessness and slaveholder fear in that state.

In Texas compared to Mississippi, the word “stolen” appears relatively more frequently, but the word “thief” relatively less frequently.

Another word which appears frequently in this category of ad is “persuaded,” which appears more frequently in Mississippi. This screenshot shows snippets of how the word “persuaded” appears in the ads.

To me, these results suggest that the Mississippi ads may have had more instances of accomplice and/or thief suspicions than Texas. Texas and Arkansas had very similar rates for the words “stolen,” “thief,” and “persuaded”.

In Voyant Tools, this embedded word trends from the Telegraph and Texas Register collection shows the correlation between the words “thief” and “stolen”. Both words follow similar patterns across the corpus.

One of the benefits of Voyant is that it allows users to copy an html link to embed Voyant tools into their own blog. Rather than a static screenshot, readers can change the settings on the embedded tool. Try playing around with it!

This tool also allows you to visualize where in the corpus peaks in slave theft occur. Patterns such as these raise questions about why theft might have spiked in the Houston area at that time. This is just one example of how a digital reading can notice patterns difficult to discern through close-reading alone, but inspire further close-reading through the questions raised.

Racial Descriptors? Differences in Racial/Ethnic involvement?

In most runaway ads, the subscriber tends to give some description of the runaway’s complexion or racial status. We were interested in tracking variations in these terms across states. We were also interested in tracking runaway slaves’ involvement with various racial or ethnic groups in their geographical area.

This embedded graph from Voyant shows trends for the words “African” and “Africans” across the Telegraph and Texas Register. Over time, occurrences of these words goes down until eventually disappearing. In class, we talked about potentially finding evidence of the illegal international slave trade continuing for a while in the early years of Texas. These trends would suggest that to be the case.

Additionally, “African” appears most frequently in Texas compared to the other states, and slightly more frequently in Mississippi than in Arkansas. This confirms my suspicions from the close-read that Texas had higher rates of Africans than the other states, as well as my hunch that Mississippi and Texas, with access to ocean ports, would have higher rates of African slaves than landlocked Arkansas.

One of the terms we both noticed in our close-readings was the French word Griff(e). “Griff” and “Griffe” occur much more frequently in Texas, followed by Mississippi, and not at all in Arkansas. Tracking the word “Griffe” alongside “French” and “Mulatto” reveals some interesting trends. While Texas and Mississippi have higher use of the word Griff(e), Arkansas has higher use of the word Mulatto. Additionally, Texas and Mississippi — the states where the French word “Griff” is used — also have higher occurrences of the word “French” suggesting a more significant presence of French people or the French language in these states. Possibly, in Texas and Mississippi, subscribers were more likely to prefer the term Griff(e) to refer to someone of part white, part black ancestry, whereas in Arkansas they were more likely to prefer the term Mulatto.

This final screenshot reveals the high relative frequency of “Mexico” and “Mexican” in Texas. In Arkansas and Mississippi, these words never occur at all. This screenshot also shows how the favorites function works in Voyant. To track several related words, use the search function in Words in the Entire Corpus, then select the word and hit the favorites heart in the bottom bar. Then you can toggle back and forth from favorites and search to either look at the list of words you want to track or select more words. From here, you can select one or more words from the favorites list to track visually chart their progress across the corpus. This is very handy tool for tracking word correlation and relation.

If you are interested in looking into these runaway trends in Voyant more closely, follow this link to a saved Voyant skin containing all of the collected ads from Texas, Arkansas, and Mississippi.

Updates on Twitter Bot

You may have noticed from my posts on Twitter that today is Day of DH 2014. To make a long story short, on #DayofDH , digital humanities scholars and teachers create special blogs to document their work for that day and to connect with like-minded scholars. Check it out if you want to learn more about the DH field writ large!

DayofDH logo

It's like a holiday, for digital humanists.

For my blog, I wrote a little bit about our Twitter bot, and particularly shared how I have now set up my computer to tweet an ad automatically every morning. As I mentioned in class yesterday, we now have around 70 followers of the Twitter account, with a couple more adding each day. Exciting times!

Now that our basic idea for the Twitter bot is up and running, perhaps we can also talk about whether there is anything else we want to add to it.

One potential limitation of our current set up is that only those who have followed us are likely to see our tweets (except when one of our followers retweets an ad, which hasn’t really happened yet). But one of our stated goals in the essay was to "surprise" people by showing them an ad in a context where they don’t expect it. We will still accomplish that with our followers, but their "surprise" will be lessened by the fact that they have decided to follow our account. Any ideas about how we can increase the distribution of and audience for the Tweets, particularly among our non-followers?

Another idea that Alyssa brought up in class was to add to our account some regular "on this day" tweets. If you have ideas about how such tweets should be worded, please share them in the comments. There may be some way to word these OTD tweets in a way that solves the problem above. Open to your suggestions!

Team 1 Progress Report Part 1: Essay Rough Draft

(note: this is a draft of the essay, so please feel free to comment with suggestions for final revisions!)

How similar were Texas runaway slave advertisements to those of Arkansas and Mississippi? A collection of runaway digitized slave advertisements from a variety of newspapers spanning the years 1800 to 1865* can help answer this question. In the end, patterns of runaway advertisements in Texas, Arkansas, and Mississippi, are on the whole very similar, with some notable distinctions.

In all three states, runaway advertisements follow a standard format, usually providing similar kinds of information. Most include a description of the runaway slave(s)’ name, age, physical characteristics (such as height and complexion), distinctive marks or injuries, and notable personality traits. The ads also provide information about slave ownership or origins, when the slave escaped (or date captured, in the case of found runaway notices), where the slave escaped from, and where they are believed to be headed. When relevant, the ads provide information about suspected accomplices or a descriptions of horses used for running away. More detailed ads sometimes describe the slave’s clothing, familial relationships, hobbies, or skilled crafts. Most advertisements concluded with the subscriber’s name, and where and how they can be contacted. And as incentive, runaway notices almost always prominently advertise that a generous reward will be given for information about or capture of the runaway slave. At first glance, this consistently short, boilerplate format of runaway ads makes it difficult to really distinguish between them. The ads from Texas, Arkansas, and Mississippi start to all look practically indistinguishable, making it difficult for close-reading alone to recognize pattern breaks between the states, without the assistance of computational data. However, there are certain distinctive details that appear more in one state vs. another.

Before describing these differences, it is worth noting the similarities in runaway advertisements across state lines. In all three states, the “typical” runaway is a young male. Ads for female runaways occur disproportionately infrequently, and it is rare to see an advertisement for a child or an elderly slave. As a means of transportation, runaways often take from their masters a horse or a mule. All states feature ads which contain a “white man” suspected of stealing or persuading slaves to run away. These ads often refer to the man as a “white villain,” clearly angered over this blatant disrespect to their property, and offer a separate reward for the apprehension and conviction of the white thief. Some ads describe a specific white man who has been seen in the neighborhood, and are able to provide details of name and appearance. Others, on the other hand, treat this “white man” as a nebulous, unnamed threat. One slaveholder in Texas, for example, found it hard to believe that his slave would run away of his own volition, stating that: “It is believed that he was instigated to run away by white persons, as he has always been treated with great kindness”. For this slaveholder it was easier to blame the outside influence of a villainous white man than question a paternalistic belief in the slave’s happiness. Unlike the other two states, in Texas, the nefarious accomplice/thief is sometimes listed as a “Mexican” as well as a “white man”.

In general, Texas runaways appear to have had more interaction with Mexico and Mexicans than in the other two states – not surprising, considering the state’s shared border with Mexico. Slaveholders in Texas were conscious of the presence of Mexico, often speculating that runaways were headed to the nearby nation. Embodying both fears of white predation and of the looming Mexican border, one Austin Gazette subscriber speculated that his runaway slave was “in company with some rascally white person. It is my impression said boy is making his way west, and will, under the guidance of white men, and with the assistance of his free pass, endeavor to reach Mexico”. While law enforcement officials and slaveholders in nearby Southern states could be depended upon to support the institution of slavery and return runaway slaves, slavery in Mexico had been abolished. This made Mexico an appealing destination for runaway slaves, and a concerning one for slaveholder subscribers.

Arkansas and Mississippi runaway ads, on the other hand, contain more mentions of interaction with Native American tribes than Texas ads. These two states much more frequently mention slaves suspected to be fleeing towards Native American tribes, slaves who are part Native American, or slaves who can speak a Native American language. Whether slaves in Arkansas were described as part Cherokee, or slaves in Texas were described as being able to speak “Mexican” (Spanish), runaway ads from all of the states suggest the diversity of the United States during the 19th century. These ads create a picture of how slaves interacted with and often benefited from the diversity of cultures in the United States and bordering nations.

It is also important to remember that regions of Texas varied both culturally and geographically, and runaway patterns across the state are not homogeneous. While ads from all the states mention instances of slaves stealing and carrying weapons upon escape, these appear more frequently among Texas ads. In particular, notices of runaway slaves carrying guns and sometimes knives appear frequently in ads listed in the Austin Gazette. Significantly, this newspaper circulated in a central Texas area, closer to the Western frontier, compared to the Houston-area based Telegraph and Texas Register. Potentially, proximity to the Western frontier, and the dangers associated with that area, gave slaves more access to weapons, or made slaves feel that taking a weapon with them on their escape was more essential to their success. Similarly, the Austin Gazette more frequently mentions slaves running for Mexico, suggesting that slaves in the central region of Texas ran for Mexico more often than the gulf region (or at least their masters suspected they did).

In all of the states, one thing to keep in mind is that patterns of runaway slave advertisements may not necessarily be the same as actual runaway patterns. There were many reasons why a slaveholder may not have placed an ad for a runaway, or would have delayed placing the ad. Maybe they believed a missing slave was merely “lying out” and would return to the plantation soon on their own. Maybe the slaveholder was using personal means to pursue and recapture runaways, such as a search team of locals, and didn’t feel the need to make a public announcement. Maybe the runaway was not valuable enough to justify the cost of running an ad. Or, in some cases, the slaveholder may not have even noticed the missing slave, if the slave ran away at an opportune time when the plantation was in chaos, such as the death of a master. However, the differences that exist between Texas runaway ads and those from Arkansas and Mississippi are enough to suggest that runaway patterns in Texas were distinct from other U.S. Southern states. These differences appear to be related in part to Texas’s proximity to Mexico and to the Western frontier.

*These included ads from the Telegraph and Texas Register from the years 1836 to 1860, hosted by the Portal to Texas History and transcribed by students at Rice University and the University of North Texas; advertisements from the Austin Gazette from 1850 to 1860, transcribed by students at UNT; and a collection of ads from several newspapers from Arkansas, 1820 to 1865, and Mississippi, 1800 to 1860, transcribed and publicly available from the Documenting Runaway Slaves project at the University of Southern Mississippi.